Evaluating UP4WS: First 5 Yolos

Child Signature Program

2012-2013 Evaluation Report




August 2014

table of contents

Introduction 1
Key Characteristics of Study Participants 6
Fostering Early Learning and Development 8
Classroom Quality 10
School Readiness 14
Recommendations 22
Next Steps 24
Appendix A: CLASS Scores by Quality Level
and Type of Care, 2012-13 25
Appendix B: ECERS Scores Quality Level
and Type of Care, 2012-13 26
Appendix C: Quality Criteria by Classroom,
2012-13 28

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 1
Introduction

reschool is important. High quality preschool programs support development during a childs formative
years. Decades of research have led to widely accepted findings that such support confers meaningful
long-term benefits for the children who participate and for society at large. According to well-known
studies, some of these benefits include:
Better chance of high school graduation;
1

Higher incomes in adulthood;
2
and
Strong return on taxpayer investment due to future public costs avoided.
3

It is in this context that the Universal Preschool for West Sacramento program operates.

What is UP4WS?

Universal Preschool for West Sacramento (UP4WS), the
Child Signature Program for Yolo County, is a collaborative
partnership among the City of West Sacramento, Washington
Unified School District, First 5 Yolo, First 5 California, and
the Yolo County Office of Education. UP4WS aims to
provide high quality, accessible child care and preschool for
all children in West Sacramento so they develop the skills they
need to be successful in school.

UP4WS funds a number of child care and preschool programs, including home-based child care and licensed
child care centers. These programs serve infants, toddlers, and preschool-age children. A representative from
each funded program attends monthly meetings to discuss program successes, lessons learned, and strategies to
strengthen early childhood development programs in West Sacramento. To increase the quality of child care
and preschool programs in West Sacramento, UP4WS:

Offers free, half-day, high-quality preschool programs to eligible children in West Sacramento;
Links teachers to professional development opportunities;
Provides coaching and mentoring to teachers from a Masters level teacher;
Provides opportunities for parent involvement;
Serves high-need families, including those who are low-income and very low-income; and
Utilizes tools to assess child needs and measures program outcomes.

1
The HighScope Perry Preschool Study. 2005. Lifetime Effects: The HighScope Perry Preschool Study through Age 40.
www.highscope.org/Content.asp?ContentId=219
2
Ibid.
3
Calman LJ and Tarr-Whelan L. 2005. Early Education for All: A Wise Investment. web.mit.edu/workplacecenter/docs/Full%20Report.pdf
P
UP4WS is a public-private
partnership created in 2003 through
a Universal Preschool Task Force
initiated by Mayor Cabaldon. UP4WS
has served over 3,500 children and
families in the past eight years.


Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 2
In fiscal year (FY) 2012-13, UP4WS funded 30 classes that include 502 slots in West Sacramento (Exhibit 1).
4

Child care and preschool sites that received funding from UP4WS include State Preschool, Head Start/Early
Head Start, community-based child development centers,
5
family child care homes, and public/private child
care centers. The majority of slots funded by UP4WS were either State Preschool (n=264) or Head Start/Early
Head Start (n=108), both of which are programs for income-eligible families.

Exhibit 1. Number of UP4WS-Funded Classes by Type, FY2012-13
Type of Care Infant/Toddler Preschool
TOTAL
Classes Slots
State Preschool -- 11 (264 slots) 11 264
Head Start/Early Head Start 2 (20 slots) 4 (88 slots) 6 108
Community Based Child
Development Centers
2 (35 slots) 2 (15 slots) 4 50
Family child care homes -- 7 (44 slots) 7 44
Public/private child care centers -- 2 (36 slots) 2 36
TOTAL 4 26 30 502

Classrooms funded by UP4WS are designated as Quality Enhancement (QE) or Maintenance of Effort (MOE),
depending on particular quality criteria selected by First 5 California.

Quality Enhanced (QE) classrooms have all
quality markers in place (see text box to the
right) and provide both a Master Teacher
who coaches and mentors classroom
teachers and a Family Support Specialist who
links families to needed resources.
Maintenance of Effort (MOE) classrooms
have not yet hit all the quality markers,
typically falling short in one or two areas.
All MOE classrooms have an improvement
plan, which requires teachers not reaching
the highest level of education to enroll in
courses and professional development
throughout the year. Among the 30
classrooms funded by UP4WS in FY 2012-
13, a greater number (n=19) were designated
as MOE compared to QE (n=11). Exhibit 2
shows the breakdown of classes by quality
level.

4
While UP4WS funded 30 classrooms in FY 2012-13, this evaluation focuses only on the 19 preschool classrooms.
5
Communicare and Yolo Crisis Nursery
UP4WS Preschool Quality Markers

Teacher education. Preschool
teachers have a BA in Early Childhood
Education (ECE) or related field;
infant/toddler teachers have an AA in
ECE or related field or at least 24 ECE
units with a BA or AA in other field.
Preschool Classroom Assessment
Scoring System (Pre-K CLASS).
Preschool classrooms receive a score of
5 or higher in Emotional Support, 4 or
higher in Classroom Organization, and
2.75 or higher in Instructional Support.
Environment Rating Scale (ERS).
Preschool and infant/toddler
classrooms receive an aggregate score of
5 or higher.



Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 3

Exhibit 2. Number of UP4WS Classes by Quality Level
6




6
Represent quality levels at the beginning of the fiscal year.
QE MOE
State Preschool 1 10
Head Start/Early Head Start 6 0
Community Based Child
Development Centers
4 0
Family child care homes 0 7
Public/private child care centers 0 2
TOTAL 11 19

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 4
The UP4WS Longitudinal Evaluation Study

The City of West Sacramento, in partnership with First 5 Yolo, sought assistance to evaluate the UP4WS
program with the goal of providing information to help improve teaching practices and child outcomes. In
June 2012, Harder+Company Community Research, a firm with technical expertise in evaluation and research,
as well as extensive knowledge of early childhood education and First 5 programs, was selected to conduct a
three-year longitudinal evaluation study of the short-term and long-term benefits of UP4WS programs. This
report presents findings from the first two years of the longitudinal study.

The purpose of the study is to examine the social, emotional, and academic impacts of the UP4WS program
over time. Specifically, the study examines the benefits of participating in this program in order to help identify
promising practices for preparing young children for school. The findings from this evaluation are intended to
help early educators, policymakers, and local stakeholders better understand the effectiveness and impact of the
UP4WS program on childrens development and will also help identify factors that may be influential to a
childs success in school.

The following questions are guiding this evaluation:

Parent Engagement: How often do UP4WS parents engage in home-based learning activities with
their children? What types of activities, if any, have parents engaged in to bridge children into a
school setting?
Classroom Quality: What are the quality levels of UP4WS preschool classrooms? How do the
quality levels vary by child care type? How do they compare to other preschool classrooms?
Child Development: Do children attending UP4WS demonstrate improved school readiness
skills? How do UP4WS children compare to a group of children who did not attend UP4WS?

Evaluation Design

The longitudinal study (Exhibit 3) was designed to prospectively follow a group of preschool-age children who
participated in an UP4WS preschool program in 2012-13 (UP4WS Group). To examine changes over time,
data are collected from the UP4WS Group in each of three years: preschool, kindergarten, and first grade. In
addition, data from the UP4WS Group are compared to another group of children who did not attend an
UP4WS program (Comparison Group). The purpose of comparing the two groups is to shed light on changes
that may be due to the UP4WS services in particular. It is important to note that while the Comparison Group
did not attend an UP4WS program, they may have participated in another preschool program or may have
different demographic and socioeconomic characteristics than the UP4WS Group. The Comparison Group
for this longitudinal study was selected using UP4WS enrollment lists and providing these lists to the
Washington Unified School District to identify incoming kindergarten children in the school district database
as either having participated in UP4WS (UP4WS Group) or not (Comparison Group). The Comparison
Group for this study was introduced in the kindergarten year.


Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 5
Exhibit 3. UP4WS School Readiness Longitudinal Study Evaluation Design

UP4WS
Group (n=191)
Comparison
Group (n=420)
Parent Survey
Desired Results
Developmental
Profile
Early Childhood
Environment Rating
Scale
Classroom
Assessment Scoring
System
Teachers Child
Report Kindergarten
Assessment
Teachers Child
Report Kindergarten
Assessment
School Level Data
Free/reduced price
meals
Attendance
Grade retention
Individualized
Education Program
(IEP)
English proficiency
Report card/grades
California English
Language
Development Test
(CELDT)
2012/13
Preschool
2013/14
Kindergarten
2014/15
Grade 1

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 6
UP4WS is reaching families in need.

Key Characteristics of Study Participants



Parents of children in the UP4WS Group were asked to complete a Parent Survey in spring 2013. This survey
(available in English, Spanish, and Russian) gathered basic demographic information on their child, their
family and themselves, in addition to information about their involvement in their childs school readiness. A
total of 129 parents/primary caregivers in the UP4WS Group completed a Parent Survey. When available,
comparison data is included.



As shown in Exhibit 4, over half (54 percent) of UP4WS children are Hispanic/Latino. The next largest groups
are multi-racial (17 percent) and White (15 percent). When compared to the race/ethnicity of Washington
Unified kindergartners and Yolo County kindergarteners overall, UP4WS served a greater percentage of
Hispanic/Latino children, multi-racial children, and Pacific Islander children and fewer White and Asian
children.

Exhibit 4. Childs Race/Ethnicity (n=127)

UP4WS Group
(n=127)
Washington Unified
School District
Kindergarteners*
(n=658)
Yolo County
Kindergarteners*
(n=2,214)
Hispanic/Latino 54% 44% 49%
Multi-Racial 17% 4% 3%
White 15% 31% 33%
Asian 6% 16% 11%
Black/African American 4% 4% 2%
Pacific Islander/Native Hawaiian 3% 1% 1%
Alaska/American Indian 1% 1% 1%
* Comparison data are from California Department of Education, 2012-13. Washington Unified School District n=658; Yolo
County n=2,214.

As shown in Exhibit 5, 69 percent of UP4WS parents reported speaking a language other than English with
their child (44 speak English and another language and 25 percent speak mostly a language other than
English). Of those who reported speaking a language other than English, the majority (79 percent) spoke
Spanish. This is similar to data from the Washington Unified School District, where the majority of English
Language Learner kindergarteners spoke Spanish (61 percent, data not shown).
7



7
California Department of Education, 2012-13.

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 7
Exhibit 5. Languages Spoken to Child at Home (n=126)


Over one quarter of UP4WS parents (28 percent) have not received a high school diploma. When comparing
UP4WS parents educational attainment to West Sacramento and Yolo County overall, UP4WS is serving a
higher percentage of parents with less than a high school education (28 percent compared to 19 percent in
West Sacramento and 15 percent in Yolo County).

Exhibit 6. Parents Educational Attainment

UP4WS Group
(n=118)
West Sacramento*
(n=31,264)
Yolo County*
(n=119,945)
Less than high school 28% 19% 15%
High school diploma or GED 29% 25% 20%
More than high school 43% 56% 65%
* Comparison data are from 2012 U.S. Census Bureau American Community Survey.

Exhibit 7 shows that one-third of
UP4WS parents indicated an annual household income
of $15,000 or less. Parallel to the educational
attainment data presented above, when comparing
UP4WS family household income to West Sacramento
and Yolo County overall, UP4WS is serving a higher
percentage of low-income families.

Exhibit 7. Family Household Income

UP4WS Group
(n=116)
West Sacramento*
n=11,548
Yolo County*
n=44,355
$15,000 or less 33% 11% 5%
$15,000 to $30,000 41%
35% 34%
$30,000 to $50,000 17%
$50,000 or more 9% 55% 61%
* Comparison data are from 2012 U.S. Census Bureau American Community Survey.
83 percent of parents whose child
participated in an UP4WS preschool
program reported household annual
incomes that were at or below the
Federal Poverty Level.
English and other
language
44%
English only
31%
Mostly another
language
25%

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 8
Parents report high levels of engagement in home-based and school-based
learning activities.
Fostering Early Learning and Development





Exhibit 8 shows data from the Parent Survey and the frequency with which UP4WS parents or other family
members engaged in specific activities with the child. For every activity listed, over two-thirds of respondents
indicated that they engaged in that activity with the child at least three times a week. The activities most
commonly done three or more times a week were talking about what happened in preschool (94 percent);
playing with toys or indoor games (90 percent); and practicing learning letters, words, or numbers (87
percent). With the exception of playing a game or sport together outside, UP4WS parents reported higher
levels of participation compared to a similar group of parents who participated in a First 5 San Joaquin funded
preschool program.
8


Exhibit 8. Family Engagement in Activities 3 or More Times in the Past Week
UP4WS
First 5 San
Joaquin
Talked about what happened in preschool (n=125) 94% --
Played with toys/indoor games (n=127) 90% 87%
Practiced learning letters, words, numbers (n=127) 87% 82%
Read stories/books (n=129) 77% 73%
Watched a childrens movie/TV program together (n=124) 75% 55%
Sang songs (n=127) 72% 63%
Played a game/sport together outside (n=125) 69% 87%

As shown in Exhibit 9, almost all respondents reported that, in the past year, they or someone else in the
household had met with the childs preschool teacher (97 percent) and had talked to the teacher about the
childs development or behavior (93 percent). Fewer respondents said they had attended a parent advisory or
council meeting (60 percent), attended a program or class event such as a play or a fair (51 percent), or helped
with preschool activities or trips (36 percent). Similar to the data above, in all but one area, UP4WS parents
reported higher levels of participation in school-based activities compared to a similar group parents who
participated in a First 5 San Joaquin funded preschool program.


8
Data are from Year 1 of the First 5 San Joaquin School Readiness Evaluation Study (2008). Parents of children who participated in a First 5
funded preschool program were asked to complete a survey in the spring of their childs preschool year.

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 9
Exhibit 9. Preschool Engagement Activities in Past Year
UP4WS
First 5 San
Joaquin
Met child's preschool teacher (n=128) 97% 91%
Talked to teacher about child's development/behavior (n=126) 93% 84%
Attended parent advisory/council meeting (n=125) 60% 32%
Attended a program or class event (n=125) 51% 27%
Helped with preschool activities or trips (n=128) 36% 43%









Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 10
In general, UP4WS classrooms receive high quality rating scores.


Classroom Quality


This evaluation made use of the framework established by First 5 California to measure the quality levels of
preschool programs. Quality standards are measured in three ways: MOE/QE designations (see Introduction),
the Classroom Assessment Scoring System (CLASS), and the Environment Rating Scale (ERS). The CLASS
focuses on the quality of teacher-child interactions, while the ERS focuses on activities and materials in the
preschool classroom that provide for a safe and stimulating learning environment. Together, data from these
assessments can inform where programs may need additional resources in order to better support a high
quality learning environment.

Classroom Assessment Scoring System (CLASS)
The Classroom Assessment Scoring System (CLASS) is an observation instrument that assesses the quality of
interactions among teachers and children across three domains: Emotional Support, Classroom Organization,
and Instructional Support. According to the Pre-K CLASS Dimension Guide:
9


Emotional Support refers to specific teaching behaviors that help children develop warm, supportive
relationships, experience enjoyment and excitement about learning, feel comfortable in the classroom,
and experience appropriate levels of autonomy or independence.
Classroom Organization describes specific teaching behaviors that help children develop skills to
regulate their own behavior, get the most learning out of each day, and maintain interest in learning
activities.
Instructional Support refers to specific teaching behaviors that support childrens cognitive
development and language growth.
Certified CLASS observers rate the classroom interactions on a scale from 1 (low) to 7 (high). To be eligible for
First 5 California Child Signature Program (CSP) funding, classrooms must meet specific domain score
standards:

A score of 2.75 for Instructional Support;
A score of 3 for Classroom Organization; and
A score of 5 for Emotional Support.


9
Teachstone Training, LLC. 2011. Pre-K CLASS Dimensions Guide.

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 11
In FY 2012-13, all 19 UP4WS-funded preschool classrooms were assessed using the CLASS 12 were MOE
and 7 were QE.
10
On average, the UP4WS preschool classrooms exceeded the CSP quality standards, with
average scores of 6.2 on Emotional Support, 5.9 on Classroom Organization, and 3.8 on Instructional Support
(Exhibit 10, purple bars). In addition, UP4WS scored higher across all three domains compared to Educare
11

preschools (light purple bars) and a national Head Start sample
12
(grey bars).

Exhibit 10. Average CLASS Scores by Domain


Although, UP4WS preschool classrooms average CLASS scores exceeded First 5 California CSP domain score
standards, five of the 19 UP4WS preschool classrooms assessed were not meeting the Instructional Support
standard (a score of 2.75 or higher). Of the five classrooms that were not meeting the Instructional Support
standard, three were QE classrooms (two Head Start classrooms and one State preschool classroom) and two
were MOE classrooms (both of which are state preschool classrooms).
13




10
Infant/toddler and family child care preschool programs were not assessed in FY 2012-13 as the Pre-K CLASS was developed for use in
preschool classrooms. The Infant Toddler CLASS was not available in FY 2012-13.
11
UNC FPG Child Development Institute. Educare Implementation Study Findings. August 2012.
http://eln.fpg.unc.edu/sites/eln.fpg.unc.edu/files/FPG-Demonstrating-Results-August-2012-Final.pdf. Educare Schools provide high-quality
instruction serving at-risk children from birth to five years old. Educare Schools are full-day and full-year. The average annual cost of the
Educare model is $20,000 per child.
12
A National Overview of Grantee CLASSTM Scores in 2013. https://eclkc.ohs.acf.hhs.gov/hslc/data/class-reports/class-data-2013.html
13
Please see Appendix A for a list of CLASS scores by classroom and quality level.
6.0
5.6
2.7
6.1
5.5
3.4
6.2
5.9
3.8
Emotional Support
Classroom Organization
Instructional Support
UP4WS (n=19)
Educare (n=64)
Head Start (n=359)
Quality
standards

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 12
ECERS-R

The Early Childhood Environment Rating Scale-Revised (ECERS-R) is a reliable and valid observational
assessment tool that measures the quality of the physical classroom environment. The ECERS-R scale was
designed specifically to measure the quality of licensed child care centers serving children ages two and a half
through five years. It consists of 43 items in seven subscales Space and Furnishings, Personal Care Routines,
Language-Reasoning, Activities, Interaction, Program Structure, and Parents and Staff. Each item is scored
with a scale ranging from one to seven with one being inadequate, five being good, and seven being
excellent. Research supports that an overall score of five or better suggests a level of program quality that has
better outcomes for children.

Exhibit 11 presents the overall average ECERS-R score and the average scores for each of the seven subscales
for the 18 UP4WS preschool classrooms that were assessed.
14
Overall, scores demonstrated that UP4WS
classrooms are meeting the quality standards for the physical classroom environment (overall score of five or
above).

Exhibit 11. Average ECERS-R scores for UP4WS Preschool Classrooms (n=18)


Similar to the CLASS data presented above, UP4WS preschool classrooms average ECERS-R overall scores
exceeded quality standards; however, of the 18 classrooms scored, two UP4WS preschool classrooms did not
receive an ECERS-R overall score of five or better. Of the two classrooms that did not meet the standards, both
were QE, Head Start classrooms.
15




14
The Yolo Crisis Nursery Preschool Program was assessed using the FCCERS. Data was not included in this section of the report. Overall
FCCERS score was 5.2.
15
Please see Appendix B for a list of ECERS scores by classroom and quality level.
6
6.7
6.5
6
5.3
5.9
5.5
6.3
ECERS-R Overall Score
Subscale 7 Parents and Staff
Subscale 6 Program Structure
Subscale 5 - Interaction
Subscale 4 - Activities
Subscale 3 Language Reasoning
Subscale 2 Personal Care Routines
Subscale 1 Space and Furnishings
Quality
standards

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 13
Summary of Quality Criteria

To understand the extent to which UP4WS preschool classrooms are meeting quality standards, data were
summarized across five quality measures:

Designated as a QE classroom;
Received a score of 2.75 or higher on the CLASS Instructional Support;
Received a score of 3 or higher on the CLASS Classroom Organization;
Received a score of 5 or higher on the CLASS Emotional Support; and
Received an overall ECERS score of five or higher.
As shown in Exhibit 12, the majority of UP4WS preschool classrooms met four of the five quality measures
(n=13) with only three classrooms meeting each of the five quality measures.
16


Exhibit 12. Number of UP4WS-Funded Preschool Classrooms Meeting Quality Measures (n=19)

Met all 5
measures
Met 4 of the 5
measures
Met 3 of the 5
measures
Number of UP4WS preschool classrooms 3 13 3


16
See Exhibit C for more detail.

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 14
Children made large developmental gains after participating in an UP4WS
preschool program.

School Readiness




Data on children who participated in an UP4WS program were collected by preschool teachers and
kindergarten teachers using two instruments. The first set of data reported comes from the DRDP-R
(preschool), followed by data from the Teachers Child Report (TCR) (kindergarten). Both instruments rely on
teacher-observed and self-reported data and were designed to assess childrens school readiness skills in a
variety of developmental domains. DRDP-R data were collected by preschool teachers in fall 2012 and spring
2013, while TCR data was collected in the fall of 2013 once the child reached kindergarten.

Desired Results Developmental Profile-Revised (DRDP-R)

The DRDP-R was completed on 259 children ages 3-5 who participated in an UP4WS preschool program in
2012-13. Through the DRDP-R, teachers rate the development of each student on a continuum ranging from
Not Yet, Exploring, Developing, Building, and Integrating. Children in the UP4WS Group made large gains
from pre to post in each area of the DRDP-R (Exhibit 13). Children made the greatest gains in the Language &
Literacy Development and Self & Social Development domains.

While large developmental gains were seen from fall to spring, nearly half of children had not yet reached the
level of Building or Integrating across all items in the Mathematical Development and Language & Literacy
Development domains by spring (48 percent and 42 percent, respectively). Although this data includes three,
four, and five year olds (i.e., children who will be getting another year of preschool and children who will be
going to kindergarten in the fall), the findings serve as an opportunity to learn more about the school readiness
needs of children in West Sacramento and to plan resources to guide continuous program improvement.

Exhibit 13. Percent of UP4WS Children Building or Integrating all items by DRDP-R Domain
(n=259)

* Measures were only assessed for preschool Dual Language Learners who are learning to speak English
(n=131 fall 2012; n=132 in spring 2013).
77%
75%
60%
58%
55%
48%
42%
23%
30%
10%
21%
6%
7%
3%
Health
Physical Development
Cognitive Development
English Language Development*
Self & Social Development
Mathematical Development
Language & Literacy Development
Pre
Post

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 15
.
Exhibit 14 compares spring DRDP-R scores between
native English speakers and English Language Learners.
With the exception of Physical Development, English
Language Learners had lower pre-DRDP-R scores
compared to Native English speakers; however, from pre to post, English Language Learners exhibited greater
change in the areas of Health and Mathematical Development.

Exhibit 14. Percent of UP4WS Children Building or Integrating all items, by DRDP-R Domain and
English Language Learner Status

Pre Post
Percentage Point
Change
Health

Native English Speakers (n=121) 33% 84% 51%
English Language Learners (n=128) 14% 70% 56%
Physical Development
Native English Speakers (n=121) 28% 81% 53%
English Language Learners (n=128) 31% 69% 38%
Cognitive Development
Native English Speakers (n=121) 12% 66% 54%
English Language Learners (n=128) 8% 54% 46%
Self & Social Development
Native English Speakers (n=121) 10% 63% 53%
English Language Learners (n=128) 2% 47% 45%
Mathematical Development
Native English Speakers (n=121) 12% 48% 36%
English Language Learners (n=128) 2% 48% 46%
Language & Literacy Development
Native English Speakers (n=121) 6% 46% 40%
English Language Learners (n=128) 0% 38% 38%






Half of children who participated in an
UP4WS preschool program were
English Language Learners.


Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 16
UP4WS Children Were Just as Ready for Kindergarten as the Comparison Group
Despite Greater Poverty and Limited English Proficiency




Kindergarten Teachers Child Report (TCR)

Preschool age children who participated in an UP4WS preschool program (UP4WS Group) in 2012-13 were
followed longitudinally into kindergarten. Kindergarten teachers completed the Teachers Child Report (TCR)
in the fall of 2013 on 183 children in the UP4WS Group (96 percent response rate). Where applicable,
comparisons are made to a group of children who did not attend an UP4WS preschool program (Comparison
Group; n=420, representing a 97 percent response rate). Note that while the Comparison Group did not attend
an UP4WS preschool program, they may have attended another program prior to kindergarten.

The TCR asks questions within four sections:

1. Emerging Cognitive Skills
17
measures motor and physical skills, literacy skills, and cognitive skills;
2. Social and Emotional Skills
18
assesses a childs social and emotional behaviors in the classroom;
3. Preschool Learning Behavior examines a childs approaches to learning; and
4. Kindergarten Transition asks one question about a childs ease of adjustment during the first weeks
of kindergarten.
UP4WS children, on average, represent a higher need population than the Comparison Group. They are more
likely to come from a family with limited English proficiency and/or qualify for free or reduced lunch. This
finding is reflective of the UP4WS recruitment strategy/eligibility requirements the majority of students
participating in UP4WS come from state-funded preschools and Head Start settings. Exhibit 15 shows that
over half (52 percent) of UP4WS children come from families with limited English proficiency compared to 33
percent in the Comparison Group. Additionally, there is a significantly higher proportion of children among
the UP4WS group that qualify for free or reduced priced lunchesover three-quarters (77 percent) of UP4WS
children qualify compared to 44 percent in the Comparison Group.
19


Exhibit 15. Percent of Children among UP4WS and Comparison Group with Socioeconomic
Characteristics

UP4WS
(n=191)
Comparison Group
(n=407)
Limited English proficiency 52% 33%
Qualify for free or reduced priced lunch 77% 44%

Despite these significant differences, findings presented below show that with the exception of emergent
literacy (an area in which UP4WS children scored significantly better than a subset of the Comparison Group;
see Exhibit 18
20
), UP4WS children were just as ready for kindergarten as their peers in the comparison group.


17
Child Accomplishments on the TCR.
18
Classroom Conduct on the TCR.
19
p<.001
20
Comparisons were made between the UP4WS Group and subset of high need children from the Comparison Group.

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 17

TCR School Readiness Assessment Areas

The following section presents data from each item on the TCR, comparing UP4WS children to the
Comparison Group.

Emerging Cognitive Skills

As shown in Exhibit 16, the majority of children in the UP4WS Group demonstrated high competency in
motor and physical skills, with percentages similar to the Comparison Group.

Exhibit 16. Teachers Child Report: Motor and Physical Skills
Percent who reported yes
Motor/Physical Skills Items
UP4WS
(n=182-183)
Comparison
Group
(n=404-406)
Does not trip, stumble or fall easily
1
95% 90%
Does not stutter or stammer
2
89% 92%
Can button his/her clothes 87% 87%
When speaking, is understandable to a stranger 82% 84%
Holds a pencil properly 81% 86%
1
Item on the TCR asks if child trips, stumbles, or falls easily. The percentage presented here represents those who answered no.
2
Item on the TCR asks if child stutters or stammers. The percentage presented here represents those who answered no.

Five key cognitive skills that teachers observed and assessed on the TCR comprise emergent literacy skills. As
shown in Exhibit 17 about three quarters or more of the children in the UP4WS Group were able to do three of
the five skills, and closer to half were able to recognize the letters of the alphabet or count to twenty. Again,
results were similar in the Comparison Group.

Exhibit 17. Teachers Child Report: Emergent Literacy Skills
Percent who reported yes
Emerging Literacy Items
UP4WS
(n=181-183)
Comparison
Group
(n=403-406)
Can write his/her own first name 86% 86%
Can identify the colors red, yellow, blue, and green by name* 79% 80%
Writes/draws rather than scribbles 74% 77%
Can recognize most or all letters of the alphabet 56% 55%
Can count to twenty 46% 52%
*Answer choices for this item were: All of them, Some of them, or None of them. The percentage presented here
represents those who answered All of them.

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 18

As described earlier, the evaluation also conducted an analysis across all TCR results of a subset of the
Comparison Group that more closely matches the UP4WS Group. Data show that across all TCR areas,
Emergent Literacy was the only one in which there was a statistically significant finding children in the
UP4WS group had significantly higher mean scores (p<.05) in the area of Emergent Literacy
21
compared to a
subset of the Comparison Group (3.38, dark purple bar, compared to 3.0, light purple bar).
22
However, both
UP4WS children and High-Need Comparison Group children, scored significantly below a group of children
in the Comparison Group who were not identified as high need (i.e. do not qualify for free or reduced price
meals, grey bar). This preliminary finding suggests that UP4WS may be helping to mitigate existing risk
factors (socio-economic gap) and is increasing the likelihood that UP4WS children are as ready for
kindergarten as their peers in the area of emergent literacy.

Exhibit 18. Emergent Literacy TCR Mean Scores


Exhibit 19 shows three other cognitive skills measured in the Child Accomplishments section of the TCR. The
majority of children in the UP4WS Group recognized their first name in print and looked at picture books,
while most did not pretend to write often, according to teachers. These findings were very similar to those of
the Comparison Group.

Exhibit 19. Teachers Child Report: Other Cognitive Skills
Percent who reported yes
Other Cognitive Skill Items
UP4WS
(n=181-183)
Comparison
Group
(n=405-407)
Recognizes his/her first name in print 92% 92%
Looks at books with pictures and pretends to read (ever) 83% 84%
Likes to write or pretend to write often 23% 26%



21
Scores for Emergent Literacy ranged from a minimum of zero to a maximum of five, which indicates high emerging literacy.
22
p<.05
3.0
3.38
4.15
High-Need Comparison Group
UP4WS
Non High-Need Comparison Group

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 19
Social and Emotional Skills

The Classroom Conduct section of the TCR examined childrens social and emotional development in the
classroom with a specific focus on problem behavior. Children were observed by teachers in three main areas:
aggressive behavior, hyperactive behavior, and withdrawn behavior. Four items pertained to aggressive behavior
as shown in Exhibit 20. On all four items, less than ten percent of children in either the UP4WS Group or the
Comparison Group were reported to exhibit the behavior often.

Exhibit 20. Teachers Child Report: Aggressive Behavior

Percent who reported very true
or often true
Aggressive Behavior Items
UP4WS
(n=180-182)
Comparison
Group
(n=402-406)
Disrupts ongoing activities 8% 7%
Disobeys rules or requests 7% 6%
Hits or fights with others 7% 4%
Has temper tantrums or hot temper 4% 2%

Exhibit 21 presents three items that measured childrens hyperactive behavior. In the UP4WS group, over ten
percent of the children were assessed as having difficulty concentrating and sitting still.

Exhibit 21. Teachers Child Report: Hyperactive Behavior

Percent who reported very true
or often true
Hyperactive Behavior Items
UP4WS
(n=182-183)
Comparison
Group
(n=402-405)
Cant concentrate, cant pay attention for long 13% 10%
Is very restless, fidgets all the time, cant sit still 12% 8%
Is nervous, high-strung, or tense 4% 3%

Teachers measured withdrawn behavior using seven items on the TCR (Exhibit 22). Very few children were
assessed by their kindergarten teacher as worrying about things or seeming unhappy. A slightly greater
percentage were assessed as acting too young for their age (seven percent), lacking confidence (eight percent),
or hard to understand when speaking (nine percent). Comparison Group results, once again, were very
similar.

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 20
Exhibit 22. Teachers Child Report: Withdrawn Behavior

Percent who reported very true
or often true
Withdrawn Behavior Items
UP4WS
(n=182-183)
Comparison
Group
(n=404-405)
Worries about things for a long time 1% 2%
Often seems unhappy, sad, or depressed 2% 2%
Keeps to him/herself, tends to withdraw 2% 4%
Often seems sleepy or tired 3% 1%
Acts too young for age 7% 7%
Lacks confidence in learning new things or trying new activities 8% 5%
Hard to understand what he/she is saying 9% 7%


Preschool Learning Behavior

The TCR also measures Preschool Learning Behavior by asking teachers to consider childrens approaches to
learning, such as attention, persistence and frustration tolerance (Exhibit 23). In general, there were positive
observations made by the teachers on the children in both the UP4WS Group and the Comparison Group.

Exhibit 23. Teachers Child Report: Preschool Learning Behavior (n=182-183)

Percent who reported very true
or often true
Preschool Learning Behavior Items
UP4WS
(n=182-183)
Comparison
Group
(n=402-406)
Cooperates in group activities 65% 70%
Sticks to an activity for as long as can be expected for a child this age 65% 65%
Pays attention to what you say 54% 58%
Is distracted too easily by what is going on in the room or seeks
distraction
18% 12%
Acts without taking sufficient time to look at the problem or work out a
solution
10% 10%
Cannot settle into an activity 8% 6%
Adopts a dont care attitude to success or failure 7% 5%
Shows little determination to complete an activity 7% 4%
Tries hard but concentration soon fades and performance deteriorates 6% 3%


Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 21
Kindergarten Transition

The final item on the TCR measures ease of transition into kindergarten by asking teachers about each childs
level of difficulty with the first weeks of kindergarten. Overall, teachers reported that 79 percent of UP4WS
children had a somewhat or very easy transition to kindergarten compared to 76 percent of the Comparison
Group (Exhibit 24).

Exhibit 24. Teachers Child Report: Kindergarten Transition
How difcult or easy were the frst weeks of
kindergarten for this child?
UP4WS
(n=182)
Comparison Group
(n=384)
Somewhat or very easy 79% 76%




Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 22
Recommendations

Continue efforts to serve high-need children in West Sacramento. As evident from data
collected through this study, UP4WS is providing services and supports to high-need children and
families in West Sacramento. These efforts are supported by research that suggests that an
achievement gap in education exits. Namely, students from a variety of racial, ethnic, and economic
backgrounds have different levels of academic performance. Therefore, UP4WS should continue its
outreach efforts to provide valuable services to the citys highest need families.
Review and monitor child assessment data to inform professional development needs.
While large developmental gains were seen on the DRDP, less than half of children had not yet
reached the level of school readiness in the Mathematical Development and Language & Literacy
Development domains. These findings serve as an opportunity for staff and teachers to learn more
about the developmental needs of children in West Sacramento and to plan resources to guide
continuous program improvement.
Increase opportunities for parents to be involved in their childs learning. Children whose
parents are involved in their childs education are more likely to succeed in school. Data from the
Parent Survey shows that while nearly all respondents reported that they had met with the childs
preschool teacher and had talked to the teacher about the childs development or behavior, fewer
respondents had attended a parent advisory or council meeting, attended a program or class event
such as a play or a fair, or helped with preschool activities or trips. To continue to increase parent
engagement, UP4WS should offer a range of opportunities, including those that are culturally and
linguistically appropriate for families receiving services from UP4WS.
Provide resources to continue to increase the number of classrooms who meet the Quality
Enhancement criteria. Classrooms funded by UP4WS are designated as Quality Enhancement
(QE) or Maintenance of Effort (MOE) depending on their quality level. To continue to provide high
quality care for all children in West Sacramento, UP4WS should continue to focus efforts on
increasing the number of classrooms designated as Quality Enhancement. Additionally, since family
child care is generally the most affordable child care option in West Sacramento; UP4WS may
consider focusing their efforts on working with these care providers. Strategies may include
identifying outreach activities to increase engagement among family child care providers, discussing
professional development needs, identifying coaching and technical assistance strategies, and working
with other First 5 counties who have successfully supported family child care providers in advancing
to a QE level.
Continue efforts to share evaluation findings. As described above, sharing this data with local child
care providers (and preschool teachers) can increase capacity in the areas of improving the early care and
education environments, improving parent involvement, and improving the overall quality of early care
and education. Additionally data sharing can, over time, improve the overall quality of early care data
such as the DRDP, CLASS, and ERS. Improved data quality can more accurately identify the strengths and
areas of opportunity of early care and education sites. Finally, data sharing can also increase awareness of
the importance of universal early education opportunities among policy makers and funders. Therefore,
UP4WS may consider developing strategies and allocating resources for sharing evaluation findings. This

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 23
may include developing classroom level reports (for both preschool and kindergarten classrooms),
presenting findings to funders and policy makers, and outreaching to participants (e.g. teachers, school
staff, and parents).



Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 24
Next Steps

As illustrated in the longitudinal evaluation study design (see Exhibit 3), children in the UP4WS Group (and
Comparison Group) will be in first grade in the fall of 2014 (Cohort 1). We will also be introducing a new
Group (Cohort 2) which will include children who attended an UP4WS preschool program in FY 13-14 who
are entering kindergarten in fall 2014. Described below are the next steps for the longitudinal evaluation.

Cohort 1

Work with Washington Unified staff to gather school level data (e.g. attendance, report cards,
free/reduced price meal participation) from the UP4WS Group and Comparison Group.

Cohort 2

Identify children who participated in an UP4WS preschool program in FY 13-14;
Develop and administer a brief Parent Survey to be completed by all parents of children entering a
Washington Unified kindergarten in fall 2014; and
Collaborate with Washington Unified school staff to administer the Teachers Child Report and
Parent Survey to all children in kindergarten in fall 2014.

Data collected from each cohort will be analyzed in order to study changes that have occurred over time and
also to consider whether early childhood experiences have an effect on school readiness. Additionally,
demographic differences within and between groups will be explored in reference to child outcomes and
program participation.






Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 25
Appendix A: CLASS Scores by Quality Level and
Type of Care, 2012-13

Classroom Name
Quality
Level
Type of
Care
Emotional
Support
Average
Classroom
Organization
Average
Instructional
Support
Average
Reyes Full Day State Preschool QE State 5.68 5.22 2.22
Gonzales AM State Preschool MOE State 6.43 6.20 3.63
Gonzales PM State Preschool MOE State 6.43 6.20 3.63
Sedilko AM State Preschool MOE State 6.25 6.33 3.78
Sedilko PM State Preschool MOE State 6.25 6.33 3.78
Cheng AM State Preschool MOE State 5.23 5.22 2.67
Cheng PM State Preschool MOE State 5.23 5.22 2.67
Ladd AM State Preschool MOE State 6.00 6.00 3.45
Ladd PM State Preschool MOE State 6.00 6.00 3.45
Morales AM State Preschool MOE State 5.42 5.78 3.55
Morales PM State Preschool MOE State 5.42 5.78 3.55
Learning Ladder Blue Room MOE Public/private 6.63 6.17 5.33
Learning Ladder Green Room MOE Public/private 6.56 5.86 4.97
Communicare Infant/Toddler QE
Community Based Child
Development Ctrs
6.48 6.53 4.63
Communicare Preschool QE
Community Based Child
Development Ctrs
6.48 6.53 4.63
Yolo Crisis Nursery Infant/Toddler QE
Community Based Child
Development Ctrs
6.77 6.53 5.64
Yolo Crisis Nursery Preschool QE
Community Based Child
Development Ctrs
6.77 6.53 5.64
Alyce Norman Head Start QE Head Start 6.50 6.11 2.67
Westfield Head Start QE Head Start 6.58 5.78 2.11
West Capitol Head Start AM QE Head Start 6.33 4.90 2.77
West Capitol Head Start PM QE Head Start 6.83 6.33 3.57

CLASS Domains
UP4WS
QE Classrooms
(n=7)
UP4WS
MOE Classrooms
(n=12)
Emotional Support 6.45 5.98
Classroom Organization 5.92 5.92
Instructional Support 3.37 3.71

Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 26
Appendix B: ECERS Scores Quality Level and
Type of Care, 2012-13

Classroom Name
Quality
Level
Type of Care Average Score
Reyes Full Day State Preschool QE State 6.2
Gonzales AM State Preschool MOE State 6.5
Gonzales PM State Preschool MOE State 6.5
Sedilko AM State Preschool MOE State 6.3
Sedilko PM State Preschool MOE State 6.3
Cheng AM State Preschool MOE State 6.1
Cheng PM State Preschool MOE State 6.1
Ladd AM State Preschool MOE State 6.2
Ladd PM State Preschool MOE State 6.2
Morales AM State Preschool MOE State 5.9
Morales PM State Preschool MOE State 5.9
Learning Ladder Blue Room MOE Public/private 7.0
Learning Ladder Green Room MOE Public/private 7.0
Communicare Preschool QE
Community Based Child
Development Centers
6.2
Alyce Norman Head Start QE Head Start 4.7
Westfield Head Start QE Head Start 5.2
West Capitol Head Start AM QE Head Start 4.9
West Capitol Head Start PM QE Head Start 5.1




Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 27

ECERS-R and Subscales
Average Scores
(n=18)
QE Scores
(n=6)
MOE Scores
(n=12)
Subscale 1 Space and Furnishings 6.3 5.46 6.75
Subscale 2 Personal Care Routines 5.5 4.31 6.07
Subscale 3 Language Reasoning 5.9 4.96 6.42
Subscale 4 - Activities 5.3 5.36 5.29
Subscale 5 - Interaction 6.0 5.43 6.20
Subscale 6 Program Structure 6.5 5.55 6.89
Subscale 7 Parents and Staff 6.7 6.67 6.67
ECERS-R Overall Score 6.0 5.40 6.33



Prepared by Harder+Company Community Research UP4WS Evaluation Report August 2014 28
Appendix C: Quality Criteria by Classroom,
2012-13

Classroom/Site
Is
classroom
QE?
Did
classroom
meet CLASS
Emotional
Support
quality
criteria?
Did
classroom
meet CLASS
Classroom
Organization
quality
criteria?
Did
classroom
meet CLASS
Instructional
Support
quality
criteria?
Did
classroom
meet
overall
ERS
quality
criteria?
23

Alyce Norman Head Start X X X

Cheng AM State Preschool

X X

X
Cheng PM State Preschool

X X

X
Communicare Preschool X X X X X
Learning Ladder Blue Room

X X X X
Learning Ladder Green Room

X X X X
Gonzales AM State Preschool

X X X X
Gonzales PM State Preschool

X X X X
Ladd AM State Preschool

X X X X
Ladd PM State Preschool

X X X X
Morales AM State Preschool

X X X X
Morales PM State Preschool

X X X X
Reyes Full Day State Preschool X X X

X
Sedilko AM State Preschool

X X X X
Sedilko PM State Preschool

X X X X
West Capitol Head Start AM X X X X

West Capitol Head Start PM X X X X X
Westfield Head Start X X X

X
Yolo Crisis Nursery Preschool X X X X X


23
With the exception of Yolo Crisis Nursery, all other ERS scores were ECERS.