Department of Psychology and Child Development California State University, Stanislaus Evaluating School Readiness Programs in Stanislaus County Methods, Challenges, and Solutions

1
Department of Psychology and Child
DevelopmentCalifornia State University,
StanislausEvaluating School
ReadinessPrograms in Stanislaus CountyMethods,
Challenges, and Solutions
Harold Stanislaw
2
The SR Evaluation Team

• Formed in 2003 to provide independent assessment
  of SR outcomes
• Harold Stanislaw and Jamie McCreary
• PhDs in psychometrics and experimental and social
  psychology
• Evaluation oversight, data analysis, preparation
  and presentation of reports
• Jennifer Esterly and Jennifer Henk
• PhDs in human development
• Instrument selection, interpretation, and training

3
The SR Evaluation Team

• Belle Jarrett
• Site coordination and data collection oversight
• Ian Summers
• Field supervisor
• Elexia Estrada, BA
• Family recruitment and teacher liaison
• Luis Molina, SCCFC
• County SR coordinator

4
Evaluationoverview
5
CommunityParticipants

• Over 3,300 participatingchildren and families
• 23 elementary schoolsfrom 11 districts
  through-out Stanislaus County

6
Evaluation Design

• Longitudinal design with assessments of SR and
  non-SR children in Grades K, 1, 3, 4, 6
• True baselines for both Cycle 1 and Cycle 2
• Obtain baseline data for each grade one year
  before SR children reach that grade
• Additional comparison data from children with no
  history of SR services

7
Instruments

• Family Background Survey
• SR services, family involvement in school, and
  home literacy activities
• Personal interview of family by teacher (usually)
  between October 1 and December 15
• Additional information from SR coordinators
• Adapted Desired Results Developmental Profile
• Rating of overall Kindergarten readiness
• Completed by teacher between weeks 6-8 of
  Kindergarten

8
Instruments

• Dynamic Indicators of Basic Early Literacy Skills
  (DIBELS)
• Timed fluency tests that predict literacy
  development
• Administered by CSU Stanislaus students between
  weeks 16-24 of Grades K, 1, and 3,and at end of
  Grades 4 and 6
• Administered in English repeated in Spanish for
  some grades and classroom types

9
Instruments

• Student Social Skills Questionnaire
• Frequency of positive behaviors(cooperating,
  showing empathy, making friends)
• Control of negative behaviors(arguing, fighting,
  getting angry)
• Completed by teacher between months 4-6 of Grades
  K, 1, and 3
• STAR scores
• Grades 2-6

10
Instruments

• Multidimensional Students Life Satisfaction
  Scale
• Ratings of satisfaction with family, friends,
  school, living environment, and self
• Predicts psychological well-being and quality of
  life
• May predict high school completion rates and
  academic success
• Completed by children towards end of Grades 4 and
  6

11
KeyOutcomes

• Percent of children with age-appropriate
• Competencies at Kindergarten entry
• Early literacy skills
• Positive social skills
• Control over negative social behaviors
• Percent of children who read at home every day

12
Challenges andsolutions
13
Working with partners

• Need assistance from 11 school districts, 23
  schools, and hundreds of teachers
• Partner roles
• Recruit families
• Locate children each year
• Provide data (e.g., conduct interviews, complete
  instruments)
• Allow access for data collection
• Provide local insight

14
Working with partners

• Meet as needed with key individuals
• District supervisors
• Principals and vice principals
• Teachers
• Accommodate as needed
• Modify instruments to facilitate completion
• Stay away during STAR testing!

15
Working with partners

• Partners deliver few if any SR services
• Provide usable feedback
• Long-term benefit that appeals to mostly to
  administrators
• Provide incentives
• Classroom supplies for Kindergarten teachers
• Pencils and stickers for all grades
• Immediate benefit that appeals mostly to teachers

16
Working with families

• Respond to concerns
• Anticipate concerns and answer questions
• Provide native language assistance
• Publicize activities to the community
• Obtain active consent to participate
• Work with schools to find solutions
• Attend Kinder Camp or Kindergarten orientation,
  Kindergarten registration, Back to School nights,
  etc.

17
Assuringdata quality

• Use validated measures whenever possible
• Train data providers to prevent drift
• Annual training workshops for teachersand CSU
  Stanislaus students
• Provide incentives to attend workshops
• Examine data for internal consistency
• Record data provider to investigate blips

18
Communicatingfindings

• Inform policy and funding decisions
• Ensure evaluation is not judgmental
• Emphasize collaboration and program development
• Use meetings or other forums to promote exchanges
  between programs and districts
• Tailor evaluation to local priorities
• Facilitate data-based decision making
• Use graphs for rapid communication of findings
• Use tables for detailed information

19
HealthServices

• Use skyscrapers to summarize service access
• All districts emphasize health screenings
  Waterford emphasized behavioral screenings

20
Early ChildhoodEducation (ECE)

• Recognize local differences
• ECE programs vary widely across districts,
  reflecting local priorities

21
Social Skills

• Compare scores to baseline look for improvement
  as service matures
• Targeting social skills has yielded strong
  benefits in Waterford

21
22
Free bookprograms

• Compare children who received service with
  children who did not
• Free book programs are highly cost-effective

23
Transition Programs

• Match by ADRDP score, language, and preschool
  experience to improve comparison
• Kinder transition programs seem beneficial

24
Preschool

• Use spider charts to illustrate program profiles
• Preschool may benefit English speakers, except
  for their social skills

25
Preschool

• Link outcomes with literature
• Preschool seems less beneficial to English
  speakers, but dual immer-sion programs may help

26
Literacy Development

• Use longitudinal approach to show long-term gains
• SR children have a literacy advantage
• SR programs may close the literacy gap for ELLs

27
Thank you!SRPEval_at_csustan.edu