Evaluation of The Childrens Aid Society Community Schools

1
Evaluation of TheChildrens Aid Society
Community Schools

• Kira Krenichyn
• ActKnowledge
• New York City

2
An Ongoing Commitment

• CAS has engaged in multiple evaluations since the
  inception of its community schools work (1992)
• Commissioned initial six-year evaluation in 1993
  (Fordham University)
• Commissioned second evaluation with Education
  Development Center in 1999
• Began work with ActKnowledge in 2000
• Participates in others evaluations as well

3
Some Basics on the Fordham Studies

• Collaborative Fordham team (Schools of Education
  and Social Services)
• Comparison group design (two experimental two
  control schools)
• Process evaluation for initial three years
• Outcome study for next three years
• Academic component followed two cohorts of
  students for three years

4
Developmental Triangle
core instructional program
enrichment educational cultural
removing barriers to learning development

• mental health
• social services

• health
• dental

5
CAS Community Schools

• Three Defining Characteristics
• Comprehensiveness
• Coherence
• Commitment

6
CAS Community Schools Results Documented by
Fordham Study

• Improved academic performance
• Higher attendance rates
• Positive school climate
• Improved school safety
• Greater parent involvement
• Improved student-teacher relationships
• Teachers able to focus on education

7
EDC Study (1999-2001)

• Commissioned as part of CAS-District 6 21st
  Century CLC grant
• Focus of study was program improvement as a
  result of 21st Century CLC investment
• Key findings Significant improvements in
  after-school program quality as a result of staff
  development and deliberate use of high quality
  enrichment curricula (e.g., KidzLit, Foundations)

8
Evaluation Since 2000

• Conducted by ActKnowledge
• First Year Outcome Evaluation of PS 50 Health
  Clinic
• Parent Involvement
• Community Study
• CS 61 First Year Implementation
• Development of CAS Theory of Change for
  community schools

9
CAS Community Schools Theory of Change
10
ActKnowledge Current Research Design

• 3-year study started September 2004
• Quasi-experimental CAS after-school/comparison
  group
• 6 community schools receiving 21st CCLC funding
• Theory-driven approach youth development is
  precondition to school attendance and achievement

11
ActKnowledge Year One Results Attendance

• Attendance significantly better for after-school
  participants for 2004-05 across all schools

12
ActKnowledge Year One Results State Citywide
Test Scores

• Students in after-school more likely to move to
  Proficiency (L2 to L3) on reading and math tests
• Students in after-school did about as well as, or
  slightly less well than, comparison group in
  moving from L1 to L2 in math and reading
• Students in after-school significantly more
  likely to move to Advanced (L3 to L4)

13
ActKnowledge Year One Results Duration
Dosage of Participation

• Above results were stronger if students were in
  after-school 2 years vs. 1 year
• Frequency of participation in after-school
  predicted increases in test scores

14
ActKnowledge Year One Results Youth Development

• As of 2005, after-school participants were
  scoring higher on community engagement and career
  aspirations
• After-school participants were significantly less
  likely to report watching TV or playing video
  games
• More results expected in Years Two and Three

15
ActKnowledge Year One Results Interviews
Focus Groups

• Relationships, fun, and skills were important for
  everyone
• But with concerns about discipline disrespect
• Partnerships with schools were also important
• Changes from the DOE brought feelings of upheaval
  and chaos

16
CAS Participation in Other Studies

• TASC evaluation conducted by Policy Studies
  Associates
• Mott-funded Best Practices Study (Reisner and
  Vandell)
• U.S. Office of Civil Rights Study of Special
  Education
• New York State Mental Health Study
• Results documented in summary factsheet