Evidence-based Evaluation for Afterschool Programs

1
Evidence-based Evaluation for Afterschool
Programs

• Denise Huang
• CRESST/UCLA
• 1/22/07

2
In the Beginning.

• after school programs were available to students
  at a much smaller scale
• The purpose is mainly to provide monitoring
• The opportunities for students to engage in
  enrichment activities
• In the 90s, the concepts of safe haven for
  at-risk youths

3
No Child Left Behind Act (2002)

• An emphasis is placed on schools to improve
  academic achievement, particularly for students
  at risk
• Schools that are not performing well can use
  after school programs as supplemental services to
  improve student performance
• More funding becomes available, both from the
  government and private sector
• For the last 3 years, the 21st Community Learning
  Center is working with a budget of over 1 billion

4
The Formation of Afterschool Alliance

• A partnership between DOE and Mott Foundation
• A coalition of public and private organizations
• Dedicated to raise awareness of the importance of
  ASP and advocating on their behalf
• The goal is to make quality ASP available to all
  children by 2010

5
The Need of Evidences to Support Expansions

• Do afterschool programs contribute to positive
  youth development?
• What aspects of the program functioning
  contribute to these positive outcomes?

6
Indicators for program effectiveness

• Student attendance in regular school
• Performance measures (achievement tests, homework
  completion, classroom grades, language
  re-designation, school retention, future
  aspirations, etc.)
• Non-cognitive measures (safety, attitudes towards
  school, relationships with adults, social
  competence, conflict resolution skills,
  self-esteem, and self-efficacy etc.)
• Parent involvement
• Professional development
• Long-term effects (drop out rate, life
  satisfaction, etc)

7
What is known about the impacts of ASP?

• Tutoring and academic
• Inconclusive

8
Personal and Social Outcomes

• Findings in general positive
• concerns about safety
• conflict resolution
• academic self-efficacy
• study habits
• family involvement
• future aspirations

9
Long-term Outcomes

• Long-term effect study
• Dropout study
• DOJ study

10
Who Benefited Most

• Students who attended more regularly
• Female students
• LEP students
• Students who scored low initially on attitudinal
  surveys (self-efficacy, work habits, conflict
  resolution skills)
• Students with low performance in test scores

11
Summary of Effects from Current Meta-analyses

• Youth who participated in quality ASP improved
  significantly in feelings and attitudes towards
  self and learning
• Positive social behaviors
• Academic improvements
• Reduce gang participation and criminal behavior

12
What practices are effective in producing student
outcomes?

• Moving away from a deficit model
• Emphasizing social capital, setting features,
  networking, collaboration, cooperation
• Purposeful staff development
• Active learning and student participation

13
Ecological ModelAfterschool programs are social
organizations embedded with cultural values
coming from families, school, and neighborhood
climate and norms.

Community/Neighborhood
School
LAs BEST
Family
Child

Space / Community / Culture
(structure/resources)
(networks) (social, norms,
sanctions)
14
Provisional Features of Positive Developmental
Settings (Larson et. al)

• Physical and Psychological Safety
• Clear and consistent structure
• Supportive relationships
• Opportunities to belong
• Positive social norms
• Support for efficacy and mattering
• Opportunities for skill building
• Integration of family, school, and community
  efforts

15
Developmental Assets (Search Institute)
Asset In Practice
Support School and other activities provide caring, encouraging environments for children
Empowerment Children are given useful roles at home and in the community
Boundaries and Expectations Schools provide clear rules and consequences
Constructive Use of Time Children participate in music, art and other creative activities at least three hours a week
Commitment to Learning Teachers encourage children to explore and engage in stimulating activities
Positive Values Children are encouraged to help others
Social Competencies Children begin to learn how to make choices at appropriate developmental levels
Positive Identity Children are hopeful about their personal future
16
High Scope Educational Research Foundation

• Points of service assessments
• Safe Environment
• Supportive Environment
• Interaction
• Engagement

17
Evidence based training approach (CASEL)

• Sequenced - a sequenced set of activities to
  achieve skill objectives
• Active - the use of active forms of learning
• Focused program component to be focused on
  specific skills
• Explicit the targeting of specific skills

18
Evidence Based Evaluation Model
Climate Psychological Safety Clear Consistence
Structure
Staff Philosophy
Shared control
Autonomy Oriented Settings
Shared control
Student choices
Critical voice
Interaction Positive Social Norm Supportive
Relationship Opportunity to belong
Active participants

Personal relevance
Use of materials
Staff Practices
Student negotiation
Classroom structure

Networks Integration of school, family
community efforts
Lifelong learning
Interaction style