Evaluation Basics for Youth Programs

1
Evaluation Basics for Youth Programs

• Ben Silliman, Ph.D.
• Dept. of 4-H Youth Development

2
Reasons to LoveProgram Evaluation

• Personal feedback on personal effectiveness
• Practical information for improving delivery
• Professional process and outcomes data to show
  accountability, impact

3
Understanding Evaluation

• Name the best thing(s) youve seen/experienced in
  a youth-at-risk program
• Name the worst thing(s) youve seen/experienced
  in a youth-at-risk program

4
Dimensions of Evaluation

• Good things come from
• A clear vision of what is happening
• A positive climate for youth
• Programs that make a difference
• Organizations that run effective programs

5
Dimensions of Evaluation

• Good things come from research-based
• Environmental scanning
• Program evaluation
• Outcome evaluation
• Organizational/
• professional evaluation

6
Environmental ScanningWhat do people need?What
resources do people have?
7
Needs and Asset Mapping

• Demographic Data
• Community Economic and Social Surveys
• Focus Groups
• Targeted Needs Assessments
• Community Coalition consensus

8
Program EvaluationHow good is the program?
9
Positive Climates/Best Practices

• Safe Environments
• Structure Appropriate for
• Supportive Relationships
• Social Inclusion
• Standards of Conduct
• Skill Learning Focused
• Contribution-Oriented
• Connections to Family, Community

10
Program Climate

• Physical and Psychological Safety
• Safe, healthy facilities
• Conflict resolution
• Wellness habits

11
Program Climate

• Appropriate Structure
• Clear consistent boundaries
• Age-based opportunities
• Effective monitoring

12
Program Climate

• Supportive Relationships
• Warmth, affirmation
• Communication
• Positive guidance

13
Program Climate

• Opportunities to Belong
• Acceptance, inclusion
• Cultural diversity/ tolerance
• Teamwork

14
Program Climate

• Positive Social Norms
• High expectations for ethics, behavior
• High achievement orientation
• Service orientation

15
Program Climate

• Efficacy and Mattering
• Growth in confidence and autonomy
• Ability to make a difference
• Meaningful changes effected with individuals,
  organizations, communities

16
Program Climate

• Opportunities for Skill Building
• Mastery of practical skills in many domains
• Ability to relate and navigate
• varied cultures
• Improvements in vocation-
• related skills

17
Program Climate

• Integration with Family, School, and Community
• Parent involvement and family activities at home
• Increased academic performance and school
  leadership
• Youth-adult partnership

18
Outcome EvaluationWhats happening with the kids?
19
Logic ModelBuilding a Game Plan

• The Big Picture Winning Legacy
• Giant Steps A Habit of Playing Well
• Small Steps Mastering the Fundamentals
• The Right Stuff Good Preparation
• Setting the Table Good Resources

20
The Logic of Evaluation

• Inputs
• Investments
• of resources
• Money
• Staff
• Volunteers
• Equipment
• Supplies
• Effects of
• constraints
• Laws
• Regulations
• Funder requirements

• Outcomes
• Short-term outcomes
• Knowledge
• Attitude
• Skills
• Aspirations
• Long-term outcomes (impacts)
• Behavioral change
• Social-economic change
• Cultural change

• Outputs
• Products of programs
• Classes taught
• Materials distributed
• Hours of service
• Participants served

21
Evaluating Outcomesin Small Steps

• Knowledge
• Attitudes
• Skills
• Aspirations

22
Outcome Domains

• Physical
• Health
• Safety
• Wellness
• Risk Reduction

23
Outcome Domains

• Intellectual/Cognitive
• Critical thinking/
• problem solving
• Academic achievement
• Project skills/mastery
• Decision-making
• Creative thinking

24
Outcome Domains

• Emotional
• Coping
• Stress management
• Optimism
• Self-efficacy
• Character

25
Organizational/ProfessionalEvaluationHow (well)
do leaders make things happen?
26
Organizational Processes

• Serving People
• Designing and Producing Better Goods and Services
• Building an Efficient and Effective Workforce

27
Six Sigma Processes

• Define the project goals and customer (internal
  and external) requirements
• Measure and determine customer needs and
  specifications benchmark competitors and
  industry
• Analyze the process options to meet the customer
  needs
• Design (detailed) the process
• to meet the customer needs
• Verify the design performance
• and ability to meet customer needs

28
Professional Evaluation

• Professional Standards
• Novice-to-Expert Rubrics and Certification for
  General Specialized Competencies
• Individual and Team goals, performance standards,
  and professional development
• Continuous quality improvement

29
Human Subjects

• Satisfaction, description is
• program feedback
• Individual performance change is consent-required
  feedback

30
Replicating and SustainingPrograms

• Stage 1 Program running as planned, having
  intended effect (Process Evaluation/Formative
  Evaluation/Quality Control)
• Stage 2 Change in knowledge, attitude, skill, or
  aspiration (Outcome Evaluation/Summative
  Evaluation/Achievements Impact)