Texas Homeless Education Office

1
Texas HomelessEducation Office
The University of Texas at AustinCharles A. Dana
Center2901 N IH 35, Room 2.200Austin, Texas
78722

• 1-800-446-3142
• http//www.utdanacenter.org/theo

2
Performance Evaluation Training

• AGENDA
• Review performance evaluation purposes and
  processes
• Create sample outcomes and activities with
  criteria and measures
• Assess YOUR projects outcome measures for
  SMART processes

3
Performance Evaluation

• Performance measurement
• a process that systematically evaluates whether
  your efforts are making an impact/change on/to
  the clients you are serving or the problem you
  are targeting.

4
Why should programs be interested in performance
measurement?
5
Why should programs be interested in performance
measurement?

• We are all in the business of helping people,
  which means we need to
• understand whether current activities are
  working to achieve intended results.

6
Why should programs be interested in performance
measurement?

• We are all in the business of helping people,
  which means we need to
• understand whether current activities are
  working to achieve intended results,
• drive program improvement and share information
  on effective practices with others.

7
Why should programs be interested in performance
measurement?

• We are all in the business of helping people,
  which means we need to
• understand whether current activities are
  working to achieve intended results,
• drive program improvement and share information
  on effective practices with others, and
• acknowledge that high-performing programs are
  more likely to receive funding through
  competitive funding processes.

8
Creating Objectives with Measurable Outcomes

• Specific.

9
Creating Objectives with Measurable Outcomes

• Specific.
• Measurable.

10
Creating Objectives with Measurable Outcomes

• Specific.
• Measurable.
• Activity-oriented..

11
Creating Objectives with Measurable Outcomes

• Specific.
• Measurable.
• Activity-oriented.
• Realistic/feasible.

12
Creating Objectives with Measurable Outcomes

• Specific.
• Measurable.
• Activity-oriented.
• Realistic/feasible
• Time-oriented..

13
Building Blocks of Performance Measurement
14
Building Blocks of Performance Measurement

• Inputs include resources dedicated to, or
  consumed by, the programe.g., money, staff and
  staff time, volunteers and volunteer time,
  facilities, equipment and supplies.

15
Building Blocks of Performance Measurement

• Inputs include resources dedicated to, or
  consumed by, the programe.g., money, staff and
  staff time, volunteers and volunteer time,
  facilities, equipment and supplies.
• Activities are what the program does with the
  inputs to fulfill its mission, such as providing
  school supplies, conducting a tutoring or summer
  program, providing staff training.

16
Building Blocks of Performance Measurement

• Inputs include resources dedicated to, or
  consumed by, the programe.g., money, staff and
  staff time, volunteers and volunteer time,
  facilities, equipment and supplies.
• Activities are what the program does with the
  inputs to fulfill its mission, such as providing
  school supplies, conducting a tutoring or summer
  program, providing staff training.
• Outputs are the direct products of program
  activities. They usually are presented in terms
  of the volume of work accomplishede.g., number
  of students served and the number of staff
  trained.

17
Building Blocks of Performance Measurement

• Inputs include resources dedicated to, or
  consumed by, the programe.g., money, staff and
  staff time, volunteers and volunteer time,
  facilities, equipment and supplies.
• Activities are what the program does with the
  inputs to fulfill its mission, such as providing
  school supplies, conducting a tutoring or summer
  program, providing staff training.
• Outputs are the direct products of program
  activities. They usually are presented in terms
  of the volume of work accomplishede.g., number
  of students served and the number of staff
  trained.
• Outcomes are benefits or changes among clients
  during or after participating in program
  activities. Outcomes relate to measurable change
  in student knowledge, behavior, skills,
  conditions, or other attributes.

18
Performance Measurement Process
Should we adjust how we spend our resources?
How do we document our efforts?
Should we add or change use of resources to
expand our impact?
What did our efforts achieve?
19
Outputs vs. Outcomes

• An outcome is

• An output is

• Focused on what the student will gain/how will
  change

• Focused on what the program will do (activity) to
  achieve the outcome.

• A way to measure the student-level impact with
  clear targets and methods for measuring change.

• A way to quantify the frequency and intensity of
  the activity.

• Reasonably attributable (a result of) to an
  output or outputs

• Specific to the activity described

• Feasible and attainable.

• Meaningful and attainable.

If outcomes show the program works, then outputs
are needed to understand how to replicate results
20
Group ExerciseOutcome, Output, or Neither?
Answers
Examples

• School supplies given to 150 students

21
Group ExerciseOutcome, Output, or Neither?
Answers
Examples
Output

• School supplies given to 150 students

22
Group ExerciseOutcome, Output, or Neither?
Answers
Examples
Output

• School supplies given to 150 students
• 90 of students pass the TAKS

23
Group ExerciseOutcome, Output, or Neither?
Answers
Examples
Output

• School supplies given to 150 students
• 90 of students pass the TAKS

Outcome
24
Group ExerciseOutcome, Output, or Neither?
Answers
Examples
Output

• School supplies given to 150 students
• 90 of students pass the TAKS
• 75 of program staff will be MV trained

Outcome
25
Group ExerciseOutcome, Output, or Neither?
Answers
Examples
Output

• School supplies given to 150 students
• 90 of students pass the TAKS
• 75 of program staff will be MV trained

Outcome
Output
26
Group ExerciseOutcome, Output, or Neither?
Answers
Examples

• School supplies given to 150 students
• 90 of students pass the TAKS
• 75 of program staff will be MV trained
• 40 of students improve reading level one grade
  level

Output
Outcome
Output
27
Group ExerciseOutcome, Output, or Neither?
Answers
Examples
Output

• School supplies given to 150 students
• 90 of students pass the TAKS
• 75 of program staff will be MV trained
• 40 of students improve reading level one grade
  level

Outcome
Output
Outcome
28
Group ExerciseOutcome, Output, or Neither?
Answers
Examples
Output

• School supplies given to 150 students
• 90 of students pass the TAKS
• 75 of program staff will be MV trained
• 40 of students improve reading level one grade
  level
• Student attendance improves by 50

Outcome
Output
Outcome
29
Group ExerciseOutcome, Output, or Neither?
Answers
Examples
Output

• School supplies given to 150 students
• 90 of students pass the TAKS
• 75 of program staff will be MV trained
• 40 of students improve reading level one grade
  level
• Student attendance improves by 50

Outcome
Output
Outcome
Outcome
30
Example Output or Outcome?

• Outputs Outcomes

100 of students in homeless situations are
enrolled immediately
All campus staff are MV trained
31
Example Output or Outcome?

• Outputs Outcomes

100 of students in homeless situations are
enrolled immediately
All campus staff are MV trained
Students in homeless situations attendance
increased by 50
100 percent of students receive school supplies
and uniforms
32
Example Output or Outcome?

• Outputs Outcomes

100 of students in homeless situations are
enrolled immediately
All campus staff are MV trained
Students in homeless situations attendance
increased by 50
100 percent of students receive school supplies
and uniforms
75 of students who attend after school tutoring
will increase reading level by one grade level
95 students attend after school tutoring
33
Can you collect the data you need in order to
prove your outcome?

• What data will you need to collect?
• How will you collect it?
• What system will you put in place to collect the
  data?
• What criteria will tell you that you are on the
  right track?
• When do you need to begin collecting data?

34
Sample MV Outcomes

• Objective What do you hope to accomplish?
• Outcome How will you know (measure) youve
  accomplished it? What will tell you that you
  have been successful?
• Activities What activities will you put in
  place to accomplish your outcome?
• Criteria What criteria will you look at to see
  if the activities are being completed?
• Measure What will be the measure (output) of
  each criteria/activity to tell you that you that
  you are on track to meet your outcome?

35
Sample MV Outcomes

• Objective Enroll students in homeless
  situations immediately.
• Outcome 95 of Students in homeless situations
  will be enrolled within 2 days on each campus.
• Activities Train campus enrollment personnel.
• Criteria number of trainings held,
• number of staff attending
  trainings,
• number of campuses sending staff to training
• Measure 3 trainings held, Elem, Middle and HS
• 200 staff attend trainings
• 100 of campuses send at least 1 staff member

36
Sample MV Outcomes

• Objective Community service providers, families
  and students in homeless situations in the
  community will know about their right to enroll
  immediately, even without records.
• Outcome Referrals to the district from
  community service providers will increase by 25
• Activities
• Criteria
• Measure

37
Sample MV Outcomes

• Objective Attendance for students in homeless
  situations will improve district wide.
• Outcome Students in homeless situations will
  increase attendance by 50
• Activities
• Criteria
• Measure

38
Sample MV Outcomes

• Objective Students in homeless situations will
  increase academic success.
• Outcome Students in homeless situations will
  increase TAKS scores by 50 on both reading and
  Math.
• Other outcomes?
• Activities
• Criteria
• Measure

39
More Sample MV Outcomes

• Objective
• Outcome
• Other outcomes?
• Activities
• Criteria
• Measure

40
Your MV OutcomesProgram Design, Evaluation
Budget

• Objective
• Outcome
• Other outcomes
• Activities
• Criteria
• Measure

41
Thanks to

• Brooke Spellman Michelle Abbenante
• For sharing their knowledge and expertise
  regarding program evaluation, and for granting
  their permission to use much of their
  presentation delivered at the
• NAEH Conference on Ending Family Homelessness -
  February 8, 2008

42
THEO
Contact Us

• Hotline [phone]
• Barbara James 512-475-8765
• Jeanne Stamp 512-475-6898
• Tim Stahlke 512-475-9709
• Patrick Lopez 512-475-9704
• Janie Phillips 512-475-9702