PolicyProgram Evaluation

1
Policy/Program Evaluation
2
The Policy Process Model
Source See Bretschneider
3
Program EvaluationYour Name __________

• Working Definition
• The _______________ of the
• ____________ or _____________ of a
• _______________ compared to a set of
• ___________________, as a means of contributing
  to the
• _____________ of the policy or program.

4
Program Evaluation Word Bank

• explicit or implicit standards
• Improvement
• operations
• outcomes
• program or policy
• systematic assessment

5
Program Evaluation

• Working Definition
• The systematic assessment of the
• operations or outcomes of a
• program or policy compared to a set of
• explicit or implicit standards, as a means of
  contributing to the
• improvement of the policy or program.

6
Evaluation from the Legislative Perspective

• Evaluation
• Internal or external
• Review of program outcomes and processes
• To determine whether it is indeed in accord with
  congressional intent as expressed in the
  authorizing legislation.
• If not working well subject to Modification or
  Termination.
• Frequently the weakest link in the policy process

Rahm 200439
7
Program Evaluations Central Task

• To successfully conduct any form of Program
  Evaluation the core issue is always to uncover
  and verify one or more CAUSE/EFFECT RELATIONSHIPS
  regardless of whether the focus is on outcomes or
  process
• Program Evaluation has this in common with
  traditional social science research

8
Examples of Cause/Effect Relationships in
Evaluation Research

• Air bags reduce traffic injuries and fatalities
• WIC produces healthier birth outcomes
• Weatherization efforts reduce energy consumption
• TQM reduces administrative costs
• Headstart reduces the gap in education attainment
  levels for poor children

9
Formative versus Summative

• The context of the distinction is that of the
  INTENTION of the Evaluation
• Formative evaluation focuses on evaluation
• aimed at enhancing the design of a program
• or policy
• Office Space (What is it you would say you do?)
• Summative evaluation focuses on results
• from a program or policy
• Outcome versus Process is a distinction of what
  stage of a program is being looked at

10
Purposes for Evaluation

• Principle Who wants what evaluated !
• fulfilling grant or funding requirements
• public relations
• window dressing
• decision making purposes

11
Purposes for Evaluation (Continuded)

• Decision Making Purposes
• Midcourse correction (process)
• Continuing, expanding or institutionalizing
• Test of new program idea
• Choosing among alternatives
• Discontinue

12
Source Michale Patton Utilizing Focused
Evaluation, Chapter 6 Being Active-Reactive-Adapti
ve1997135
13
Purposes for Evaluation (Continued)

• Organizational Learning Purposes
• Documenting Program History
• Feedback to practitioners
• Highlighting Program Goals
• Accountability
• Understanding social intervention

14

• Does this look familiar?
• You worked on developing an implementation plan
  for this last week.
• Now it is time for the next step, program
  evaluation.
• How might you evaluate whether or not the program
  you implemented, are implementing, is working?
  Lets re-visit the goals.

15
(No Transcript)
16
(No Transcript)
17
PRWORA TANF Issues To be Concerned With

• Create Measures to Evaluate the Following
• Design at least one Formative and one Summative.
• Which is more appropriate?
• 1. Caseloads Decrease caseloads by getting
  people into the labor force, and admitting less
  applicants.
• 2. Work Participation All receiving TANF must be
  working within 1 year.
• 3. Child Care Remove Childcare as a barrier to
  employment. You can use you save from
  decreased funding and diverted applicants for
  child care.
• 4. Family Formation you want to encourage
  2-parent families, and punish non-custodial
  (absent) parents.

Source Brookings
18
Who Does the Evaluation?

• Inside versus outside evaluation
• Things to Consider
• Objectivity
• Autonomy
• Understanding the program

19
Consulting Joke
20

• A shepherd was herding his/her flocks in a
  remote pasture when suddenly a brand new Jeep
  Cherokee advanced out of a dust cloud towards
  him. The driver, a young man in a Brioni suit,
  Gucci shoes, Ray Ban sunglasses and a YSL tie,
  leaned out of the window and asked the shepherd
  "If I tell you exactly how many sheep you have in
  your flock, will you give me one?"
• The shepherd looks at the yuppie, then at his
  peacefully grazing flock and calmly answers,
  "sure!" The yuppie parks the car, whips out his
  notebook, connects it to a cell-phone, surfs to a
  NASA page on the Internet where he calls up a GPS
  satellite navigation system, scans the area,
  opens up a database and 60 Excel spreadsheets
  with complex formulas. Finally he prints out a
  150 page report on his hi-tech miniaturized
  printer, turns round to the shepherd, and says
  "you have exactly 1586 sheep!"

21

• "That is correct, take one of the sheep," says
  the shepherd. He watches the young man select a
  sheep and bundle it in his Cherokee.
• Then he says "If I can tell you exactly what
  your business is, will you give me my sheep
  back?"
• "Okay, why not," answers the young man.
• "You are a consultant," says the shepherd.
• "That is correct," says the yuppie. "How did
  you guess that?"
• "Easy," answers the shepherd. "You turn up
  here although nobody called you.
• You want to be paid for the answer to a
  question I already knew. And you don't know
  anything about my business because you took my
  dog."