Evaluation Research

1
Evaluation Research

• Introduction to Evaluation Research
• SWk 5983 Program Evaluation
• January 12, 2004

2
Evaluation Research Defined

• Program evaluation is the use of social science
  research methods to systematically investigate
  the effectiveness of social intervention programs

3
Questions for Evaluation Research

• What is the social or health problem being
  addressed?
• What feasible interventions will help ameliorate
  the problem?
• Has the intervention been effective in reaching
  goals and objectives?
• Is the intervention implemented as intended?
• Are the benefits of the intervention worth the
  cost?

4
Evaluation covers

• Planning
• Needs assessment
• Social problem definition
• Budgeting
• Quality control
• System modeling
• Accountability
• Cost benefits analysis

5
Evaluation Practice

• Evaluation is practiced in three arenas
• In policy development
• Program management
• Client or consumer advocacy

6
Need for Evaluation Research

• Limited resources will perpetuate need for
  evaluation research
• Evaluation of existing programs will lead to
  changes in health social servicesand thus
  require additional evaluation to promote more
  effective and less costly programs

7
Two central evaluation tasks

• Description of the program being evaluated
• A standard or criteria against which program
  performance is measured

8
Valid descriptions of programs

• Must describe distinctly and precisely
• Allows for discrimination for meaningful
  variations in level of performance to be detected
• Avoids unduly vague or equivocal description
• Provides a precise measure for comparison to a
  standard

9
Keys to evaluation

• Needs to be based on social science research
  methods
• Needs to be systematic
• However, will need to be innovative in using
  different methodologies
• Social programs are an inhospitable environment
  for evaluation

10
Five domains of program evaluation

• Need for the program
• Design of the program
• Program implementation service delivery
• Program impact or outcomes
• Program efficiency (cost effectiveness)

11

• Evaluation may cover one or more of these
  activities,
• However, an evaluation that covers all five areas
  is called a comprehensive evaluation

12
Evaluating Programs

• Evaluating public health or social programs means
  that we want to consider effectiveness as
  ultimately making a contribution to improving
  social conditions
• Not just profit or efficiency

13
Evaluation Planning

• Evaluation plans must be specific for each
  program evaluated
• Typically, evaluation plans will need to be
  modified revised during implementation
• Will need to start with evaluation questions
• Consider ability to be flexible with questions in
  contracts and with funding source
• Develop questions to show program success

14
Planning (cont)

• Know why evaluation questions are developed
• the political context
• the purpose of the evaluation
• Evaluation will need to serve administrative
  needs

15
Conflicts in evaluation

• Evaluation is an administrative drain on funds
  and program resources
• Social programs change focus frequently
• Scientific methods may be required but not
  feasible
• There is little consensus about evaluation methods

16

• End of Introduction