Evaluation Research

1
Chapter 12

• Evaluation Research

2
Chapter Outline

• Introduction
• Topics Appropriate To Evaluation Research
• Formulating the Problem Issues of Measurement
• Types of Evaluation Research Designs
• The Social Context
• Social Indicators Research

3
Evaluation Research

• Appropriate for any study of planned or actual
  social intervention.
• Goal is to determine whether a social
  intervention has produced the intended result.
• Results are not always well received.

4
Evaluation Research

• Research undertaken for the purpose of
  determining the impact of some social
  intervention, such as a program aimed at solving
  a social problem.

5
Question

• Why has evaluation research grown in popularity?
• federal requirements demanding program
  evaluations
• the availability of research funds to fulfill
  federal requirements
• both a and b
• none of these choices

6
Answer C

• Evaluation research has grown in popularity
  because of federal requirements demanding program
  evaluations, and the availability of research
  funds to fulfill federal requirements.

7
Needs Assessment Studies

• Studies that aim to determine the existence and
  extent of problems, typically among a segment of
  the population, such as the elderly.

8
Cost-benefit Studies

• Studies that determine whether the results of a
  program can be justified by its expense (both
  financial and other).

9
Monitoring Studies

• Studies that provide a steady flow of information
  about something of interest, such as crime rates
  or the outbreak of an epidemic.
• Program Evaluation/outcome Assessment
• The determination of whether a social
  intervention is producing the intended result.

10
Question

• ___________________ aim to determine the
  existence and extent of problems.
• longitudinal studies
• needs assessment studies
• cost-benefit studies
• monitoring studies

11
Answer B

• Needs assessment studies aim to determine the
  existence and extent of problems.

12
Question

• _________________ provide a steady flow of
  information about something of interest.
• longitudinal studies
• needs assessment studies
• cost-benefit studies
• monitoring studies

13
Answer D

• Monitoring studies provide a steady flow of
  information about something of interest.

14
Types of Measurement in Evaluation Research

• Outcome (response variable)
• Experimental Context - aspects of the context of
  an experiment that might affect the experiment.

15
Types of Measurement in Evaluation Research

• Experimental Stimulus (interventions)
• Population - demographic variables as well as
  variables defining the population.

16
Evaluation Research Designs

• Experimental designs
• Quasi-experimental designs
• Time-series design
• Nonequivalent control groups
• Multiple Time-Series designs
• Qualitative evaluations

17
Question

• Besides making measurements relevant to the
  outcomes of a program, researchers must measure
  the __________________.
• primary cause
• program intervention
• tertiary effect
• all of these choices
• none of these choices

18
Answer B

• Besides making measurements relevant to the
  outcomes of a program, researchers must measure
  the program intervention.

19
Observations of Class Participation Before and
After an Open Discussion
20
(No Transcript)
21
Quasi Experiments

• Nonrigorous inquiries somewhat resembling
  controlled experiments but lacking key elements
  such as pre- and posttesting and/or control
  groups.

22
Time-Series Designs

• Studies that involve measurements taken over
  time.

23
Non-equivalent Control Group

• A control group that is similar to the
  experimental group but is not created by the
  random assignment of subjects.

24
Question

• A research design that involves measurements made
  over some period may be defined as
• time series designs
• quasi experiments
• multiple time series designs
• none of these choices

25
Answer A

• A research design that involves measurements made
  over some period may be defined as time series
  designs.

26
Active Initiatives over Time
27
Ethical Issues

• Social interventions being evaluated may raise
  ethical issues.
• Evaluation research may be a mask for unethical
  behavior.

28
Why Results Are Ignored

• Implications may not be presented in a way that
  nonresearchers can understand.
• Results sometimes contradict deeply held beliefs.
• Vested interest in a program.

29
Social Indicators

• Aggregated statistics that reflect the social
  condition of a society or social subgroup.

30
Social Indicators Research

• Provides an understanding of broader social
  processes.
• Researchers are developing more refined
  indicators.
• Research is being devoted to discovering the
  relationships among variables within whole
  societies.

31
Rate per 100,000 Population of 1st- and 2nd-
Degree Murders
32
Quick Quiz
33

• 1. ______________ is a form of applied research.
• evaluation
• experimentation
• interviewing
• field research

34
Answer A

• Evaluation is a form of applied research.

35

• 2. __________________ determine whether the
  results of a program can be justified by its
  expense.
• longitudinal studies
• needs assessment studies
• cost-benefit studies
• monitoring studies

36
Answer C

• Cost-benefit studies determine whether the
  results of a program can be justified by its
  expense.

37

• 3. A key variable for evaluation researchers is
  to measure the ___________.
• independent variable
• dependent variable
• response variable
• none of these choices

38
Answer C

• A key variable for evaluation researchers is to
  measure the response variable.

39

• 4. ________________ are distinguished from
  true experiments primarily by the lack of
  random assignment of subjects to an experimental
  and a control group.
• time series designs
• quasi experiments
• multiple time series designs
• none of these choices

40
Answer B

• Quasi experiments are distinguished from true
  experiments primarily by the lack of random
  assignment of subjects to an experimental and a
  control group.

41

• 5. Professor Yee wants to do an evaluation study
  of the effects of a patient education program on
  patient anxiety. He uses one wing in a hospital
  for the experiment and compares the results with
  a similar group of patients in a similar wing in
  another hospital. Which design would be best?
• classical
• nonequivalent control group
• time-series
• posttest-only control group design
• factorial

42
Answer B

• Professor Yee wants to do an evaluation study of
  the effects of a patient education program on
  patient anxiety. He uses one wing in a hospital
  for the experiment and compares the results with
  a similar group of patients in a similar wing in
  another hospital.
• The nonequivalent control group design would be
  best.

43

• 6. Evaluation researchers encounter more
  logistical problems than other researchers
  because evaluation research
• occurs in the context of real life.
• takes longer.
• is more costly.
• has more measurement problems.
• examines more variables.

44
Answer A

• Evaluation researchers encounter more logistical
  problems than other researchers because
  evaluation research occurs in the context of real
  life.