Objectives for Session Thirteen

1
Objectives for Session Thirteen

• Discuss Ethical Considerations in Evaluation
  Research
• Politics and Evaluations
• Utility of Formative Evaluations
• Evaluating the Evaluation Course

2
Ethics in Research Designs and Evaluation

• Confidentiality and Anonymity
• Human subjects review
• Informed consent
• Nonexploitative
• Emphasis on beneficence
• Maximizing good outcomes
• Minimizing harm or risk
• Competence and capacity?
• Honesty and integrity in designs/analyses

3
Studying At-Risk Populations

• Ethical dilemmas that program evaluators might
  encounter when studying at-risk groups?
• Poverty Pimping
• Causing distress to young children
• Time lag between traumatic event and study
• No support or assistance offered
• Risk of sanction and/or prosecution

4
Consent Forms for Participants in Child
Protective Services Evaluation

• If the Study learns that I, my child, or others
  are (or were) in danger of serious harm, the
  information will be shared with an appropriate
  authority"
• "You need to be told that we are required by law
  to report to CPS any new or suspected incidents,
  or past unreported incidents, of child abuse and
  neglect. We are also required to report any
  information your child gives us that indicates
  he/she might be at risk of other harms, such as
  suicidal thoughts."

5
How to Ensure Our Actions Are Not Unethical or
Exploitative

• Consent forms
• Commitment to helping those we study
• Streets of Hope, Medoff and Sklar
• Compensation
• Building Relationships with Participants
• Link to Assistance or Support Services
• Participatory Methods

6
Politics and Evaluation

• The evaluation should be planned and conducted
  with anticipation of the different positions of
  various interest groups, so that possible
  attempts by any of these groups to curtail
  evaluation operations or to bias or misapply the
  results can be averted or counteracted.
• Patton, p. 343

7
Politics in Evaluation

• Evaluations are necessarily political in nature
• Provide information and assessment of performance
• Control or direction of program
• Allocation of scarce resources
• Career, prestige, reputation
• External funding
• Evaluations require interpretation
• Interpretations affect program decisions
• Interpretation of causality, implementation,
  solutions
• Value-laden exercise of power

8
Brown and Kiernan

• What is the impact of formative evaluations?
• Mix of methods in two-stage experimental design
  evaluation
• Lessons about program implementation drawn from
  the first stage

9
Did Implementation Affect Program Outcomes?
10
Brown and Kiernan

• Why might formative evaluations not have easily
  discerned effects on outcomes?
• What is the value or likely impact of
  education-based services?
• Challenges of asking participants for feedback on
  program implementation

11
Course Objectives 1st Half

• Summative versus formative evaluations
• Different qualitative data collection techniques
• Develop a formative evaluation strategy
• Maintain your familiarity with statistics

12
Readings for Next Class with Prof. Duncombe

• Schroeder, part 1
• Mason, Chapter 12
• Trochim, Chapter 10
• Notes 1 and 2
• Evaluation example Evaluating the WIC program.
  Read pages 123-136 from Douglas Besharov and
  Peter Germanis. 2000. "Evaluating WIC."
  Evaluation Review. 24 (April). (coursepack)