Developing Evaluation Questions

1
Developing Evaluation Questions

• Module 5

2
Overview

• Framework for evaluation questions
• Examples of questions
• Characteristics of good evaluation questions

3
The Cycle for Conducting an Evaluation
Establish Need
Use Results
Plan
Communicate Results
Resources
Collect Data
Recommend Action
Analyze Data
Interpret Data
4
What can an evaluation tell us?

• Success
• Are we achieving the results that we were hoping
  for?
• Are results being produced as planned at all
  levels of our logic model?
• Relevance
• Do stakeholders care about what we are doing? Are
  we solving the problem?
• Cost-Effectiveness - Are we getting value for
  money?

5
Generic Evaluation Issues
Success
Relevance
Cost-effectiveness
6
From Issues to Questions

• Issues guide the evaluator in
• Asking the right questions!

7
Relevance Questions

• Are program objectives still valid?
• Do stakeholders support the program?

?
8
Cost Effectiveness Questions

• Is there optimal conversion of inputs into
  outputs?
• Faster, cheaper, better
• Do the costs conform to accepted standards for
  similar processes?
• Does the program conform to contractual
  requirements?

9
Success Questions

• Are the intended results being achieved?
• If not, why not?
• What are the unintended results?
• Positive
• Negative
• Will results hold over time?

10
When formulating questions

• Ensure a clear link between questions and the
  issues of the evaluation
• Be realistic in the number of questions asked
• Choose questions selectively
• Ensure questions can be answered
• Focus on the important issues

11
Checking the feasibility of evaluation questions

• Specific is there a topic and a stem for each
  question?
• Measurable can the question be answered?
• Actionable will the answers provide the
  information needed for decision-making?
• Relevant nice to know or need to know?
• Timely is it important to ask this question now?