Evaluation Defined

1
Evaluation Defined

• FCS 5470
• M. Burns

2
What do you think?

• Draw a visual of your definition of evaluation.
• As a group, write a definition of evaluation

3
Evaluation as defined by some of the evaluation
experts

• Michael Scriven
• Rossi and Freeman
• Patton
• Preskill and Torres
• Weiss

4
Evaluation is

• A systematic assessment of the operation and/or
  outcomes of a program or policy, compared to a
  set of explicit or implicit standards, as a means
  of contributing to the improvement of the program
  or policy.

5
What is evaluated?

• Program
• Policy
• Process
• Product
• People
• Service

Evaluand
6
My definition

• Use of scientific methods to judge and improve
  something

7
Evaluation can

• Contribute to increased knowledge among members
• Help prioritize resources
• Assist in planning/delivering organizational
  initiatives
• Promote accountability
• Fulfill grant responsibility
• Contribute to improved quality

8
Reasons Evaluation is Neglected

• Organization members misunderstand evaluations
  purpose and role.
• Organization members fear the impact of findings.
• There is a real or perceived lack of evaluation
  skill.

9
Reasons Evaluation is Neglected

• Evaluation is considered an add-on activity.
• Limited utilization of results.
• Viewed as time-consuming and laborious.
• Perceived costs outweigh perceived benefits.

10
Reasons Evaluation is Neglected

• Administration doesnt buy-in to evaluation.
• Disappointing/disastrous previous experiences.
• No one has asked for it!

11
When not to conduct evaluation

• Results are not going to be utilized.
• Program has limited stability
• Limited resources

12
Kinds of Evaluation

• Developmental
• Formative
• Summative
• Monitoring/ auditing
• Outcome
• Impact
• Performance

13
Developmental Evaluation

• Positions the evaluator as a guide throughout the
  programs development
• Used frequently in a needs assessment

14
Needs assessment

• Systematic set of procedures undertaken for the
  purpose of setting priorities and making
  decisions about program improvement and
  allocation of resourcesthe priorities are based
  on the identified needs

15
What do you think?

• Several members of a small community have
  identified a need for a mentoring program for the
  communitys youth.
• What developmental questions need to be asked
  by the evaluators as a mentoring program is
  established?

16
Formative Evaluation

• Typically conducted
• during the development or improvement of a
  program by internal evaluators
• More than once
• Intent is to improve the program as the it is
  being implemented

17
What do you think?

• The MAYN program has been in place for 10 years,
  but the state funding has been severely cut.
  Obtaining money from alternative sources is
  critical.
• What questions might be addressed in a formative
  evaluation of the program?

18
Summative Evaluation

• Typically conducted
• After program completion for the benefit of some
  external audience or decision maker
• By external evaluators
• Used to judge worth or value of the program

19
Types of summative evaluation

• Monitoring and auditing
• Implementation issues
• Accountability
• Outcome
• Changes in knowledge, attitudes, values, skills,
  etc due to intervention

20
Types of summative evaluation

• Impact
• Form of outcome evaluation
• Assesses the net effect of a program by comparing
  outcomes to an estimate of what would have
  happened without the program

21
Types of summative evaluation

• Performance
• Used mostly by government
• Looks at the process in comparison to the output
• Documentation of the results

22
Final thoughts

• Based on the information presented and your
  previous thoughts, how has your definition of
  evaluation changed?