EVALUATION AND ACCOUNTABILITY

1
EVALUATION AND ACCOUNTABILITY
2

• Presented By
• David S. Anderson, Ph.D.Associate Professor
• George Mason UniversityCenter for the
  Advancement of Public HealthSchool of
  Recreation, Health and Tourism
• College of Education and Human Development

3
OBJECTIVES

• To provide project staff with some broader
  perspectives about evaluation

• To stress the role that accountability has with
  evaluation

• To identify specific ways for enhancing a
  projects accountability through evaluation

• To enhance the desire for being reflective about
  the public trust placed in us

• To provide greater science and grounding
  regarding whether our programs are making a
  difference

4
THREE OVERALL SEGMENTS

• Evaluations Role
• The Context of Evaluation
• Accountability and Evaluation

5
We must educate people today for a future in
which the choices to be faced cannot be
anticipated by even the wisest now among us.
John F. Kennedy
6
(No Transcript)
7
(No Transcript)
8
Evaluation can help in . . .
9
HOW EVALUATION CAN HELP

• Planning

Clarify what we want to accomplish
Gain consensus and agreement
Refining the planning
Assessing accomplishment of desired outcomes
Review program goal attainment
Keeping on track
Review milestones
Justification
Prepare project documentation
10
HOW EVALUATION CAN HELP

• Enhancement and refinement

Improve the project
Continuation
Sustain the project
Making budget decisions
Allocate or reallocate resources
Diffusion of innovation
Share insights with others
Test or refine theories
Advancing knowledge
11
HOW EVALUATION CAN HELP

• Enhancing the sophistication of our work

Help build the science of prevention
Increasing the public trust
Maintain accountability
12
I have not failed. I've just found 10,000 ways
that won't work. Thomas Alva Edison
13
Twenty Misperceptions About Evaluationor Why I
Really Dont Want to Do Evaluationand a
Different Reality!!!
14
You gain strength, courage and confidence by
every experience in which you really stop to look
fear in the face You must do the thing which you
think you cannot do. Eleanor Roosevelt
15
ATTITUDE

• Evaluation is too complicated.

Evaluation is expensive our funding can be
better spent elsewhere.
Evaluation reports are boring.
Evaluation can never be perfect, so why bother?
16
ATTITUDE

• Evaluation is too complicated.
• Wide range of components and elements
• Numerous strategies and approaches
• Blends quantitative and qualitative

17
ATTITUDE

• Evaluation is expensive our funding can be
  better spent elsewhere.
• Doesnt have to be costly
• Investment in project
• Properly manage and thought through early

18
ATTITUDE

• Evaluation reports are boring.
• Reports can come alive and tell a story
• Get the reports to tell you what you want to know
• Blend approaches
• Customize reports and presentations

19
ATTITUDE

• Evaluation can never be perfect, so why bother?
• Nothing can be perfect
• Use reasonable and manageable processes

20
Obstacles don't have to stop you. If you run
into a wall, don't turn around and give up.
Figure out how to climb it, go through it, or
work around it. Michael Jordan
21
PROCESS

• Evaluation is really long-term its a
  longitudinal process.

Evaluation includes lots of different approaches
so its not clear what to do.
Evaluation includes one standard instrument how
will that help me?
Evaluation is supposed to be scientific,
including random and representative approaches.
Evaluation requires lots of procedures, like red
tape.
22
PROCESS

• Evaluation is really long-term its a
  longitudinal process.
• Evaluation can continue over time
• It can provide some short-term results
• Focus on what we want to know

23
PROCESS

• Evaluation includes lots of different approaches
  so its not clear what to do.
• Address projects unique elements
• Blend a range of measures
• Triangulate the results
• Get your questions addressed

24
PROCESS

• Evaluation includes one standard instrument how
  will that help me?
• A standard approach can cut across all projects
• Dont use just one approach
• Get your questions answered

25
PROCESS

• Evaluation is supposed to be scientific,
  including random and representative approaches.
• Incorporate as much science as possible
• Maintain standards of quality
• Use science-based approaches

26
PROCESS

• Evaluation requires lots of procedures and lots
  of red tape.
• Procedures and policies help protect those served
  by us
• Identify these as early as possible

27
If you only have a hammer, you tend to see
every problem as a nail. Abraham Maslow
28
INTERPRETATION

• Evaluation reports are confusing. They are not
  written in a way that is usable to me or my
  project personnel.

Evaluation results are subject to interpretation.
So, why should I do it, since different people
will interpret the results differently.
Evaluation results can be dangerous, since the
media often distorts these and thats just not
worth it.
Evaluation never ends all reports say that more
studies are needed.
29
INTERPRETATION

• Evaluation reports are confusing. They are not
  written in a way that is usable to me or my
  project personnel.
• Negotiate and spell out the report content
• Do this at the beginning of the project
• Clearly identify the questions you want answered
• Acknowledge that all questions cannot reasonably
  be answered

30
INTERPRETATION

• Evaluation results are subject to interpretation.
  So, why should I do it, since different people
  will interpret the results differently.
• Different interpretations can be helpful
• These help understand the truth (or truths) about
  what is occurring

31
INTERPRETATION

• Evaluation results can be dangerous, since the
  media often distorts these and thats just not
  worth it.
• Be clear with what you learn and what you didnt
  learn from the evaluation
• Learn skills for dealing with the media

32
INTERPRETATION

• Evaluation never ends all reports say that more
  studies are needed.
• Insights can be gathered through any process
• Additional insights will help address the current
  uncertainties

33
The most difficult part of any journey is
deciding on the destination.
34
RESULTS

• Evaluation may tell me that what Ive been doing
  is wrong.

Evaluation results may affect my job.
Evaluation often reveals no notable differences.
So, why should I have bothered?
35
RESULTS

• Evaluation may tell me that what Ive been doing
  is wrong.
• It gives insights about the project
• This can help redirect the project

36
RESULTS

• Evaluation results may affect my job.
• This should be affected only if you dont
  incorporate evaluation
• Respect the partnership role that it can play

37
RESULTS

• Evaluation often reveals no notable differences.
  So, why should I have bothered?
• With no information, you wouldnt know that no
  differences existed
• Ongoing monitoring is important in this process

38
They always say time changes things, but you
actually have to change them yourself. Andy
Warhol
39
PRACTICALITY

• Evaluation is something that we dont need to
  worry about now thats what is done at the end
  of the program.

Evaluation opens up so much new information that
I will then have to deal with.
Evaluation is beyond my skill level, and no one
will help.
Evaluation interferes with the implementation of
the program it might control how the project
gets done.
40
PRACTICALITY

• Evaluation is something that we dont need to
  worry about now thats what is done at the end
  of the program.
• Evaluation is a strategic planning tool
• Think through issues at the onset of a program to
  maximize its effects
• It helps modify project efforts as appropriate

41
PRACTICALITY

• Evaluation opens up so much new information that
  I will then have to deal with.
• Its more helpful to have the information than to
  go without it
• Have a clear idea of what you want to know, and
  what youll be able to use
• New information can revise delivery approaches
  for future projects

42
PRACTICALITY

• Evaluation is beyond my skill level, and no one
  will help.
• Resources are always available
• Engage them in a thoughtful and managed way
• Outline evaluation goals in the beginning

43
PRACTICALITY

• Evaluation interferes with the implementation of
  the program it might control how the project
  gets done.
• The project implementation takes primary focus
• Engage evaluation and respect its role
• This is a continuous negotiating task

44
No pessimist ever discovered the secret of the
stars, or sailed to an uncharted land, or opened
a new doorway for the human spirit. Helen Keller
45
(No Transcript)
46
www.ttac.org
47
Tools Understanding Negotiating Effort Us Present
ation
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
The Role of Accountability
54
(No Transcript)
55
How to be Accountable Through the Use of
Evaluation

• Think long-term

• Incorporate multiple components

• Be reflective

• Acknowledge that this is a process

• Think about institutionalization

56
How to be Accountable Through the Use of
Evaluation

• Engage others

• Report both positive outcomes as well as concerns

• Incorporate results into the project as it goes
  along

• Initiate at the beginning of the project

• Remember that we are stewards of the public trust

57
THREE OVERALL SEGMENTS

• Evaluations Role
• The Context of Evaluation
• Accountability and Evaluation

58
www.caph.gmu.edu
George Mason University Center for the
Advancement of Public Health School of
Recreation, Health and Tourism College of
Education and Human Development 4400 University
Dr. MS 1F5 Fairfax, VA 22030-4444
Phone [phone] Fax
703-993-3763 E-mail caph_at_gmu.edu