AEAs Evaluation Policy Initiative

1
AEAs Evaluation Policy Initiative

• William Trochim, 2008 President of the
• American Evaluation Association
• George Grob, Consultant to the
• AEA Evaluation Policy Task Force

2
What Is Evaluation Policy?

• Action or Inaction by Government Policy Makers
  and Others That Affect
• How evaluation is defined
• When it gets employed
• On what programs
• What kind of methods are used
• Who does it
• What resources are available for it
• How it is used
• And much more

3
Why Is It Important?

• Evaluation policy affects
• The scope and nature of evaluations that are
  performed
• The effectiveness and efficiency of public
  programs which are or are not evaluated
• The quality of services and benefits that
  beneficiaries and the public at large receive
• Our professional practice

4
Some Examples

• Government Performance and Results Act (GPRA)
• Program Assessment Rating Tool (PART) and
  ExpectMore.gov
• HHS one percent evaluation set-aside for many
  health and human services programs
• Federal education evaluation requirements for
• High risk testing
• Randomized experimental designs for evaluating
  program effectiveness

5
What Can AEA Do About It?

• Many evaluation policy decisions are made without
  the systematic input and advice of evaluators
• AEA is well positioned to be the source of such
  advice
• AEA Board creates the Evaluation Policy Task
  Force (EPTF)
• Two year charge
• Reports to the AEA Executive Committee

6
EPTF Goal

To assist AEA in developing an ongoing capability
to influence evaluation polices that are
critically import to the practice of evaluation
7
Evaluation Policy Task Force Members

• Eleanor Chelimsky
• Leslie Cooksy
• Katherine Dawes
• Patrick Grasso
• Susan Kistler
• Mel Mark
• Stephanie Shipman
• William Trochim, Chair
• George Grob (Consultant)

8
How Will the EPTF Work?

• Consultative Campaign
• Public Presence Initiative
• Proactive and Reactive Activities

9
Consultative Campaign

• Scan emerging Federal legislative, regulatory,
  and administrative policies to identify targets
  of opportunity
• Brief congressional and executive staff about the
  value of evaluation and how to effectively use it
• Network with contacts both within and outside of
  government that can assist AEA in influencing
  evaluation policy
• Respond to requests for input on policy
  formulation

10
Public Presence Campaign

• Consultations. Consult with evaluators and
  others experienced in policy development
• Materials. Develop collateral materials, e.g.
  white paper, brochures, AEA Guiding Principles
• Website. Advise on AEA website enhancements for
  policy influence

11
Proactive and Reactive

• Target a few specific legislative and executive
  policies
• Focus on selected substantive areas
• Be appropriately opportunistic

12
Responsibilities and Participation

• Broad input from and consultation with AEA
  Members
• EPTF will guide the initiative and make policy
  recommendations to the AEA Executive Committee
  and Board
• The Consultant will assist and support the EPTF
  and be the lead on implementation

13
Next Steps

• Identify highly influential evaluations
• Learn from
• Experienced Evaluators
• Professional Organizations
• Establish Policy Scan function
• Prepare briefing kits
• Consult with experienced federal evaluators
• Meet the policy makers
• Congressional
• Executive Branch
• Identify evaluation policy opportunities

14

• DISCUSSION

15

• Big Questions
• Federal Evaluators Suggestions
• Working Together

16
Big Questions

• What is the best way to get our message across to
  evaluators, policy makers, and program managers?
• What are the ripe policy opportunities that we
  should pursue?

17
Comments Received From Federal Evaluators

• Outcomes
• Communications with Agency and Congressional
  Staff
• Coalition for Evidence-Based Policy
• Training
• Other Professional Association
• PART and GPRA
• Longer Term Efforts
•  

18
Working Together

• Entre to Senior Evaluators and Agency Executives
• Policy Scan
• Discussion Group

19
Staying In Touch

• Email Evaluationpolicy_at_eval.org
• Website http//www.eval.org/EPTF.asp
• Discussion Group
• http//www.eval.org/EPTF.signup.discussion.asp