AEAs Evaluation Policy Initiative

1
AEAs Evaluation Policy Initiative

• 2007 AEA National Conference
• Baltimore, Maryland
• Hallie Preskill, 2007 President
• William Trochim, 2008 President

2
What Is Evaluation Policy?

• Action or Inaction by Government Policy Makers
  and Others That Affect
• How evaluation is defined
• When it gets employed
• On what programs
• What kind of methods are used
• Who does it
• What resources are available for it
• How it is used
• And much more

3
Why Is It Important?

• Evaluation policy affects
• The scope and nature of evaluations that are
  performed
• The effectiveness and efficiency of public
  programs which are or are not evaluated
• The quality of services and benefits that
  beneficiaries and the public at large receive
• Our professional practice

4
Some Examples

• Government Performance and Results Act (GPRA)
• Program Assessment Rating Tool (PART) and
  ExpectMore.gov
• HHS one percent evaluation set-aside for many
  health and human services programs
• Federal education evaluation requirements for
• High risk testing
• Randomized experimental designs for evaluating
  program effectiveness

5
What Can AEA Do About It?

• Many evaluation policy decisions are made without
  the systematic input and advice of evaluators
• AEA is well positioned to be the source of such
  advice
• AEA Board creates the Evaluation Policy Task
  Force (EPTF)
• Two year charge
• Reports to the AEA Executive Committee

6
EPTF Goal

To assist AEA in developing an ongoing capability
to influence evaluation polices that are
critically import to the practice of evaluation
7
Evaluation Policy Task Force Members

• Eleanor Chelimsky
• Leslie Cooksy
• Katherine Dawes
• Patrick Grasso
• Susan Kistler
• Mel Mark
• Stephanie Shipman
• William Trochim, Chair
• George Grob (Consultant)

8
How Will the EPTF Work?

• Consultative Campaign
• Public Presence Initiative
• Proactive and Reactive Activities

9
Consultative Campaign

• Scan emerging Federal legislative, regulatory,
  and administrative policies to identify targets
  of opportunity
• Brief congressional and executive staff about the
  value of evaluation and how to effectively use it
• Network with contacts in government that can
  assist AEA in influencing evaluation policy
• Respond to requests for input on policy
  formulation

10
Public Presence Campaign

• Materials. Develop collateral materials, e.g.
  white paper, brochures, AEA Guiding Principles
• Consultations. Consult with evaluators and
  others experienced in policy development
• Talking Points. Advise consultant on talking
  points on evaluation and its potential
• Website. Advise on AEA website enhancements for
  policy influence
• Policy Scan. Establish a policy scanning system
  f to increase awareness of important policy
  developments.

11
Proactive and Reactive

• Target a few specific legislative and executive
  policies
• Focus on selected substantive areas
• Be appropriately opportunistic

12
Responsibilities and Participation

• Broad input from and consultation with AEA
  Members
• EPTF will guide the initiative and make policy
  recommendations to the AEA Executive Committee
  and Board
• The Consultant will assist and support the EPTF
  and be the lead on implementation

13
Next Steps

• Identify highly influential evaluations
• Learn from
• Experienced Evaluators
• Professional Organizations
• Establish Policy Scan function
• Prepare briefing kits
• Consult with experienced federal evaluators
• Meet the policy makers
• Congressional
• Executive Branch
• Identify evaluation policy opportunities

14
Staying In Touch

• Email Evaluationpolicy_at_eval.org
• Website http//www.eval.org/EPTF.asp