James Avery, Ph'D'

1
Overview of Multi-Year Plans

• James Avery, Ph.D.
• Research Coordination Staff
• Office of Science Policy
• January 19, 2005

2
What are Multi-Year Plans?

• Provide a framework that integrates research
  across ORDs laboratories and centers and
  Government Performance and Results Act (GPRA)
  goals in support of the Agencys mission to
  protect human health and the environment
• Describe the direction of ORDs research program
  in a topic area over the next 5-10 years

3
Structure of Multi-Year Plans

• Narrative
• Introduction and background that familiarize the
  reader with MYPs and Agency goals and objectives
• Progress to date from previous versions
• Overview of the Long Term Goals (LTGs)
• Description of the flow diagrams and the
  rationale used to sequence the performance
  measures
• Relationship to other research, e.g., work in
  other MYPs or by other Federal entities
• Performance and accountability information
• Logic Model
• Performance measures
• Flow diagrams that illustrate steps to attainment
  of the LTGs

4
ORD Multi-Year PlansCurrent Topic Areas

• Goal 4
• Safe Pesticides/Safe Products
• Ecological Research
• Global Change
• Mercury
• Human Health
• Endocrine Disruptors
• Goal 5
• Pollution Prevention and New Technologies
  (transitioning to Science and Technology for
  Sustainability)
• Economics and Decision Sciences

• Goal 1
• PM Tropospheric Ozone
• Air Toxics
• Goal 2
• Drinking Water
• Water Quality
• Goal 3
• Contaminated Sites RCRA

5
ORDs Inclusive MYP Process

• Customer/User Needs
• EPA Program Offices
• Regions/States
• Federal research partners
• Private Sector

• EPA Strategic Plan
• Government Performance and Results Act Goals
• Clean Air
• Clean and Safe Water
• Preserve and Restore the Land
• Healthy Communities and Ecosystems
• Compliance and Environmental Stewardship

Multi-Year Research Plans

• Outside Peer Advice
• Science Advisory Board
• Board of Scientific Counselors
• National Research Council
• Scientific Peer Reviews

• ORD Strategic Planning
• ORD Strategic Plans
• Peer Reviewed Research Strategies and
  Plans

6
How are MYPs developed?

• Written by a lead author and writing team
• Writing team consists of
• Laboratory and center experts
• Program and regional office representatives
• Office of Science Policy (OSP) staff
• Office of Resource and Management Acquisition
  (ORMA) staff
• Assume level resources using the most recent
  Presidents budget
• Updated biennially or on an as needed basis

7
MYP Development

• Agency Strategic Direction
• GPRA goals/objectives/outcomes
• Program/Regional science needs

Develop key science questions to be addressed by
research

• Develop long-term goals (LTGs)
• Identify time-frame to deliver work
• Determine ORD role and role of others

• Determine annual performance goals to reach LTGs
• Identify sequence to provide results
• Integrate research from all sources

• Develop annual performance measures
• Determine who will accomplish work (in-house lab
  or center or STAR research)
• Ensure work can be done with available resources

8
How are MYPs used?

• MYPs are used to
• Provide the link between Agency and ORD Strategic
  Plans and Research Strategies and Laboratory
  Implementation Plans
• Develop performance goals and measures
• Provide critical evidence on implementation of
  the Research and Development Investment Criteria
  and for the Program Assessment Rating Tool (PART)
  
• Plan the annual budget and explain the context
  and impact of budget decisions
• Communicate the direction of ORDs research
  program internally and externally

9
EPA
Multi-Year Planning
EPA Strategic Plan
Annual Planning
ORD Strategic Plan
Lab/Center Research Plans
Research Strategy (if available)
EPA Priorities Budget Guidance
10
Link between Agency and ORD Strategic Plans

• Agency and ORD Strategic Plans
• Set major areas of emphasis
• Provide GPRA goals that define environmental
  outcomes and ORDs interpretation of science
  needed to achieve them
• Inform research strategies and plans, as well as
  MYPs, that translate overall direction into more
  specific strategic approaches, priorities and
  outcomes

11
Develop Performance Goals and Measures

• Outcomes
• External to the research program
• Relevant to, and important for, the intended
  client, the agency, and the public (e.g.,
  reduction in exposure to pollutants)
• Performance goals measures
• Describe the most important aspects of the
  programs mission and priorities
• Include clear links to a program outcome
• Include a logic model, long term goals, annual
  performance goals and annual performance measures
• MYP guidance on performance measures is evolving
  based on continuing dialogue with OMB

12
Develop Performance Goals and Measures Logic
Model

• Logic Model links
• Specific environmental problems
• EPAs goals and objectives
• Environmental decisions needed
• To the research required to strengthen
  decision-making

13
Logic Model Communicating How Environmental
Research Contributes to Outcomes

14
Develop Performance Goals and Measures Long
Term Goals

• Long Term Goals
• Define the endpoints to be achieved from ORD
  research
• Should be
• Outcome-oriented and directly relevant to the
  research program in the MYP
• Not overly broad (e.g., improve science), but
  narrow enough to show contribution of the
  research
• Achievable by ORD in 5-8 years
• Defined so that success can be measured

15
Develop Performance Goals and Measures APGs and
APMs

• Annual Performance Goals
• Major steps necessary to achieve the LTG
• Described in the context of the outcome to which
  the research outputs contribute
• Typically multi-laboratory/center
• Annual Performance Measures
• Most important research outputs that measure the
  progress to achieve APGs and LTGs
• Examples include models, reports and
  publications, etc.
• Laboratory/center-specific for accountability
  purposes
• APGS and APMs should represent all of ORDs
  projected significant accomplishments in a
  research program

16
Develop Performance Goals and Measures Flow
Diagram
17
ORD Multi-Year Research Plans Prospective
Evidence that the Research Program Responds to
the OSTP/OMB RD Investment Criteria
Criterion 1 Relevance. RD programs must be
able to articulate why their investment is
important, relevant, and appropriate. Programs
must have well-conceived plans that identify
program goals and priorities and identify
linkages to national and client needs and to
agency mission.
Advances in research scientific knowledge
18
ORD Multi-Year Research Plans Prospective
Evidence that the Research Program Responds to
the OSTP/OMB RD Investment Criteria
Criterion 2 Performance. RD programs must have
the plans and management processes in place to
monitor and document how well their investment is
performing. Program managers must define
appropriate outcome measures and milestones that
can be used to track progress towards goals, and
assess whether funding should be enhanced or
redirected

Please note that Performance Goals include key
research outputs , a synthesis product, plans
for effective transfer to intended clients to
bring about short-term outcomes. The synthesis
product addresses a key research question, and
compares the accomplishment represented by the
Performance Goal to baseline conditions and to
related Performance Goals in future years needed
to achieve outcomes.
19
ORD Multi-Year Research Plans Prospective
Evidence that the Research Program Responds to
the OSTP/OMB RD Investment Criteria
Criterion 3 Quality. RD programs must justify
how funds will be allocated to ensure quality
RD. Programs allocating funds through means
other than a competitive, merit-based process
must justify these exceptions and document how
quality is maintained.

Research Program
Clients
Results
Research Activities
Resources
Effective Transfer
Long-Term Outcomes
Specific Clients
Research Outputs
Short-Term Outcomes
Intermediate Outcomes
Strategic Objectives
Strategic Goals
8. . ... which contribute to measurable changes
in environmental contaminants, stressors, or
exposures
3. to create the research knowledge, tools,
processes, technologies etc. and the key outputs
that must be developed over time
4. for intended clients (such as senior
scientists or decision makers in government, the
regulated community, or the public)
7. which result in changes in client decisions
or actions
6. client reactions and to changes in knowledge,
attitudes, skills, or aspirations ...
5. and to prepare for the effective transfer
of key research outputs that will lead to . . .
2. to coordinate and conduct the research,
development, administrative activities that
will be needed
9. to demonstrate measurable long-term
improvements in human or ecosystem health
1. We use resources (, FTE, infrastructure),
partnerships, and guidance from stakeholders
Examples of Quality Indicators external expert
review of STAR research
external expert review of
scientific progress performance
20
MYPs Provide Critical Evidence for OMBs Program
Assessment Rating Tool (PART) Agencies must
submit evidence that support a research
programs design and purpose, strategic
planning, management, and results
Performance Goals
Program Results
Outcomes
Clients Reached
Outcomes
Research Activities
Outputs
Outcomes
Resources
Program Design
APMs
20 of total score
Section I Program Purpose Design
PART
10 of total score
Section II Strategic Planning
20 of total score
Section III Program Management
50 of total score
Section IV Program Results

OMB requires agencies to submit results from
independent, expert, evaluation
21
MYPs and the Budget

• Serve as the basis for ORDs budget request
• Provide information for the research budget
  Congressional justification
• Description of the planned research and
  application of the RD investment criteria
• Used to prioritize work within and across MYPs
  for budget planning
• Provide information on the context and impact of
  budget decisions (increases or decreases)
• Form the structure for ORD accounting

22
Communication

• Used to communicate with the program and regional
  offices about the research program
• Planned work
• Progress to date
• Used to communicate with external audiences,
  e.g., public and other research organizations
• Available on the Internet
• http//www.epa.gov/osp/myp.htm