Managing for Results Objectives

1
Managing for Results Objectives and Indicators

• Session rationale
• Consistent with sound development practice,
  program designs must have clearly stated
  objectives and indicators for how the
  institutional capacity of the PVO and its local
  partners will be developed under their program,
  and how sustainability will be achieved

2
Noted problems

• With objectives
• Set too low, output oriented
• Often dont describe change
• Technical objectives are pretty good, but
  capacity building and management arent
• With indicators
• Arent a direct match(nor reasonable proxy) of
  theobjective

3
Managing for Results Objectives and Indicators

• Session purpose
• Focus PVO attention on the attributes of good
  objectives and indicators, and their contribution
  to effective development program management

4
Managing for Results Objectives and Indicators

• Session objectives
• A better understanding of what USAID means by
  results, particularly related to institutional
  strengthening
• Clarity on how the Agency defines objectives,
  indicators and activities
• A small contribution to increasing the soundness
  of PVO program design

5
Todays agenda

• Use a results flow chart to
• Delineate the reasoning that underpins
  development programming
• Give meaning to the elements (i.e., inputs ?
  results) that define development programming
• Differentiate between these elements and
  categorically between activities and objectives
• Build on the results flow chart to
• Define indicatorswhat they are, what they do,
  and what characteristics they must have

6
Results flow chart
Activities
Objectives
Inputs
Processes
Outputs
Outcomes
Results
Causality
Planning
7
Activities
Objectives
Inputs
Processes
Outputs
Outcomes
Results

• Any action, funded or unfunded, undertaken to
  achieve a program result
• Assignments, tasks, stuff we do to achieve our
  objectives!
• For example, training for NGO counterparts in
  institutional capacity assessment

8
Activities
Objectives
Inputs
Processes
Outputs
Outcomes
Results

• Output level objectives are those which result
  from activities
• Immediate goods and services
• In most instances dont represent future state
  change
• Usually not your BHR/PVC Planning Matrix
  objectives
• Frequently bean counting
• Useful for program management purposes
• And because audits hold you accountable for the
  beans

9
Objectives
Activity Training for NGO counterparts in
institutional capacity assessment (ICA)
Outputs NGO counterparts competent in ICA
Results Flow Chart Update
10
Activities
Objectives
Inputs
Processes
Outputs
Outcomes
Results

• Anything that is clearly intended tobe achieved
• Intermediate effects
• Often occur at several levels in a series of
  cause and effect relationships
• Effects on the target population/customers

11
Objectives
Activity Training for NGO counterparts in
institutional capacity assessment (ICA)
Outputs NGO counterparts competent in ICA
Outcome Increased NGO counterparts capability to
measure community level institutional capacity
Activity Technical assistance to NGO counterparts
in administering and interpreting ICA with
community level institutions
Results Flow Chart Update
12
Activities
Objectives
Inputs
Processes
Outputs
Outcomes
Results

• Results and outcomes are
• Changes in the condition of a customer orchanges
  in the host country condition that affect a
  customer
• Whether an objective is a result or outcome
  depends on where in the chain of results it is
• Results describe the long-term impact on your
  customers
• Outcomes describe the immediate effects on your
  customers
• Performance accountability (i.e., for outcomes,
  results, or both) depends on your resources,
  program duration, span of influence over partners
  and the gap between the real and the ideal

13
Objectives
Activity Training for NGO counterparts in
institutional capacity assessment (ICA)
Outputs NGO counterparts competent in ICA
Outcome Increased NGO counterparts capability to
measure community level institutional capacity
Activity Technical assistance to NGO counterparts
in administering and interpreting ICA with
community level institutions
Result NGO counterparts direct changes in
community level institutional capacity to improve
community based health management
Results Flow Chart Update
14
Results Flow Chart Chain of Objectives Goals
Objectives
Activities
Goals
Outputs
Outcomes
Results

• Goals are at the highest level and are not
  measured in the program context
• For example, reducing morbidity and mortality in
  the general population are usually considered to
  be at the goal level
• Sustainable service delivery (post project
  activities) will assist local people to continue
  to meet objectives and move toward goal attainment

15
Activities
Objectives
Inputs
Processes
Outputs
Outcomes
Results

• Your Planning Matrix objectives should bethe
  highest order objectives for which youcan be
  held accountable
• In your programs manageable interest
• They should
• Be specific
• Clearly state the type of change expected
• Be clear about who and what is changing
• Individuals, groups, organizations
• Be unidimensional, time-bound and measurable

16
Child Survival Strategic Objective Strengthen
Capacity for Sustainable Service Delivery

• Sectoral specific interventions for example
  timely and complete immunization of young
  children or increased antenatal care coverage
• Strengthen national/local capacity so local
  people can maintain, improve or extend health
  care service delivery
• Strengthen PVO capacity to strategically manage,
  evaluate and plan its program portfolio across
  projects

17
Example of PVO Headquarters and Field Capacity
Strengthening for M E

• Activity Hire analytical staff at headquarters
• Develop training for field staff
• Output Country/project level staff trained to
  collect, analyze and use country/project level
  data to improve projects
• Outcome Changes are made in country level
  program to collect and analyze country/project
  level data
• Result At country/project level data will be
  routinely used for decision-making and project
  improvement
• Result PVO headquarters uses data from
  multiple evaluations across countries to
  strategically plan

18
Activities
Objectives
Inputs
Processes
Outputs
Outcomes
Results

• Your Planning Matrix objectives should bethe
  highest order objectives for which youcan be
  held accountable
• In your programs manageable interest
• They should
• Be specific
• Clearly state the type of change expected
• Be clear about who and what is changing
• Individuals, groups, organizations
• Be unidimensional, time-bound and measurable

19
Results flow chart
A clear set of indicators to measure program
performance
Whats missing?
20
Results flow chart

• Indicators
• Signal the achievement of the objective
• Evidence that the objective has been achieved
• Answer the question, How will we know
  achievement when we see it?

21
Selecting Indicators

• Key indicator selection to intended use
• Consistent with requirements of management
• Use USAIDs criteria
• Some technical areas meet most of these criteria
  easily others have a rough time
• Institutional capacity development has a rough
  time

22

• Who to involve?
• Partners and customers
• Donors
• Those familiarwith relevant data
• Those with technical data collection expertise
  inthe particular area
• When?
• Iterative - As work on program planning proceeds
• Check data sources before finalizing
• If you cant get the data at all, or you can get
  them but theyre bad or too expensive, you really
  dont have an indicator (and may not have a
  result)

23
High Quality Performance Indicators are...

• Direct
• Objective
• Adequate tomeasure the result
• Practical

24

• Indicators are straightforward and valid measures
  of the objective or result
• If direct indicators are not feasible, use
  credible proxy measures

25
High Quality Performance Indicators are...

• Direct
• Objective
• Adequate tomeasure the result
• Practical

26
Objective indicators
Unidimensional

• Unambiguous about
• ? what is being measured
• ? what data are being collected

Consistent over time
Quantitative (where possible)
27
High Quality Performance Indicators are...

• Direct
• Objective
• Adequate tomeasure the result
• Practical

28
Adequate indicators
Sensitive to change

• A performance indicator (and companion
  indicators) should adequately measure the result

Adequate coverage of the result
Disaggregated (when required)
29
High Quality Performance Indicators are...

• Direct
• Objective
• Adequate tomeasure the result
• Practical

30
Practicality

• Are the data associated with the indicator
  practical? Ask whether...
• Quality data are currently available
• The data can be procured on a regular and timely
  basis
• Primary data collection, when necessary, is
  feasible and cost-effective

31
of NGO counterparts own institutional
development initiatives undertaken resulting from
ICA
of community level institutional capacity
assessments conducted by counterpart NGOs
of NGO counterpart targeted communities
that_____?
Results Flow Chart Update
or of NGO counterparts applying ICA to their
organization
32
Communities establish self-managed and
self-reliant groups at the community level so
that people can improve their own health through
collective action
eg. Health committees Growth monitoring groups
Mothers/grandmothers clubs Fathers clubs,
etc. Community gardening groups Grain banks



33
Results flow chart
Indicators
34
Activities
Objectives
Inputs
Processes
Outputs
Outcomes
Results

• Your Planning Matrix objectives should bethe
  highest order objectives for which youcan be
  held accountable
• In your programs manageable interest
• They should
• Be specific
• Clearly state the type of change expected
• Be clear about who and what is changing
• Individuals, groups, organizations
• Be unidimensional, time-bound and measurable

35
Noted problems

• With objectives
• Set too low, output oriented
• Often dont describe change
• Technical objectives are pretty good, but
  capacity building and management arent
• With indicators
• Arent a direct match(nor reasonable proxy) of
  theobjective

36
Final thoughts shared values for common-sense
performance measurement

• More indicators arent necessarily better you
  need enough information to get an honest
  reckoning
• Indicators are only as good as the results they
  measure indicators are only meaningful to the
  extent they measure our results
• Data quality presumes indicator quality
• Be aware of the limitations (regarding validity,
  reliability, and timeliness) of your data
• Performance monitoring is only one source of
  performance information evaluation, observation,
  and experience complete the performancepicture

• Use data for managementits the main reason we
  measure performance
• Take a practical approachget adequate
  information that is reasonably current at a
  reasonable cost
• Indicators should make sense to a development
  professional, but be understandable to interested
  persons
• Choose the best indicators and data among the
  practical alternatives the development setting
  limits what we measure, and how we can measure it
• Meaningful qualitative indicators are usually
  okay and sometimes preferable