Setting Key performance indicators to monitor progress

1
Setting Key performance indicators to monitor
progress

• What are indicators
• Types of indicators
• Indicators at different levels
• Qualities of a good indicator
• Setting up indicators

2
Definition of an Indicator

• Specific information that provides evidence about
  the achievement of planned impacts, results and
  activities

Ideally indicators should be reported
quantitatively but this will not always be
possible - dont limit ME to only what can be
measured
3
What are indicators

• Indicators are yardsticks that can be used to
  demonstrate that changes has or has not taken
  place.
• They provide meaningful and comparable
  information to changes.
• They are measurable or tangible signs that
  something has been done or that something has
  been achieved.

4
Examples of indicators

• Indicator for improved standard of
  living-increased number of television aerials
• Indicator for community empowerment-increased
  frequency of community members speaking at
  community meetings
• Indicators are an essential part of a monitoring
  and evaluation system because that is what you
  measure.

5
Types of indicators

• Indicators are either qualitative or quantitative
  criteria used to check whether planned changes
  have taken place as intended.
• They (indicators) are designed to provide a
  standard against which to measure or assess or
  even show the success or progress of a programme
  against stated targets
• Indicators have to be suggested, negotiated,
• adapted and approved.

6
Types of indicators

• Quantitative indicators
• Should be reported in terms of a specific number
  (number, mean, or median) or percentage.
• Assessing the significance of an outcome requires
  data on both number and percent.
• Qualitative indicators
• Qualitative statements
• Measure perceptions
• Measure attitude, behavior

7
Quantitative indicators

• Examples
• Number of
• Proportion of
• Percentage of
• Amount of
• The ratio of
• Length of distance
• Weight of
• Size of
• Areas of/spread of
• Value of
• etc.

8
Qualitative Indicators

• Examples
• Level of
• Presence of
• Evidence of
• Availability of
• Quality of
• Accessibility of
• Existence of
• Sustainability of
• Improvement of
• Ability to (e.g. skills)
• Potential of
• etc.

9

• Pre-designed indicators
• The pros and cons of pre-designed indicators
  (Indicators that are established independently ).
• MDGs, World Bank, IMF)
• Pros
• They can be aggregated across similar projects
  and policies
• Reduce costs of building multiple unique
  measurement systems
• Make possible greater harmonization

10
Pre-designed Indicators

• Cons of predetermined indicators
• They dont address country specific or
  organizational goals
• They are often viewed as imposed
• They dont promote key stakeholder participation
• They can lead to the adoption of multiple
  competing indicators.

11
Indicators at different levels

• Indicators are needed to monitor
• Inputs,
• Activities
• Outputs
• Outcomes
• Goals

12
Outcome Indicators

• Outcome indicators assess progress towards the
  project purpose and development goal
• Is the project making a difference
• Target response
• Benefits

13
Outcome indicators

• Development Goal
• Rural households on
• communal lands
• increase their average
• household income by 25

• Indicators
• Increase in number of children from rural
  households enrolled in school
• Decrease in incidences of malnourishment

14
Outcome indicators

• Project purpose
• Subsistence farmers Increase their crop yields by
  implementing more appropriate crop production
  methods

• Indicators
• Farmers fertilize and fence their land.
• Crop farmers plant and harvest a more diverse
  range of high grade vegetables.
• Farmers sell meat milk and vegetables in the
  district markets

15
Output indicators

• Is the project making any progress towards its
  objectives
• How efficiently is the project making use of its
  resources.
• Performance
• Quantity, quality, timing, location, people
• Efficiency
• Were too many resources used

16
Indicators at Different Levels in the Objective
Hierarchy

• Impact indicators - indicators that show to what
  extent the project has contributed towards its
  goals
• Result (Outcome and Output) indicators -
  indicators that show to what extent planned
  results (outputs and outcomes) have been achieved

17
Indicators at different levels

• Activities (Process indicators) - indicators that
  show what activities have been completed
• Input indicators - indicators that show what
  resources have been used by the project

18
Qualities of a good indicator

• The CREAM of good performance indicators.
• Clear- Precise and unambiguous
• Relevant- Appropriate to the subject at hand
• Economic-Available at a reasonable cost
• Adequate-Provide a sufficient basis to assess
  performance
• Monitor able-Amenable to independent validation

19
Qualities of a good indicator

• Assessing proposed indicators
• Is the indicator..
• As direct as possible a reflection of the outcome
  itself?
• Sufficiently precise to ensure objective
  measurement?
• Calling for the most practical, cost-effective
  collection of data ?
• Sensitive to change in the outcome, but
  relatively unaffected by other changes?
• Disaggregated as needed when reporting on the
  outcome?

20
Qualities of a good indicator

• In general, indicators should
• be verifiable (where and how to we get
  information about the indicator)
• measure what is important and not what is easy to
  measure
• measure only changes that can be linked or
  attributed to the project/programme
• be targeted in terms of quantity, quality and
  timing
• be useful in that valid and reliable (see p. )
  data are available
• measure either quantitative or qualitative change.

21
How SMART?

• As far as possible results should be
• Specific
• Measurable
• Achievable
• Relevant (to the project purpose and goal)
• Timeframe
• But dont get too SMART
• What is achievable may need to be developed from
  experience
• Good ideas take time to develop
• Not everything that is worth doing can be easily
  measured

22
Ideal number of indicators

• What is the ideal number of indicators?
• The minimum number that answers the question
• Has the outcome been achieved
• Has the output been achieved

23
Developing Indicators

• Step 1
• Identify the problem situation you are
• trying to address
• Economic
• Social

24
Developing Indicators

• Step 2
• Develop a vision for how you would like
• the problem areas to look like
• What will tell you that the vision has been
  achieved?
• What signs will you see that will prove the
  vision has been achieved
• This will give you impact indicators
• If you can answer yes to these questions then
  progress is being made.

25
Developing Indicators

• Step 3
• Develop a process vision for how you
• want things to be achieved
• Participation
• Enhancing community empowerment
• This will give you process indicators

26
Developing Indicators

• Step 4
• Develop indicators for effectiveness
• Evidence compared to baseline
• Indicators to show how you have effectively
  changed for example peoples attitudes and
  behaviors
• This will give you outcome and output indicators

27
Developing Indicators

• Step 5
• Develop indicators for efficiency targets
• Indicators of time management
• Resource utilization
• With this steps you are in a position to
• monitor and evaluate efficiency, impact ,

28
Weaknesses in Indicator design

• Indicators are usually discussed and defined late
  in the planning process (especially when this is
  confined to workshops).
• Indicators are designed to meet scientific
  research needs and are therefore less appropriate
  and meaningful for the beneficiaries.
• They are often not measurable in terms of
  cost-effectiveness.
• There are usually too many indicators.
• Indicators which are easy to measure are
  preferred.
• Quantitative indicators are favoured to produce
  hard and reliable statistics.

29
Weaknesses in Indicator design

• They are usually not developed by stakeholders
  and/or beneficiaries and therefore do not
  represent their reality.
• Indicators are donor driven since they have to
  legitimise the support for the respective
  project/programme.
• Indicators are often an outcome of a desk study
  to prepare a proposal, satisfy funding
  requirements or to establish a baseline.