Methodologies of monitoring and evaluating capacity development

1
Methodologies of monitoring and evaluating
capacity development
UNFCCC Expert Workshop on Monitoring and
Evaluating Capacity-Building in Developing
CountriesRio de Janeiro, Brazil, 6-7 November,
2008

• Makoto Kato
• Japan
• kato_at_oecc.or.jp

2
Outline

• Defining Capacity, and Capacity Development
• Methodologies and Tools for Monitoring and
  Evaluating Capacity Development
• Comparison? ME at national level?
• Uniqueness of Capacity Development in Climate
  Change area

3
Defining Capacity, and Capacity Development

• Capacity
• Developing countries ability to cope with
  challenges by their own organization and
  actions.
• Capacity development
• Process in which such capacity is being
  strengthened and sustained at individual,
  institutional, and society levels as a whole.

3
Source JICA 2006
4
Capacity at different levels(Capacity
Development must be comprehensive)
Consensus in Burden sharing by different
stakeholders
New Waste Segregation Rules
Society Level
Policy Goals toward Waste Reduction
Institutional Level
Awareness raised about Waste Reduction
Strengthened Management of Waste Control Authority
Individual Level
Administrators Capacity Enhancement
Source JICA 2006
5
Different Approaches to Capacity
Development(Capacity Development should be
country-driven and sustainable)
Not Sustainable
Level of Capacity necessary to cope with
challenges
More Sustainable
External Input (Incl. Knowledge Sharing)
External Capital Transfer
Existing Capacity
Existing Capacity
Approach B Developing capacity with external
facilitation(building capacity under the
ownership of host countries/communities)
Approach A Transfer of capacity from external
player (Filling a Gap with external capital
transfer)
Source JICA 2006
6
Key Drivers of Capacity Development

• Ownership
• Enabling Environment
• Incentives
• Leadership
• Knowledge

7
Methodologies and Tools for Monitoring and
Evaluating Capacity-Building (1)
Flow of MEConceptual Figure of Project Cycle
Management(PCM)
Plan
Evaluation
Identification
Evaluation
Formulation
Implementation / Monitoring
Preparation
Implementation
8
Methodologies and Tools for Monitoring and
Evaluating Capacity-Building (2)
Where indicators are used Format of Project
Design Matrix (PDM)
-
-
-
Vertical Logical Relationship
9
Methodologies and Tools for Monitoring and
Evaluating Capacity-Building (3)
How Overalls Goals/project objectives and
Indicators are made? Problem Tree
Effect
Core Problem
Cause
10
Methodologies and Tools for Monitoring and
Evaluating Capacity-Building (3)
How Overalls Goals/project objectives and
Indicators are made? Objective Tree
Ends
Core Objective
Feasibility
Means
Expected Situations
11
Methodologies and Tools for Monitoring and
Evaluating Capacity-Building

• Evaluation is heavily dependent on Target
  Setting(based on capacity assessment)
• Target in different levels Overall Goal,
  Objectives of Efforts (eg. Policy/Programme/Projec
  t objectives)
• Indicators are useful only if/when they are
  elaborated with means of verification, important
  assumption and pre-condition (used in PDM)
• ME are properly done by Project Owners (and
  contracted external evaluators)

12
Can we compare?

• Same type Projects in 2 different countries

Country B
Country A
Project X
Project Y
Comparison is only useful for drawing lessons
from Good Practices (Simple replication does
not happen)
13
Can we compare?

• Same type Projects in the same country

Country C
Project P
Project Q

• (Internal Factors)
• Willingness of Stakeholders
• Entry point of Efforts etc.

• (Internal Factors)
• Willingness of Stakeholders
• Entry point of Efforts etc.

Easier to compare the two, since the external
factors are the same or similar. But still
internal factors of projects should be taken into
account.
14
Can we compare? ME at National Level

• Accuracy of Comparison (Evaluation)
• Project levelgtProgramme levelgtPolicy levelgt
• 
  Regional/International Network(At higher level,
  quantified indicators may omit useful qualified
  information)
• Methods of ME
• We use the same ME methods(some variety), but
  indicators are more tailor-made
• Pre-condition of Defining Indicators
• Capacity Assessment (jointly done by D-ingD-ed
  countries) is crucial
• Assessment(BAU)? Monitoring(Indicators)?
  Evaluation (Indicators) must be coherent, and
  reflect a specific context.

15
Uniqueness of Capacity Development in Climate
Change area

• This Area is already decided by 2/CP7.
• Fill in one of the 15 scopese.g. (promotion of)
  CDM

-
-
-
Entry points of such efforts are different by
host countries. Host countries chose such entry
point, jointly with our Agencies.
16
Conclusion from our practice

• A single methodology (with variation) for ME is
  used.
• Indicators are used and functions in specific
  context (within PDM), and simple aggregation of
  evaluation results is strictly avoided.
• Indicators are useful to interpret lessons from
  Good Practices.
• Evaluation at larger level employs more
  qualitative/narrative way.
• Entry points of efforts are different by
  countries, and it affects selection of
  indicators.
• For climate change, we approach both from
  Overall Goal and from the bottom side of PDM(
  and it still works).