Evaluation Unit EuropeAid

1
Evaluation Unit EuropeAid
Workshop on Lessons Learned from International
Joint Evaluations French Ministry of Economy,
Finance and Industry 6th February, 2012.

• Martyn Pennington
• Head of Evaluation Unit- Devco B2

2
Budget Support Evaluation
MUTUAL TRUST
3
Consensus Building Process

• Creation of two groups

Present work for further discussion and approval
Steering Group
Core Group
Head of evaluation services
OECD/ DAC
Delegate Tasks
4
Activities Include
Donor Country

• Agreement on the approach
• Standard ToR for the evaluation of Budget support
• Criteria for the choice of test evaluations
• Inventory of BS support interventions and
  proposal for the tests
• Introductory Note to the Partner country to ease
  involvement
• Choice of (a) lead donor(s) for each evaluation
• Ongoing work on Institutional Arrangements
  (multiannual evaluation plan, training issues,
  quality control, harmonization and capitalization)

5
Partner Country
Activities Include

•  

• First contact through the representatives of the
  donors in partner country leader in
• country is the lead donor at the head quarter
  level.
• Agreement from the partner countries to undertake
  such an evaluation
• Different level of experience in regards to
  getting Govts on board

6
What worked well?
The Case of Mali

• Good involvement of the partner country,
  participation of different stakeholders
• Participation of some donors in a Reference Group
  at country level
•   
• Good discussion among the Management Group at HQ
  level
•  
• Important work provided by a small and strong
  team of consultants
• Rapid agreement on the conclusions and
  recommendations of the report
• Clear separation between lead donor and
  evaluation team (consultants)

7
Important Challenges
Management Necessity to have a strong
leadership at the HQ level of the lead donor
Not enough involvement of the partner
country in reading and/or commenting the reports.
and how have they been addressed
Final decision taken by the lead donor,
well documented In-depth work only with
the donors available
8
Important Challenges
and how have they been addressed
Methodology

• Desk analysis, field mission, synthesis phase and
  the 3 compulsory reports writing (inception,
  desk, final)
• Too many versions of the report before reaching
  final agreement
• Complexity of the 3 step approach

Compromises on the less important
issues Acceptable for pilot evaluation but
not for regular evaluations. Continuous
refinement for simplification and time
efficiency purposes. But, budget support is a
complex instrument and the evaluation of its
impacts is even more complex.
9
Important Challenges
and how have they been addressed them
Timing
Basis of work, no compromise Tailored
according to the need Work necessary

• Finalisation of the ToR (inclusion of the
  information from every donors time consuming)
• Lack of information and too short foreseen field
  mission
• Significant difficulties to consolidate all the
  comments in an understandable and useful
  presentation for the consultants

10
Next Steps

• Simplification of the approach
• Current development of framework for Policy
  Dialogue for improved understanding (internal and
  external) and better documentation (back to
  office reports, e-mail exchanges, etc)
• Raising awareness among Government officials,
  donors, NSAs
• on the approach and how to improve the
  efficiency and effectiveness
• of these evaluations.
• First intervention the workshop to be
  hosted by the Belgium government
• ( 21/22 March 2012)
• Increase pool of experts that can carry out such
  evaluation development
• and implementation of training workshops