Title: Delegation constituted in La Paz
1European CommissionJoint Evaluation Unit common
to EuropeAid, Relex and Development
Methodology for Evaluation of Budget support
operations at Country levelConceptual and
Methodological Note
2EXISTING BUDGET SUPPORT EVALUATIONS 2004
Tanzania 2006 Joint Evaluation of 7 countries
from 1994 to 2004 budget support 2007
Ghana
- Evaluations addressing mainly THE PROCESS
- And not enough the CHAIN OF RESULTS
- IMPACTS AT THE GLOBAL LEVEL
- IMPACTS ON SECTOR POLICIES
3- Expected Outputs of
- the Budget Support Evaluation
- An inventory of the inputs provided through
budget support during a given period. - An assessment of the performance of these inputs
in terms of direct outputs and induced outputs. - An identification of the changes which have
occurred during the period in the partner country
and the key causal factors driving those changes
related to the budget support. - An assessment of the extent to which budget
support may have contributed to the results
identified at the outcome and impact levels. - An assessment of the overall relevance of budget
support arrangements in the light of the evolving
country and sector specific context, the aid
policy and the related goals.
4- Comprehensive Evaluation Methodology
- A Three components approach methodology combining
- An evaluation of the budget support itself
- An assessment of the budget support related
outcomes and impacts of the government strategy - An exploration of the linkages between the
budget support process and some outcomes and
impacts of the government strategy - Focused on the main topics of the budget support
- Each topic is associated with one or several
Evaluation Questions, Judgement Criteria and
Indicators - To ensure harmonisation of the evaluations in
various contexts
5GBS/SBS COMPREHENSIVE EVALUATION FRAMEWORK
6EVALUATION OF THE BUDGET SUPPORT
Overview of the national context and its evolution
Entry conditions for budget support
Volume of funds, policy dialogue and
conditionalities, technical assistance and
capacity building, harmonisation and alignment
Assessment of the inputs provided by the budget
support
Assessment of the direct outputs of the budget
support
Effects of the identified inputs including
effects on transaction costs
Assessment of the induced outputs and the
corresponding government actions
Changes that budget support has induced in
comparison to what was expected
7ASSESSMENT OF IMPACTS AND OUTCOMES
Government
- Economic growth
- Incidence of income and non income poverty
- Degree of empowerment and social inclusion
- Evolution of MDG indicators
- Achieving sector development objectives for SBS
Review of the impacts of the government
strategy specifically those related to the
budget support deliverables
e.g.
8Causal factors
- Changes in business confidence and
competitiveness - Changes in the use and appreciation by the
beneficiaries of goods and services provided by
the public sector - Changes in the degree of confidence of the
general public sector
Identification of the factors that have
contributed to the key changes
e.g.
Identification of the factors external to the
government strategy which could have had an
impact on the mentioned changes
- Regional and international economic
- Political environment
- Other unforeseen event
e.g.
9 Relationship
- Positive linkages between government actions and
improved outcomes - and
- Failure or inappropriate actions which can have
jeopardised the quality of the outcome
Identification of the outputs of the government
strategy which appear to have been of greatest
importance in generating change at outcomes level
10 ASSESSMENT OF THE INFLUENCE OF BUDGET
SUPPORT INTEGRATION AND CONSISTENCIES BETWEEN
COMPONENTS 1 2
- Comparison of the results of component 1 and
component 2 - Improved understanding of the influence of
budget support on the institutional and policy
changes - Increased consideration of the role of dominant
factors in the determination of outcomes and
impacts
If consistencies indicate causal links showing
positive conclusion for budget support
If no consistency find a possible explanation
11GBS/SBS COMPREHENSIVE EVALUATION FRAMEWORK
Component 1
Component 3
Component 2
12HOW TO MANAGE ANDIMPLEMENT THIS
EVALUATION?CHALLENGES AND PROPOSALS
- We must take into account
- THE STAKEHOLDERS
- THE LEAD OF THE EVALUATION
- THE CONSISTENCY, HARMONISATION AND DISSEMINATION
ISSUES - THE IMPLEMENTATION
- THE CAPACITY BUILDING ISSUE
13STAKEHOLDERS
- To whom this evaluation is for?
- Are the donors the only stakeholders?
- Proposal 1
- All donors, the government and other main
stakeholders involved in the budget support
evaluated should be part of the evaluation. - The evaluation of budget support operation in a
designed country should be part of a multi-donor
evaluation programme.
14EVALUATION LEAD
- Who will take the lead for managing the
evaluation? - From where will the evaluation be managed?
- Proposal 2
- One of the main donors in the country should be
chosen by the other donors to take the lead and
consequently for contracting with the evaluators.
Special arrangements should be set up to involve
representatives in the country.
15CONSISTENCY, HARMONISATION AND DISSEMINATION
ISSUES
- Proposal 3
- To ensure consistency, the headquarter
Evaluation Service of the lead donor should be
responsible for the evaluation methodology and
quality assurance. Each donor can have dedicated
persons in his Evaluation Service to go into the
countries where the evaluation takes place, at
crucial moments. - Proposal 4
- A network of donors should be set up to ensure
harmonisation and dissemination.
16IMPLEMENTATION
- Who will do the evaluation?
- Who has currently the competencies to evaluate
budget support and how to manage it? - Proposal 5
- A team of external independent consultants
familiar with the main topics of the evaluation
should be contracted for each evaluation.
17CAPACITY BUILDING
- Do we need a specific training on budget support
evaluation? - Proposal 6
- Make sure that few consultants are immediately
operational and have been already trained in
existing trainings - Proposal 7
- Train the identified people within the
Evaluation services of the Member States, some
external independent consultants and the main
governmental stakeholders of the recipient
countries.
18- Issue paper (May 2008)
- Steering group at OCDE / DAC level (June 2008)
- 1st meeting of a core group (September 2008)
- DAC Evaluation Network (18 19 November 2008)
- 2nd meeting of core group (December 2008)
- Common methodological document and standard terms
of reference (early 2009) - Testing methodological approach with partners
agreement (2009)