Title: IFC
1Post-evaluation in IFCNovember, 2003
William E. Stevenson Roland Michelitsch Operations
Evaluation Group (OEG)
2Overview
- OEG in the World Bank Group
- Project evaluation in IFC
- Beyond project evaluation
3The World Bank Group
Board
President World Bank Group
International Finance Corporation
Multilateral Inv. Guarantee Agency
World Bank
4International Finance Corporation
Board
President World Bank Group
International Finance Corporation
Multilateral Inv. Guarantee Agency
World Bank
5IFCs purpose
- Article 1
- further economic development
- by encouraging the growth of
- productive private enterprise
6IFCs instruments
- IFC loans equity for viable projects
- Capital mobilized from other sources
- Advisory services
7IFCs mission
- Promote sustainable
- private sector investment
- in developing countries,
- helping to reduce poverty
- and improve peoples lives
8IFCs strategy sustainable development
- Support enterprises that
- can prosper in the marketplace
- dont rely on distortions
- are environmentally benign or beneficial
- tangibly improve peoples lives
9WBG independent evaluation
Board
Director-General, Operations Evaluation
President World Bank Group
International Finance Corporation
Multilateral Inv. Guarantee Agency
World Bank
Operations Evaluation Group
Operations Evaluation Unit
Operations Evaluation Department
10OEG independent evaluation in IFC
Board
Director-General, Operations Evaluation
President World Bank Group
International Finance Corporation
Multilateral Inv. Guarantee Agency
World Bank
Operations Evaluation Group
Operations Evaluation Unit
Operations Evaluation Department
11OEGs mission
- Contributing to sustainable
- private sector investment
- through excellence in evaluation
12Evaluation principles
- Learning accountability
- Relevance
- Independence
- Coherence
- Incentives
13Overview
- OEG in the World Bank Group
- Project evaluation in IFC
14Project-level evaluation in IFC
- Self-evaluation
- Expanded Project Spvn. Reports (XPSR)
- Other PSRs, portfolio review meetings
- Independent evaluation
- XPSRs
- System
- Sampling
- Framework
- Validation (ratings lessons)
- LRN system (lessons dissemination)
15Project evaluation system
- Purpose learning accountability
- Design relevance, efficiency, coherence
- Evaluation framework derived from IFCs purpose
- Random, representative sample
- Investment departments write XPSRs
- OEG reviews and validates findings
- IFC management discusses 30 per year
- Lessons easily accessible through LRN
- Incentives for good execution
- Results feed into investment dept. scorecards
16Evaluation framework
- Derived from IFCs purpose mission
- In place since 1996
- Aligned with front-end sustainability measures
- Best practice among private sector MDBs
17MDBs portfolios evaluation reports
18Evaluation framework Performance dimensions
- Development outcome
- IFCs investment outcome
- IFCs effectiveness
19Evaluation framework Development outcome
- Synthesis of projects overall impact on
countrys development - How well a project has contributed to IFCs
purpose mission - Compared to what would have happened without the
project - Based on 4 indicators
20Development outcome Synthesis of 4 indicators
( their success standards)
- Business performance ?Financial viability
(FRR gt WACC) - Economic sustainability ?Economic viability
(ERR gt 10, plus qualitative aspects) - Environmental (ESHS) ?Compliance
witheffects IFCs safeguards - PSD ?Demonstration effects (e.g.
technology, governance), linkages,
changes in legal and regulatory framework
21IFCs investment outcome Synthesis of 2
indicators
- Gross contribution loan ?Loan will earn
expected interest income - Gross contribution equity ?Return on equity
(ROE) exceeds fixed rate loan plus
premium
22IFCs effectiveness (work quality) Synthesis of
3 indicators
- Screening, appraisal, ?Materially sound
structuring execution (e.g. operating
policies and procedures, credit notes,
etc.) - Supervision, ?Appropriate monitoring
administration (e.g. compliance with
covenants, - Role, contribution ?IFCs operating
principles/strategy IFCs added
value Client satisfaction
(timeliness, etc.)
23Project scorecard (sample)
24Incentives are essential Awards for good XPSR
execution
25Work quality drives outcomes
- Departments with the best work quality generally
achieved the best scores for development outcome - Development outcome is a key ingredient of the
Scorecard
- Investment outcome similar pattern
26Incentives are essentialInvestment department
scorecard
- Development Impact
- Profitability
- Client Satisfaction
- Average XPSR development impact scores
- New commitments which meet criteria for high
additionality
- Department net income
- Department cost/income ratios
- Volume of commitments
- Quality of IFCs services
- Responsiveness and timeliness of IFCs services
- Overall satisfaction with IFCs services
27Incentives are essentialInvestment department
scorecard
- Development Impact
- Profitability
- Client Satisfaction
- Average XPSR development impact scores
- New commitments which meet criteria for high
additionality
- Department net income
- Department cost/income ratios
- Volume of commitments
- Quality of IFCs services
- Responsiveness and timeliness of IFCs services
- Overall satisfaction with IFCs services
28Project evaluation - summary
- Tailored to IFCs needs
- 3 key performance dimensions
- Aggregatable results
- Coherent measures incentives
29Overview
- OEG in the World Bank Group
- Project evaluation in IFC
- Beyond project evaluation
30Evaluation learning from experience
OEGs synthesis
Generation
Annual reviews
XPSRs OEGs mini XPSRs
OEG special studies
The basic building blocks
IFCs
Evaluation
Learning
Cycle
Dissemination
Application
Policy and strategy
Screening
Appraisal
Structuring
Board approval
Supervision and evaluation
31XPSRs contributing to learning in IFC
- Case Studies
- Training
- Publications
Top Lessons (country, sector, subject)
Portfolio Presentations
Annual Review
Special Studies
Lessons Retrieval Network (LRN)
Learning by doing
IOs using XPSRs
XPSR
32OEGs annual reporting to Board
- Annual Review of IFCs Evaluation Findings
- Draws from all evaluation findings
- Recommendations to improve IFCs results
- Annual Report on Operations Evaluation in IFC
- Functioning of evaluation processes
- Recommendations to improve processes
- Work Program and Budget
- Agreed among Management, DGO, Board
33OEGs special studies
- Country
- Brazil (2003)
- Russia (2002)
- Bulgaria (2002 mini)
- Vietnam (2001)
- Kazakhstan (2000)
- Tanzania (2000)
34OEGs special studies
- Sector
- Extractive industries (2003)
- Agribusiness (2003)
- PSD in energy (2003)
- Information infrastructure (2000)
- African financial intermediaries (1999)
35OEGs special studies
- Program and thematic evaluation
- Project development facilities (ongoing)
- Equity rating reliability (2003)
- Project economic analysis (2000)
- Africa Enterprise Fund (2000)
- Forest strategy in IFC (2000)
- Foreign Investment Advisory Service (1998)
- Problem projects (1997)
- Market assessments (1997)
36OEG Vision Embedded evaluation
- Supervision provides aggregatable, real-time
information about entire portfolio - Financial viability
- Economic viability
- Environmental compliance
- Achievement of PSD objectives
- Management reports portfolio results
- OEG evaluation shifts focus
- Synthesis analysis and dissemination dominate
- Quality control function reduced
37An evaluation system should provide
- Relevant learning
- Credible accountability
- Coherent measurement
- Learning incentives