Using Knowledge to Improve Development Effectiveness: An Evaluation of World Bank Economic and Secto

1
Using Knowledge to Improve Development
Effectiveness An Evaluation of World Bank
Economic and Sector Work and Technical
Assistance, 2000-06

• Helena Tang
• Lead Evaluation Officer
• September 2008

2
Why did we evaluate ESW and TA?

• The Bank considers knowledge important for
  development
• ESW and TA are the Banks main knowledge products
• Knowledge long-standing agenda for the Bank
• Global knowledge bank (1996)
• One of 6 strategic pillars going forward
• First comprehensive evaluation of these products
• Inform Banks future strategy on knowledge and
  learning

3
What are ESW and TA?

• ESW
• Economic reports (53 types)
• Inform Bank activities (strategy and lending)
• Influence clients policies and/or programs
• TA
• Technical Advice
• Implement reforms and strengthen institutions
  (drafting legislation, training in data analysis,
  knowledge sharing, etc.)

4
85 percent of AAA, FY00-06
5
One-quarter of spending on country services,
FY00-06
6
What questions did we ask?

• To what extent did ESW and TA meet their
  objectives?
• To what extent did the following affect the
  achievement of ESW and TA objectives?
• Origination (client-requested or not)
• Partnership in production with local institutions
  (government or others)
• Technical quality
• Dissemination

7
What evaluation tools did we use?

• Five Sets of Evidence
• 12 Country Reviews

8
Country Review Selection
Serbia
Romania
Jordan
Bangladesh
Mali
Guyana
Vietnam
Malaysia
Democratic Republic of Congo
Peru
Mauritius
Lesotho
9
What evaluation tools did we use?

• Five Sets of Evidence
• 12 Country Reviews
• Electronic Surveys of in-country stakeholders
• Specific ESW
• Specific TA
• General
• Electronic Surveys of all ESW and TA TTLs
• Electronic Survey of loan TTLs
• Statistical and econometric analysis

10

• MAIN FINDINGS

11
Client views on ESW and TA

• Clients find Bank ESW and TA more useful than
  those provided by other institutions
• Clients value Bank ESW and TA for their high
  quality, objectivity, and provision of
  international perspectives
• Clients generally prefer TA over ESW (IBRD IDA)
• Middle income country (MIC) clients prefer TA and
  ESW over lending
• Some MIC clients clearly prefer de-linking TA and
  ESW from lending

12
Client country respondents had a range of views
on the effectiveness of ESW and TA
At least two-thirds gave an above average rating
13
Examples of effective ESW Vietnam PER

• Budget legislation
• MTEF
• Capacity

14
Examples of effective ESW ICAs

• Privatization (Serbia)
• Competitiveness strategy (Guyana)
• Labor law, property registration, deregulation of
  public service delivery (Malaysia)

15
Example of effective TA Mauritius Aid for Trade

• Just-in-time advice on trade reform program
• Analysis of reform scenarios and effects
• Recommendations incorporated in government reform
  program

16
ESW improved Bank activities

• Shaped country assistance strategies
• Improved lending quality
• Presence of relevant ESW associated with better
  loan quality at entry
• Around 90 percent of DPL but only around 60
  percent of investment loans

17
What made ESW and TA effective?

• Technical Quality
• Good quality ESW requires resources
• ESW better resourced in IBRD than in IDA
  countries
• Bank budget and not trust-fund matters
• Origination
• Client interest and buy-in essential but products
  can be originated by the Bank

18
What made ESW and TA effective?

• Partnership
• Close collaboration with clients throughout the
  process but not necessarily co-production
• Collaboration takes time (completion of forest
  sector review in DRC delayed nearly 2 years)
• Dissemination
• Sustained engagement beyond one-off dissemination
  
• Broad vs targeted
• Language and translation

19
What made ESW and TA effective?

• Government capacity
• Lower in post-conflict and some low income
  countries (DRC, Lesotho)
• Lower in countries with high turnover of senior
  government officials (Jordan, Serbia, and in the
  sector ministries in Peru)

20

• RECOMMENDATIONS

21
Recommendation 1

• Reinvigorate the mandate (underpinned FY99 ESW
  reforms) of a strong knowledge base on countries
  where Bank is providing (planning to provide)
  funds

22
Recommendation 2

• For IDA countries, ensure ESW is adequately
  resourced, even if it means fewer ESW in some
  countries

23
Recommendation 3

• Enhance institutional arrangements substantive
  task team presence in country offices, and
  include a clear strategy for sustained
  post-delivery engagement

24
Recommendation 4

• Recognize and build on client feedback to help
  counter-balance current Bank incentives for
  lending over non-lending, and ESW over TA

Tell Us What You Think
Feedback
25
Recommendation 5

• Take results tracking framework more seriously,
  including systematizing client-feedback