Assessing the Effectiveness of Multilateral Organisations

1
Assessing the Effectiveness of Multilateral
Organisations

• Simon Burall
• Research Fellow
• Centre for Aid and Public Expenditure
• Overseas Development Institute
• London, UK
• s.burall_at_odi.org.uk

2

• Context
• Previous attempts to assess effectiveness
• Describe the methodology
• The global results
• The Bangladeshi results
• Discussion

3
Context Aid Volumes Rising
DAC Members net ODA 1990 2006 and DAC
Secretariat simulations of net ODA to 2010
4
The Paris Declaration

• Signed at the Paris High-Level Forum on Aid
  Effectiveness (2005) by
• 35 donor countries
• 26 multilateral donor agencies
• 56 countries that receive aid
• 14 Civil society organisations acting as
  observers
• Follow-up summit in Accra in 2008 and another in
  2010

5
Paris Declaration
6
Whats new about Paris declaration?

• Number and level of Participants
• Detailed set of 56 specific commitments
• 12 agreed measurable indicators to be monitored
• Potential impact on behaviour and practice at
  country level

7
Other challenges
8
Assessments of Effectiveness

• Project Evaluations
• Individual Donor Evaluations
• Bilateral Donor Assessments
• Intergovernmental Assessments
• Some work by civil society
• Most by and for Donors
• Results not always publicly available

9
The Perceptions Survey

• DFID commissioned this pilot survey
• Why?
• Interested in stakeholder perceptions
• Comprehensive spending review
• Implications

10
The Survey - Countries
Bangladesh
India
Ghana
Tanzania
Zambia
South Africa
11
The Survey - Organisations

• Regional Development Banks
• African and Asian Development Banks
• European Commission
• Global Fund to Fight TB, HIVAIDS and Malaria
• United Nations Childrens Fund
• United Nations Development Programme
• World Bank

12
The Survey - Apples and Pears

• Global Fund
• UNDP
• UNICEF

• AfDB
• AsDB
• European Commission
• World Bank

13
The Survey - Stakeholder Groups

• Government Officials
• Civil Servants
• Parliamentarians
• Business Leaders
• Civil Society Leaders

14
The Survey - Questionnaire

• 17 questions covering 4 broad areas
• Paris Declaration
• 15 effectiveness criteria
• Effectiveness of MOs
• Disbursement preferences
• Respondents asked to rate, rank and comment

15
The Survey Who Answered?The countries
16
The Survey Who Answered? The Stakeholders
17
The Results Ranking Funding Performance
18
The Results Ranking Policies and Procedures
19
The Results Paris Declaration

• Little Difference
• UN Agencies tend to be rated higher, but this is
  country specific

20
The Results Overall Effectiveness
Mean rating very low (1) very high (5)

• No difference in aggregate and limited at country
  level
• This is surprising
• Other factors taken into account

21
Comments on Effectiveness

• AfDB support tend to be visible in
  infrastructure. (Zambia Civil Service)
• GFATM and UNICEF's funding especially as regards
  to HIV/AIDS and children, its effectiveness can
  be seen through activities currently implemented
  in these two areas (Tanzanian Civil Service)
• Organisations such as the EC hide behind their
  regulations forcing recipients to use their
  procedures arguing that these cannot be changed
  quickly because of their bureaucracy
  etc... (South Africa Government)
• The WB and AsDB monitors the projects very
  effectively and continuously (Bangladesh Civil
  Service)
• The European Commission is very bureaucratic and
  takes a long time ... UNDP and World Bank are
  much better, even though there are delays also
  (Ghana Business)
• The World Bank gives loans as a general rule
  which have to be paid at some future date. We
  need more grants (Zambia Business)

22
The Results Disbursement Preferences
Specialised Agencies
Large scale funders

• Clear Preferences
• Weakly correlated to effectiveness indicators
• RDBs above EC and WB
• Governance appears to be important

23
Comments on Disbursement

• The AfDB is an African Development Bank it
  therefore has a better understanding of Africa's
  situation (Zambia Business)
• AsDB and World Bank will be best in areas of
  physical development e.g. infrastructure related
  works (Bangladesh Government)
• Aid is more effective for the EC ranked high,
  less transaction costs, less unnecessary travel
  costs, more flexible and programmes more aligned
  to government plans (Zambia Government)
• WB's loan disbursement policy is guided by the
  head office. Their prescription never give
  results for the LDCs. (Bangladesh Business)
• UN agencies are not cost effective and their
  efforts have limited penetration compared to WB,
  EC and AsDB. (India Parliament)
• UNDP works most closely with the Governments
  and Civil society compared to the others,
  therefore more aid to UNDP may mean more support
  to Govenments and CSOs (Tanzanian Civil Society)

24
The Results

• Stakeholders want to be heard
• 261 people
• Over 2300 comments