Evaluating the Governance of Global and Regional Partnership Programs (GRPPs)

1
Evaluating the Governance of Global and Regional
Partnership Programs (GRPPs)

• Anna Aghumian and Chris Gerrard
• IEG World Bank
• November 13, 2009

2
Main Messages

• Assessing legitimacy and effectiveness of
  governance and management is essential since
  shared governance is a key characteristic of all
  GRPPs
• As international public sector organizations,
  GRPPs should be expected to comply with generally
  accepted principles of public sector governance
• Understanding how governance is actually
  practiced requires more than just a cursory
  examination of a programs charter,
  organizational chart and TORs

3
Prevailing Governance Models among the 60
Programs Reviewed

•  

Shareholder Model Stakeholder Model Prominent Individuals Hybrid
Global Partnership programs 9 35 4 2
Regional Partnership Programs 3 7 -- --
4
Special Features of GRPPs in Relation to
Governance Management

• Often have complex governance and management
  structures
• Need to establish their legitimacy on a basis
  other than shareholder rights
• Often have a long chain of accountability from
  global to local
• Have a global community clientele, making
  transparency in planning and implementation
  particularly important
• Often housed in existing international
  organizations

5
Assessing Governance Management Suggested
Criteria

• Legitimacy in the exercise of authority in
  relation to those with a legitimate interest in
  the program
• Efficiency governance management structures
  facilitate efficient allocation and use of
  resources
• Accountability up and down the internal chain
  of command and control
• Responsibility to stakeholders outside the
  internal chain of command and control
• Transparency in relation to decision-making,
  reporting, and evaluation
• Fairness equal opportunity for partners and
  participants, similarly situated, to influence
  and benefit from the program

6
Governance Management Suggested Approach

• Start from a clear understanding of the G M
  arrangements and processes
• The extent to which these are well articulated
  and working well to bring about legitimate and
  effective governance and management of the
  program
• Host arrangements, if any
• Primary focus on governance. Focus on those
  aspects of management that most directly affect
  program performance
• Build upon and add to the assessments of
  relevance, efficacy, and efficiency of the Program

7
Evaluating Governance and Management of GRPPs
Tools and Instruments

• Desk review of key founding documents
• Interviews with key partners and other
  stakeholders. Good practice evaluations use
  interview protocols, semi- structured surveys
  (e.g. GAVIs interview guide for Board members
  GDNs Board Survey questionnaire)
• Surveys of members of the governing bodies, wider
  circle of stakeholders, beneficiaries
• Other review of meeting minutes of the
  governing, executive, and advisory bodies Board
  meeting attendance rate
• Good practice evaluations use a mix of evaluation
  instruments

8
Extent to Which 60 Evaluations Assessed Different
Dimensions of G M
9
Good Practice Examples Legitimacy

• Global Fund to Fight AIDS, Tuberculosis and
  Malaria
• Evaluation question To what extent the Board
  is representative and to what extent its decision
  making is in accord with the GFs founding
  principles
• Methodology Board interviews and stakeholders
  survey, Board meeting minutes reviews,
  assessment of Board and committee meeting
  attendance rates
• Findings
• Board is formally representative. Yet, some
  constituencies are not participating effectively
  and do not have equal voice
• Poor communication with beneficiaries, linguistic
  barriers, lack of adequate financial resources
  for beneficiary representation
• Recommendation The Board should improve the
  quality of representation by enhancing
  communication with all constituencies and by
  favorably considering proposals for assistance
  from constituencies with limited resources

10
Efficiency

• Association for the Development of Education in
  Africa (ADEA)
• Evaluation question Are the ADEA structures
  adequate and functioning properly in light of
  their assigned tasks and available resources?
• Methodology Organizational assessment based on
  interviews with stakeholders and staff, on-line
  survey, desk review of documents and field visits
• Findings
• The Steering Committee is overburdened with the
  management of the program, and no time is left
  for considering strategic issues.
• Decision-by-consensus model, while provides
  equal access to decision making to all members of
  the SC, slows down its responsiveness
• Host arrangements with IIEP reduces the
  efficiency of G M of ADEA
• Recommendations The SC should revise
  organizational structure of ADEA, improve
  division of the roles and responsibilities
  between the SC and the Secretariat, conduct a
  cost-benefit analysis of the host arrangements
  with IIEP

11
Accountability

• Global Alliance for Vaccines and Immunization
  (GAVI)
• Evaluation questions How well has the GAVI Fund
  governance structure worked? Is there clarity of
  role/responsibilities between various entities?
• Methodology Interviews with key informants, desk
  review of documents, prior assessments of GAVIs
  governance
• Findings
• Lack of accountability due to separation of
  programmatic and fiduciary responsibilities
  between GAVI Alliance and GAVI Fund
• Unclear and weak accountability chain within each
  of the governing bodies (vertical accountability)
  
• Unclear view among partners regarding their
  respective roles and responsibilities
• Recommendation Drastic changes in governance
  structure

12
Responsibility

• Medicines for Malaria Venture
• Evaluation question The extent to which the
  program accepts and exercises responsibility to
  stakeholders who are not directly involved in
  governance
• Methodology Desk review of documents,
  interviews
• Findings
• MMV has increased engagement of researchers and
  research institutions in endemic countries
• Has held key meetings in countries where malaria
  is widespread and MMV-sponsored research is
  underway
• Has included a majority of beneficiary country
  members on its Access and Delivery Advisory
  Committee
• Recommendation To engage more advocacy NGOs in
  the design and execution of MMVs access and
  delivery work program.

13
Transparency

• Partnership for Maternal, Newborn and Child
  Health
• Evaluation question Is there adequate
  transparency in governance?
• Methodology Systematic review of the programs
  website
• Findings
• Board meeting minutes are accessible on the
  website, but other important background material
  is not provided
• Financial information is not fully available
• Work plans are not available on the website,
  leading to potential duplication of activities by
  partners
• Recommendations Develop a Board-approved
  disclosure policy

14
Fairness

• Cities Alliance
• Evaluation question The extent to which
  participants similarly situated have equal
  opportunity to receive benefits from the program
• Methodology Desk review of documents, interviews
• Findings
• All grant applications have to be sponsored by a
  board member and about 90 of grant applications
  are approved
• Some kind of screening or pre-selection process
  was taking place which was not transparent and
  potentially unfair to potential recipients
• Recommendation To make the grant management
  criteria and process more transparent

15
Host Arrangements

• International Land Coalition
• Evaluation question The extent to which the
  relationship between ILC and IFAD, as host and
  international focal point, is mutually
  beneficial?
• Methodology Desk review, interviews, survey of
  partners
• Findings
• There are gaps and conflicting clauses in the
  legal and administrative agreements regulating
  IFAD-ILC relations
• Dominant role of IFAD reduces the independence of
  ILC and the incentives of other partners to
  participate effectively in the program
• Host arrangements contribute to ILC efficiency in
  the short term, but hinder its financial
  sustainability in the long term.
• Recommendation Coalition Council should prepare
  a strategy for transition from an IFAD-hosted
  institution to an independent legal entity with
  international status.

16
Conclusions

• Most evaluations assess some aspects of GM,
  such as accountability, but few assess all
  aspects
• Using a consistent approach that focuses on
  compliance with generally excepted principles of
  public sector governance facilitates comparison
  across programs
• Evaluators are developing innovative ways to
  apply this approach and find out how governance
  is actually being practiced in individual GRPPs