Prepared by:

1
Design of the Evaluation FrameworkGAVI Alliance
GAVI Fund Phase 1 (2000-2005)

• Prepared by
• Stein-Erik Kruse (HeSo), Emma Denton (HLSP) and
  Martine Donoghue (HSLP)

2
Design Phase Objectives

• To define the scope and approach of the
  evaluation
• To provide the Terms of Reference of the
  evaluation

3

4
Methods used to frame the evaluation scope

• Steps taken to identify and select the evaluation
  questions
• Step 1 Stakeholder Consultation
• 67 GAVI Fund GAVI Alliance stakeholders were
  consulted via individual semi-structured
  telephones interviews. A sub-set (n15)
  contributed by email questionnaire).
• 73 questions were identified from this
  consultation
• Step 2 Similar questions were grouped and
  aggregated into overarching questions 23
  questions emerged.
• Step 3 Question Selection 23 questions
  assessed in terms of importance for
  accountability, relevance to 2006-2010 GAVI
  Strategy and feasibility timeliness.
• 20 questions satisfied these criteria.

5
Number of Questions by Thematic Area (n20)
Structures Roles GAVI Alliance 5Q GAVI Fund
3Q
Strategic Objectives 3 Questions
Performance Impact 3 Questions
Activities GAVI Alliance 6 Q GAVI Fund 2 Q
6
Conducting the Evaluation Some Key Points (1 /4)

• It is recommended that a separate expert
  fiduciary and asset management appraisal of the
  GAVI Fund (i.e. to address Q 7 and Q 16) be
  included in the Phase 1 Evaluation.
• Some questions (e.g. Q1, Q6, Q14, Q17) are being
  fully or partially addressed by other
  evaluations. For example, the Governance
  Evaluation, ISS Evaluation, ADIP Evaluation etc.

Fiduciary Financial Management of the GAVI Fund
GAVI Alliance GAVI Fund Phase 1 Evaluation
ADIPs Evaluation
ISS Evaluation
Governance Evaluation
7
Conducting the Evaluation Some Key Points (2 /4)

• The 20 evaluation questions need to be examined
  using a range of methods
• Stakeholder consultation (e.g. Q4, Q9 )
• Quantitative data analysis at global and country
  level for
• programmatic and financial data
  sets (e.g. Q17, Q19, Q10)
• Comparative analysis via external benchmarking
  with
• appropriate comparators (e.g. Q6,
  Q4)
• Documentary review (e.g.Q2, Q12)
• Country Case Studies (e.g. Q3, Q5, Q8, Q18)
• Innovative and appropriate methodological
  approach to be a selection criteria for the
  winning proposal

8
Conducting the Evaluation Some Key Points (3/4)

• The need to survey critical voices
• The need to survey non uptakers in the
  evaluation
• defined as those 4 countries eligible for GAVI
  funds who have not received funds, vaccine
  manufacturers who have not engaged or responded
  to market signals, as well as potential donors
  who have not contributed to GAVI

9
Conducting the Evaluation Some Key Points (4/4)

• The team conducting the overall Phase 1
  evaluation will require a diverse and
  complimentary skill set with national/regional
  and international experience
• Organisational management and evaluation
• Vaccine and immunisation knowledge
• Health financing
• Fund raising
• Health systems
• Monitoring Evaluation
• Quantitative data analytical skills
• Governance

10
The Boards are requested to

• Approve the scope and methodology of the GAVI
  phase 1 evaluation including
• Draft Terms of Reference for the GAVI Fund GAVI
  Alliance Phase 1 Evaluation (2000-2005)
• Draft Terms of Reference for an Independent
  Fiduciary Financial Management Evaluation of
  GAVI Fund in Phase 1 (2000-2005)