Good Practice Government

1
Good Practice Government Systems for ME
The Cases of Chile and Colombia Presentation to
a LAC Region Summer Seminar 9 August 2006 Keith
Mackay Independent Evaluation Group (kmckay_at_world
bank.org)
2
What Does Success Look Like?-- Why Countries
Want an ME System

• To support budget decision-making
  performance-based budgeting
• To support national and sectoral planning
• To design policies and programs
• To assist sector ministries / agencies in their
  management
• To strengthen accountability relationships

3
Chiles ME System -- Architecture

• Designed, managed and used by Hacienda
• Developed incrementally, over past decade
• Performance indicators (?1,600) for all
  government programs (1994)
• Government program evaluations (? 160) --
  these are desk reviews (1996)
• Rigorous impact evaluations (?14) (2001)
• Comprehensive Spending Reviews -- desk reviews of
  all programs in a functional area (2002)

4
Chiles ME System -- Strengths (1)

• Graduated approach to ME
• Evaluations conducted externally, in fully
  transparent process, and are highly credible
• All ME findings reported publicly and sent to
  Congress
• ME system closely linked to the information
  needs of Hacienda, especially for budget process
• Performance information used to set performance
  targets for ministries -- these are largely met

5
Chiles ME System -- Strengths (2)

• High utilization of ME findings by Hacienda in
  the budget process and to impose management
  improvements on ministries / agencies (see Table)

Utilization of government evaluations -- 2000 to 2004 Utilization of government evaluations -- 2000 to 2004 Utilization of government evaluations -- 2000 to 2004 Utilization of government evaluations -- 2000 to 2004 Utilization of government evaluations -- 2000 to 2004 Utilization of government evaluations -- 2000 to 2004
Minor adjustment, modification, relocation of program Change in program design or manage-ment process Institutional relocation of program Substantial redesign of program or of internal management processes Program termination TOTAL
25 39 5 21 10 100
6
Chiles ME System -- Challenges

• Unevenness in quality of evaluations -- due to
  cost and time constraints
• Chile probably not spending enough on evaluations
• Low utilization -- low ownership -- of
  Haciendas evaluations by sector ministries

7
Colombias ME System (SINERGIA) -- Architecture

• SINERGIA is managed by the Department of National
  Planning (DNP), with strong support from the
  Presidencia
• On-line sub-system -- SIGOB -- for monitoring and
  reporting government progress vis-à-vis
  Presidential Goals
• Ambitious agenda of impact evaluations (?15
  underway)
• DNP provides technical assistance to a few
  ministries/agencies to develop ME, and to
  municipalities to pilot SIGOB and
  performance-based budgeting

8
Colombias ME System -- Strengths

• Very high utilization of SIGOB by President for
  oversight of ministers and ministries -- via
  performance targets -- and for accountability,
  i.e. social control
• Rigorous impact evaluations conducted externally,
  and have high credibility
• Collaborative approach between DNP and sector
  ministries/ agencies, and with municipalities
• Performance budget reports efforts to further
  strengthen performance budgeting
• Efforts to engage with civil society

9
Colombias ME System -- Challenges

• Too high reliance on donor funding for SINERGIA
  -- low level of government funding support
• Insufficient reliance on ME information to
  support national planning and budget
  decision-making -- this may now be changing
• Weak coordination of ME roles / functions within
  DNP and with central and sector ministries
• SIGOB data quality perceived as low

10
How to Define a Government ME System as Being
Good Practice

• Can be dangerous concept -- each countrys
  starting point and desired end-point are unique
• What a successful ME system is NOT
• complex set of laws, decrees, regulations
• performance indicators collected
• rigorous impact evaluations conducted
• rapid evaluations conducted
• these simply reflect the architecture of the
  system or are measures of ME effort

11
How to Define a Government ME System as Being
Good Practice

• Chile and Colombia have good-practice ME systems
  because
• the quality of their ME work is generally
  reliable, and
• in particular, the monitoring information and
  evaluation findings which they produce are used
  intensively
• High utilization reflects strong demand for ME,
  and can be viewed as a predictor of an ME
  systems sustainability

12
Lessons (1)

• Lessons from Chile and Colombia are consistent
  with other countries experience
• Key role of powerful champion of ME
• Opportunistic development of ME systems, via
  continuous review and modification non-linear
  development of the systems
• Centrally-driven, by capable ministry
• Incentives are key -- cultural change to
  strengthen demand, achieve high utilization
• Avoid competing systems Planning, Finance

13
Lessons (2)

• Build reliable ministry data systems
• Role of structural arrangements to ensure ME
  objectivity and quality
• Long-haul effort, requiring patience
• Limitations of relying on laws, decrees,
  regulations
• An ME system can be built and operated are
  relatively low cost
• Chiles ME system costs 0.75m p.a.
• Colombias system costs about 2m p.a.

14
Useful Resources (1)

• Chile www.dipres.cl/fr_control.html
• Colombia www.dnp.gov.co/paginas_detalle.aspx?idp
  266
• Ernesto May et al. (eds.), Towards the
  Institutionalization of Monitoring and Evaluation
  Systems in Latin America and the Caribbean, World
  Bank/IADB, 2006. http//web.worldbank.org/WBSITE/E
  XTERNAL/COUNTRIES/LACEXT/0,,contentMDK20893139pa
  gePK146736piPK146830theSitePK258554,00.html
  (Disponible en Español)
• Keith Mackay, Institutionalization of Monitoring
  and Evaluation Systems to Improve Public Sector
  Management, Independent Evaluation Group, World
  Bank, 2006. www.worldbank.org/ieg/ecd/institutiona
  lizing_me.html (Disponible en Español)

15
Useful Resources (2)

• Ariel Zaltsman, Experience with
  Institutionalizing ME Systems in Five Latin
  American Countries Argentina, Chile, Colombia,
  Costa Rica and Uruguay, Independent Evaluation
  Group, World Bank, 2006. www.worldbank.org/ieg/ecd
  /experience_five_la.html
• Fernando Rojas et al, Chile Análisis del
  Programa de Evaluación del Gasto Publico, World
  Bank, 2005. http//iris37.worldbank.org/domdoc/PRD
  /Other/PRDDContainer.nsf/WB_ViewAttachments?ReadFo
  rmID85256D2400766CC785257155005CB26B
• World Bank website on Building Government ME
  Systems www.worldbank.org/ieg/ecd/