The Cases of Chile and Colombia

1
The Cases of Chile and Colombia
Presentation to a LAC Region Summer Seminar 9 A
ugust 2006 Keith Mackay Independent Evaluation
Group (kmckay_at_worldbank.org)
2
What Does Success Look Like?-- Why Countries
Want an ME System

• To support budget decision-making
  performance-based budgeting
  
• To support national and sectoral planning
• To design policies and programs
• To assist sector ministries / agencies in their
  management
  
• To strengthen accountability relationships

3
Chiles ME System -- Architecture

• Designed, managed and used by Hacienda
• Developed incrementally, over past decade
• Performance indicators (?1,600) for all
  government programs (1994)
  
• Government program evaluations (? 160) --
  these are desk reviews (1996)
  
• Rigorous impact evaluations (?14) (2001)
• Comprehensive Spending Reviews -- desk reviews of
  all programs in a functional area (2002)

4
Chiles ME System -- Strengths (1)

• Graduated approach to ME
• Evaluations conducted externally, in fully
  transparent process, and are highly credible
  
• All ME findings reported publicly and sent to
  Congress
  
• ME system closely linked to the information
  needs of Hacienda, especially for budget process
  
• Performance information used to set performance
  targets for ministries -- these are largely met

5
Chiles ME System -- Strengths (2)

• High utilization of ME findings by Hacienda in
  the budget process and to impose management
  improvements on ministries / agencies (see Table)

6
Chiles ME System -- Challenges

• Unevenness in quality of evaluations -- due to
  cost and time constraints
  
• Chile probably not spending enough on
  evaluations
  
• Low utilization -- low ownership -- of
  Haciendas evaluations by sector ministries

7
Colombias ME System (SINERGIA) -- Architecture

• SINERGIA is managed by the Department of National
  Planning (DNP), with strong support from the
  Presidencia
  
• On-line sub-system -- SIGOB -- for monitoring and
  reporting government progress vis-à-vis
  Presidential Goals
  
• Ambitious agenda of impact evaluations (?15
  underway)
  
• DNP provides technical assistance to a few
  ministries/agencies to develop ME, and to
  municipalities to pilot SIGOB and
  performance-based budgeting

8
Colombias ME System -- Strengths

• Very high utilization of SIGOB by President for
  oversight of ministers and ministries -- via
  performance targets -- and for accountability,
  i.e. social control
• Rigorous impact evaluations conducted externally,
  and have high credibility
  
• Collaborative approach between DNP and sector
  ministries/ agencies, and with municipalities
  
• Performance budget reports efforts to further
  strengthen performance budgeting
  
• Efforts to engage with civil society

9
Colombias ME System -- Challenges

• Too high reliance on donor funding for SINERGIA
  -- low level of government funding support
  
• Insufficient reliance on ME information to
  support national planning and budget
  decision-making -- this may now be changing
  
• Weak coordination of ME roles / functions within
  DNP and with central and sector ministries
  
• SIGOB data quality perceived as low

10
How to Define a Government ME System as Being
Good Practice

• Can be dangerous concept -- each countrys
  starting point and desired end-point are unique
  
• What a successful ME system is NOT
• complex set of laws, decrees, regulations
• performance indicators collected
• rigorous impact evaluations conducted
• rapid evaluations conducted
• these simply reflect the architecture of the
  system or are measures of ME effort

11
How to Define a Government ME System as Being
Good Practice

• Chile and Colombia have good-practice ME systems
  because
  
• the quality of their ME work is generally
  reliable, and
  
• in particular, the monitoring information and
  evaluation findings which they produce are used
  intensively
  
• High utilization reflects strong demand for ME,
  and can be viewed as a predictor of an ME
  systems sustainability

12
Lessons (1)

• Lessons from Chile and Colombia are consistent
  with other countries experience
  
• Key role of powerful champion of ME
• Opportunistic development of ME systems, via
  continuous review and modification non-linear
  development of the systems
  
• Centrally-driven, by capable ministry
• Incentives are key -- cultural change to
  strengthen demand, achieve high utilization
  
• Avoid competing systems Planning, Finance

13
Lessons (2)

• Build reliable ministry data systems
• Role of structural arrangements to ensure ME
  objectivity and quality
  
• Long-haul effort, requiring patience
• Limitations of relying on laws, decrees,
  regulations
  
• An ME system can be built and operated are
  relatively low cost
  
• Chiles ME system costs 0.75m p.a.
• Colombias system costs about 2m p.a.

14
Useful Resources (1)

• Chile www.dipres.cl/fr_control.html
• Colombia www.dnp.gov.co/paginas_detalle.aspx?idp
  266
  
• Ernesto May et al. (eds.), Towards the
  Institutionalization of Monitoring and Evaluation
  Systems in Latin America and the Caribbean, World
  Bank/IADB, 2006. http//web.worldbank.org/WBSITE/E
  XTERNAL/COUNTRIES/LACEXT/0,,contentMDK20893139pa
  gePK146736piPK146830theSitePK258554,00.html
  (Disponible en Español)
• Keith Mackay, Institutionalization of Monitoring
  and Evaluation Systems to Improve Public Sector
  Management, Independent Evaluation Group, World
  Bank, 2006. www.worldbank.org/ieg/ecd/institutiona
  lizing_me.html (Disponible en Español)

15
Useful Resources (2)

• Ariel Zaltsman, Experience with
  Institutionalizing ME Systems in Five Latin
  American Countries Argentina, Chile, Colombia,
  Costa Rica and Uruguay, Independent Evaluation
  Group, World Bank, 2006. www.worldbank.org/ieg/ecd
  /experience_five_la.html
• Fernando Rojas et al, Chile Análisis del
  Programa de Evaluación del Gasto Publico, World
  Bank, 2005. http//iris37.worldbank.org/domdoc/PRD
  /Other/PRDDContainer.nsf/WB_ViewAttachments?ReadFo
  rmID85256D2400766CC785257155005CB26B
• World Bank website on Building Government ME
  Systems www.worldbank.org/ieg/ecd/