Internship Programme in UVER

1
Internship Programme in UVER
State Planning Organization DG Regional
Development and Structural Adjustment Department
of Monitoring and Evaluation
5-16 November 2007 Zeynep ERSAYIN Volkan Idris
SARI
2
Content

• Definition of monitoring
• Overview of major studies in Monitoring and
  Evaluation Department of SPO
• Internship Subjects
• Perspectives

3
Definition of monitoring (parallel approach)

• Monitoring can be defined as collection, analysis
  and use of information for the purpose of
  management and decision-making systematically.
  Monitoring tracks the actual performance or
  situation against what was planned or expected
  according to pre-determined targets (CMM, 2006)
• Continuous function that uses the systematic
  collection of data on specified indicators to
  provide management and stakeholders of an ongoing
  intervention with indicators of the extent of
  progress and achievement of objectives and
  progress in the use of allocated funds (OECD,
  2002)

4
Regional Programme Implementations and RDAs
Izmir
Çukurova
5
Overview of major studies in ME Dep. of SPO

• Ongoing studies
• Design and coordination of the monitoring system
  for EU Regional Development Programmes (RDPs)
• Establishment of monitoring and coordination of
  the monitoring system for RDAs
• Design of the monitoring system for Public
  Investments (small-scale, local)
• Contribution to design of the monitoring system
  for National Development Plan
  (large-scale, national)

• Future studies
• Effectiveness evaluation at project level
• Common monitoring system programmes designed,
  implemented and monitored at regional and local
  level.
• Review of the project fiche to incorporate future
  plan
• Forecasting system and database integration

6
Internship Subjects

• Data Collection
• Data Integration
• Monitoring and Information System
• Project Fiche
• Data Anaysis
• Project Monitoring and Reporting
• Verification of Public Investments
• Data Interpretation
• Benchmarking and Ranking
• Effectiveness Evaluation

7
Data Collection
Italian case Turkish case Opportunities
Data collected from four main different DBs (APQ, Monit, AVCP, Public tenders) Importance of the data quality New DB for the Public Investment Projects No effective monitoring system for public investments For EU programmes, monitoring system based on declaration and validation SPO is extending this system for public interventions Italian experience can be the good example for the development of Turkish system Standardized data collection and systematic data analysis
8
Data Integration
Italian case Turkish case Opportunities
Integration of data collected at project level and tender database Relevant data collected from different actors for further analysis Statistical tools and definition of unique key field Textual analysis of project titles ME Department intends to integrate data at central, regional and local levels Tender DB is maintained by Public Procurement Institution No integration yet Integration of the different DBs increases the quality, variety and reliability of the analysis
9
Monitoring Information System
Italian case Turkish case Opportunities
EU Funds APQ Effectiveness Evaluation Completamenti Integration of systems Not functional for public investments yet Common monitoring system for EU RDPs MIS for KÖYDES (Village infrastructure) System design for monitoring of Public Investment System for Project proposal launched System for provincial projects designed System design for public investments ongoing Importance of the integrated MIS
10
Project Fiche
Sec. Italian case Turkish case Differences
1 Project Details Project Details New fields can be added
2 Time scale Planning stages, other approval stages, other activities, Implementation stages - Will be incorporated
3 Economic plan Expenditures Future perspective will be incorporated
4 Financial plan Financial plan Availability of data on sources of funds
5 Accounting data Liabilities, Payments, Savings - Will be considered
6 Targets Indicators Similar
7 Other information
8 Problems and needs
11
Data Analysis
Italian case Turkish case Opportunities
Several types of analysis made by UVER Promotion of different aspects of projects and programmes at different levels Territorial analysis and spatial interactions Short-run this kind of studies can be applied to EU RDPs within the limits of available data and procedures Long-run Can be applied in Turkey only after developing a sound data collection and monitoring system for public investments Integrating different DBs and using other source of data enable the monitoring and evaluation unit to understand past behaviors of the system to produce analyses for identifying problems and needs in current situation to promote possible future interventions
12
Project Monitoring and Reporting
Italian case Turkish case Opportunities
Continuous data collection with project fiche (core data of the system) Low quality problem in early stages of the system DPS more focused on data collection and procedures Monitoring has to have financial outputs Project reporting without IS, on paper for public investment New system is on progress Lessons learnt in EU Programmes (KÖYDES, Provincial investments) Similar data quality problems Discussions on monitoring of the Annual Investment Plan Review of the project fiche During discussion ME Department is designing data collection and monitoring mechanisms An asset for the design process
13
Verification of Public Investments
Italian case Turkish case Opportunities
Two-level control in the verification of the public investments Every single programme under EU funds has MA, CA and AA. UVER is an AA for some of the National Operational Programmes Different management structure of the EU funds in Turkey For grants first level of control - PIUs and CFCU Overall control Treasury Auditors Ex-ante approval ECD For other tenders verification and auditing procedures are similar to Italian case For the Instrument for Pre-Accession 2007-13, verification procedures will be very similar with Member states
14
Data Interpretation
Italian case Turkish case Opportunities
Forecasting system Early-warning system Data interpretations are limited to current data collections Not allowing forecasting For EU programmes, because of the nature of the system there are no need for forecasting Short-run forecasting for the DAs projects
15
Benchmarking and Ranking
Italian case Turkish case Opportunities
Producing inputs to decision makers for the evaluation of the public administration According to developments in methodology, the analysis will be carried out periodically within the limits of data sets Rating of the administrations Three group of areas for the ranking critical one is institutional contest New concept was developed with strategic planning perspective for the evaluation of the public bodies There is no specific analysis on the project implementation performance of the administrations Partnership between different institutions during data collection
16
Effectiveness Evaluation
Italian case Turkish case Opportunities
Based on outcomes and results of the projects by Eff. Area of UVER Integrated information system for monitoring and evaluation Positive and negative assessment Problems in performance indicators and baseline analysis Differences between EU and national evaluation perspective No ex-post evaluation for the completed projects under investment plans Final evaluation of the EU RDPs will be carried out Similar problems in baseline analysis and selection of performance indicators Project- and programme-level approaches to the evaluation Performance indicators can be shared Future studies can be focused on this specific issue
17
Perspectives

• Presentation of project work in Final
  Conference
• Dissemination of internship experiences within
  SPO
• Setup of a contact list of participants for
  exchange
• Definition of a project fiche for future
  collaboration between SPO and DPS

18
GRAZIE PER TUTTO...ARRIVEDERCI

• Zeynep ERSAYIN ztokoglu_at_dpt.gov.tr
• Volkan Idris SARI isari_at_dpt.gov.tr