Impact evaluation for: National Irrigation Rehabilitation Project

1
Impact evaluation for National Irrigation
Rehabilitation Project

• Case study 3
• NONIE meeting in Lisbon
• October 1, 2008

2
Section A Presentation of National Irrigation
Rehabilitation Project
3
Section A Presentation of National Irrigation
Rehabilitation Project

• Background and rationale
• Project concept and strategy
• Project Goal Purpose
• Components
• Selection of Scheme
• Project Implementation

4
National Irrigation Rehabilitation Project

• 1.1Background and Rationale
• Agriculture performance specially paddy
  production depended heavily on irrigation
• Many irrigation schemes were old and deteriorated
  facilities resulted in poor water management
  practices led to loss of paddy
• The irrigation facilities did not function
  properly due to poor attention on OM
• Need to encourage farmers to assume full
  responsibility for OM and thereby reduce the
  role of Public Sector Involvement in irrigation
  management

5
Project Goal

• Stabilize agricultural production and income and
  to raise the standard of living of farmers
  through rehabilitation and improved OM of
  existing irrigation schemes.

6
Project purpose

• Stabilized and increase agricultural production
  on rehabilitated schemes through sustained and
  improved water management with OM of selected
  existing of irrigation schemes.

7
National Irrigation Rehabilitation Project

• 1.2 The Project Concept and Strategy
• In this backdrop the project strategy was
  inline with the Government National Irrigation
  subsector strategy to improve the performance of
  existing irrigation capacity and promote
  increased participation of farmers in the
  management and maintenance of schemes.
• Rehabilitation
• OM FOs
• Instititional Strengthening

8
Project components

• The proposed 7 year project (1991 to 1997)
  includes
• Rehabilitation and improvement of 1000 minor and
  60 medium/major schemes covering about 37 500 ha
• (b) Creating viable/healthy Farmer Organizations
  (FOs)
• for managing the rehabilitated schemes and
  introduction of
• improved OM practices in all
  rehabilitation schemes
• (c) Upgrading the skills of farmers and staff of
  the implementing
• agencies

9
Program Theory and results-chain
Network of Networks on Impact Evaluation
10
Project Management

• Executing Agency - Min. Irrigation
• Implementing Agencies - ID, ASD, PC
• PMU PD , Consultants
• National PCC Sec.Min Irrigation, Donor, IAs
• Ptovincial PCC Chief Sec

Network of Networks on Impact Evaluation
11
Selection of Schemes

• The schemes for rehabilitation were selected
  after reaching prior agreement with the
  respective FOs to undertake responsibility for
  the full cost of OM of minor schemes and
  Distribution and field cannels of major/ medium
  schemes after rehabilitation.
• Minor Schemes 750 per ha
• Participatory Planning, Management and Post Impl.

12
Project Implementation

• Project reached physical rehabilitation targets
  civil works of 1048 minor schemes and 34
  major/medium schemes (completed in June 1999)
• Established scheme levels FOs in all minor
  schemes and in all major/medium schemes.
• Handed over 64 of completed schemes (ie 2/3) to
  registered FOs and 32 of DF cannels (ie 1/3)
  major/medium scheme to FOs

13
However the organizational and financial
viability of FOs remain uncertain

• Handing-over process in many schemes is supply
  driven
• (written agreement signed between IAs and
  FOs) and
• remains a risk
• Quality Assurance of Civil Works Project
  supported 14 Quality Control Labs. Equipments
  arrived after 80 work completed.

14
Findings of ICR

• FOs should have been established before
  rehabilitation
• Organisational and Ginancial Viability of FOs
  -Uncertain
• Handing-over process in many schemes is supply
  driven
• Unrealistic Assumtions Major Schemes Headworks
  and Main canals
• Project Implementation Delays
• Quality Assurance of Civil Works Project
  supported 14 Quality Control Labs. Equipments
  arrived after 80 work completed.
• Cost Criteria Proprities only attended
  Partially completed schemes

Network of Networks on Impact Evaluation
15
Section B Impact evaluation of National
Irrigation Rehabilitation Project
16
Section-B Impact Evaluation of National
Irrigation Rehabilitation Project

• Evaluation Framework and Context
• Evaluation Questionnaires
• Outcome Indicators
• Theory and results framework
• Evaluation Design
• Challengers and Risk
• Methodology
• Implementation
• Management
• Expected Results
• Expected Conclusions

17
Program Theory and results-chain
18
Impact Evaluation of National Irrigation
Rehabilitation Project

• Evaluation Questions
• 1.Relevence
• 1.1 Is the project consistent National
  Irrigation
• subsector strategies and policies of the
  government?
• 1.2 Has the project included the need and
  priorities of
• the farmers?
• 1.3 Is the project components and activities are
  balanced
• and relevant to the Goals and Purpose?
• 1.4 Are the selected schemes, target groups,
  target
• areas appropriate?

19
Evaluation Questions

• 2.Efficiency
• 2.1 Were the schemes rehabilitated successfully
  covering all civil works
• specially Headwork and Delivery system
  (Cannels)?
• 2.2 Is the cost incurred justify the degree of
  achievement of outputs and
• outcomes? (Comparison with similar
  projects)
• 2.3 Any alternative strategies that will achieve
  the same results?
• 2.4 What factors that constrain or contribute to
  the efficiency of the project
• implementation process?

20
Evaluation Questions

• 3.Effectiveness
• 3.1 Is the project strategy and sequence of
  activities appropriate?
• 3.2 Are the schemes functional?.
  
• State of the maintenance of irrigation
  structures
• 3.3 Are the FOs are ready to undertake OM?
• 3.4 Do the FOs have ability to generate funds to
  meet OM?
• 3.5 How is the performance of FOs?.
• 3.6 What are the hindering or contributing
  factors for project effectiveness?

21
Evaluation Questions

• 4. Impact
• 4.1 Has the overall goal being achieved?
• 4.2 Has the intervention institutionalized
  improved water management
• practices?
• 4.3 Was there an increase in the Yield, Cropping
  intensity and Paddy
• production in the scheme?
• 4.4 Did the intervention improve the income and
  living standards of farmers in
• the scheme?
• 4.5 Any impacts on the institutions?

22
Cont Evaluation Questions

• 4. Impact
• 4.6 Any external factors which influenced the
  impacts positively or
• negatively?
• 4.7 Any unintended effects and impacts of the
  project?
• 4.8 Did the project had any influence on the
  policy?

23
Evaluation Questions

• 5.Sustainability
• 5.1 Will the termination of the project affect
  the outcomes and impacts?
• 5.2 How far the schemes were successfully handed
  over to FOs for
• maintenance?
• 5.3 Have the FOs taken over OM responsibility
  of minor schemes and
• selected FD Cannels in major/medium
  schemes?
• 5.4 Ability of FOs to manage schemes, resolve
  water disputes, venture
• into the marketing and input suppliers
  etc. and whether FOs are
• economically and organizationally
  empowered through commercial
• activities?

24
Evaluation framework and context
25
Outcome indicators
26
Evaluation design

• Impact Evaluation
• Focus - Program outcomes and Impact
• - Program Effectiveness
• Method - Outcome/Impact Measurement
  (Results)
• - Before/After and with and without
  Analysis Framework (Treatment Group
  with Comparison Group/Counterfactual
  analysis)
• - Cost benefit Analysis -
  EIRR
• DAC/OECD - Relevance, Efficiency,
  Effectiveness, Impact, Sustainability
• Criteria
• Data - Secondary sources/Administration
  Records
• - Primary sources
• - Key informant interviews, Focus
  group discussions, Community
• interviews, Semi
  structured survey interviews Direct
• Observations.

27
Challenges and risks

• Other Services are assured e.g. Extension
  Services, Seed Paddy, Fertilizer, Credit
• FOs undertake contracts are financially
  benefitted
• No political interferences in selection of
  schemes
• Quality assurance system is in operation (civil
  works, soil esters etc.)
• Stable and improved prices for agricultural
  products
• No deterioration in catchment hydrology and
  rainfall pattern

28
Evaluation Methodology

• Highly successful
• Successful
• Partly Successful
• Unsuccessful
• Judgment of the
• Achievement
• Verification of
• achievement and results
• Question designed to
• addressed to criteria
• Relevance, efficiency, effectiveness,
  impact
• sustainability

29
Implementation

• Evaluability Assessment (Program theory,
  availability of data, intended use)
• Evaluation Plan
• Appointment of the Management Group and Reference
  Group
• Defining the TOR
• Contracting the Evaluation and selecting
  evaluators
• Desk Research
• Scoping Session
• Field Study
• Analysis and drafting the report
• Dissemination
• Management response

30
Management

• Reference Group
• Management Group
• National Evaluation Coordinator
• Peer Reviews

31
Expected results

• Insert text on expected evaluation results here

32
Expected conclusions

• Insert text on expected evaluation conclusions
  here