Aliou Diagne

1
Ex-post Impact Assessment at the Africa Rice
Center

• Aliou Diagne
• Program Leader
• Policy and Impact Assesment
• Africa Rice Center (WARDA)
• SPIA Meeting
• Defining and Refining Good Practice in Ex-post
  Impact Assessment
• Alvorada Brasilia Hotel, Brasilia, Brazil
• November 10-11, 2008

2
Outline of presentation

• Objectives and Scope of epIA at WARDA
• Methodology
• Highlights of epIA results

3
Objectives and Scope of epIA at WARDA
4
Impact Assessment at WARDA Objectives and Scope

• To assess the potential and actual diffusion and
  adoption/usage of Rice research outputs in Africa
• To assess the potential and actual impact of rice
  research and policies in Africa
• To enhance capacity in impact assessment in Africa

5
Impact Assessment at WARDA Objectives and Scope

• Types of Rice Research outputs assessed
• Improved varieties
• NRM/ICM/IPM options,
• post-harvest technologies
• Learning tools
• Policy research

6
Impact Assessment at WARDA Objectives and Scope

• Impact indicators assessed
• Livelihoods and welfare of rice farmers in Africa
  with focus on the MDG indicators
• Yield, crop income,
• food security, nutrition
• Poverty, gender equality
• Health, boys girls schooling
• Biodiversity, etc
• National economies of African countries
• National food supply
• National income

7
Impact Assessment at WARDA Implementation

• Joint implementation through and with NARS
• Backstopping in the field and by e-mail
• Use of common methodologies
• Sharing of questionnaires and survey designs
• Sharing of analysis tools
• Training
• Methodology-learning workshops
• One-one training on the job
• Development of tutorials

8
Impact Assessment at WARDA Methodology
9
Impact Assessment at WARDA Methodology

• Household and plot level surveys data
• Secondary countrywide agricultural census data
• FAO data on rice areas and farm populations

10
Impact Assessment at WARDA Methodology

• Counterfactual Potential outcomes framework
• Outcomes with and without intervention/treatment
• Estimation of Impact parameters that have causal
  interpretations
• Average Treatment Effect (ATE)
• Local Average Treatment Effect (LATE)
• ATE estimation of potential and actual adoption
  rates
• ATE/LATE estimation of actual and potential
  impacts
• Disaggregation of estimates by socio-demographic
  groups
• Aggregation of estimates to national levels using
  population weights

11
Adoption outcomes as results of
intervention/treatment

• The Intervention
• exposure and access to the technology
• The two counterfactual states
• being exposed to the technology
• not being exposed to the technology
• The two counterfactual outcomes
• adoption outcome under exposure
• adoption outcome under non-exposure

12
Impact of adoption

• The Intervention
• exposure and access to the technology
• The two counterfactual states
• Adoption of output
• Non-adoption of output
• The two counterfactual outcomes
• Impact indicator outcome under adoption
• Impact indicator outcome under non-adoption

13
Population Impact parameters

• ATE Average Treatment Effect
• The mean impact in full population
• expected impact on a person randomly selected for
  treatment from population
• ATE1 Average Treatment Effect on the Treated
• The mean impact in the treated subpopulation
• The impact on a person randomly selected from the
  treated
• subpopulation
• ATE0 Average Treatment Effect on the Treated
• The mean impact in the untreated subpopulation
• The impact on a person randomly selected from the
  
• untreated subpopulation

14
Population impact parameters

• The Local Average Treatment Effect (LATE)
• The mean impact in the subpopulation of
  compliers
• The expected impact on a person who was induced
  to change her behavior by the treatment/interventi
  on

15
Individual-level Impact parameters Example of
over-estimation of impact of adoption
Y1
Realized impact ATE1
A progressist farmer who has adopted the
technology
Y0
Outcome
Difference of observed outcomes MD
Y1
Mean potential impact ATE
Selection bias
Y0
A non-progressist farmer who has not adopted
the technology
Potential impact ATE0
Time
t

• Realized impact Y1 Y0 under adoption (D1)
  ATE1
• Potential impact Y1 Y0 under non adoption
  (D0) ATE0
• Mean potential impact (Y1 Y0 )P(D1) (Y1
  Y0)P(D1) ATE
• Difference of observed outcomes Y1 Y0
  (Biased impact estimate) MD
• Selection bias Y0 Y0 gt 0 (Overestimation
  of impact)

16
Impact parameters with causal meaning under
different types of treatment/intervention

• Under randomized treatment/intervention
• MDATE1ATE0ATELATE
• Under non-randomized but exogenous
  treatment/intervention
• ATELATE
• ATE1
• ATE0
• Under non-randomized and endogenous
  treatment/intervention
• LATE

17
Estimation methods

• Conditional Independence-based methods
• Pure regression-based methods
• Non-Parametric
• Parametric (OLS/NLS)
• Propensity score-based methods
• Non/semi/fully parametric
• Matching methods
• Instrumental variables based methods
• Extended Instrumental variables based methods (1
  and 2)

18
Impact Assessment at WARDA Highlights from
Recent Studies
19
Nerica Diffusion and Adoption
20
Nerica Impact on rice yield and income
21
Nerica Impact on the MDG indicators

• Impact on poverty
• Consumption spending 0.30 adult equivalent
• Daily calories intake 36 kcal/adult
  equivalent/day
• Consumption expenditure deficit ratio -19
• (compared to the poverty line)
• Impact on child schooling
• 6 increase in school attendance rate
• About 20 increase per child in school
  expenditure
• Impact on child health
• 5 increase in the hospital attendance frequency
  when sick
• About 12 increase in health expenses per sick
  child

22
Adoption and Impact of the ASI thresher

• The potential population adoption rate (ATE) of
  the ASI thresher estimated at 86 in 2006
• The adoption of ASI has a positive impact on
  labour a net gain of 22 person-days per hectare
  for the subpopulation of potential ASI adopters

23
Main conclusions on NERICA and ASI adoption and
impact

• A low diffusion rate and lack of seed have so far
  limited the adoption of the NERICA
• NERICA adoption impact on rice yields is
  heterogeneous with some farmers not experiencing
  any yield increase when adopting NERICA
• Positive impact on income, child schooling,
  health, food consumption and poverty
• Nerica impact significantly higher for women than
  for men
• The ASI thresher reduces labour needed for
  threshing

24
Thank youMerci