Impact evaluation Applying Analytical Methods: Microfinance Case Study

1
Impact evaluation Applying Analytical Methods
Microfinance Case Study

• Evaluating the Impact of Projects and Programs
• Beijing, China, April 10-14, 2006
• Shahid Khandker, WBI

2
Micro-finance Impact Evaluation An example

• Micro-finance refers to financial and
  non-financial services (often small size) for
  low-income people
• Micro-finance is a response to market failure
• Micro-finance relies on social mechanisms to
  enforce contract and to reduce the impacts of
  capital market imperfection and asymmetric
  information
• Micro-finance usually depends on grants and soft
  money for institutional development.

3
What does micro-finance provide?

• Financial services 
• Loans
• Savings
• Insurance
• Remittance transfer
• Non-financial services 
• Awareness and conscious building
• Employment and skill development training
• Extension and infrastructure development
• Legal and regulatory services

4
How does micro-finance provide services?

•   Collateral-free lending
• Small loan size
• Small amount of savings
• Group-or individual-based lending/savings
• Peer monitoring
• Targeted lending
• Market interest rate policy
• Social intermediation

5
Why is micro-finance needed?

• Micro-finance helps expand income and strengthen
  
• financial sector.
• Micro-finance supports poverty alleviation
  effort.
• Micro-finance increases business opportunities
• through financial innovations that
  otherwise would
• not be adopted.
• Micro-finance helps decentralize economic power
  
• and strengthen community groups.
• Micro-finance empowers women and other socially
  excluded groups

6
Expected benefits of micro-finance

• Short-term benefits
• Consumption and its smoothing
• Employment
• Income
• Poverty and vulnerability
• Long-term benefits
• Household asset and net worth
• Human capital
• Social capital
• Community assets

7
Policy Issues Regarding Micro-finance

• Micro-finance borrowing has a high recovery rate
  and the potential for reducing poverty,
  particularly among women
• But these programs are dependent on subsidy which
  might have alternate uses
• Policy issues focus on the efficiency and
  cost-effectiveness of micro-finance programs to
  justify their subsidy dependence.

8
Notation

• C is credit demand
• Y is outcome variable (consumption, assets,
  employment, schooling, family planning)
• Subscripts household i village j male m
  female f time t
• Want to allow separate effects on outcome of male
  and female credit
• X represents individual or household
  characteristics.

9
Measuring Impacts Cross-Sectional data

• Credit demand equations
• Cijm Xijmßc Zijm?c µcjm ecijm
• Cijf Xijfßc Zijf?c µcjf ecijf
• Outcome equation
• Yij Xij ßy Cijm dm Cijf df µyj eyij
• Note the Z represent variables assumed to
• affect credit demand but to have no direct
• effect on the outcome.

10
Endogeneity Issues

• Correlation among µcjm, µcjf and µyj, and among
  ecijm, ecijf and eyij
• Estimation that ignores these correlations have
  endogeneity bias.
• Endogeneity arises from three sources
• 1) Non-random placement of credit programs
• 2) Unmeasured village attributes that affect
  both program credit demand and household
  outcomes
• 3) Unmeasured household attributes that affect
  both program credit demand and household
  outcomes.

11
Resolving endogeneity

• Village-level endogeneity resolved by village
  FE
• Household-level endogeneity resolved by
  instrumental variables (IV)
• In credit demand equation Zijm and Zijf represent
  instrumental variables
• Selecting Zij variables is difficult
• Possible solution identification using
  quasi-experimental survey design.

12
Quasi-experimental design

• Households are sampled from program and
  non-program villages
• Both eligible and non-eligible households are
  sampled from both types of villages
• Both participants and non-participants are
  sampled from eligible households.
• Identification conditions
• Exogenous land-holding
• Gender-based program design.

13
Quasi-experimental Design (contd.)

• Exogenous land-holding criteria
• - Only households owning up to 0.5 acre of land
  qualify for program participation. In practice
  there is some deviation from this cutoff.
• Gender-based program participation criteria
• - Male members of qualifying households cannot
  participate in program if village does not have a
  male program group.
• - Female members of qualifying households
  cannot participate in program if village does not
  have a female program group.

14
Quasi-experimental Survey DesignConstruction of
Zij variables

• Male choice1 if household has up to 0.5 acre
• of land and village has male credit group
• 0 otherwise
• Female choice1 if household has up to 0.5
• acre of land and village has female credit group
• 0 otherwise
• Male and female choice variables are interacted
• with household characteristics to form Zij
• Variables.

15
What this means..

• Intuitively, the outcome regression now
• relates variation in Y to variation in C
• associated with variation in Z.

16
Alternative Models for Estimation

• (--) Ordinary Least Squares (OLS) estimation of
  outcome equation with no correction for weights
• (-) OLS on outcome equation with correction for
  sampling weights
• () IV (2-stage) estimation of outcome equation
  with weights correction
• () IV, weights, village fixed effects.

17
Comparing Results among Alternative Models
Log-log impacts of GB womens credit on HH per
capita consumption
18
Panel Data Analysis Rationale

• Results based on cross-sectional data may not be
  robust
• Impacts based on cross-sectional data may be
  short-term
• Cross-sectional data analysis cannot separate
  credit effects from non-credit effects
• Panel data analysis can assess whether credit has
  increasing or diminishing returns
• Panel data analysis can assess whether
  diseconomies or market saturation exists.

19
Concerns with Panel Data

• Household attrition
• Not a problem if it is random
• Efforts should be taken to minimize it.
•  
• Households split-off
• As many component households as possible should
  be traced
• Component households are logically combined.

20
Estimation with Panel Data Resolving
Endogeneity

• Household FE resolves both household- and
  village-level endogeneity.
• So IV (quasi-experimental) method is not required
  for resolving endogeneity.
• Outcome equation
• Yijt Xijt ßy Cijmt dm Cijft df µyj
  eyijt
• After differencing between two time-points
  
• ?Yij ?Xij ßy ?Cijmdm ?Cijftdf ?eyij

21
Estimation with Panel Data New Biases

• Household-level unmeasured biases that were
  assumed as time-invariant may actually change
  over time (that is we should use ?yjt instead of
  ?yj)
• If credit is measured with error, that error gets
  amplified when differenced between two time
  points
• This measurement error results in attenuation
  bias of the credit coefficients, meaning that
  impact estimates will be biased towards zero.

22
Estimation with Panel Data

• Re-introduction of instrumental variables (IV)
• Outcome equation
• ?Yij ?Xijßy ?Cijmdm ?Cijfdf ??yj ?eyij
• Credit demand equations after differencing
• ?Cijm ?Xijmßc (Zijm2?2 Zijm1?1) ?ecijm
• ?Cijf ?Xijfßc (Zijf2?2 Zijf1?1) ?ecijf
• Since credits are cumulative at two periods,
• ?C in above equations represents borrowing
• after first period.

23
Purpose of Instruments

• Same instruments as in cross-section analysis,
  but with a new purpose
• Before, they controlled for unobserved fixed
  household specific effects
• Now, they control for unobserved non-constant
  household specific effects, and measurement
  error
• Household panel structure is now taking care of
  the fixed effects.

24
FE-IV method over FE method

• Durban-Hausman-Wu test was conducted to see if
  the estimated coefficients of FE or FE-IV model
  are significantly different.
• Results suggest that changes in credit volume
  used in the FE method are somewhat determined by
  the time-varying heterogeneity or the measurement
  errors in credit variables. That means, the FE
  method may not be as reliable as one would
  expect.
• For comparison, both results are presented.

25
Comparing Results Among Alternative Models
Log-log impacts of womens current credit on HH
per capita consumption
26
Propensity Score method

• Run a program participation equation
• Predict propensity score for each household
• Match non-participant household with participant
  household
• Take the difference of outcomes of interest
  between participant and non-participant
• Weighted mean difference is the average program
  effect
• Use both cross-sectional and panel data analysis

27
Results of PSM with Cross-sectional Data Set
Average treatment effect of female borrowing

• 1991/92 1998/98 panel
• Per capita expenditure -140.9 455.8
  587.7
• Moderate poverty 2.30 -5.40 -7.46
• Extreme poverty 0.30 -6.70 -14.84
• Note refers to significance level of 10 or
  better results shown based on nearest neighbor
  matching.

28
Selected References

• Ravallion, Martin. The Mystery of Vanishing
  Benefits The World Bank Economic Review. Volume
  15. No. 1. pp. 115-140.
• Washington D.C. The World Bank, 2001.
• Khandker, Shahidur R. Micro-finance and Poverty
  Evidence Using Panel Data from Bangladesh The
  World Bank Economic Review, October, 2005, PP
  263-286.
• Khandker, Shahidur R. Fighting Poverty with
  Micro-credit Experience in Bangladesh New York,
  NY Oxford University Press, 1998.
• Pitt, Mark M. and Shahidur R. Khandker. The
  Impact of Group-Based Credit Programs on Poor
  Households in Bangladesh Does the Gender of
  Participants Matter? Journal of Political
  Economy 106 (October) 958-96, 1998.