Impact of Micro Finance and other research questions

1
Impact of Micro Financeand other research
questions

• June 10, 2008

2
Most of the poor are in deep trouble due to
these loans. Poor people are committing suicides
because of peer pressure of the organizations for
repayment Andra Bhoomi, 9/4/06
Micro-credit has been changing people's lives
and revitalizing communities
UN, 2005, Year of micro-credit
3

• What is an impact evaluation?
• Why impact evaluations?
• Doing an impact evaluations is hard
• So what to do?
• Impact evaluation design
• Randomization in practice
• Other research issues

4
Impact evaluation is different from monitoring

• Impact evaluation and monitoring are different
• Monitoring
• To monitor the development of the program as a
  whole, and of its component projects, in relation
  to changes in the context and circumstances of
  their implementation
• To monitor the development of the program as a
  whole, and of its component projects, in regard
  to goals, timelines, and any unforeseen
  circumstances that may occur

Source google
5
Impact.. On what?
On community
On household

• Financial Services usage
• Consumption level
• Consumption smoothing capacity
• Income, Business scale and profitability
• Labor supply
• Intra Household decision Making
• Women empowerment
• Asset Ownership
• Children education

• Informal lenders
• Labor supply and demand
• Markets
• Women empowerment
• Schooling
• Prices

6
What do you think is the impact of microfinance
on..

• Could be positive if income generating activity
• Could also be zero

Income

• Could be positive if income generating activity
  and if women bargaining power increases
• Could be zero on level, but positive on smoothing

Consumption
Women empowerment

• Could be positive
• Could also be negative (violence may increase)

• Could be positive if women have more bargaining
  power or if previous inability to pay fee
• Could also be zero

Children education and health
7

• What is an impact evaluation?
• Why impact evaluations?
• Doing an impact evaluations is hard
• So what to do?
• Impact evaluation design
• Randomization in practice
• Other research issues

8
Why is it important to measure impact?

• Current scenario implicitly subsidizes
  microcredit, but it may not last without evidence
• So do benefits justify substantial costs?
• Even if clients repay and continue to come,
  microcredit may not be beneficial to them (they
  may not be rational when they make the decision
  to borrow)
• Micro Finance is subject to political influences
  and reactions are very emotional rigorous impact
  evaluations are essential
• Social investors should also be interested

9
Understanding impact

• It is not only important to understand final
  impact, but also to understand how this happened.
  
• This will help draw very useful lessons for
  improving programmes and designing new programmes
• Also, it is important tot understand on whom
  microfinance has the most impact
• For ex, does it have a positive impact on the
  poorest of the poor?
• Morduch lower impact on them
• Some claim they should be served by other
  services, some that they can benefit from
  microfinance
• How can programmes be designed that have an
  impact on them?

10

• What is an impact evaluation?
• Why impact evaluations?
• Doing an impact evaluations is hard
• So what to do?
• Impact evaluation design
• Randomization in practice
• Other research issues

11
Imagine..

• In 2000, Spandana clients have a monthly income
  of 1000Rs in average
• In 2001, micro-credit programme is introduced.
• In 2002, the monthly income of Spandana clients
  is 1500Rs. in average
• What do you conclude?

The before-after difference may be due to other
things, in addition to the programme effect
12
Now.. Look at these facts

• The top quarter Grameen borrowers (in terms of
  loan size) enjoys 15 higher consumption per
  capita than households in the bottom quarter.
• 62 of the school-age sons of Grameen Bank
  borrowers are enrolled in school versus 34 of
  the sons of eligible households that do not
  borrow.
• For daughters, the Grameen advantage is 55
  versus 40.
• What do you conclude?

Borrowers and non borrowers are different before
the Programme this is the SELECTION BIAS
13
A nutrition programme
B
A
Selected
D
C
Non Selected
14
A nutrition programme
Small, Selected
Tall, Non Selected
C
B
15
What is the difference between B and C?
B-C
Programme impact
Intrinsic difference
But we are not able to isolate one from the
other.
16
To show you this formally

• We want B-A (difference between same children
  with and without programme at the same time)
• We observe B-C (difference between small children
  with nutrition and tall children without
  nutrition)
• Add A-A to B-C
• B-CA-A B-AA-C

Selection bias
Programme impact
17
Why are treatment and control groups
intrinsically different?
A-C
Observable characteristics
Non Observable characteristics
Can control for
Can not control for
18
Crude estimations and well estimated results can
give very different results..

• In the Grameen case Once appropriate comparisons
  with control groups are made, access to the three
  microfinance programs does not yield meaningful
  increases in per capita consumption, the
  education of sons, nor the education of daughters
  (Morduch 1999)

19
Then, lets do one thing..

• Spandana went to a village, Balliguda
• There is another village nearby, Vizag, where
  they did not go.
• In Balliguda, the income of people is 1500 Rs in
  average
• In Vizag it is 2000Rs.
• What do you conclude?

Organizations do not place their programme in
villages at random. This is called NON RANDOM
PLACEMENT
20
How to evaluate impact of microfinance?

• Compare before and after
• Other things may be going on
• Compare borrowers to non-borrowers
• Selection bias
• Compare villages with microfinance and villages
  with microfinance
• Non random placement

21
What do we really want to know?

What would happen to the SAME person in two
different states of the world?
22
But in reality all we have is..

We need to find proper comparison groups that
puts us in a situation similar to the one before
23

• What is an impact evaluation?
• Why impact evaluations?
• Doing an impact evaluations is hard
• So what to do?
• Impact evaluation design
• Randomization in practice
• Other research issues

24
Coleman Old and New villages

• Coleman study

participants
participants
Non participants
Non participants
Old villages with credit
Newly identified villages
25
Coleman Outcomes of interest

• Targeting and participation
• Selection of members
• Borrowing by members
• Impact
• Physical assets
• Savings debt and lending
• Production expenses sales and labor time
• Health care and education

26
Coleman However

• Problems with the methodology?
• Villages where they have entered first may be
  different
• Future borrowers have the time to adjust their
  behavior (investment etc) since they know they
  will get a loan

27
PK Eligibility Rule

• Pitt and Khandker study
• Need an exogenous rule the restriction that
  households owning more than 0.5 acres of land are
  precluded from joining any of the three credit
  programs considered (BRAC, Grameen and BRDB)
• Use a village-level fixed-effects method to
  control differences between villages
• With access to group-based credit randomly
  allocated to one sex or another, and which
  controls for self-selection by these randomly
  chosen household members into the program.

28
PK Methodology
consumption
Land size
0.5 ha
29
PK Results

• Credit is a significant determinant of many of
  these outcomes.
• Credit provided to women was more likely to
  influence these behaviors than credit provided to
  men.
• Credit provided women significantly affects all 6
  of the behaviors studied.
• Credit provided men does so in only 1 of 6 cases.
• Annual household consumption expenditure, the
  most comprehensive measure available of program
  impact, increased 18 taka for every 100
  additional taka borrowed by women from these
  credit program, compared with 11 taka for men.

30
Morduch Can we believe these results?

• The program rule can be the basis of a plausible
  econometric strategy if the eligibility
  requirement is strictly enforced
• But, Frequent violations of the rules.
• For example, 30 of Grameen borrowers own more
  land than the half-acre cut-off, with
  landholdings as large as fourteen acres.
• overall, 20-30 of borrowers are over the
  mandated land cut-off.
• The mistargeting also creates problems when
  comparing differences across villages.
• Unlike in program villages, the eligible in
  control villages were chosen by the survey team
  strictly on the basis of having total land
  holdings below 0.5 acre.

31
Morduch Problems in eligibility rule
32
Morduch Can we believe these results?

• Loans to males are larger on average
• The difference between impact on men and on women
  can also be explained by the standard theory of
  declining marginal returns to capital.

Returns
Loan size
33
Morduch compare all eligible and non eligible in
treatment and control villages
34
Morduch Difference in difference
35
Morduch Difference in difference

• We observe in treatment villages that average
  income of elligible is 80, and average income of
  non eligible is 100
• Can we conclude that programme had a negative
  effect?
• No, because difference must have been higher
  before the programme
• We need to know what was this difference
• For that we can use difference between eligible
  and non eligible in control villages
• Even in control villages are better off in
  average, if difference between eligible and non
  eligible is similar, we are fine

36
Morduch Difference in difference
37
Morduch is there an issue as well?

• His difference-in-difference approach does not
  deal with the key issues about program placement.
• Remember the original assumption

38
Morduch Difference and difference

• We observe in treatment villages that average
  income of elligible is 80, and average income of
  non eligible is 100
• Can we conclude that programme had a negative
  effect?
• No, because difference must have been higher
  before the programme
• We need to know what was this difference
• For that we can use difference between eligible
  and non eligible in control villages
• Even in control villages are better off in
  average, if difference between eligible and non
  eligible is similar, we are fine

39
Morduch is there an issue as well?

• His difference-in-difference approach does not
  deal with the key issues about program placement.
• Remember the original assumption
• If Grameen Bank focuses on areas where the
  inequality between the rich and the poor is the
  greatest, how will Morduchs estimate will be
  biased?

40
Morduch Difference in difference
40
50
41
Difference in difference
If effect of time is same for treated and non
treated, it means that the difference between non
treated and treated would be the same before and
after (without programme)
Income
b
a-b
a
Control
Treatment
Time
Programme
Evaluation
42
Difference in difference

• Problem is that it may not be true that treatment
  and control would have evolved the same way.
• Since they are intrinsically different, they may
  have different patterns over time. Income of
  control may have been growing faster, or slower,
  than income of the treatment.
• For example, entrepreneurial people may grow
  their income faster anyway.
• We can not verify this assumption..

43
Difference in difference
Income
b
a-b
a
c
Control
c-b
Treatment
Time
Programme
Evaluation
44
So what to do?

• We could compare new borrowers to old borrowers
• Takes care of selection effect and of non random
  placement
• AIMS tool
• Assessing the Impact of Microenterprise Services,
  USAID

45
But..

• Are old and new clients the same?
• Old clients may have been more entrepreneurial,
  and new clients are the ones who imitate
• MFI may have targeted poor people first and then
  everybody in the village (reverse may be true
  too)
• Dean Karlan

46
Lessons

• When the researcher comes ex post (the programme
  has been implemented already), it is very
  difficult to find a situation where borrowers and
  non borrowers are difference only because they
  benefit from the programme or not
• There is always another difference going on
• The researcher needs to decide ex ante who will
  receive the programme or not, in order to make
  sure that the only difference is the programme

47
This is why we randomize..

• So we want to have A-C0, or AC so that
• B-CB-A
• I,e, we want to compare two groups that are
  identical before the programme is introduced so
  that what we observe after programme is equal to
  programme impact.
• If your sample is large enough and you divide
  your group randomly into two groups before the
  programme, the two groups will be identical on
  observables AND non observables

48
Remember

• We observe B-C (difference between small children
  with nutrition and tall children without
  nutrition)
• B-CA-A B-AA-C

Programme impact
Selection bias

• Since Treatment has been randomly assigned, A-C
  0

49

• What is an impact evaluation?
• Why impact evaluations?
• Doing an impact evaluations is hard
• So what to do?
• Impact evaluation design
• Randomization in practice
• Other research issues

50
Background

• Spandana an MFI in Andra Pradesh, rural and
  urban
• 800,000 clients
• Offers group loans (10 members groups, weekly
  repayments), individual loans with daily
  repayment, consumption loans.
• Expanded micro-credit in Hyderabad recently
• First loan Rs 7000, one year cycle
• Early repayment is possible and additional Rs
  2,000 is available after 6 months.

51
Research objectives

• Assess the impact of Access to Credit on
• Consumption
• Business scale and profitability
• Labour supply
• Income
• Intra Household decision Making
• Asset Ownership
• Financial Services usage

52
Decision steps

• Unit of randomization
• How do we randomize?
• Sample size

53
Unit of randomization

• We can randomize at different levels
• Individual level
• Group level (school, class, mf group etc.)
• Village or slum level
• This depends on programme, presence of
  externalities etc.
• What should we do in Spandana case?
• Has to be at slum level, because of feasibility,
  and externalities

54
How do we randomize in practice?

• We can randomize
• By phasing in the programme
• During the pilot phase
• By encouragement design
• By lottery when funds are limited
• In what situation are we with Spandana?
• Take advantage of the fact that Spandana was
  starting the programme in Hyderabad

55
What sample size do we need?

• Remember we said that sample size needs to be
  large enough.
• Look at the excel sheet the larger the sample,
  the closer is your control and treatment
• How large is large enough?
• With your sample size the objective is to
  minimize the probability of doing Type I and Type
  II errors
• Type I Find an effect when in fact there is none
  (confidence level proba of Type I error)
• Type II Not find an effect when there is one
  (power 1 minus proba of type II error)
• You want a sample which is large enough that you
  will see the impact its your glasses

56
Ingredients for a power calculation in a simple
study
57

• What is an impact evaluation?
• Why impact evaluations?
• Doing an impact evaluations is hard
• So what to do?
• Impact evaluation design
• Randomization in practice
• Other research issues

58
What problems can we face on the field?

• And what can we do about them?

59
Take-up of credit (or compliance)

• The take-up will not be 100
• In the follow up survey we can over-sample the
  people who are the most likely to borrow
• Spandana Rule at least 2 renters in a group of
  10
• According to a sample survey, owners are more
  likely to get loans
• We can also use this to measure externalities
• Even though, take-up will not be 100

60
What to do about no full compliance

• Denote Z (0,1) the initial randomization, and T
  (0,1) the treatment status.
• Z is randomly assigned, T is not unless TZ.
• Comparison of outcomes for Z1, Z0 will give
  Intention to treat, which may or may not be the
  policy parameter of interest.
• You can use the randomization like an instrument,
  and just like IV, you get the treatment effect on
  the compliers.
• Simplest case is Wald estimate (difference in
  average effect divided by difference in proba of
  being treated)
• EYZ1- EYZ0/(ETZ1-
  ETZ0)

61
Invasion of the control slums

• Control slums invaded by Spandana
• Control people going to treatment slums to get
  credit
• It is important to keep the initial assignment as
  the treatment, as this is the only assignment
  that was randomly assigned
• As seen before the assignment to treatment can be
  used as instrument
• So if take-up is 50 in treatment, 5 in control,
  and average difference in expenditure per hh is
  1000Rs, your wald estimator will be 1000/0.45

62
Invasion of the control slums

• The problem is that if there is leakage across
  the groups, the treatment-control difference in
  impact will be small, but the difference in the
  probability of being assigned is also small.
• What happens to your effect on treated?
• What does it do to sample size required?

63
Other issues

• Attrition
• Non random attrition is a problem
• Need to check whether attrition is different in
  T/C
• Might be difficult when experiment is over a long
  period of time
• Externalities randomization design is key
• Behavioral response to experiment etc.
• We need to be aware of these things if they
  happen

64
Things to remember on the field

• Make sure nobody gets treated in control group
• Keep the original assignment intact
• Keep track of compliance while intervention is
  going on who gets treated, who does not etc.
• Be very clear from beginning where is control and
  where is treatment
• Spandana example
• If large risk of attrition, find a way to trace
  as many people as possible (need to know who goes
  away, and possibly find them)
• Bandhan example
• In any case, keep track of attrition
• The follow-up survey is the most important, make
  sure you put lots of ressource and energy so data
  collected is of high quality
• Not only outcome variables are important, but
  also intermediary variables

65
Lessons

• Implementation of the randomized experiment is
  key
• One has to be on the field the entire period to
• Coordinate with the partner organization and
  explain the research design continuously
• Guard the control slums!!
• Do mid-term surveys (ex take-up surveys)

66

• What is an impact evaluation?
• Why impact evaluations?
• Doing an impact evaluations is hard
• So what to do?
• Impact evaluation design
• Randomization in practice
• Other research issues

67
What works and what does not?

• Research so far has mostly focused on
• Documenting the sector evolution and delivery
  models
• Impact and women empowerment issues
• Case studies of successful organizations
• There is a lot of theoretical literature that
  shows how joint liability solves adverse
  selection problems
• However there is little experimental research
  that aims to rigorously design ways to improve
  existing services, for a higher impact
• Randomization can also be used for this purpose

68
Maximizing impact programme and product design

• Programmes and products could be improved so that
  their impact on the organization, and on the
  clients, is maximized
• Organizations often experiment new products.
  However if they expand it to the entire
  organization, it is risky for both the
  organization and the beneficiaries
• It seems a win-win situation to evaluate the
  differential impact of the new product through a
  rigorous evaluation before launching it full
  scale.
• Instead of comparing micro-credit to no
  micro-credit, we can compare one product to
  another variation of this product

69
Maximizing impact non financial interventions

• Impact can also be maximized by non financial
  interventions
• Business training or financial literacy training
• Health interventions bed nets
• Health education freedom from Hunger
• It is important to evaluate the additional impact
  of these interventions
• Especially because they cost money so is the
  investment worthwhile?

70
Basic data sometimes lacking

• Impact evaluations and experiments on product
  design are long-term or medium-term studies
• But often, policy makers lack even more basic
  information that could inform policy decisions,
  on a variety of basic indicators and descriptive
  data
• Number and characteristics of existing and
  potential microfinance clientele in the country
• For example, how poor are SHG and JLG clients,
  how do they compare to each other? How do they
  use their loans? Who is NOT enrolled and why?
• Short, descriptive surveys and/or focus group
  discussions can add value to the policy-making
  process, and allow us to rapidly assess
  situations and react to unexpected
  exogenous events that affect the entire sector.

71
Data quality

• Collecting data of quality requires lots of
  efforts and monitoring
• Market research companies are not used to the
  kind of household data that researchers want to
  collect
• Whether one outsources data collection or builds
  own team, it requires a lot of monitoring
• Issue costs while it seems the first best to
  collect your own data, it is also more expensive

72
Some Pressing issues for microfinance programmes
and policies
Policy
Programme design

• Impact
• Competition
• Effects and implications (for loan size, debt
  capacity)
• Interest rates
• Demand elasticity
• Transaction costs
• Clients business returns
• Transparency/clients understanding
• Savings
• Is NBFCs collecting deposits risky? Alternative
  channels
• Impact of savings
• MFIs financing - mission drift risk?

• Reach the unreached
• Take-up issues delivery channel difference?
• demand
• Repayment schedules
• Frequency
• Flexibility
• Drop-outs
• Risk mitigation bundling
• Production vs consumer credit
• Does it matter?
• Fin. and business training
• Individual lending/male clients

73

• Dean Karlan Microfinance Impact Assessments The
  Perils of Using New Members as a Control Group
  (December 2001) Journal of Microfinance.
• http//research.yale.edu/karlan/downloads/impactpe
  rils.pdf
• Pitt and Khandker The Impact of Group-Based
  Credit Programs on Poor Households in Bangladesh
  Does the Gender of Participants Matter? (1997)
• http//www.pstc.brown.edu/mp/jour14.pdf
• Jonathan Morduch "Does Microfinance Really Help
  the Poor? New Evidence on Flagship Programs in
  Bangladesh" (PDF), June 1998. Presented at
  Stanford, UC-Berkeley, University of Washington,
  RAND, University of Toronto, Princeton, and Yale.
• http//www.nyu.edu/projects/morduch/documents/micr
  ofinance/Does_Microfinance_Really_Help.pdf
• Brett Coleman The impact of group lending in
  Northeast Thailand, Journal of Development
  Economics 60 105-42.
• http//dx.doi.org/10.1016/S0304-3878(99)00038-3