Test of New Master

1
A Randomized Evaluation of a School Scholarship
Program in Rural India

• Karthik Muralidharan
• (with Michael Kremer, Venkatesh Sundararaman)
• Conference on Public Private Partnerships in
  Education
• World Bank, Washington DC
• 7-8 June, 2007

2
Outline

• Background
• Research Questions Experimental Design
• Questions for Discussion/Feedback

3
Private Schools in Rural India

• Private fee-charging schools are widespread in
  rural India
• Especially in areas with poorly performing public
  schools
• 28 of villages had private schools in 2003
  (probably much higher now)
• 18 of school enrollment in rural India is in
  private schools (2006)
• Measures of private school performance are
  superior to public schools especially with
  village fixed effects
• Teacher attendance, teaching activity, student
  attendance, test scores
• Children attending private schools come from
  advantaged backgrounds relative to those
  attending public schools
• Difficult to infer causal effect of private
  schools on performance
• Omitted variables and also additional years of
  schooling
• High levels of elite exit from public schooling
• Several important questions need to be answered
  to assess the suitability of a voucher/scholarship
  /PPP model for improving school education
  outcomes in India

4
Andhra Pradesh Randomized Evaluation Study

• 3-way partnership between Government of Andhra
  Pradesh, Azim Premji Foundation, and the World
  Bank
• 5 year MoU with GoAP to systematically study the
  most promising policy options to improve
  developing country education via randomized
  allocation of programs
• 4 major interventions have been implemented and
  evaluated so far (Muralidharan Sundararaman
  2006)
• Group and Individual-level performance pay for
  teachers
• Use of contract teachers as opposed to
  civil-service teachers
• Cash block grants to schools
• Starting a long-term study on the impact of
  school choice/scholarships/vouchers for
  disadvantaged children to attend private schools
  under AP RESt

5
Existing Research on School Vouchers

• Several studies consider the impact of randomly
  offering students a chance to attend
  private/charter schools
• Rouse in Milwaukee
• Peterson et. al in New York City, Washington DC,
  and Dayton, Ohio
• Angrist et. al in Colombia
• Cullen et. al in Chicago
• Hoxby Rockoff in Chicago
• These report varying extent of benefits to
  voucher recipients
• No one finds voucher recipients doing worse off
• But more evidence is needed on the overall impact
  of school vouchers on all the students in a
  system
• Hoxby (2003), Lavy (2006), Hsieh and Urquiola
  (forthcoming)

6
Decomposing Peer Effects

• Randomly allocated voucher programs typically
  create 4 distinct categories of students
• In public school, dont apply for voucher
• Apply for voucher, but dont win (and typically
  go to public school, but sometimes go to private
  school anyway)
• Apply for voucher, and win (and likely to go to
  private school)
• In private school, independent of the voucher
  program
• Best studies typically compare the 2nd and 3rd
  groups
• ITT and ToT measures
• But we dont know if the first and fourth groups
  are worse off as a result of peer effects
• The public school potentially loses its most
  motivated students
• The private school gets students below its
  existing average
• Ideally, we would want to randomize entire
  communities into voucher programs and compare the
  result for all students with the results of all
  students in similar control communities without
  vouchers

7
Proposed Research Design

• An Indian village is pretty close to a closed
  economy in terms of school choice especially
  for primary schooling
• Identify 200 villages that all have an existing
  private school
• Two stage randomization
• Randomly select half the villages to receive
  school vouchers
• Randomly select children in voucher villages to
  receive them
• Track learning outcomes of all children in all
  schools in both treatment and control villages
• Child-level comparison gives the impact on
  participants
• Village-level comparison gives the overall effect
  of the voucher program
• Phase 1 (Pilot) with 32 villages starting now,
  with a planned expansion to 200 villages next
  year

8
Scholarship Program Design (1 of 2)

• The universe of children eligible for the
  scholarship will be those currently in government
  schools (grades 1-3)
• Provides a sample of disadvantaged children
  without having to do means testing
• The number of scholarships to be offered in a
  village will be capped at 30 of the enrollment
  of all the government schools
• Dont want to empty out the government school
• Parents will be required to apply for the
  scholarship to be eligible (should provide higher
  first stage but we will find out)
• Once a child receives a scholarship, he/she will
  continue to receive one till the end of primary
  school subject to meeting attendance requirements
  and taking the end of year tests
• Differential exposure to program across cohorts
  of recipients

9
Scholarship Program Design (2 of 2)

• No topping up scholarship amount will be set at
  around the 80th percentile of the private school
  fee distribution across all villages
• Private schools can determine the number of such
  places at this scholarship rate (but must accept
  all students who are allocated to these places by
  a lottery limits cream skimming)
• All expenses for books, uniforms, and school
  supplies are being covered by the scholarship.
• A transport subsidy may be provided in some
  cases, but may not be required if the choice is
  being exercised within the village.
• The total scholarship spending per child (all
  inclusive) is expected to be around Rs.
  3,200/child per year (USD 80/year).
• This is significantly less than the spending per
  child in the government schooling system
• At least Rs. 4,000/year counting only pure
  variable costs
• Over Rs. 5,000/year including various overhead
  costs

10
Resource Equalization

• Even if the scholarship villages do better it
  could be a reflection of additional resources in
  these villages
• Resource equalization is a problem for many other
  studies as well
• Possible solutions
• Make public school lose money for every student
  who leaves
• Not easy since the main expense is very lumpy
  (teacher salaries)
• Also politically much more difficult
• Provide matching resources to the public schools
  in control villages
• Doubles scholarship cost of program
• Operationally more difficult because it becomes
  another treatment
• Use estimates from other studies to net out the
  effect of additional resources in public schools
• Most likely course of action especially since
  this in Andhra Pradesh
• Existing studies in AP will provide estimates of
  the returns to the 2 main categories of inputs to
  public schools additional teachers, and cash
  grants with school-level flexibility on spending
  (no infrastructure though)

11
Questions We Are Getting At

• Do private schools perform better even after
  accounting for the unobserved variables that
  might determine private school enrollment?
• Answer by comparing scholarship winners to losers
  in voucher villages
• Can think of the question as whether marginal
  spending on education is best routed via the
  private sector
• What is the aggregate impact of the program (and
  is any group worse off)?
• Answer by comparing voucher villages to
  non-voucher villages
• Unit of analysis is the grade-level average score
• What is the functional form of peer effects?
• How do parents exercise choice?

12
Questions We Are Not (Yet) Getting At

• Effects of competition on productivity of public
  schools?
• Not at this point negative incentives for
  public schools are quite difficult practically
  (lumpy inputs), politically, and potentially
  ethically
• Positive incentives for retention could be
  considered (your thoughts?)
• Benefits of better matching?
• Unlikely to be the focus at the primary level
• Adequacy and nature of supply response?
• Not at this point
• Scholarships will only be redeemable at schools
  existing prior to study
• Combining choice with the effects of information
  to parents on school performance?
• Not in Phase 1, but could be an
  orthogonal/additional treatment

13
Issues for Discussion (1 of 2)

• Take up
• Aspiration gap, uncertainty of funding/unanticipat
  ed expenses
• Larger distances to travel
• Have required an expression of interest in
  applying (want a good first stage), and the
  number appears high (90)
• Medium of Instruction/SR Adjustments
• Most private schools are English Medium
• Need to allow enough time for SR adjustments to
  be made
• What grades to target?
• Had initially intended grades 1-4 (moving to 2-5)
• Now only looking at KG, 1, 2 (moving to 1, 2, 3)
• Scholarship amount?
• Dont want top ups, 80th percentile makes it
  attractive for most schools and covers marginal
  costs for almost all schools

14
Questions for Discussion (2 of 2)

• Transport - to provide or not?
• More competition versus greater complexity
• Very hard to provide a standard option for
  transport.
• Best option may be to provide a subsidy to
  parents
• Piloting both options
• Thoughts on ways to provide incentives for the
  public school to retain children without explicit
  penalties
• What expectations should the public school
  teachers be operating under?
• How much and what kind of information should we
  provide to parents?
• Thoughts on verification/payment/fraud prevention?