The Governance Triad

1
Assessing the Distributional Impact of Social
Programs
Public Expenditure Analysis and Manage Core Co
urse
Presented by Dominique van de Walle DECRG
March 21-24, 2005
Presented to
2
The evaluation problem

• Who gains from the programs?
• Who uses public services? At what cost?
• Who benefits from subsidies?
• Who are the target groups?
• How should transfers be allocated?
• 2. How much do they gain?
• Is there more poverty with or without a policy?
• How much impact will programs have on poverty?

3
The evaluation problem

•  Impact is the difference between the outcome
  indicator with the program and that without it.
  
• However, we can never simultaneously observe
  someone in two different states of nature
  
• So, while a post-intervention indicator is
  observed, its value in the absence of the program
  the counterfactual is not.
  
• The essential problem in evaluation is one of
  missing data on the counterfactual of what would
  have happened in the absence of the intervention
  

4

• To measure impacts rigorously we need ex-post
  impact evaluation techniques
  
• Econometric generally need baseline or panel
  data
  
• Experimental require randomized assignment
• Often we must instead turn to other quick
  dirty approaches that examine spending
  "incidence
  
• The most commonly used is Benefit Incidence
  Analysis (BIA)

5
What is benefit incidence analysis?

• Step 1 rank individuals by welfare indicator
• Access to services
•   
• Step 2 identify usage/participation
• Utilization
•   
• Step 3 attribute "gain" or benefit identified by
  unit
  
• cost of providing service
• Incidence of spending

6
STEP 1 Access to infrastructure in rural Vietnam
( rural population with the infrastructure)
7
  Step 2 Participation in public works and a
means-tested credit subsidy in Maharashtra,
India
Consumption expenditure per person
8
Step 3 A typical example of a benefit
incidence analysis
9
Health spending in Kenya, 1992

100
80
Primary
Hospital
60
All health
Cumulative subsidy/income
Income
40
20
0
0
20
40
60
80
100
Cumulative population
10

• Advantages of traditional benefit incidence
  analysis
  
• Easy to do and to present (with caveats)
•   
• Disadvantages and limitations
•  
• Strong assumptions
• Do not explain incidence outcomes
• No specific policy implications

11
Traditional benefit incidence analysis may...

• 1. Wrongly assume that the cost of provision
  reflects the benefit to user
  
• 2. Be sensitive to method of ranking households
  in the original position
  
•   spatial prices, comprehensiveness of welfare
• indicator, demographics

12
The welfare measure matters for primary education
in Ghana

100
80
Per capita expenditure
60
Adult equivalent
Cumulative subsidy
expenditure
40
20
0
0
20
40
60
80
100
Cumulative population
13
How quintiles are defined matters for health in
Ghana

100
80
60
Household quintiles
Population quintiles
Cumulative subsidy
40
20
0
0
20
40
60
80
100
Cumulative population
14
Traditional benefit incidence analysis may...

• 3. Mispecify the counterfactual BIA ignores
  behavioral responses
  
•  
• ex figure of transfers in Yemen
• Conclusions about targeting incidence depend on
  how the counterfactual is defined
  

15
Distribution of public and private transfers in
Yemen, 1998, by deciles of per capita
expenditures excluding transfers (annual YR per
capita)
16
Distribution of public and private transfers in
Yemen, 1998, by deciles of per capita
expenditures including transfers (annual YR per
capita)  
17
Traditional benefit incidence analysis may...

• 4. Give an incomplete picture of welfare
  effects
  
• how did other dimensions of welfare (eg health
• literacy, nutrition) improve as a result of
  subsidies?
  
• 5. Be unable to assess some important public
  goods and services
  
• eg safe water, sanitation, vector control,
  physical infrastructure
  

18
Traditional benefit incidence analysis may...

• Ignore general equilibrium indirect effects on
  poor
  
• eg indirect benefits from tertiary education
• 7. Confound average and marginal incidence
• Distribution of gains 

19
7. Average versus marginal incidence

• Standard benefit incidence estimates the
  distribution of average incidence at one point in
  time
  
• This can be deceptive about how changes in public
  spending will be distributed
  
• Marginal incidence is an example of a behavioral
  incidence analysis where one measures the
  incidence of actual increases or cuts in program
  spending

20
How will gains from social program expansion be
distributed across groups?

• Non-poor often capture benefits of (even
  targeted) social programs
  
• information, incentive political problems
  make perfect targeting hard
  
• But, program capture by non-poor can differ
  according to how costs benefits of
  participation vary with program scale
  
• eg fees, opportunity costs of time, transport
  costs etc
  
•  

21
How will gains from social program expansion be
distributed across groups?

• Model 1 Early capture by non-poor net gains to
  non-poor are positive initially, but fall with
  program expansion
  
• Model 2 Late capture by non-poor cost initially
  too high for the non-poor, but net gains rise
  over time
  
• Average participation rates may be deceptive for
  inferring how gains losses from program
  expansion or contraction will be distributed.

22

• One way to identify marginal incidence is to
  compare incidence across geographic areas with
  different program sizes
  
•  Marginal odds of participation (MOP)
•  increment to group-specific participation rate
  with a change in overall participation
  
•  
• The income group specific MOP is estimated by
  regressing income group specific participation
  rate across regions on the average rate for the
  region.
• MOP shows incidence of a change in spending

23
Average and Marginal Odds of Primary School
enrollment, India 1993-94
Note odds of enrollment ratio of
quintile-specific enrollment rate to the mean
rate.
24
In conclusion

• Extreme care is needed when interpreting average
  incidence and traditional benefit incidence
  analysis
  
•   
• Beware of reform recommendations based solely on
  BIA and concentration curves as conventionally
  calculated.  
  

25
Q A