An Overview of Empirical Methods for Assessing the Impact of Policies on the Distribution of Income

1
An Overview of Empirical Methods for Assessing
the Impact of Policies on the Distribution of
Income

• Francisco H.G. Ferreira
• World Bank
• Summer School on Inequality, Growth and Human
  Development
• Civita Castellana, July 2007

2
Plan of Lecture

• Understanding changes in the distribution of
  incomes
• A unified framework around the GIC.
• statistical counterfactual decompositions
• The impact of policies on the distribution
• Assigned programs
• Ex-post impact evaluation and heterogeneous
  impact
• Ex-ante evaluation towards partial equilibrium
  economic decompositions?
• Economy-wide policies
• General equilibrium economic decompositions?

3
1. Understanding changes in the distribution of
incomes
Growth (in the mean), poverty dynamics and
inequality dynamics as different aggregations of
the information contained in growth incidence
curves (GICs).
Growth in Thailand, 1975-1992, seen as rightward
shifts in the Cumulative Distribution Function.
Source Ahuja et al. 1997
4
Growth in mean incomes

• Growth in mean incomes is simply a weighted
  average of income growth along the distribution,
  with weights given by relative incomes.
• This can be written in terms of the growth
  incidence curve (GIC)
• So growth (in the mean) is simply a particular
  aggregation of the percentile-specific growth
  rates in the GIC.

The Growth Incidence Curve was first formally
described by Ravallion and Chen, 2003. The
version in discrete time is
Which is, of course, just the proportional change
in the Pen parade F-1(p), at every p.
5
Changes in Poverty and Inequality Drawing (in
part) on Kraay (2003)
Write a general poverty measure formulation as
gives you the FGT class, for instance, and
where
gives you the Watts index.
Differentiating with respect to time yields
and
with
(holding z constant.)
6
Changes in Inequality

• Like poverty measures, many inequality indices
  can be written as functions of a sum of
  individual distance indicators

for example, gives the GE class.
gives the Atkinson class.
Differentiating relative measures with respect to
time yields
and
with
So poverty and inequality changes are also
transformations of the information in the GIC.
7
So economic growth, changes in poverty and
changes in inequality are effectively different
ways of weighing the income changes along the
distribution which are presented in a growth
incidence curve.
?P0 2pp
?P0 -9pp
8
Statistical counterfactual decompositions.

• To seek an understanding of changes in the
  distribution of incomes is to seek an
  understanding of why the GIC looks the way it
  does.
• To understand the nature and determinants of the
  incidence or distribution of economic growth.
• The first step is statistical

Counterfactual income distribution
Counterfactual GIC
Residual
9
Statistical counterfactual decompositions(continu
ed)

• Of course, this is just another way of describing
  generalized Oaxaca-Blinder decompositions such as
• Where the counterfactual distribution is
  constructed from
• By simulating a change in either the conditional
  distribution of y on X, or on the joint
  distribution of X.
• For example

10
Statistical counterfactual decompositions(continu
ed)

• There are a number of ways to implement such
  simulations in practice.
• They may be based simply on reweighing the
  sample, so as to reproduce the changes in the
  distribution of some exogenous characteristic,
  such as the age composition of the labor force,
  or the number of people receiving the minimum
  wage.
• DiNardo, Fortin and Lemieux (1996)
• Hyslop and Maré (2005)
• They may be based in importing parameters from
  models estimated in one year to the other.
• Bourguignon, Ferreira and Lustig (2004)

11
The origins of statistical counterfactual
decompositionsa. The Oaxaca-Blinder Decomposition

• These approaches draw on the standard
  Oaxaca-Blinder Decompositions (Oaxaca, 1973
  Blinder, 1973)
• Let there be two groups denoted by r w, b.
• Then and
• So that
• Or
• Caveats (i) means only (ii) path-dependence
  (iii) statistical decomposition not suitable for
  GE interpretation.

returns component
characteristics component
12
Modern applications parametric method for wage
distributions.b. Juhn, Murphy and Pierce (1993)
Juhn, Murphy and Pierce (1993)
where
Define
Then
Returns component
Unobserved charac. component
Observed charac. Component.
13
Modern applications non-parametric method for
wage distributions.c. DiNardo, Fortin and
Lemieux (1996)
Essentially, DFL propose estimating a
counterfactual income distribution such as
By appropriately reweighing the sample, as
follows.
where
A variant of this approach is applied to HPCY
distributions by Hyslop and Maré.
14
Modern applications parametric and
semi-parametric mixed methods for HPCY
distributions.d. Bourguignon, Ferreira and
Lustig (2005)

• Depart from
• Note that this can be written

15
Let kA gA, hA and kB gB, hB be ordered
sets of conditional distributions. Define a
counterfactual (ordered) set of conditional
distributions ks, the dimension of which is ?1,
(like kA and kB) whose elements are drawn either
from kA or kB.
Define a counterfactual distribution fsA?B(y ks,
?A)    
For example, the counterfactual distribution
fsA?B(y gA, h1B, h-1A, ?A) is given by  
  For each counterfactual distribution fs, the
difference between fA and fB can be decomposed as
follows  
16
The next step is to estimate those conditional
distributions. We do so through a set of
parametric models, built around three
blocks (1) Earnings and self-employment
equations
(2) Occupational structure equations
If e has a Weibull distribution, the probability
of individual i choosing occupation s is given by
which is estimated through a standard multinomial
logit model.
17
(3) Conditional distributions of education and
family size. Education MLE (E?A, R, r, g, nah
?)   Fertility MLC ( nch ?E, A, R, r, g, nah
?)   Other Incomes T ( y0h ?E, A, R, r, g, nah
?) Household incomes are then aggregated as
follows
18
In practice

• After one estimates those models for both t and
  t, various counterfactual distributions are
  constructed by
• Importing the relevant set of parameter estimates
  from t to t (or vice-versa).
• Importing the (actual or simulated) residual
  terms for each individual.
• Predicting the counterfactual income levels (and
  occupations or educations or family structures,
  as needed) for each individual.
• Computing the desired counterfactual statistics,
  such as inequality or poverty measures, for the
  resulting counterfactual distribution.
• Graphing changes in the distribution for each
  step of the decomposition.

19
An Example The Brazilian Slippery Slope,
1976-1996
20
(No Transcript)
21
(No Transcript)
22
The Brazilian Slippery Slope Price, Occupation
and Endowment Effects(Disaggregated into
Education and Fertility)
Source Ferreira and Paes de Barros (1999)
23
2. Impact of Policies on the Distribution

• Generalized Oaxaca-Blinder decompositions such as
  those discussed above, whether parametric or
  semi-parametric, suffer from two shortcomings
• Path-dependence
• The counterfactuals do not correspond to an
  economic equilibrium. There is no guarantee that
  those counterfactual incomes would be sustained
  after agents were allowed to respond and the
  economy reached a new equilibrium.
• I.e. no causal inference is possible. Assessing
  the impact of a particular policy requires moving
  towards an economic decomposition.

24
Impact of Policies on the Distributiona. Assigne
d programs

• Are policies / programs targeted to particular
  groups or individuals. Some people receive them,
  others do not.
• The theoretical impact of policy P (implemented
  between t0 and t1) on individual i is given by
• Impact evaluation techniques are a set of
  techniques to estimate a counterfactual value for

Missing data problem unobservable counterfactual
25
Impact of Policies on the Distributiona. Assigne
d programs

• If treatment assignment is exogenous and
  independent of any other factors correlated with
  y, then
• I.e. A single difference estimator yields an
  unbiased estimate of impact if a program is
  randomly assigned to a treatment group, with data
  available on a control group.
• If take-up rates are less than 100, then the
  above is an ITT. TT and ATE require multiplying
  ITT by take-up rates.

26
Impact of Policies on the Distributiona. Assigne
d programs

• Two problems
• What if treatment is not randomly assigned?
• If all individual characteristics which correlate
  with treatment assignment are observed, the
  problem can be fully resolved by controlling for
  those observables in the estimates of conditional
  impact
• Which can be achieved by linear regression
  models, or by more flexible methods, such as
  propensity score matching.
• But these do rely on the assumption that
  selection is only on observables.
• Alternative approaches, like RDD, exploit
  continuity assumptions about unobservables to
  attenuate this problem.

27
Impact of Policies on the Distributiona. Assigne
d programs

• Two problems
• All of the above methods estimate average impacts
  (conditional or unconditional). What about the
  distribution of impacts along the distribution of
  incomes?
• This is one particular case of impact
  heterogeneity, a subject of growing interest in
  the field.

28
2. Impact of Policies on the Distributiona. Assig
ned programs

• I will focus, in this lecture, on the ex-ante
  evaluations approach a comparison of a
  (counterfactual) prediction of the outcomes of a
  program, with the (actual) absence of the
  program.
• Compare how an ex-post evaluation approximates
  the counterfactual...
• With how an ex-ante evaluation does it

29
What are ex-ante evaluations for?

• For simulating assigned programs before they are
  implemented. (Should be combined with a credible
  ex-post evaluation design from the outset.)
• For simulating alternative designs (and costs)
  for an existing program, before reforming it.
  (Should be combined with a credible ex-post
  evaluation design from the outset.)
• For evaluating an economy-wide policy, for
  which no credible control or comparison group is
  available. E.g. trade liberalization, change in
  the exchange rate change in the tax system.
• Simulation requires a model
• Simple arithmetic simulations
• Behavioral (partial equilibrium) simulations
• General equilibrium models
• Models may also be structural or reduced-form,
  provided reduced form does not change under the
  new policy and one is not interested in the deep
  structural parameters.

30
Ex-Ante Evaluation of an Assigned
ProgramImplementation A Five-Step Process

• Step 1 Identify a well-defined, tractable policy
  reform question.
• Step 2 Write the simplest economic model able to
  capture the factors that are likely to determine
  the agents response to the policy reform.
• Step 3 Find a data set that contains reliable
  information on the variables that need to be
  included in the model.
• Step 4 Estimate the model on the data set.
• Step 5 Simulate the policy reform using the
  empirical estimate of the model.

31
An Example from Brazil(Bourguignon, Ferreira and
Leite, WBER, 2003.1. The policy reform
question

• How would the introduction of a conditional cash
  transfer perform with respect to its twin stated
  objectives the reduction of current and future
  poverty?
• Are the school enrollment incentives built into
  CCTs effective? (Do households change their
  behavior in response to the program?)
• What is the impact of the program on current
  poverty and/or inequality?

32
The policy reform question The Bolsa
Escola Program

• Means-test income per capita less than R90
  (50 of the 1999 minimum wage)
• Conditionality 6-15 year-olds must
• Be enrolled in school.
• Attend at least 85 of classes.
• Transfer R15 per child in school
• Limit R45 per household
• Monitoring at the local and federal levels
• Introduced in July 2001. No ex-post evaluations
  by 2003.

33
Ex-Ante Evaluation of an Assigned Program2. The
Model

• For simplicity, we make four key simplifying
  assumptions
• Gloss over debate on who makes the childs
  occupational decision.
• Adult behaviour unaffected by child-level
  variables
• Sibling interactions ignored
• Household composition exogenous.

34
Ex-Ante Evaluation of an Assigned Program2. The
Model

• Childs occupational choice
• (0) Not going to school (paid or unpaid work)
• (1) Going to school and paid work
• (2) Going to school and no paid work
• Ui(0) Zi.?0 ?0.(Y-i yj0) vi0
• Ui(1) Zi.?1 ?1.(Y-i yj1) vi1
• Ui(2) Zi.?2 ?2.(Y-i yj2) vi2

35
Ex-Ante Evaluation of an Assigned Program2. The
Model

• Child is Contribution to Household Income in
  state j 0, 1 or 2
• yi0 wi yi1 Mwi yi2 Dwi
• with wi market earnings
• Log wi Xi .? mI(Sj1) ui
• and M Exp (m)

36
Ex-Ante Evaluation of an Assigned Program2. The
Model

• Household (kid) i chooses the alternative j that
  yields the highest utility Ui(j)
• Ui(0) Zi.?0 ?0.Y-i ?0.wi vi0
• Ui(1) Zi.?1 ?1.Y-i ?1.wi vi1
• Ui(2) Zi.?2 ?2.Y-i ?2.wi vi2
• with ?0 ?0 ?1 ?1M ?2 ?2 D

37
Ex-Ante Evaluation of an Assigned Program3. The
Data

• Pesquisa Nacional por Amostra de Domicílios
  (PNAD, 1999).
• Approx. 60,000 households.
• Representative of country, except for rural areas
  of Acre, Amazonas, Pará, Rondônia and Roraima.
• Labor status questions asked if age gt 10.
• Enrolment (but no attendance) questions.
• Reasons to suspect income variables in rural
  areas (see FLN, 2003) but staple hh survey in
  Brazil.
• Bolsa Escola Program not in effect!

38
Ex-Ante Evaluation of an Assigned Program4.
Estimation (by Age)

• If vij are i.i.d. and have a double exponential
  distribution, then this discrete choice model can
  be estimated as a multinomial logit
• Earnings estimation

Log wi Xi .? mInd(Sj1) ui
39
Ex-Ante Evaluation of an Assigned Program4.
Estimation (by Age)

• Usual estimation of ML generates estimates of
  differences ?j-?0, aj-a0, ßj-ß0 only (j 1,2).
• Since income variables are asymmetric across
  occupational alternatives, this is insufficient.
  (Need all three as).
• But

40
Ex-Ante Evaluation of an Assigned
ProgramDescriptive Statistics and Estimation
Results

41

42

43

44
Ex-Ante Evaluation of an Assigned Program5.
Simulation

• Introducing the State-ConditionalTransfer

45
Ex-Ante Evaluation of an Assigned Program5.
Simulation

• 40 currently not enrolled would have the
  incentive to change status and enroll
• Impact on children currently working is smaller
• Impacts stronger for the poor (means test)

46

• No conditionality vs. Bolsa Escola
  conditionality is key
• Impact quite sensitive to changes in T, the
  transfer amount
• Impact less sensitive to changes in the means
  test Y0

47
(No Transcript)
48
Conclusions from Ex-Ante Evaluation

• Bolsa Escola was rather effective in inducing
  additional enrollment (60 of poor kids out of
  school enroll in simulation)
• Role of conditionality is key substitution
  rather than income effect
• Likely to INCREASE number of children studying
  and working. (School duration?)
• The programs impact on current poverty and
  inequality is modest, due largely to low transfer
  amounts.
• Dynamics See Attanasio, Meghir, Santiago (2005),
  IFS, and Todd, Wolpin (AER, 2006) for dynamic
  simulations using Progresa in Mexico.
• Use the randomized experiment to validate dynamic
  structural models and simulate alternative
  designs

49
2. Impact of Policies on the Distributionb. Econo
my-wide policies

• Another Brazilian example (!)
• Ferreira, Leite and Wai-Poi (2007) Trade
  Liberalization, Employment Flows and Wage
  Inequality in Brazil (WB PRWP4108)
• This paper asks whether trade liberalization
  contributed to the decline in wage inequality
  and, if so, how.
• Extends Goldberg Pavnick (2005) two-stage
  estimation of effects of changes in trade
  variables on industry and wage-premia to
  employment model and combines it with JMP
  (1993)-style simulations

50
1. Motivation

• Brazils trade liberalization episode took place
  between 1988 and 1995, when tariff and non-tariff
  barriers were considerably reduced.

51
1. Motivation

• During the same period, wage inequality declined.
  Is there a causal link?

52
2. Methodology

• Estimation
• First Stage
• Second Stage

53
2. Methodology

• Decomposition of changes in the wage distribution
  
• (using the two-stage trade effects framework)

54
5. Results (Wage Decomposition)
55
5. Results (Wage Decomposition)
56
5. Results (Wage Decomposition)
57
5. Results (Wage Decomposition)
58
5. Results (Wage Decomposition)
59
5. Results (Wage Decomposition)
60
5. Results (Wage Decomposition)
61
5. Results (Wage Decomposition)
62
2. Impact of Policies on the Distributionb. Econo
my-wide policies

• This paper adopted a reduced-form approach to
  the general equilibrium processes through which
  price changes (in tradable goods markets)
  affected other prices, wages, output and
  employment levels in the economy. An alternative
  is to model these general equilibrium processes
  explicitly.
• There are two basic approaches to generating
  GE-compatible counterfactual income distributions
  (and thus counterfactual GICs)
• Fully disaggregated CGE models, where each
  household is individually linked to the
  production and consumption modules. E.g. Chen and
  Ravallion, 2003, for China.
• Leaner macroeconomic models linked to
  microsimulation modules on a household survey
  dataset. E.g. Bourguignon, Robilliard and
  Robinson, 2005, for Indonesia.

63
Impact of Policies on the Distributionb. Economy
-wide policies
Distributional Impact of Chinas accession to the
WTO. (Chen Ravallion, 2003) GE-compatible
counterfactual GICs corresponding to a specific
policy.
64
2. Impact of Policies on the Distributionb. Econo
my-wide policies

• In the Macro-Micro approach, some key
  counterfactual linkage variables are generated in
  a leaner macro model, whose parameters may have
  been calibrated or estimated from a time-series,
  and then fed into sector-specific equations
  estimated in the household survey, to generate a
  counterfactual GIC.

Macro model
Linkage AggregatedVariables (prices, wages,
employment levels)
Household income micro-simulation model
65
The LAV structure(Wages One for Urban one for
Rural)
Note In rural areas, intermediate and high skill
groups were pooled.
Occupations Urban x Rural household heads,
spouses, others.
66
Micro-simulations

• Solution of system of 21 equations

(
)
g
g
"



w
p
e
b
a

å
å
g
x
Exp
...
ih
g
ih
gs
Î
Î
g
i
g
s
67
Simulation

• Solve the system of 21 equations changing all
  constant (?0 and ?) terms.
• Calibrated so that micro-simulation reproduces
  changes in aggregate structure of employment
  obtained in macro-economic framework.
• Newton-Rapshon algorithm.
• Minimize the sum of squared differences between
  the left- and the right-hand side of equations.

68
Results Changes in Unemployment Levels by Skill
Note In rural areas, intermediate and high skill
groups were pooled.
69
Results Changes in Nominal Wages by Skill
Note In rural areas, intermediate and high skill
groups were pooled.
70
Results Household Incomes (I)
71
Results Household Incomes (II)
72
Results Household Incomes (III)
73
Conclusions

• Growth, changes in poverty and changes in
  inequality are all summary measures of changes in
  the disaggregated distribution of incomes.
• Understanding these changes requires
  understanding the determinants of changes in the
  growth incidence curve.
• Counterfactual simulations that isolate the
  individual impacts of changes in prices, in
  occupational structure, in the distribution of
  household endowments, or in transfers, are a
  useful first step.
• Counterfactual GICs that are consistent with
  (partial or general) economic equilibria are more
  difficult to estimate, as they involve modeling
  behavior. But starts have been made.
• Beware of Lucas critique and the black-box
  critique.