Program Evaluation 3

1
Program Evaluation (3)
Feng ,Jin Fudan University
2
Matching
3
Problems

• We are interested Average treatment effect on
  treated
• E(Y1 X, D1) E(Y0X, D1) E( (Y1-Y0) X,
  D1)
• We observe
• E(Y1 X, D1) E(Y0X, D0)
• The bias
• BE(Y0 X, D1) E(Y0X, D0)

4
Matching Assumptions

• Need a treatment group and a non-treated group,
  from which you can select a control group
• The distribution of observable variables is as
  similar as possible
• Assume all the difference between groups can be
  captured by observable variables X
• Having high quality and extensive non-treatment
  group is crucial
• Common support the extent of overlap in the
  treatment and control groups in terms of
  observable variables

5
Matching on observables

• Having the advantage of making the comparisons
  transparent to non-specialists
• Matching is most practical where the causing
  variable takes two values, as in union status and
  training program examples
• For example, the random treatment conditional on
  a set of variables X, people with equal value of
  each X allocated to each other to a room

6
Matching estimators

• Matching is a weighted average of effects for
  each value of the causing variable
• Individual matching
• Block matching

Zi,D1 number of observations in the group
7
Decomposition of the bias

• sources of bias
• a. differences in supportdistribution of X for
  treated and untreated persons
• b. mismatching, or mis-weighting, of the data
• c. selection bias (selection on unobservable
  variables)

Define B (bias) to be the following
8
Bias B can be decomposed into the following
B1 is the bias due to non-overlapping
support different distribution of X for treated
(D1) and untreated (D0) S1\S10 indicates that
the integral is taken over the region of X that
is unique to treated S0\S10 indicates that the
integral is taken over the region of X that is
unique to untreated B2 is the bias due to
different distributions of X between treated and
untreated within the common support B3 is
selection on unobservable variables
Matching can eliminate B1 and B2
9
Questions

• How to do matching?
• How to get the weights (wi)?
• How to computer the standard errors?

10
Covariate matching

• With discrete variables and large samples, we may
  be able to look within actual covariate bins
• With small sample, or continuous variables, we
  will not able to match exactly

11
Propensity Score Matching
12
The method of psmatch

• Instead exact matching on covariates, we use
  match on propensity score to reduce the
  dimensionality
• Probability of receiving treatment conditional on
  covariates
• (1)we are trying to condition on X (2) we are
  trying to condition just on PS, because same PS
  have the same distribution of the full vector of
  covariates X

13
The assumption of psmatch

• The important assumption if one can control for
  observable differences in characteristics between
  the treated and non-treated group, the outcome in
  the absence of treatment is the same in both case

E(Y0X,D1) E(Y0X,D0)
P(D1Y0,Y1,X) P(D1,X)
Conditional on PS, each individual has the same
probability of assignment to treatment, as in a
randomized experiment.
14
psmatch

• Using bins with given width
• Nearest neighbor march (with or without
  replacement)

15
Psmatch algorithms

• Caliper matching

16
Psmatch algorithms

• Kernel matching

match treated observations with a local average
of non-treated observation based on the value of
p(X)
17
Steps to do ps matching
18
Steps to do ps matching
19
Steps to do ps matching
20
Bootstrap

• Parametric assume a kind of distribution
• Non-parametric bootstrapping standard errors

21
(No Transcript)
22
An example of psmatch (DW,1999 and 2002)

• Discuss the PSM methods and compare the result to
  that of experiment. The paper shows PSM succeed
  in comparing subset of the comparison units with
  the treated units. PSM can alleviate the bias due
  to systematic difference between treated units
  and comparison units
• National supported work program (NSW)
• Candidates either randomly assigned to or
  excluded from a training program (185 treated vs
  260 controlled)
• Two non-experimental comparison groups drawn from
  CPS and PSID

23
An example of psmatch (DW, 2002)

• The randomization table 1
• Differences between treatment groups and
  comparison groups table 2 and table 3, row 1-2
• Estimating Propensity score of different groups
  figure 1 and 2 very few comparison units are
  comparable to treated one
• Match results figure 3-6
• Choose sub sample table 2 row 3-5,dofference
  between CPS and PSID
• Test the assumption of PSM

24
Advantages Compared to regression method

• (1) Common support
• (2) No specification
• Whether it is better to selection based on
  unobservable method depends on the available data
  and the nature of selection process

25
Understand matching
26

• May need to use other methods, Difference in
  Difference. Matched DID

27
stata

• ssc install pscore
• ssc install psmatch2

28
Further readings

• Heckman, Ichimura and Todd, 1997, Matching as an
  econometric evaluation estimator evidence from
  evaluating a job training program. RES.
• Heckman, Ichimura and Todd, 1998, Matching as an
  econometric evaluation estimator. RES