Matching Methods - 2

1
Matching Methods - 2

• Methods of Economic Investigation
• Lecture 12

2
Last Time

• Ways to define a control group if you dont
  have an experiment
• Difference-in-Differences
• Assume Fixed Differences over time
• Attribute any change in trend to treatment
• Propensity Score Matching
• Assume Treatment, conditional on observables, is
  as if randomly assigned
• Attribute any difference in outcomes to treatment

3
Choices when doing p-score matching

• Sample with or without replacement
• One-to-one or one-to-many matching
• How many observations to use for a match
• What criteria to just how close is close enough

4
How close is close enough?

• No right answer in these choiceswill depend
  heavily on sample issues
• How deep is the common support (i.e. are there
  lots of people in both control and treatment
  group at all the p-score values
• Should all be the same asymptotically but in
  finite samples (which is everything) may differ

5
Tradeoffs in different methods
Source Caliendo and Kopeinig, 2005
6
How to estimate a p-score

• Typically use a logit
• Specific, useful functional form for estimating
  discrete choice models
• You havent learned these yet but you will
• For now, think of running a regular OLS
  regression where the outcome is 1 if you got the
  treatment and zero if you didnt
• Take the ET X and thats your propensity score

7
The Treatment Effect

• CIA holds and sufficient region of of common
  support
• Difference in outcome between treated individual
  i and weighted comparison group J, with weight
  generated by the p-score distribution in the
  common support region

J is comparison group with J is the number of
comparison group units matched to i
N is the treatment group and N is the size of
the treatment group
8
General Procedure

• Run Regression
• Dependent variable T1, if participate T 0,
  otherwise.
• Choose appropriate conditioning variables, X
• Obtain propensity score predicted probability
  (p)

• 1-to-1 match
• estimate difference in outcomes for each pair
• Take average difference as treatment effect

• 1-to-n Match
• Nearest neighbor matching
• Caliper matching
• Nonparametric/kernel matching

Multivariate analysis based on new sample
9
Standard Errors

• Problem Estimated variance of treatment effect
  should include additional variance from
  estimating p
• Typically people bootstrap which is a
  non-parametric form of estimating your
  coefficients over and over until you get a
  distribution of those coefficientsuse the
  variance from that
• Will do this in a few weeks

10
Some concerns about Matching

• Data intensive in propensity score estimation
• May reduce dimensionality of treatment effect
  estimation but still need enough of a sample to
  estimate propensity score over common support
• Need LOTS of Xs for this to be believable
• Inflexible in how p-score is related to treatment
  
• Worry about heterogeneity
• Bias terms much more difficult to sign
  (non-linear p-score bias)

11
Matching Diff-in-Diff

• Worry that unobservables causing selection
  because matching on X not sufficient
• Can combine this with difference and difference
  estimates
• Take control group J for each individual i
• Estimate difference before treatment
• If the groups are truly as if random should be
  zero
• If its not zero can assume fixed differences
  over time and take before after difference in
  treatment and control groups

12
Bottom Line

• Matching Methods used to replicate experimental
  methods
• Need to believe independence, conditional on Xs
• If matching assumption is right, can estimate the
  TOT without worrying about selection bias