Designing Quasi Scientific Political Research

1
Designing (Quasi) ScientificPolitical Research

• PS 102
• Introduction to Political Inquiry
• Lecture 7

2
Non-experimental Design

• Most Political Science research does not use
  experimental design
• Inability to manipulate treatments
• (e.g. randomly assigning states to alliances)
• Ethical difficulties with treatments
• (Causing, war, poverty, etc. is bad)
• Concerns over external validity
• (Do presidents think like college students)
• Non-experimental designs have at least one of the
  following weaknesses
• No control group, no control over group
  assignment, no control over application of
  treatment

3
Non-Experimental Design

• Lack of randomized treatment and control leads to
  key threats to causal inference
• Spuriousness
• Endogeneity
• Selection bias
• ALL Non-experimental designs must cope with these
  threats (qualitative and quantitative)
• Leads to use of advanced statistics
• Experimental data is EASY to analyze
• Non-Experimental designs generally weaker on
  internal validity and stronger on external
  validity

4
ExploratoryNon-Experimental Designs

• Small-N

• Focus Groups

• AKA case studies or comparative cases studies
• Involve rich analysis of a small number of cases
• Rely on Mills method of difference
• Important role in scientific research
• Theory building
• Verify causal mechanisms
• Weak for causal inference
• Spuriousness
• Representativeness

• Common tactic in market research
• Has role in scientific research
• Theory building
• Measurement verification
• VERY weak for causal inference

5
SystematicNon-Experimental Designs

• Cross-sectional designs
• Measure independent and dependent variables
  simultaneously
• Data analysis is necessary for causal inferences
• Strong ability to account for spuriousness
• Weak ability to account for endogeneity
• Example Does watching Comedy Central reduce
  support for the Iraq War?

6
The Stewart/Colbert Effect?
--------------------------------------------------
---------------------------- IraqRight
Coef. Std. Err. t Pgtt 95 Conf.
Interval ---------------------------------------
--------------------------------------
comcen1 -.8788581 .1304979 -6.73 0.000
-1.134785 -.6229314 _cons
1.31706 .02757 47.77 0.000 1.262991
1.371129 ---------------------------------------
---------------------------------------
But is Colbert Nation just like the rest of the
nation?
COMEDY CENTRAL WATCHERS Variable Obs
Mean Std. Dev. Min
Max ---------------------------------------------
------------------------ age
89 35.1573 14.5007 18
75 partyid7pt 88 2.647727
1.742239 1 7 ideology
79 2.240506 1.088579 1
5 NON COMEDY CENTRAL WATCHERS Variable
Obs Mean Std. Dev. Min
Max ---------------------------------------------
------------------------ age
1911 45.5259 15.20771 18
86 partyid7pt 1860 3.804301
2.285766 1 7 ideology
1718 3.21362 1.120727 1
5
7
The Stewart/Colbert Effect!
--------------------------------------------------
---------------------------- iraqright
Coef. Std. Err. t Pgtt 95 Conf.
Interval ---------------------------------------
--------------------------------------
comcen1 -.3654761 .1040692 -3.51 0.000
-.569588 -.1613642 age
-.0013567 .0013926 -0.97 0.330
-.0040879 .0013746 partyid7pt .2803524
.0119637 23.43 0.000 .256888
.3038169 ideology .2411823 .0245547
9.82 0.000 .193023 .2893417
_cons -.4499053 .0857604 -5.25 0.000
-.618108 -.2817026 -------------------------
--------------------------------------------------
---
But what assumptions must we make about the
effects of party and ideology?
How many conservative Republican retirees are
Colbert fans?
8
The Stewart/Colbert Effect!
Variable Obs Mean Std. Dev.
Min Max --------------------------
-------------------------------------------
Age_NO 2001 44.11394 15.56801
18 86 Age_YES
2001 42.95052 14.98916 18
75 partyid7NO 2001 3.638181
2.300657 1 7 partyid7YES
2001 3.650675 2.333326 1
7 ideologyNO 2001 3.070465
1.15392 1 5 ideologyYES
2001 3.0005 1.104762 1 5
Number of obs 1770 Number of
matches (m) 1 -------------------------
--------------------------------------------------
--- iraqright Coef. Std. Err.
z Pgtz 95 Conf. Interval ------------
-------------------------------------------------
---------------- SATE
-.5566714 .1675329 -3.32 0.001
-.8850298 -.228313 ----------------------------
--------------------------------------------------
Matching variables age partyid7pt ideology
BUTWhat about endogeneity?
9
SystematicNon-Experimental Designs

• Panel studies
• Cross-sectional designs that include a time
  element
• Rely on measurement of the same units of analysis
  at different points in timecreating waves of
  data for analysis over time
• Longitudinal designs
• Allow for the measurement of variables at
  different points in time
• Can model change across time examine the time
  order of a causal relationship and estimate age,
  cohort, and period effects

10
Casualties and Support for the Iraq War
11
SystematicNon-Experimental Designs

• Intervention analysis
• Measurements of a dependent variable before and
  after the introduction of an independent variable
  that is observed but not controlled by the
  researcher
• Connecticut crackdown on speeding
• Did severe traffic penalties prevent traffic
  deaths?
• Often referred to as a quasi-experiment or
  natural experiment
• Not a useful label
• Data either do or do not have experimental
  qualities

12
The Connecticut Crackdown
13
The Connecticut Crackdown
14
The Connecticut Crackdown?