Political Science 30 Political Inquiry

1
Political Science 30Political Inquiry

• Experiments

2
Experiments

• The Beauty of Random Assignment
• How Experiments Work
• Strengths and Weakness

3
The Beauty of Random Assignment

• Problem In non-experimental studies, what
  determines the values that an independent
  variable takes on? Often, a confounding variable
  determines these values, and affects the DV.
• For instance, the confound of Are you a serious
  student may determine where you will sit in a
  class.

4
An Example of a Confound from Recent Research

• Getting Out the Vote!, a recent book by Don Green
  and Alan Gerber, asks
• Does contacting registered voters through phone
  calls or visits make them more likely to turn out
  on election day?
• Potential Confound Previous participation
  records. Campaigns with limited resources
  concentrate their mobilization efforts on voters
  who have turned out in the past.

5
The Beauty of Random Assignment

• Solution Interrupt the causal path that leads
  from the confound to the independent variable by
  randomly assigning the values that the IV takes
  on in each case.
• Randomly assign seats so that there are just as
  many serious students and slackers in each part
  of the lecture hall.
• Green and Gerber randomly assigned some voters to
  be contacted, in order to measure the actual
  effects of mobilization (which are quite weak).

6
The Beauty of Random Assignment

• Treatment Group
• All of the cases in this group have been assigned
  one value of the IV (sit in front, take medicine,
  etc.)

• Control Group
• All of the cases have been assigned a different
  value of the IV (in most cases, left alone or
  given placebo)

In all other respects (including the values of
confounding variables that they take on), these
groups are similar.
7
How Experiments WorkStep 1 Random Assignment

• Begin by splitting your cases into two or more
  groups of 30 or more through a process that is
  truly random.
• Using something like a random number generator is
  key, because many seemingly neutral assignment
  processes may be correlated with a confounding
  variable.
• Examples arrival times, last names, section
  times, bleeding hearts.

8
How Experiments WorkStep 2 (Optional) Pre-Test

• To check how the random assignment process
  worked, measure the value that the DV takes on
  for each case before any treatment has been
  applied.
• Each group should average about the same values
  on the dependent variable.
• Even if something went wrong, we can still learn
  from the time-series comparison.
• Often it is hard to pre-test.

9
How Experiments WorkStep 3 Apply the Treatment

• Change the value of the independent variable that
  cases in at least the treatment group take on.
• Administer the medicine or the placebo, put
  students in their seats, request that subjects
  administer an electric shock.
• This is where ethical issues arise.

10
How Experiments WorkStep 4 Post-Test

• Measure the value that the DV takes on for each
  case after the treatment has been applied.
• Comparing values of the DV in treatment group vs.
  control group tells us the effect of the
  treatment, if random assignment worked.
• Comparing shifts from pre-test to post-test is
  helpful when random assignment failed.

11
Schematic of an Experiment

• Treatment Group (pre-test)
  Treatment (post-test)
• Random Compare
• Assignment
• Control Group (pre-test) (post-test)

12
Strength of ExperimentsHigh Internal Validity

• Internal validity judges how well a research
  design has tested a causal relationship, in the
  cases examined.
• Among the cases in our study, do we have reason
  to believe that IV 1 causes DV?
• Because random assignment takes away our fear of
  confounds, experiments have high internal
  validity.

13
Weakness of ExperimentsLow External Validity

• External validity judges how confident we can be
  that a causal relationship identified in our
  cases can be generalized to the outside world.
• Our cases may be different than the general
  population, or our cases may react differently to
  treatments, or our treatments may be very
  artificial.
• College sophomore problem makes external
  validity the flaw in experiments.
• You cant assign every treatment gender, race.