Research Design

1
Research Design

• The program that guides the investigator in the
  process of collecting, analyzing, and
  interpreting observations. It is a logical model
  of proof that allows the researcher to draw
  inferences concerning causal relations among the
  variables under investigation (Nachmias and
  Nachmias).

2
Research Design and Causality

• Relationships between variables Two variables
  are related to one another (i.e. are correlated)
  if one or more values of one variable tend to be
  associated with one or more values of the other
  variable.
• Causal relationship A relationship in which one
  variable directly causes/explains the other
  variable.

3
3 Criteria for Establishing Causality

• X is correlated with Y
• X precedes Y in time
• The observed relationship between X and Y is not
  spurious
• Spurious Relationship An observed relationship
  between X and Y is said to be spurious (or partly
  spurious) if there exists a third variable Z,
  which is both a cause of Y AND is correlated with
  X.

4
Example of Spuriousness
5
Experimental Designs

• Select a sample
• Randomly assign subjects into 2 or more groups.
• Observe (measure) DV for all groups (if design
  includes pretest)
• Introduce the stimulus (IV)
• Observe (measure) DV for each group
• If the change in the value of the dependent
  variable varies significantly across groups, then
  we conclude that X ? Y

6
Simple Experimental Designs

• 2-Group Pretest - Posttest Design (Classical or
  Simple Experiment)
• R Mexp1 X Mexp2
• R Mcontrol1 Mcontrol2

7
Simple Experimental Designs

• 2-Group Pretest - Posttest Design (Classical or
  Simple Experiment)
• If two different treatments (and no pure control
  group)
• R M1A XA M2A
• R M1B XB M2B

8
Experimental Designs

• Key distinguishing feature of the experimental
  design
• Randomization (random assignment of subjects to
  groups)

9
An Example Rosenberg and McCafferty

• Hypothesis Candidate presentation (appearance)
  is related to candidate image (and therefore vote
  choice).
• Sample University students
• Dependent Variable Vote Choice (simulated
  ballot)
• Independent Variable Candidate Appearance
  (picture in campaign flyer)

10
Research Design (RM)

• 2-group Posttest Only
• Generally
• R XA M1A
• R XB M1B
• R XA (Good/Bad picture) M1A (Vote)
• R XB (Bad/Good picture) M1B (Vote)

11
The Independent Variable (Stimulus or
Treatment)
12
The Independent Variable (Stimulus or
Treatment)
13
Results
14
Experiments and Causality

• Correlation?

15
Experiments and Causality

• Correlation?
• Comparison of two or more groups (on dependent
  variable) experiencing different levels of
  exposure to the causal (explanatory) variable
  (X). This establishes correlation.

16
Experiments and Causality

• Temporal Precedence?

17
Experiments and Causality

• Temporal Precedence?
• The introduction of the independent variable
  (stimulus) is manipulated by the researcher to
  insure temporal precedence.

18
Experiments and Causality

• Spuriousness?

19
Experiments and Causality

• Spuriousness?
• Random assignment insures that rival hypotheses
  are ruled out, thus eliminating the threat of
  spuriousness. (How?)

20
Simple Experimental Designs

• 2-Group Pretest - Posttest Design (Classical or
  Simple Experiment)
• R Mexp1 X Mexp2
• R Mcontrol1 Mcontrol2
• OR
• R M1A XA M2A
• R M1B XB M2B

21
Simple Experimental Designs

• 2-Group Posttest Only Design
• R X M1exp
• R M1control
• OR
• R XA M1A
• R XB M1B

22
Other Types of Experimental Designs

• Multigroup designs more than two groups
• Multiple Group Pretest - Posttest Design
• Multiple Group Posttest Only Design

23
Other Types of Experimental Designs

• Time series design
• Multiple observations over time

24
Field Experiments

• Experiments that occur outside the artificial
  setting of laboratory (occur in the real world)
• Example Voter mobilization strategies

25

• Previous Research
• Phone contact positively related to turnout
• Research design compare turnout rate of those
  contacted to those who were not

26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
Determine the following

• Hypothesis
• Theory
• Research design (Table 1)

30
(No Transcript)
31
Zilber and Niven (SSQ)

• Table 1 2-Group Posttest Only
• R X1(black) M1
• R X2(A-A) M2

32
Zilber and Niven (SSQ)

• Table 3 2X2 Factorial Design
• R (black/liberal) M1
• R (A-A/liberal) M2
• R (black/conserv) M3
• R (A-A/conserv) M4
• To see how the effect of racial label varies as a
  function of ideology, we compare
• M1-M2 to M3-M4

33

• Conclusion The choice of racial labels does
  affect white attitudes toward blacks, but only
  among liberals.

34
Evaluating Research DesignsInternal Validity

• Internal Validity - the degree to which we can be
  sure that the independent variable caused the
  dependent variable within the current sample

35
Evaluating Research DesignsInternal Validity

• Experimental designs - randomization of
  subjects/units across values of the independent
  variable greatly reduce (eliminate?) the
  potential for spuriousness to threaten internal
  validity

36
Evaluating Research DesignsInternal Validity

• Specific factors threatening internal validity
  (experimental or nonexperemental) include
• History
• Maturation
• Experimental mortality
• Instrumentation
• Testing

37
Evaluating Research DesignsExternal Validity

• External Validity - the degree to which the
  results of the analysis can be generalized beyond
  the current sample/study. Can be maximized by
• Using subjects (units) that are representative of
  the population to which ones theory applies
• Using a laboratory that is as close to real
  life conditions as possible
• Field experiments

38
Nonexperimental Designs

• Deviate in some important way(s) from true
  experimental design
• All nonexperimental designs lack random
  assignment of subjects to groups
• But some nonexperimental designs may lack other
  features too

39
Quasi-Experiments

• Lack random assignment, but otherwise similar to
  a true experiment
• Example

40
(No Transcript)
41
Matched Pair Designs (precision matching)

• Effort to overcome nonrandom assignment of
  subjects to treatment groups (i.e. to equalize
  the comparison groups in a research design)
• 1. Form matched pairs pairs of subjects that
  are matched based on variable(s) known to affect
  the DV
• 2. Assign one member of each pair to treatment
  and control groups

42
Matched Pair Designs (precision matching)

• Example
• The Diversification of the Federal Bench Policy
  and Process Ramifications
• Thomas G. Walker Deborah J. Barrow
• The Journal of Politics, Vol. 47, No. 2. (Jun.,
  1985), pp. 596-617.

43
(No Transcript)
44
Nonexperimental Designs

• Cross-Sectional Designs
• No manipulation of IV by researcher
• Observations for IV and DV recorded at the same
  time

45
Example Wine and Health

• Hypothesis Drinking wine causes individuals to
  be healthier (esp. heart)
• Existing studies compared the health of wine
  drinkers to the health of those who do not drink
  wine
• Research design
• XA (Wine drinkers) M1A (Health)
• XB (Non-drinkers) M1B (Health)

46
Spurious Results?
47
Controlling for Affluence

• Research design
• XA (Affluent Wine drinkers) M1A (Health)
• XB (Affluent Non-drinkers) M1B (Health)
• XC (Poor Wine drinkers) M1C (Health)
• XD (Poor Non-drinkers) M1D (Health)

48
Another (Possible?) Example
49
Stack Gundlach

• Hypothesis There is a positive relationship
  between exposure to country music and suicide
  rates
• Research design
• XA (no country music) MA (suicide rate)
• XB (1 station) MB (suicide rate)
• XC (2 stations) MC (suicide rate)
• XD (3 stations) MD (suicide rate)
• Xi ( etc.) Mi (suicide
  rate)

50
Stack Gundlach

• Findings 51 of the variation in urban white
  suicide rates can be explained by variation in
  airtime devoted to country music
• Is this result spurious?

51
Nonexperimental Designs

• Time Series Design repeated observations for X
  and Y for a single unit
• Panel Time Series Design repeated observations
  for X and Y for a group

52
Nonexperimental Designs

• Case Study
• X O1
• Inference made by examining one case.

53
Historical Development of Ethical Standards in
Observational Research

• 1974 - National Research Act
• National Commission for the Protection of Human
  Subjects of Biomedical and Behavioral Research
• identify the basic ethical principles that should
  underlie the conduct of biomedical and behavioral
  research involving human subjects and to develop
  guidelines which should be followed to assure
  that such research is conducted in accordance
  with those principles
• Belmont Report (1979)

54
Milgrams Obedience Experiments

• Experiment advertised as test of memorization
  training through electric shock
• Those being tested were actually actors
• Lab assistants hired to administer shocks were
  the true subjects
• 65 of lab assistants were willing to administer
  shocks of up to 450 volts

55
The Stanford Prison Experiment

• Phillip Zimbardo, 1971
• Goal to study the psychological effects of
  imprisonment on guards and prisoners