Alternative Research Designs

1
Alternative Research Designs
2
Visualizing Designs

• Block notation
• Useful for visualizing conditions, interactions
• Design notation
• Useful for seeing the temporal sequence of the
  experiment

3
Design Notation

• Elements
• Observations/Measures O
• Treatments/Programs X
• Conditions Each condition has its own line
• Assignment to group
• R Random assignment
• N Nonequivalent groups
• C Assignment by cutoff
• Time Moves from left to right

4
Design Notation

• Kasser Sheldon (2000)
• Bransford Johnson (1972)?

R XMS F O R Xmusic F O
5
Alternative Research Designs

• Quasi-experimental designs
• Nonequivalent groups
• Interrupted time series
• Small-N designs

6
Quasi-Experimental Designs

• Strictly speakingtrue experiments use random
  assignment or counterbalancing
• Quasi-experiments
• You cannot randomly assign experimental
  participants to groups
• You DO manipulate an IV
• You DO measure a DV

7
When to use quasi-experimental design?

• Study participants in certain groups
• Evaluate an ongoing or completed
  program/intervention
• Study social conditions (examples poverty, race,
  unemployment)
• Expense, time, or monitoring difficulties
• Ethical considerations

8
Nonequivalent Groups Design

• Structured like a pretest-postest randomized
  experiment
• Key is to create as equal a comparison group as
  possible through our selection criteria

N O X O (treatment) N O O (comparison)
9
Nonequivalent Groups Design

• Example Geronimous (1991)
• Typical outcomes for teen mothers
• Poverty, high-school dropout rates increase,
  higher infant mortality
• Geronimous believed that family factors, like
  SES, were better predictors of outcomes than teen
  pregnancy
• How to find a comparison group as similar as
  possible?

10
Nonequivalent Groups Design Analysis

• Example Fry pan company wants to institute a
  flextime schedule
• Pittsburgh plant (experimental group)
• Cleveland plant (comparison group)

NPitt O Xflextime O NClev O O
11
(No Transcript)
12
Nonequivalent Groups Design Analysis

• Your groups began the experiment as not
  equivalent
• So, the important question is not whether there
  was a difference
• Is the difference the same as before the
  experiment?

13
Nonequivalent Groups Design

• Threats to Internal Validity
• Maturation
• Instrumentation
• Interaction between selection and history
• Statistical regression

14
Regression and Matching

• Use matching as a control procedure to create
  equivalent groups
• BUT! What if the groups come from populations
  that are different on the factor thats used as
  the matching variable?

15
Regression and Matching

• Example Improving reading skills in
  disadvantaged youth
• Treatment (reading program) Most in need
• Comparison group Similar neighborhood in other
  cities
• Match groups according to initial reading skill

16
Regression and Matching
Experimental Group Pretest 25 Reading
program Posttest 25
Comparison Group Pretest 25 Posttest 29
17
Nonequivalent Groups Design

• Interpretation of findings must be more cautious
  than for a true experiment, but a strong research
  design

18
Interrupted Time-Series Design

• Measure a group of participants repeatedly over
  time
• Interrupt with a treatment
• Measure participants repeatedly again

O1 O2 O3 O4 X O5 O6 O7 O8
19
(No Transcript)
20
Interrupted Time-Series Design

• Threats to Internal Validity
• History
• Maturation
• Instrumentation

21
Interrupted Time-Series Design

• How to control for history threats?
• Frequent measurement intervals
• Comparison group

22
Interrupted Time-Series Design

• How to control for history threats?
• Frequent measurement intervals
• Comparison group
• Measure, treatment, measure, then undo the
  treatment, measure again
• Not always feasible

23
Small-N Designs

• Single-subject or very few subjects
• Why use?
• Sample may exhaust the population
• A single negative refutes a theory
• Participant may be extremely rare
• Research is time consuming, expensive, requires
  lots of training
• How is this different from a case study?

24
Small-N Designs

• Behavior of the individual must be shown to
  change as a result of treatment
• Characteristics
• Repeated measures
• Baseline measurement
• Change one variable at a time

25
Small-N Designs

• A-B design
• Problems?
• A-B-A design
• Problems?
• A-B-A-B design