Lecture 8: Quasiexperiments

1
Lecture 8 Quasi-experiments

• Aims Objectives
• To differentiate between true and
  quasi-experiments
• To discuss the nature of random allocation
• To examine threats to experimental validity
• To examine some basic quasi-experimental designs

2
Type of general approaches to design

• Descriptive
• What, where, when and to whom
• Relational
• Co-variaton
• Experimental
• Causal analysis via random allocation
• Quasi-experimental
• Causal statements when groups are not equivalent
  no random allocation

3
Random allocation

• Every potential subject has an equal chance of
  being in any condition
• Simple randomisation
• Block randomisation
• Blocks AB, produce sequences e.g., AABB, ABAB.
  Sequences are selected at random and subjects
  selected at random into that block
• Stratified randomisation
• Select on a characteristic that influences the
  groups and have block randomisation lists within
  those blocks

4
Internal validity I

• Ruling out a third cause
• Randomisation controls for
• History effects
• Maturation effects
• Mortality
• Statistical regression to the mean
• Randomisation does not control for
• Effects equalising groups
• Diffusion of treatment effects
• Compensatory rivalry
• Compensatory equalisation
• Effect separating groups
• Resentful demoralisation

5
Statistical validity

• Risk of making a type 1 error
• Power
• Fishing
• Reliability of measures, treatments
• Random irrelevance
• Random heterogeneity of respondents

6
External validitygeneralisation

• Is the effect stable
• Over time
• Across individuals
• Across IVs DVs
• Across places

7
Mook

• Research is not always about generalizability of
  findings
• Conceptualisation of generalizability are base
  don an agricultural model
• Experiments are about generalizability of theory
  not findings

8
Construct validity

• Experimenter effects
• Structural
• Mono-operation bias
• Mono-method bias
• Poor explication of constructs
• Interpersonal
• Demand characteristics
• Apprehension evaluation
• Rosenthal effect

9
Quasi-experiments
Nomenclature
X a treatment O Observation Not
randomly assigned
10
Uninterrupted designs
One group pre- post test design
O X O
Threats history, maturation regression
11
Non-equivalent groups
Untreated control group with pre and post test
O X O O O
12
Reverse treatments
O X O . O x- O
13
ITSDs
OOOOXOOOO .. OOOO OOOO
OO OOOXOOO OOXOOO OOO
With switch replication
14
ARIMA

• OOOXOOO
• 456 Upward drift
• 444 Upward constant
• 466 Gradual upwards
• 333 333 No change

15
Regression discontinuity
Depression
Poverty
Short
Long
16
Randomized field trials

• Randomisation by independent group
• Make seek treatment elsewhere
• Within condition effects
• Placebo-control

17
Experiments the last word

• Experiments are important because they allow us
  to show what can or ought to happen
• Bio feedback
• Milgram
• Sherrifs boys camp study