Experimental design 2:

1
Experimental design 2
2
Good experimental designs have high internal
validity To unequivocally establish causality,
we need to ensure that groups in our study differ
systematically only on our intended independent
variable(s) and not on other confounding
variables as well.
3
Threats to the internal validity of an
experiment's results (e.g. Campbell and Stanley
1969) Time threats History Maturation Selection
-maturation interaction Repeated
testing Instrument change Group threats Initial
non-equivalence of groups Regression to the
mean Differential mortality Control group
awareness of its status. Participant reactivity
threats Experimenter effects, reactivity,
evaluation apprehension.
4
Types of experimental design 1.
Quasi-experimental designs No control over
allocation of subjects to groups, or timing of
manipulations of the independent variable. (a)
One-group post-test" design
Prone to time effects, and no baseline against
which to measure effects - pretty useless!
5
(b) "One group pre-test/post-test" design
Now have a baseline against which to measure
effects of treatment. Still prone to time effects.

Statistics marks 2008
Statistics marks 2009



course change
6
(c) "Interrupted time-series" design

measurement

measurement

time

measurement

treatment

measurement

measurement

measurement

Still prone to time effects.
7
(c) "Interrupted time-series" design (cont.)
Deaths for Friday nights, 10-12 pm Saturday and
Sunday nights, 10 pm - 4 am. Vertical line
implementation of British Road Safety Act, Oct.
1967 (Ross, Campbell Glass, 1970).
8
(d) Static group comparison" design

group A

measurement

treatment

(experimental gp.)

group B

measurement

no
treatment

(control gp.)

Subjects are not allocated randomly to groups
therefore observed differences may be due to
pre-existing group differences.
9
2. True experimental designs (a) "Post-test
only/control group" design

group A

measurement

treatme
nt

(experimental
random
gp.)

allocation

group B

measurement

no treatment

(control gp.)

Random allocation of subjects to groups should
ensure that observed differences are not due to
pre-existing group differences - but can't be
certain!
10
(b) "Pre-test/post-test control group" design

gro
up
measurement

measurem
ent

treatment

A


random
allocation

group

measurement

measurement

no treatment

B

Ensures that groups are indeed comparable before
the experimental manipulation was administered.
11
(c) "Solomon four group" design

measurement

treatment

measurement

group

A

measurement

no treatment

measurement

group

B

random
allocation

treatment

measurement

group

C

no treatment

group

measurement

D

Ensures that groups are indeed comparable before
the experimental manipulation was administered,
and that pre-testing hasn't affected performance.
(Uses lots of subjects, so rarely used).
12
Between-groups versus within-subjects
designs Between-groups (independent measures)
- Each subject participates in only one condition
of the study. e.g. sex differences in
memory. Within-subjects (repeated measures) -
Each subject does all of the conditions in a
study. e.g. effects of alcohol on memory. Mixed
designs - Mixture of both. e.g, sex differences
in effects of alcohol on memory.
13
Advantages and disadvantages of between-groups
and within-subjects designs
14
Within-subjects designs and order effects Order
effects practice, fatigue, boredom. A fixed
order of conditions would cause order to vary
systematically with condition - results are
uninterpretable, because they could be due to
order effects, experimental manipulations or
both. Solutions (a) Randomise order of
conditions e.g. with 3 conditions, subjects
randomly get orders ABC, BCA, ACB, CBA, CAB,
BAC. (b) Counterbalance order of conditions e.g.
equal numbers of subjects get each order.
15
A simple within-subjects design
subject 2
time
treatment A
measurement A
subject 1
treatment B
treatment B
measurement B
measurement A
treatment A
measurement A
16
Are threatening faces detected faster than happy
ones?
subject 2
time
threatening faces
detection time
subject 1
happy faces
happy faces
detection time
detection time
threatening faces
detection time
17
Disadvantages of the experimental
method Intrusive - participants know they are
being observed, and this may affect their
behaviour. Experimenter effects. Not all
phenomena are amenable to experimentation, for
practical or ethical reasons (e.g. post-traumatic
stress disorder, near-death experiences, effects
of physical and social deprivation, etc.). Some
phenomena (e.g. personality, age or sex
differences) can only be investigated by methods
which are, strictly speaking, quasi-experimental.
18
Conclusion Experiments are a useful tool for
establishing cause and effect - but other methods
(e.g. observation) are also important in
science. A good experimental design ensures that
the only variable that varies is the independent
variable chosen by the experimenter - the effects
of alternative confounding variables are
eliminated (or at least rendered unsystematic by
randomisation).