Experimental Design:

1
Experimental Design Threats toValidity
2
EXPERIMENTS   The independent variable is
manipulated todetermine its effect on the
dependent variable(s)whilst holding all other
potential influences onthe dependent variable(s)
constant
3
VALIDITY OF A STUDY
INTERNAL VALIDITY To what extent does the
design of the studyallow us to attribute changes
in the dependentvariable(s) to the effects of
changes in theindependent variable?
4
THREATS TO INTERNAL VALIDITY
Changes in the dependent variable may be dueto a
variety of extraneous factors (confoundingor
potential independent variables), rather thanto
the manipulation of the independent variable
5
VALIDITY OF A STUDY
EXTERNAL (ecological) VALIDITY
To what extent does the design of the studyallow
us to generalise the results to populationsother
than that from which the sample was drawn,or to
similar populations in different settings orat
different times?
6
THREATS TO INTERNAL VALIDITY
Threats relating to
The passage of time Selection of
participantsTesting and manipulations
7
THREATS RELATING TO THE PASSAGE OF TIME
MATURATIONChanges within the participants due to
thepassage of time (developmental,
ageing,hunger, fatigue etc.)
8
THREATS RELATING TO THE PASSAGE OF TIME
HISTORY Events occurring between pre- and
post-test in addition to the experimental
manipulation
9
THREATS RELATING TO THE PASSAGE OF TIME
INSTRUMENTATIONChanges in the way the dependent
variableis measured (measurement
errors,different tests, calibration problems,
etc)
10
(No Transcript)
11
MORTALITY Differential drop-out from groups
betweenpre- and post-test
12
RANDOMISED PRE-TEST POST-TESTCONTROL GROUP
DESIGN
R O1 X O2 R O3 O4
Controls for threats to internal validity due
tothe passage of time provided that
Randomisation works (groups equated at
pre-test) There is no differential mortality
between groups
13
RANDOMISED PRE-TEST POST-TESTCONTROL GROUP
DESIGN
R O1 X O2 R O3 O4
Threats relating to the passage of timeare
controlled because they shouldmanifest
themselves equally in each group
14
R O1 X O2 R O3 O4
15
R O1 X O2 R O3 O4
Dependent Variable
Pre-test
Post-test
16
THREATS RELATING TO SELECTIONOF PARTICIPANTS
SELECTIONBias resulting from differential
selection when assigning participants to groups
17
Pre-operative anxiety and recovery from
anaesthesia.
A study was conducted to determine whether a
brief pre-operative relaxationprocedure could
reduce recovery time in surgical patients with
sports injuries.Athletes about to undergo three
different types of surgery were assigned toone
of three conditions. Those about to receive a
steroid injection for frozenshoulder were
assigned to a relaxation condition in which they
listened to a30 min. relaxation tape immediately
prior to transfer to the operating
theatre.Athletes about to undergo repair of
ruptured anterior cruciate ligament wereassigned
to a placebo condition in which they listened to
a 30 min. tape-recordedshort story. Athletes
about to undergo reduction of dislocated shoulder
actedas no-treatment controls.
18
Dissociation and endurance performance (Morgan et
al., 1983). A study was performed to evaluate
whether a mental dissociation strategy
couldfacilitate endurance performance. 27 navy
personnel underwent a submaximalexercise test to
determine VO2 max. Participants returned to the
lab 48hours later and ran to exhaustion on a
treadmill at an exercise intensity of80 VO2
max. Time to exhaustion was recorded.
Participants returned again48 hours later and
were randomly assigned to a treatment group or a
controlgroup. The treatment group were
instructed in how to employ a dissociationstrateg
y whilst running. Time to exhaustion at 80 VO2
max was assessed again. Results showed that
significantly more of the dissociation
participantsincreased their endurance time
between trials two and three compared tocontrol
participants (78 of dissociation participants
versus 31 of controlparticipants p lt .05).
This suggests that the dissociation strategy
facilitatesendurance performance.
19
Dissociation and endurance performance (Morgan et
al., 1983). A study was performed to evaluate
whether a mental dissociation strategy
couldfacilitate endurance performance. 27 navy
personnel underwent a submaximalexercise test to
determine VO2 max. Participants returned to the
lab 48hours later and ran to exhaustion on a
treadmill at an exercise intensity of80 VO2
max. Time to exhaustion was recorded.
Participants returned again48 hours later and
were randomly assigned to a treatment group or a
controlgroup. The treatment group were
instructed in how to employ a dissociationstrateg
y whilst running. Time to exhaustion at 80 VO2
max was assessed again. Results showed that
significantly more of the dissociation
participantsincreased their endurance time
between trials two and three compared tocontrol
participants (78 of dissociation participants
versus 31 of controlparticipants p lt .05).
This suggests that the dissociation strategy
facilitatesendurance performance.
20
THREATS RELATING TO SELECTIONOF PARTICIPANTS
SELECTION X MANIPULATION Effects of the
manipulation only hold forthe particular
population sampled A threat to external validity
21
(No Transcript)
22
THREATS RELATING TO TESTINGAND MANIPULATIONS
EFFECTS OF TESTINGThe effects of taking the
pre-test on scoreson the post-test Post-test
scores may be affected by practice,memory
etc. Pre-test sensitisation
23
Attitudes (mean)
Pre-test
Post-test
24
THREATS RELATING TO TESTINGAND MANIPULATIONS
REACTIVE EFFECTS OF TESTINGTesting X
manipulation interactionThreat to external
validity Pre-testing changes the participants
responsesto the manipulation. The results
obtained frompre-tested participants may be
unrepresentativeof results that would be
obtained from thosewho only received the
manipulation
25
REACTIVE ARRANGEMENTS
The experimental setting influences
participantsto respond differently to how they
would respondin a real-life setting
Research settings are unnatural situationsso we
might expect participants to behaveunnaturally
26
Dissociation and endurance performance(Morgan et
al., 1983).
Long distance runners in Tibet have used
thistechnique to run 300 miles non-stop in 30
hoursand many successful marathoners are known
to use similar techniques By dissociating in
thisway you will not feel the same amount of
fatigue,pain or discomfort you experienced
during the lasttrial
27
REACTIVE ARRANGEMENTS
DEMAND CHARACTERISTICS (Orne, 1962) the
totality of cues which convey an experimental
hypothesis to the subject.
28
Dissociation and endurance performance(Morgan et
al., 1983).
Long distance runners in Tibet have used
thistechnique to run 300 miles non-stop in 30
hoursand many successful marathoners are known
to use similar techniques By dissociating in
thisway you will not feel the same amount of
fatigue,pain or discomfort you experienced
during the lasttrial Please go as long as you
can in order to makeour experiment a success.
29
NON-SPECIFIC TREATMENT EFFECTS
Placebo/attention effects Hawthorne
effects Expectations of benefit Credibility
of the treatment
30
EXPERIMENTER EFFECTS
The Pygmalion effect Rosenthal Jacobson
(1968)
31
RANDOMISED PRE-TEST POST-TESTCONTROL GROUP
DESIGN
R O1 X O2 R O3 O4
Controls for threats relating to the passage of
timeprovided that
Randomisation works (groups equated at
pre-test) There is no differential mortality
between groups All potential independent
variables includingnon-specific treatment
effects are held constant
32
R O1 X O2 R O3 O4
Threats relating to selection and testingare
controlled because participants arerandomly
assigned to groups
EXCEPT Because there is a pre-test, the
reactiveeffects of testing are not controlled
33
POST-TEST ONLY CONTROL GROUP DESIGN
R X O1 R O2
34
R X O1 R O2
PLUSThis design also controls for reactive
effectsof testing because there is no pre-test
BUTWe dont know if the groups were equated at
pre-test
35
EXTERNAL VALIDITY
With the exception of the post-test only
designscontrol for reactive effects of testing,
logically, wecannot control for threats to
external validity In an internally valid study
we can onlydemonstrate that the effects (of X)
hold underthe specific conditions of the
study e.g. The effects of X hold for
(pre-tested)participants from the particular
populationsampled at this point in time in this
particularplace etc.