Within subjects designs

1
Within subjects designs

• Definition
• Reasons for using within subjects designs
• Stage of Practice effects
• Definition
• Two types of practice effects
• Order effects
• Sequence effects

2
Within subjects designs

• Stage of practice effects (continued)
• Remedies
• Complete W.S. design
• Incomplete W.S. design
• Limitations of within subjects designs
• Examples of W.S. designs
• Grice Hunter (1964)
• Kahneman et al. (1993)
• Lee Katz (1998)

3
Within subjects designs

• Definition

4
Within subjects designs definition

• When a variable is manipulated within subjects,
  all subjects receive all levels of that variable.

• A given study can use only between groups
  variables, only within-subjects variables, or a
  combination of the two.

5
Within subjects designs definition

• For example, suppose you want to know which of
  three kinds of car is most comfortable to drive
  on a long journey.

• You have a Ford, a Chevy, and a Toyota, and 10
  drivers (the subjects)
• Each driver drives each car on the same length
  journey and rates each for comfort

6
Within subjects designs

• Definition
• Reasons for using within subjects designs

7
Within subjects designs

• Reasons for using within subjects designs

• Few subjects are available
• Increase efficiency
• No acceptable matching procedure
• Increase sensitivity
• Study differences in subjects over time
• Compare to between groups design

8
Reasons for using the W.S. design

• Few subjects are available

• E.g., research with patients with particular
  impairments that are important but uncommon, such
  as deep dyslexia or prosopagnosia

9
Reasons for using the W.S. design

• Few subjects are available
• Increase efficiency

• Answer more questions with the same number of
  subjects
• E.g., instead of dividing 40 subjects among two
  treatment groups for one study, use them in two
  separate studies.

10
Reasons for using the W.S. design

• Few subjects are available
• Increase efficiency
• No acceptable matching procedure

• For example, if you cannot measure enthusiasm,
  speed of processing, efficiency of attention, etc.

11
Reasons for using the W.S. design

• Few subjects are available
• Increase efficiency
• No acceptable matching procedure
• Increase sensitivity

• Sensitivity refers to the ability to detect
  differences in performance produced by the
  treatment
• Analogous to turning up the magnification of a
  microscope

12
Reasons for using the W.S. design

• Few subjects are available
• Increase efficiency
• No acceptable matching procedure
• Increase sensitivity
• Study differences in subjects over time

• Learning
• Psychophysics
• Whenever you want subjects to compare two or
  more stimuli relative to one another
• E.g., Kahneman et al. (1993)

13
Reasons for using the W.S. design

• Few subjects are available
• Increase efficiency
• No acceptable matching procedure
• Increase sensitivity
• Study differences in subjects over time
• Compare to between groups design

• Treatment might have different effect in within
  subjects vs. between groups designs.
• E.g., Grice Hunter (1964)

14
Within subjects designs

• Stage of practice effects
• Definition
• Two types of stage of practice effects
• Order effects
• Sequence effects
• Remedies
• Complete within subjects design
• Incomplete within subjects design

15
Stage of Practice Effects Definition

• The changes subjects undergo with repeated
  testing are called stage of practice effects.

• With repeated testing, subjects performance on a
  task may get
• better if a skill is being developed
• worse if fatigue or boredom increase.

16
Within subjects designs

• Stage of practice effects
• Definition
• Two types of stage of practice effects
• Order effects
• Sequence effects
• Remedies
• Complete within subjects design
• Incomplete within subjects design

17
Two types of stage of practice effects

• Order effects

• these result from the position in the sequence of
  treatments that a particular treatment has.

18
Order effects

• If B and D give different results, is that
  treatment effect?
• Subjects might just be more tired, or more
  skilled, when they get D

A B C D A B C D
19
Two types of stage of practice effects

• Sequence effects

• These result from interactions among the
  treatments (also known as differential transfer
  effects).

20
Sequence effects

• B follows A vs. B follows C
• This difference could produce sequence effects
  is a B / C difference due to treatment or due to
  what they follow?

A B C D C B A D
21
Within subjects designs

• Stage of practice effects
• Definition
• Two types of stage of practice effects
• Order effects
• Sequence effects
• Remedies
• Complete within subjects design
• Incomplete within subjects design

22
Stage of Practice effects Remedies

• Before considering remedies, we have to
  distinguish between two types of W.S. design

• Complete within subjects design
• Incomplete within subjects design

23
Stage of Practice effects Remedies

• Complete within subjects design

• Subjects get each treatment often enough to
  balance stage of practice effects for each
  subject.

24
Stage of Practice effects Remedies

• Incomplete within subjects design

• Subjects get each treatment only once.
• Levels of I.V. are confounded with order levels
  are presented in

25
Within subjects designs

• Remedies
• Complete within subjects design
• Block randomization
• ABBA counterbalancing
• Incomplete within subjects design

26
Complete Within Subjects Designs

• There are two approaches to arranging the order
  of treatments in a complete within subjects
  design.

• Block randomization
• ABBA counterbalancing

27
Block randomization

• Each block of trials contains one trial for each
  treatment.

• Number of blocks number of times each treatment
  is administered.
• Order of treatments randomized within a block
• Works better with many trials per treatment

28
ABBA counterbalancing

• In general, counterbalancing controls for
  practice effects by presenting the treatments in
  multiple sequences

• ABBA Counter-balancing presents treatments in a
  sequence, then presents them in the reverse
  sequence.
• Repeat as often as needed to generate desired
  amount of data per treatment

29
ABBA counterbalancing

• Can be used with any of treatments and repeated
  any of times within an experiment

• For 3 treatments, use ABCCBA, etc.
• Must repeat whole sequence, not just a part of it

30
ABBA counterbalancing

• Anticipation effects may be a problem, especially
  if there are many cycles through the sequence.

• Works well when practice effects are linear.
• Does not work with non-linear practice effects.
  For non-linear effects, stabilize performance
  with practice trials before recording data

31
Trial RT Practice effect 1 550 -- 2 525 25 3
500 25 4 475 25 5 450 25 6 425 25 This
shows a linear practice effect increase in
speed is the same every trial. ABBA
counter-balancing works in this case.
32
Linear practice effect
RT
Trial
33
Trial RT Practice effect 1 550 -- 2 500 50 3
470 30 4 460 10 5 455 5 6 453
2 This shows a nonlinear practice effect
increase in speed is larger in the early trials.
ABBA counterbalancing is no help in this case.
34
Non-linear practice effect
RT
Trial
35
Within subjects designs

• Incomplete within subjects designs

• Definition
• All possible orders
• Selected orders
• Latin square
• Random starting order with rotation

36
Incomplete W.S. design definition

• Each subject gets each treatment once.
• Practice effects are balanced across subjects
  rather than within subjects.

• Levels of the I.V. are confounded with order of
  presentation within any subject
• Thus data for individual subjects are not
  interpretable

37
Incomplete W.S. design definition

• In this design

• Hypothesis is tested within subjects.
• Practice effects are controlled between groups of
  subjects.

38
Incomplete W.S. design

• General rule for these designs
• Each treatment condition must appear in each
  ordinal position of the sequence equally often.

• The techniques that follow vary in what
  additional counter-balancing effects they
  achieve, but all achieve this effect, so all
  produce interpretable data.

39
All possible orders

• Preferred incomplete W.S. design technique

• All treatments appear in each ordinal position
  equally often.
• Each treatment precedes follows every other one
  equally often at each ordinal position

40

• For 3 treatments (A, B, and C)
• treatment order
• Subj 1st 2nd 3rd
• 1 A B C
• 2 A C B
• 3 B A C
• 4 B C A
• 5 C A B
• 6 C B A

41
Selected orders

• We often have 5 or more treatments in one study.
• 5 treatments 120 possible orders.
• 6 treatments 720 possible orders.
• Too many subjects!

• When we have many treatments, we use selected
  orders.
• That is, from the set of all possible orders we
  use only a subset.

42
Selected orders Latin square

• Each treatment appears equally often at each
  ordinal position
• Each treatment precedes follow every other
  treatment exactly once

• Limited to experiments with an even number of
  treatments
• Procedures for creating Latin Squares appear in
  advanced texts.

43
Selected orders random starting order with
rotation

• Start with any order
• With each new subject, rotate each treatment one
  position to the left in the sequence

• each condition appears in each ordinal position
  an equal number of times
• but each condition precedes follows same
  conditions throughout
• advantages simplicity, applicability

44
Random starting order with rotation example
with four treatments

• Subj Treatment order
• 1 D A C B
• 2 A C B D
• 3 C B D A
• 4 B D A C
• 5 D A C B

45
Limitations of W.S. designs

• W.S. designs cannot be used
• On subject variables such as age and sex.

• With unfolding sequences of successive events
  (for example, animal in Operation condition
  cannot also be in Anesthesia-only condition).
• If each treatment takes a long time (e.g., 1
  year).

46
Examples of within subjects designs

• Grice Hunter (1964)
• Kahneman et al. (1993)
• Lee Katz (1998)

47
Grice Hunter (1964)

• Classical conditioning study
• Two different intensities of sound as C.S.s
• In general, a more intense C.S. gives stronger
  classical conditioning

• G H found stronger effect of sound intensity in
  a within subjects version of the study than in a
  between groups version

48
Kahneman et al. (1993)

• Examined effects of three pain characteristics on
  the memory for pain.

• Duration of pain
• Worst moment
• Final moment

49
Kahneman et al. (1993)

• Condition A
• Subject keeps hand in 14 C water for 60 seconds

• Condition B
• Subject keeps hand in water for 90 seconds
• 60 seconds at 14 C plus 30 extra seconds during
  which temperature rises gradually to 15 C

50
Kahneman et al. (1993)

• One trial per condition
• Half of subjects got A first then B
• Half of subjects got B first then A

• 7 minute distracter task
• Subjects asked which condition they preferred to
  repeat
• 60 chose B

51
Kahneman et al. (1993)

• D.V. was choice of pain.

• You can only use this D.V. with a within-subjects
  design
• Subjects must get both conditions if they are to
  choose between them

52
Kahneman et al. (1993)

• You could do this experiment with a different
  D.V. say, pain ratings which would allow a
  between groups design

• But would groups be comparable?
• More sissies in one group than the other?

53
Lee Katz (1998)

• Study of figurative language
• Distinguished between irony and sarcasm
• Both figures involve saying something you know
  is not true

• Lee Katz sarcasm has a victim irony does not

54
Lee Katz (1998)

• Example
• What a sunny day

• Made on a rainy day irony
• Made on a rainy day to someone who predicted
  sunshine sarcasm

55
Lee Katz (1998)

• Manipulation
• Subjects read eight passages and rate each for
  sarcasm on a 7-point scale

• Two I.V.s manipulated within subjects prediction
  and victim identity
• Well look at victim identity today

56
Lee Katz (1998)

• Prediction
• A prediction made in the passage was either true
  or false
• E.g., prediction that it will be a sunny day

• Victim identity
• Either the speaker or the listener
• E.g., either the speaker or the listener had
  predicted sunshine

57
Lee Katz (1998)

• Speaker as victim
• Mean rating 4.90
• S.d. 1.34

• Listener as victim
• Mean rating 6.43
• S.d. 0.73

58
Lee Katz (1998)

• Same passage was rated as a better example of
  sarcasm when listener was the victim

• Why? Perhaps because people dont usually make
  sarcastic remarks about themselves

59
Lee Katz (1998)

• Subjects are expressing an opinion is a remark
  sarcastic?
• They may vary in sensitivity to sarcasm or the
  probability they would use sarcasm

• Comparing rated sarcasm for Speaker and Listener
  conditions between groups would let group
  differences on sensitivity or probability of use
  affect means