Statistical Analysis of Factorial Designs

1
Statistical Analysis of Factorial Designs

• Review of Interactions
• Kinds of Factorial Designs
• Causal Interpretability of Factorial Designs
• The F-tests of a Factorial ANOVA
• Using LSD to describe the pattern of an
  interaction

2
Review 1-- interaction Task
Presentation Paper
Computer Task Difficulty Easy 90
70 Hard 40 60
1) Put in a lt, gt or to indicate the pattern of
the simple effect of Task Presentation for Easy
Tasks
gt
2) Put in a lt, gt or to indicate the pattern of
the simple effect of Task Presentation for Hard
Tasks
gt
3) Is there an interaction? Is the simple effect
of Task Presentation the same for Easy and for
Hard Tasks ???
4) Describe the pattern of the interaction...
3
Review 1-- 1st main effects Task
Presentation Paper
Computer Task Difficulty Easy 90
70 Hard 40
60
1) Compute the marginal means for Task
Presentation.
2) Put in a lt, gt or to indicate the pattern of
the main effect of Task Presentation
gt
gt
3) Is there a main effect of Task Presentation
???
65 65

4) Is the pattern of the main effect for Task
Presentation descriptive or potentially
misleading (is the pattern of the main effect or
Task Presentation the same as the patterns of the
simple main effects of Task Presentations for
both Easy and Hard Tasks) ?
5) Describe the main effect
4
Review 1-- 2nd main effects Task
Presentation Paper
Computer Task Difficulty Easy 90
70 Hard 40 60

1) Compute the marginal means for Task Difficulty
2) Put in a lt, gt or to indicate the pattern of
the main effect of Task Difficulty
80 50
gt
gt
gt
3) Is there a main effect of Task Difficulty ???
4) Is the pattern of the main effect of Task
Difficulty descriptive or potentially misleading
(is the pattern of the main effect or Task
Difficulty the same as the patterns of the simple
main effects of Task Difficulty for both Computer
and Paper Presentations) ?
5) Describe the main effect
5

• Kinds of 2-factor Designs
• BETWEEN GROUPS FACTORIAL DESIGN
• each IV uses a between groups comparison
• each participant completes only one condition of
  the design
• WITHIN-GROUPS FACTORIAL DESIGN
• each IV uses a within-groups comparison
• each participant completes all conditions of the
  design
• MIXED FACTORIAL DESIGN
• one IVs uses a between groups comparison and the
  other IV uses a within-groups comparison.
• each participant completes both conditions of
  the within- groups IV, but completes only one
  condition of the between groups IV.
• it is important to specify which IV is BG and
  which is WG

6
Between groups factorial design ? experimental or
natural grps designs used to
study differences Each participant is in only
one condition, having a particular combination
of Initial Diagnosis and Type of Treatment.

Type of Treatment Initial Diagnosis
Individual
Group
Therapy Therapy
Clients diagnosed
Clients diagnosed Depression as depressed
who as depressed who
are treated with are
treated with individual
therapy group therapy
Clients diagnosed
Clients diagnosed Social Anxiety with
social anxiety with social anxiety
who are treated with
who are treated with
individual therapy group therapy
7
Mixed group factorial design ? natural groups
designs used to study different
changes or changing differences Species
was a between groups IV (a turtle can only be a
member of one species). Each turtle
participated in both the mid-morning dusk
conditions of the Time of Day IV.
Species of
Turtle Time of Day
Snapping Turtle Painted Turtle
Each snapping turtle Each
painted turtle Mid-morning completed a trial
completed a trial
during mid-morning during mid-morning
Each snapping turtle
Each painted turtle Evening completed
a trial completed a trial
during the evening during the
evening
8
Mixed group factorial design ? experimental
designs used to increase data
collection efficiency or statistical
power Type of Evidence was a between groups IV
-- people cant read the same study twice
give independent ratings. Each participant rated
the guilt of both Defendants -- a within-groups
IV --as they would in this type of case.
Type of
Evidence Defendant
DNA Eye Witness
Major Actor DNA evidence
was An eye witness testified
presented against to seeing the
major the major actor
commit the crime Conspirator DNA
evidence was An eye witness testified
presented against to seeing
the conspirator the conspirator
commit the crime
9
Within-groups factorial design experimental
designs used to increase data
collection efficiency or statistical
power Each participant completed four trials,
one of each combination of Retention Interval
and Word Type.
Retention Interval Word Type
Immediate Test Delayed Test
The test was given The test was
given Familiar immediately after the 5
minutes after the study of a
list of study of a list of
40 familiar words. 40 familiar words.
The test was given The test was
given Unfamiliar immediately after the 5
minutes after the study of a
list of study of a list of
40 unfamiliar words. 40 unfamiliar words.
10
Within-groups factorial design natural designs
to study changing changes Each
participant was observed in both School Home
(WG) settings both when they were 12 16 (WG)
Age Setting
12 years old
16 years old School
Participants were Participants
were observed in a school
observed in a school setting
at age 12. setting at age 16. Home
Participants were Participants
were observed in a home
observed in a home setting at
age 12. setting at age 16.
11
Practice Identifying Types of Factorial Designs -
answers next page The purpose of the study was
to examine the possible influence of two
variables upon maze-learning by rats, length of
the maze (either 10 feet or 30 feet) and the size
of the reward (either 1sugar pellet or 5 sugar
pellets). Here are three versions of the
study tell which is BG, WG MG a. Each rat
completed one trial. Each was assigned to
either the longer or the shorter maze, and also
assigned to receive either 1 or 5 sugar pellets
upon completing the maze. b. Each rat completed
two trials in either the longer or the shorter
maze. Following one trial in the assigned maze,
each received 1 pellet reward, after the other
trial they received the 5 pellets. c. Each rat
completed four trials, two in the shorter maze
and two in the longer maze. Each received 1
pellet after one of the short-maze trials and 5
pellets after the other, and also 1 pellet after
one of the long-maze trials and 5 pellets after
the other.
BG
MG
WG
12
Another Example -- 3 versions of the same
study The researcher wanted to investigate
infant's startle responses to loud sounds. The
two variables of interest were the Position of
the Sound (in front of versus behind the infant)
and the Type of Sound (a hand-clap versus deep
male voice saying "Hey"). Here are three
versions of the study tell which is BG, WG
MG a. Each infant completed trials all involving
a hand-clap or all involving the voice saying
"Hey". During some of the trials, the
appropriate type of sound was made in front of
the infant. During other trials, the
appropriate type of sound was made behind the
infant. b. Each infant had some trials during
which the sound was made in front of then and
some during which the sound was made behind them.
Some of the sounds were the hand-clap and the
others were the voice saying "Hey". c. Each
infant always heard either the hand-clap or the
Hey, and whatever sound they heard was always
played either in front of them or behind them.
MG
WG
BG
13
Remember about the causal interpretation of
effects of a factorial design Start by
assessing the causal interpretability of each
main effect Remember, in order to causally
interpret an interaction, you must be able to
casually interpret BOTH main effects. For each of
the following Tell the IVs and tell what effects
could be causally interpreted (assuming proper
RA, IV manip. and confound control were used)
1. Male and female participants who were African
American, Mexican American, or European American
were asked to complete a questionnaire about
satisfaction with their Senators. 2. Children
played with either a toy gun, a toy car or a
puzzle, some while their parents were in the room
and some not. The DV was the amount of
aggressive behavior they exhibited. 3.
Participants played with either a simple puzzle
or a complex puzzle in pairs made up of two boys,
two girls or one boy one girl.
zilcho-causo
All three !
Puzzle type only.
14

• Statistical Analysis of 2x2 Factorial Designs
• Like a description of the results based upon
  inspection of the means, formal statistical
  analyses of factorial designs has five basic
  steps
• 1. Tell IVs and DV 2. Present data in
  table or figure
• 3. Determine if the interaction is significant
• if it is, describe it in terms of one of the
  sets of simple effects.
• 4. Determine whether or not the first main
  effect is significant
• if it is, describe it
• determine if that main effect is descriptive or
  misleading
• 5. Determine whether or not the second main
  effect is significant
• if it is, describe it
• determine if that main effect is descriptive or
  misleading

15

• Interpreting Factorial Effects
• Important things to remember
• main effects and the interaction are 3 separate
  effects each must be separately interpreted --
  three parts to the story
• most common error -- interaction is different
  main effects
• best thing -- be sure to carefully separate the
  three parts of the story and tell each
  completely
• Be careful of causal words when interpreting
  main effects and interactions (only use when
  really appropriate).
• caused, effected influenced, produced, changed
  .
• Consider more than the significance
• consider effect sizes, confidence intervals,
  etc. when describing the results

16
Statistical Analysis of a 2x2 Design
Task Presentation (a) SE of
Presentation Paper
Computer for Easy Tasks Task
Difficulty (b) Easy 90
70 80 Hard 40
60 50 65 65
SE for Presentation for Hard
Tasks Presentation Difficulty
Interaction Main Effect
Main Effect Effect
SSPresentation SSDificulty
SSInteraction 65 vs. 65 80
vs. 50 SEEasy vs. SEHard
17
Constructing F-tests for a 2x2 Factorial
FPresentation ( SSPresentation /
dfPresentation )
( SSError / dfError) FDifficulty
( SSDifficulty / dfDifficulty )
( SSError / dfError ) FInteraction
( SSInteraction / dfInteraction )
( SSError / dfError)
18
Statistical Analyses Necessary to Describe Main
Effects of a 2x2 Design

• In a 2x2 Design, the Main effects F-tests are
  sufficient to tell us about the relationship of
  each IV to the DV
• since each main effect involves the comparison
  of two marginal means -- the corresponding
  significance test tells us what we need to
  know
• whether or not those two marginal means are
  significantly different
• Dont forget to examine the means to see if a
  significant difference is in the hypothesized
  direction !!!

19
Statistical Analyses Necessary to Describe the
Interaction of a 2x2 Design

• However, the F-test of the interaction only tells
  us whether or not there is a statistically
  significant interaction
• it does not tell use the pattern of that
  interaction
• to determine the pattern of the interaction we
  have to compare the simple effects
• to describe each simple effect, we must be able
  to compare the cell means
• we need to know how much of a cell mean
  difference is statistically significant

20
Using LSD to Compare cell means to describe the
simple effects of a 2x2 Factorial design

• LSD can be used to determine how large of a cell
  mean difference is required to treat it as a
  statistically significant mean difference
• Will need to know three values to use the
  computator
• dferror -- look on the printout or use N 4
• MSerror look on the printout
• n N / 4 -- use the decimal value do not
  round to the nearest whole
  number!

Remember only use the lsdmmd to compare cell
means. Marginal means are compared using the man
effect F-tests.
21
Applying lsdmmd to 2x2 BG ANOVA
Task Presentation
Paper Computer Task Difficulty
for the interaction Easy 60
90 F(1,56) 6.5, p .023 Hard
60 70 lsdmmd
14 Is there a Task Difficulty main effect?
Based on what? for the following, tell the
mean difference and apply the lsdmmd
Yes! F-test of Int
30 gt
Simple effect of Task Presentation SE of
Task Presentation for Easy Tasks SE of Task
Presentation for Hard Tasks Simple effects of
Task Difficulty SE of Task Difficulty for Paper
Pres. SE of Task Difficulty for Comp. Pres.
10
0

20
gt
22
Applying lsdmmd to 2x2 BG ANOVA
Task Presentation
Paper Computer Task Difficulty
for Difficulty ME Easy 60
90 75 F(1,56) 4.5, p .041
Hard 60 70 65
lsdmmd 14 Is there a Task Difficulty
main effect? Based on what? Is main effect
descriptive (unconditional) or potentially
misleading (conditional)?
Yes! F-test of ME
Simple effects of Task Difficulty SE of Task
Difficulty for Paper Pres. SE of Task
Difficulty for Comp. Pres.
0

20
gt
Descriptive only for Computer presentation
misleading for Paper presentations.
23
Applying lsdmmd to 2x2 BG ANOVA
Task Presentation
Paper Computer Task Difficulty
for Presentation ME Easy 60
90 F(1,56) 7.2, p .011 Hard
60 70 lsdmmd 14
60
80 Is there a Task Difficulty main effect?
Based on what? Is main effect descriptive
(unconditional) or potentially misleading
(conditional)?
Yes! F-test of ME
Simple effects of Task Difficulty SE
of Task Presentation for Easy Tasks
SE of Task Presentation for Hard Tasks
30 lt
10
Descriptive only for Easy tasks misleading for
Difficult tasks.
24
Effect Sizes for 2x2 BG Factorial designs For
Main Effects Interaction (each w/ df1) r
? F / (F dferror) For Main Effects Simple
Effects d (M1 - M2 ) / ? Mserror d²
r ----------
? d² 4 (This is an
approximation formula)