Mixed Design ANOVA GLM 5

1
Chapter_12

• Mixed Design ANOVA (GLM 5)?

2
What can you do with a Mixed ANOVA?

• In a Mixed ANOVA you can combine
• Between-subjects variables and
• Within-subjects variables
• Although you can choose any number of independent
  variables, bear in mind that you still want to be
  able to interpret interactions
• (It is hardly possible to interpret more than a
  three-way interactions, as you will see...)?

3
Gentle reminder variance in within-subject
designs

• In a repeated ANOVA, the experimental effect
  shows up in the within-subjects variance, between
  the conditions.
• The overall within-subjects variance is composed
  of the exp variance and error variance
• The exp variance derives from the different
  treatment subjects have incurred in the various
  within-subjects conditions.
• The error variance derives from random factors
  making subjects behave differently in the various
  conditions, apart from the exp effect
  (interaction!)

Compare in a between subject design, the
within- groups variance is the residual
variance SSR!
4
Gentle reminder variance in between-subject
designs

• In an between-subjects, independent ANOVA, the
  experimental effect shows up in the
  between-subjects variance, between the
  conditions.
• The overall between-subjects variance is composed
  of the exp variance and error variance
• The exp variance derives from the different
  treatment subjects have incurred in the various
  between-subjects conditions.
• The error variance derives from inter-individual
  differences between subjects, unrelated to the
  experimental manipulation.

5
Partitioning the variance for repeated ANOVA
SST total Sum of squares
SSW within participants
SSBG between groups/ subjects
SSR Error Variance
SSM Effect of Experiment (Model Variance)?
Model and residual variance both arise from
within subjects
6
Partitioning the variance for repeated ANOVA,
expl attitudes
SSM Model Sum of squares (within subj and
interactions with betw subj)?
3 factors A (within) looks B(within)
charisma C (between) gender
SSB Factor B charisma
SSA Factor A looks
SSBxC Interaction charismagender
SSAxC Interaction looksgender
SSAxBxC Interaction lookscharismagender
SSAxB Interaction lookscharisma?
7
Breaking down the Total sum of squares (SST) in a
between subj (indep) ANOVA
SST, total variance
SSR Unexplained variance
SSM Model variance expl gender (Factor C)?

• ? In a between subj design the model variance
  arises from the between subjects variance, here
  gender.
• ??In a?within subj design the model variance
  arises from the within subjects variance, here
  looks and charisma
• ?In a mixed design, the model variance arises
  from both sources plus their interaction(s)?

8
No further theory let's do an example

• Research Q How do male and female subjects rate
  persons of opposite sex who they date when these
  persons vary with respect to
• Personality high, some, no charisma
• Looks attractive, average, ugly?
• Sex of rater Between subj variable, 2 levels
• Personality Within-subj variable, 3 levels
• Looks Within-subj variable, 3 levels

9
Data from LooksOrPersonality.sav
10
Data entry for LooksOrPersonality.sav
For the between- subj variable 1 column
with dummy variables
For each within subj variable 1 column
Each subject 1 row
11
Analyze ? General Linear Model ? Repeated
Measures...
1st within factor looks number of levels 3 2nd
within factor charisma number of levels
3 Click add to transfer to window
After having defined the two factors and their
levels and having added them to the window, click
on Define
12
Define within-subjects variables
Thinking for the contrasts which is the
neutral level for each factor? Looks
average Charisma some

• For the 'Looks' factor, define 'average' as 3rd
  category,
• for the 'Charisma' factor, 'some'.

13
Rearranging the within-subjects variables

• 'av' is the reference category 3 for
  'attraction'
• 'att' is 1 'ugly' is 2
• 'some' is the reference category 3 for
  'charisma'
• 'high' is 1 'none' is 2.

14
Specify the levels of the 2 within-subj factors
according to the matrix and enter 'gender' as
between-subj variable

• Go on defining the contrasts

15
Contrasts
Since 'gender' has only 2 levels, actually, no
contrasts have to be specified

• Change all contrasts from 'polynomial' to
  'simple' for all factors.

16
No post hoc tests

• For the within-subj factors, no post hoc tests
  can be specified in the post hoc window
• For between-subj factors, post hoc tests can be
  defined in this window.
• However, since our factor 'gender' has only 2
  levels anyway, we do not have to specify anything
  here.

17
Plots
Transfer 'looks' to the horizontal axis and
'charisma' to the separate lines. SPSS will plot
an interaction graph where for each level
of 'looks' the values for each level of 'charism'
will be produced
Transfer 'looks' to the horizontal axis and
'charisma' to the separate lines. SPSS will plot
an interaction graph where for each level
of 'looks' the values for each level of 'charism'
will be produced
Drag 'gender' into the 'separate plots'
window. SPSS will produce 2 plots for the above
interaction, one for males, one for females.
After dragging the variables into their windows,
click 'add'
18
2- and 3-way interactions
After having dragged all variables into
their appropriate windows, the 'Plots'
window should look like that.
With 3 independent variables, there will be three
2-way interactions lookscharisma looksgender
charismagender as well as one 3-way interaction
(between all 3 var)? lookscharismagender
Plots for 2-way interactions can be requested as
well!
19
Options
Drag all main effects and interactions in the
'Display Means' window. Choose the indicated
options. If your version of SPSS has 'Estimated
marginal means for all effects', choose it.
Partial eta square ?p2

• Press 'continue' and then finally OK to run the
  analysis.

20
Output of Mixed ANOVA - Descriptives

• SPSS lists the 2 within-subj variables and the 1
  between-subj variable and their levels

21
Output of Mixed ANOVA - Descriptives

• Means, SD's, and N's for all 3 variables and
  their respective levels are shown

22
Sphericity Mauchley's test

• Mauchley's test is n.s. for both within-subj
  variables as well as for their interaction
• ? we can interpret the uncorrected F-values in
  the Main ANOVA

23
Homogeneity of variancesLevene's test

• None of Levene's tests is significant
• ? homogeneity of variances can be assumed

24
Main table Test of within-subj effects
All main effects, all 2-way interactions, as well
as the 3-way interaction are significant!
25
Tests of within-subjects contrasts
All contrasts are simple 1vs3 2vs3
26
Main effect of between-subj variable 'gender'
2,222E-02 means 0.022

• The main effect for 'gender' is not significant
• (its SS and MS are very small). That means that
  overall male and female raters gave the same
  judgements.

27
Main effect 'gender'
Double-click on the output table, highlight the
two mean values, make a right mouse click and
select 'Create graph'? bar.

• In the 'Estimates' table, the means for males and
  females are displayed. The plot is derived from
  these values.

28
Main effect 'looks'
Snippet of Tests of within-subj effects

• There is a main effect of 'looks' (F (2,36)
  423.73, p
• Irrespective of any other variable, raters rated
  attractive, average, and ugly dates differently .
• In the Estimates, you can see that ratings
  decrease as attractiveness decreases.

29
Plot for main effect 'looks'

• In the 'Estimates' table, the means for
  attractive, average, and ugly are displayed. The
  plot is derived from these values.
• As attractiveness goes down, ratings go down.
• Raters' desire to go out with a date depends
  significantly on the date's 'look'.

30
Contrasts for 'looks'
Snippet from big table 'Test of within- subjects
contrasts'

• In simple contrasts,
• attractive vs. average (1 vs. 3) and
• average vs. Ugly (2 vs. 3)?
• was compared. Both contrasts are significant.

Attractive dates (1) were rated significantly
higher than average dates (3), F (1,18) 226.99,
p significantly higher than ugly ones (2), F (1,18)
160.01, p
31
Main effect 'charisma'
Snippet from the Tests of Within Subjects
Effects

• There is a main effect of charisma, F (2,36)
  328.25, p
• Irrespective of the other variables, raters judge
  dates with high, some, or no charisma differently
  .

32
Plot for main effect 'charisma'

• In the 'Estimates' table, the means for high,
  some, and no charisma are displayed. The plot is
  derived from these values.
• As charisma goes down, ratings go down.
• Raters' desire to go out with a date thus depends
  significantly on the date's 'charisma'.

33
Contrasts for 'charisma'
Snippet of the 'Test of Within- Subj Contrasts'

• In simple contrasts,
• high vs. some (1 vs. 3) and
• no vs. some (2 vs. 3)?
• was compared. Both contrasts are significant.

Dates with high charisma (1) were rated
significantly higher than dates with some
charisma (3), F (1,18) 109.94, p dates with some charisma (3) were rated
significantly higher than dates with none (2), F
(1,18) 227.94, p
34
The interaction LooksGender
Snippet from the Test of Within-subject effects

• There is a significant interaction between gender
  and looks (F (2,36) 80.43, p means that male and female raters rated 'Looks'
  of a date differently.

35
Interactions
Note the following graphs have been created in
Excel with the original order of the
levels Looks Attr-average-ugly Charisma high,
some, dullard The level ordering in the
Estimates tables are different! Looks
Attr-ugly-average Charisma high, dullard,
some Remember that the rearrangement was
necessary for the simple contrasts contrasts
always had to be specified wrt to the last level
which happened to be the second in the original
data file
36
LooksGender
'Estimates' table, from which the graphics below
is derived
'Estimates' table, from which the graphics below
is derived

• The significant interaction looksgender is due
  to differences in ratings of male and female
  raters for attractive and ugly dates
• Male raters prefer more strongly to go out with
  attractive dates and disprefer more strongly to
  go out with ugly dates than female raters do.
  They rate average looking dates similarly.

37
1st Contrast of interaction level 1
(attractive) vs. Level 3 (average) for male and
female raters

• The contrast between attractive vs.
  Average-looking dates for male and female raters
  is significant (F (1,18) 43.26, p means that the increased interest in attractive
  dates as compared to average-looking dates is
  more pronounced for men than for women, hence the
  steeper decline in the graph for men.

Snippet from the Tests of within-subjects contrast
s
38
2nd Contrast of interaction level 2 (ugly) vs.
level 3 (average) for male and female raters

• The contrast between ugly vs. Average-looking
  dates for male and female raters is significant
  (F (1,18) 30.23, p women are less inclined not to go out with ugly
  dates as compared to average-looking dates than
  men, hence the crossing of lines compared to the
  first contrast.

Snippet from the Tests of within-subjects contrast
s
39
There is a significant interaction
Charismagender (F (2,36) 62.45, p This means that men and women differ in their
ratings of charisma.
The interaction CharismaGender
Snippet from the Test of Within-subject effects
40
CharismaGender
Estimates table. The plot is derived from these
numbers
Estimates table. The plot is derived from these
numbers

• Ignoring the looks of the date, men and women
  rate dates differently with respect to Charisma.
• Men's willingness to go out with a date depending
  on charisma is not as different for various
  levels of charisma (high, some, dullard) as it is
  for women who show a much steeper graph.

Bug note Fig. 12.10 (p 503 in Fields, 2005) is
based on a wrong number for male/high charisma!
41
1st contrast for the charismagender interaction
high vs. Some charisma, male vs. female
Snippet from the Test of within- subjects
contrasts

• The 1st contrast for the charismagender
  interaction comparing high vs. Some charisma for
  men and women is significant (F (1,18) 27.2, p
  charisma equally, women rate highly charismatic
  dates higher than men.

42
2nd contrast for the charismagender interaction
some charisma vs. dullard, male vs. female
Snippet from the Test of within- subjects
contrasts

• The 2nd contrast for the charismagender
  interaction comparing some charisma vs. dullard
  for men and women is significant (F (1,18)
  33.69, p with some charisma equally, men rate dullards
  higher than women.

43
Interaction between lookscharisma
Snippet from the Test of Within- subjects Effects

• There is a significant interaction between
  lookscharisma (F (4,72) 36.63, p means that irrespective of gender of the raters,
  the profile of ratings across dates of different
  levels of charisma was different for attractive,
  average, and ugly dates.

44
Interaction lookscharisma
Estimates table. The plot is derived from these
numbers
Estimates table. The plot is derived from these
numbers

• Ignoring gender of the rater, rating of charisma
  is dependent on looks.
• For dates with some charisma (red line) the
  rating declines equally with decreasing levels of
  looks .

For dates with high charisma (blue line), levels
of looks make little difference, esp. no
difference between high and average looks. Esp.
ugly dates are still rated quite highly. For
dullards, attractiveness raises ratings whereas
average and ugly dates are rated equally low.
45
1st contrast attractive vs. average, high vs.
some charisma
This contrast is (F (1,18), 21.94, p

'Is the difference between high and some charisma
the same for attractive vs. average-looking
people?

If you are very attractive, having high or some
charisma results in the same high rating

If you are average-looking and have high
charisma, this is as good as being very
attractive

If you are average-looking and have only some
charisma the ratings decrease.

The lines are not parallel ? interaction!
46
2nd contrast attractive vs. average, dullard
vs. some charisma
This contrast is n.s. (F (1,18), 4.09, p

Is the difference between dullard and some
charisma the same for attractive vs.
average-looking people?

Yes, for attractive and average-looking dates
alike ratings drop alike when charisma drops from
some C to being a dullard.

The lines are parallel ? no interaction!
47
3rd contrast ugly vs. average, high vs. some
charisma

• 'Is the difference between high and some charisma
  the same for ugly vs. average-looking people?
• If you are average-looking having high charisma
  gives you higher ratings than having only some
  charisma
• If you are ugly, having only some charisma is
  worse than having some charisma when you are
  average-looking (slightly steeper decline)?

This contrast is . (F (1,18), 6.23, p
The lines are not really parallel ?
interaction! (although hard to see!)?
48
4th contrast ugly vs. average, some charisma
vs. dullard

• 'Is the difference between having some charisma
  and being a dullard the same for ugly vs.
  average-looking people?
• If you are average-looking having some charisma
  gives you higher ratings than being a dullard.
• If you are ugly, it does not help you having at
  least some charisma it is as worse as being an
  ugly dullard.

This contrast is . (F (1,18), 88.6, p
The lines are not parallel at all ?
interaction!
49
The 3-way interactiongenderlookscharisma

• The 3-way ANOVA tells us whether the above
  interaction between lookscharisma is the same
  for male and female raters. It is significant
• (F (4,72) 24.12, p

Snippet from the Test of Within-subjects Effects
50
genderlookscharisma
Estimates table, from which the two plots are
derived
Estimates table, from which the two plots are
derived
51
genderlookscharisma

• Men
• If dates are attractive, level of charisma does
  not matter
• If dates are average good-looking, charisma
  matters it boosts ratings for high C and reduces
  ratings for low C
• If dates are ugly, charisma can't help

• Women
• If dates are attractive, level of C matters high
  and some C boost ratings, whereas being a dullard
  greatly reduces it
• If dates are average good-looking, C matters it
  boosts ratings for high C and reduces ratings for
  low C
• If dates are ugly, high C can fully compensate
  for this lack. Some and low levels of C however
  reduce ratings.

52
Contrasts in the 3-way interactions

• If you look at the contrasts in the three-way
  interaction 'looks x charisma x gender' you judge
  whether two pairs of graphs (for levels of 'looks
  x charisma') differ between the two gender.
• So you look whether the pattern of these graph
  pairs look alike.

53
genderlookscharisma, contrast 1attractive vs
average, high vs. some C
Male and female raters alike make no
difference between high and some C when the date
is attractive or average.
Both pairs of lines are highly similar (F (1,18)
.928, p .348, n.s)? No interaction!

• Men and women judge attractive and
  average-looking dates with high C equally high.
  For average-looking dates, when they have only
  some C, ratings decline

54
genderlookscharisma, contrast 2attractive vs
average, dullard vs. some C
Male and female raters differ in their
judgements between dullards and some C when the
date is attractive or average.
Both pairs of lines are dissimilar (F (1,18)
60.67, p

Men jugde dullards and dates with some C equally
if only they are attractive.

Women strongly disprefer dullards as compared to
dates with some C.

55
genderlookscharisma, contrast 3ugly vs
average, high vs. some C
Male and female raters differ in their
judgements between high and some C when the date
is ugly vs. average.
Both pairs of lines are dissimilar (F (1,18)
11.7, p

For men looks matters as much for High and some
C. Ratings drop for ugly dates.

Women jugde average and ugly looking dates
equally when they have high C. For some C,
attractive dates are rated higher than ugly ones

56
genderlookscharisma, contrast 4ugly vs
average, dullard vs. some C
Male and female raters have similar
judgements for dullard vs some C when the date is
ugly vs. average.
Both pairs of lines are similar (F (1,18)
1.33, p

For men and women alike, being a dullard or
having some C does not matter when the date is
ugly they are judged very low in any case

Men and women also agree in that having some C
boosts ratings for average-looking dates.

57
Summary genderlookscharisma
What's the difference between men and women's
rating of a date?

• Men and women judge dates alike when they are
  average good-looking they prefer high C,
  disprefer dullards, and rate some C somewhere in
  the middle.
• However, men and woman are the opposite of each
  other wrt to the overall factor that influences
  their rating most
• Men are enthusiastic about attractive dates
  irrespective of their personality
• Women are almost the mirror-image they are
  enthusiastic about high charisma irrespective
  of looks.

58
Conclusions

• The various contrasts did not exhaust the full
  range of possible contrasts. The missing
  contrasts could be calculated using a different
  contrasting scheme.
• Interactions for 3 variables are very hard to
  interpret! A three-way interaction is at the
  upper limit of what we can understand on-line.
• Contrasts have to be carefully chosen.

59
Df's of the various F-statistics
Note since we entered the between subjects
variable 'gender' (p), the df's will always take
p into account, in the error df's, even when we
only look at the within-subjects variables q and
r!
p gender q looks r charisma
Left out here df's for the single contrasts
(all df18)?
60
Calculating effect sizes(only for the contrasts,
for comparisons of 2 groups, df (1,18))?
r ? F (1,dfR) F (1,dfR) dfR

• Main effect 'gender'
• rGender ? 0.005/0.005 18 .02

Tiny effect
61
Calculating effect sizes(only for the contrasts,
for comparisons of 2 groups, df (1,18))?

• Contrasts for 'looks'
• rAttractive vs. Average ?226.99/226.9918
  .96
• rUgly vs. Average ?160.07/160.0718 .95

huge effects
62
Calculating effect sizes

• Contrasts for Interaction LooksGender
• rAttractive vs. Agerage, Male vs. Female
• ?43.26/43.26 18 .84
• rUgly vs. Average, Male vs. Female
• 30.23/30.23 18 .79

big effects
63
Calculating effect sizes

• Contrasts for 'Charisma'
• rHigh vs. Some ?109.94/109.94 18 .93
• rDullard vs. Some ?227.94/227.94 18 .96

Huge effects
64
Calculating effect sizes

• Contrasts for Interaction 'Charismagender'
• rHigh vs. Some, Male vs. Female
• ?27.2/27.2 18 .78
• rUgly vs. Average, Male vs. Female
• ?33.69/33.69 18 .81

big effects
65
Calculating effect sizes

• Contrasts for Interaction 'LooksCharisma'
• rAttractive vs. Average, High vs. Some
• ?21.94/21.94 18 .74
• rAttractive vs. Average, Dullard vs. some
• ?4.09/4.09 18 .43
• rUgly vs. Average, High vs. Some
• ?6.23/6.23 18 .51
• rUgly vs. Average, Dullard vs. some
• ?88.6/88.6 18 .91

Medium to large effects
All large effects
66
Calculating effect sizes

• Contrasts for 3-way interaction
  'GenderLooksCharisma'
• rAttractive vs. Average, High vs. Some, Male vs.
  Female
• ?.93/.93 18 .22
• rAttractive vs. Average, Dullard vs. Some, Male
  vs. Female
• ?60.67/60.67 18 .88

Small to medium to large effects
Small and large effects
67
Calculating effect sizes

• Contrasts for 3-way interaction
  'GenderLooksCharisma'
• rUgly vs. Average, High vs. Some, Male vs. Female
• ?11.7/11.7 18 .51
• rUgly vs. Average, Dullard vs. Some, Male vs.
  Female
• ?1.33/1.33 18 .26

Small to medium to large effects
Small and medium effects
68
Effect Size Measures in Analysis of Variance
'Measures of effect size in ANOVA are measures of
the degree of association between an effect
(e.g., a main effect, an interaction, a linear
contrast) and the dependent variable. They can be
thought of as the correlation between an effect
and the dependent variable. If the value of the
measure of association is squared it can be
interpreted as the proportion of variance in the
dependent variable that is attributable to each
effect. Two of the commonly used measures of
effect size in AVOVA are Eta squared,
?2 proportion of the total variance that is
attributed to an effect partial
Eta squared, ?p2proportion of the effect
error variance that is attributable
to the effect Eta squared and partial Eta
squared are estimates of the degree of
association for the sample. SPSS for Windows
displays the partial Eta squared ?p2 when you
check the "display effect size" option in GLM.'
(In the 'Options' Window)?
http//web.uccs.edu/lbecker/SPSS/glm_effectsize.ht
m
69
What SPSS computes Partial Eta squared, ?p2
Partial Eta squared, ?p2 The partial Eta squared
is the proportion of the effect error
variance that is attributable to the effect.
The formula differs from the Eta squared formula
in that the denominator includes the SSeffect
plus the SSerror rather than the SStotal ?2?
SSeffect / SS error ?p2 SSeffect / (SSeffect
SSerror)?
Quoted /adapted from http//web.uccs.edu/lbecker/S
PSS/glm_effectsize.htm
70
What SPSS computes Partial Eta squaredNote it
IS the partial eta squared, however, SPSS CALLS
it simply eta squared ?2
For the main effect 'Looks', the partial ?2 is
.948.?2 SSeffect / (SSeffect SSerror) ?2
23233.6/(23233.61274.044) .948
71
What SPSS computes Partial Eta squared, ?2
For the interaction 'Charisma x Gender', the
partial ?2 is .817.?2 SSeffect / (SSeffect
SSerror) ?2 3944.1/(3944.1882.711)
.817 Note in the interaction between a within
subjects factor ('charisma') and a between
subjects factor ('gender'), the error term is
taken from the within subjects factor
('charisma').
72
What SPSS computes Partial Eta squared, ?2
For the twoway within-subjects factor 'Looks x
Charisma' interaction, the partial ?2 is .671.?2
4055.267/(4055.267199.622) .671For the
threeway mixed subjects factor 'Looks x Charisma
x Gender' interaction the partial ?2 is .573
?2 2669.667/(2669.6671992.622) .573 In both
cases, the error term is taken from the 'looks x
charisma' interaction.
73
What SPSS computes Partial Eta squared, ?2

• E2 for the between subjects factor 'gender' is
  0.00
• ?2 0.022/(0.02284.47) 0.00

74
Effect Size Measures in Analysis of Variance
'Two further commonly used measures of effect
size in AVOVA are omega squared, ?2
the Intraclass correlation, (??) Omega
squared and the intraclass correlation are
estimates of the degree of association in the
population.'
Quoted and/or adapted from http//web.uccs.edu/lb
ecker/SPSS/glm_effectsize.htm
75
Effect Size Measures in Analysis of Variance
'Omega squared, ?2 Omega squared is an estimate
of the dependent variance accounted for by the
independent variable in the population for a
fixed effects model. The between-subjects, fixed
effects, form of the w2 formula is -- ??
SSeffect-(dfeffect)(MSerror)? -----------------
-------------------- MSerror SStotal (Note
Do not use this formula for repeated measures
designs)'
Quoted and/or adapted from http//web.uccs.edu/lb
ecker/Psy590/es.htm
76
Effect Size Measures in Analysis of Variance
'Because ?2 and ?p2 are sample estimates and ??
is a population estimate, ?? is always going to
be smaller than either ?? or ?p2. '
Quoted and/or adapted from http//web.uccs.edu/lb
ecker/Psy590/es.htm
77
Effect Size Measures in Analysis of Variance
'Intraclass correlation (?? )? The intraclass
correlation is an estimate of the degree of
association between the independent variable and
the dependent variable in the population for a
random effects model. Because it is for a random
effects model it is not commonly used in
psychology experiments. The formula for ?? is
-- ?? (MSeffect - MSerror) / (MSeffect
(dfeffect)(MSerror)) . The square of the
intraclass correlation is an estimate of the
amount of dependent variable variance accounted
for by the independent variable.'
Quoted/adapted from http//web.uccs.edu/lbecker/SP
SS/glm_effectsize.htm