Computational shortcuts and additional post hoc tests

1
Chapter 13

• Computational shortcuts and additional post hoc
  tests

2
Calculating Fishers test

• Apply ANOVA
• Test pairs from the largest to the smallest
• Why?
• Use table A2, to find critical t
• Degrees of freedom dfW
• Compute t
• Compare t to tcrit

3
ShortcutsLSD

• These shortcuts only apply to Fisher when ni are
  all equal.
• They can always be used with Tukey.
• Key the denominator is the same for all pairs of
  means.
• Goal Find the smallest difference of means which
  is significant.
• This difference is called the least significant
  difference or LSD.

4
ShortcutsLSD

• If the numerator is the smallest difference of
  means that gives statistical significance
• Then, by definition, t becomes what special t?

5
ShortcutsLSD

• Answer tcrit
• Our next trick is to solve for LSD and use it as
  a critical test statistic! (multiply both sides
  by the square root expression).

6
ShortcutsLSD

• Now all we have to do is compare all the pairwise
  differences of means to the LSD.
• If Xi - Xj gt LSD the difference is
  significant.
• The only purpose in using the LSD approach is
  that one does not need to repeatedly compute the
  denominator.

7
LSDExample

• Using the colored cookie data from last lecture

8
LSDExample

• Running an ANOVA gives us the following MSW and
  differences of means.
• MSW6.367
• Xgreen - Xred 3.7-4.7 1
• Xgreen - Xblue 3.7-2.8 .9
• Xred - Xblue 4.7-2.8 1.9
• n 6

9
LSDExample

• Compute the LSD as follows
• n6
• Look up tcrit for dfW (666)-3 15,
• ? .05, two tailed in table A2.
• tcrit 2.131
• Plug and chug.

10
LSDExample

• Comparing the LSD to the differences of means,
  one sees that none are significantly different.
• Xgreen - Xred 3.7-4.7 1
• Xgreen - Xblue 3.7-2.8 .9
• Xred - Xblue 4.7-2.8 1.9

11
ShortcutsHSD

• Since Tukey is used with a single n, either n all
  equal or the harmonic mean of the nis,
• we can always use this shortcut.
• Things will go pretty much the same way as with
  Fisher.
• Key the denominator is the same for all pairs of
  means.
• Goal Find the smallest difference of means which
  is significant.
• This difference is called the honestly
  significant difference or HSD.

12
ShortcutsHSD

• If the numerator is the smallest difference of
  means that gives statistical significance
• Then, by definition, q becomes what special q?

13
ShortcutsHSD

• Answer qcrit
• Our next trick is to solve for HSD and use it as
  a critical test statistic! (multiply both sides
  by the square root expression).

14
ShortcutsHSD

• Now all we have to do is compare all the pairwise
  differences of means to the HSD.
• If Xi - Xj gt HSD the difference is
  significant.
• The only purpose in using the HSD approach is
  that one does not need to repeatedly compute the
  denominator.

15
HSDExample

• Using the colored cookie data from last lecture

16
HSDExample

• Running an ANOVA gives us the following MSW and
  differences of means.
• MSW6.367
• Xgreen - Xred 3.7-4.7 1
• Xgreen - Xblue 3.7-2.8 .9
• Xred - Xblue 4.7-2.8 1.9
• n 6

17
HSDExample

• Compute the HSD as follows
• n6
• Look up qcrit for k3 dfW (666)-3 15,
• in table A11.
• qcrit 3.67
• Plug and chug.

18
HSDExample

• Comparing the HSD to the differences of means,
  one sees that none are significantly different.
• One can also see that Tukey has set the bar
  higher then Fisher (Tukey is more conservative).
• Xgreen - Xred 3.7-4.7 1
• Xgreen - Xblue 3.7-2.8 .9
• Xred - Xblue 4.7-2.8 1.9

19
Other post hoc tests

• Newman-Keuls
• A compromise between the liberal Fisher and the
  conservative Tukey
• A trade off
• Nothing gained overall
• Duncans
• Like fisher
• Too liberal in accepting HA for most k
• Dunnetts
• Useful for complex comparisons,
• where the groups are classified into categories.

20
Other post hoc tests

• Scheffes
• Also used in complex comparisons.
• Bonferrononis (or Dunns)
• Very conservative unless,
• used with planned comparisons.

21
Other post hoc tests

• REGWQ
• More power than Tukey
• No increased probability of type I error
• Difficult to create tables for (not a problem
  with SPSS)
• Overall a very good choice when doing
  computerized post hocs.

22
Post hoc testsBottom line

• Assuming you are not doing planned comparisons
  (planning certain pair-wise tests before doing
  your ANOVA), or complex comparisons.
• If you have homogeneity of variance( Levene sig gt
  .05 )
• If you are computing with calculator and tables
• And if k3
• Use Fisher
• Else if kgt3 and group sizes are similar
• Use Tukey
• If you are using a computer and group sizes are
  similar
• Use REGWQ

23
Post hoc testsBottom line

• If you do not have homogeneity of variance
• Use Dunnett C

24
Doing post hoc tests with SPSS

• Use divorce.sav from the George data sets.
• Analyze-gtCompare Means-gtOne-Way ANOVA
• Current marital status as factor
• Age in Dependent List
• Click Post hoc button
• Select LSD, Tukey, REGWQ. Dunnett C
• Continue

25
Doing post hoc tests with SPSS

• Click options.
• Analyze-gtCompare Means-gtOne-Way ANOVA
• Select Homogeneity of variance test and Welch
• Continue
• OK

26
Doing post hoc tests with SPSSInterpreting output

• Note that Sig. for the Levene Statistic is gt .05,
  so the variances are not significantly different
  (this is good).
• F is significant, indicating that people with
  different marital statuses tend to have different
  ages.
• Welch is significant but we didnt need it.

27
Doing post hoc tests with SPSSInterpreting output

• Note that the output for post hocs are different.
• According to Tukey there are two significantly
  different pairs.
• Married - separated
• Married - cohabitation

28
Doing post hoc tests with SPSSInterpreting output

• According to Fisher LSD there are four
  significantly different pairs.
• Married - separated
• Married - cohabitation
• Married - divorced
• Divorced - cohabitating
• Fisher is more liberal as expected.
• We didnt need Dunnett on the basis of lacking
  homogeneity.
• However, it gives one significantly different
  pair
• Married - cohabitation
• Notice that Sigs are not given for Dunnett, but
  indicates that significance is reached.

29
Doing post hoc tests with SPSSInterpreting output

• REGWQ results are in the Homogeneous Subsets
  table.
• Homogeneous subsets are groups of groups that are
  the same.
• Note that the only two groups that do not share a
  subset are the married - cohabitation.
• This means that married and cohabitating groups
  are the only distinct groups.
• Thus, Tukey appears to be more liberal than
  REGWQ.
• This is not what we expected.
• REGWQ is supposed to have more power to detect
  effects.
• How to explain this?
• Which of these 4 tests do you believe and why?

30
Doing post hoc tests with SPSSInterpreting output

• Always look at the means.
• Who do you expect to have a larger mean age,
  married or cohabitating?
• Why?
• Does the data agree?

31
Exercises

• Page 382
• 1,2,3,5,8
• Page 348, use SPSS to perform post hocs (Fisher,
  Tukey, REGWQ) for data in
• 4, 6, 7, 8
• In each case, which post hoc do you believe most,
  and why?