Action Research FTests and ANOVA

1
Action ResearchF-Tests and ANOVA

• INFO 515
• Glenn Booker

2
F-Tests and ANOVA

• Analysis of variance (ANOVA) allows comparison of
  several means in one test
• Strong assumptions about the nature of the data
  are made
• The groups being compared are assumed to be
  random samples from normal populations with the
  same variance
• A few ANOVA tests can handle differing variances,
  but most cant

3
F-Tests and ANOVA

• The groups are from the independent variables
  values we wish to compare
• An estimate of the variance between groups is
  compared to an estimate of the variance within
  groups this is the core concept of Analysis of
  variance (ANOVA)
• This is also called an F test, since thats the
  name of the main output variable

4
One-Way ANOVA

• Compares the means of two or more unrelated
  (independent) samples
• The null hypothesis in n-way analysis of variance
  (ANOVA) is that the means of the n groups are
  equal (do not differ) in the population

5
F Test

• This is like extending the Independent T-test to
  more than two groups or variables
• ANOVA is comparing multiple means to see if
  theyre statistically homogeneous
• This example will look at a set of data on
  political views, and determine if there is 1)
  differences in age, and later 2) a trend for
  people to become more conservative as they get
  older

6
F Test Example

• The data set is GSS91 political.sav
• The variables of interest are age and
  polviews (political views)
• So were seeing if the mean age (variable)
  differs by political view (group)
• Use the command Analyze / Compare Means / One-Way
  ANOVA /
• The age is Dependent, and partyid is the
  Factor (group)

7
F Test Example
8
F Test Example

• Use the Options button to show Descriptive
  statistics and Homogeneity-of-variance test
• The Descriptives output are fairly self
  explanatory
• The Age of Respondent is the dependent variable
  whose variance being tested for each of the
  political party groups
• Mean, std dev, std error, etc. are found for each
  group, as well as for the Total sample

9
F Test Example
10
F Test Example

• The Levene statistic looks for homogeneity of
  variances across the data set
• As seen earlier, the Levene statistic is a test
  of the null hypothesis that the variances are
  homogeneous
• In this case, its significance passes the 0.050
  test at 0.000, hence the prerequisite assumption
  of equal variances is NOT justified
• Here, this is bad!

11
F Test Example

• Should use ANOVA only when Levene test supports
  its null hypothesis, namely Sig. gt the critical
  value, e.g. 0.050
• We should be cautious about using ANOVA results
  when the prerequisite condition has not been
  met, such as here
• To fix this, could investigate transforming the
  data into something with homogeneous variances,
  or only use tests that allow for unequal
  variances

12
F Test Example

• The values for df (next slide) are the number of
  degrees of freedom in ANOVA
• df Between groups is the number of groups (k)
  minus 1 dfbg k - 1 8 - 1 7
• df Within Groups is the sample size (n) minus
  k dfwg n - k 1492 - 8 1484
• Notice that n is 1492, even though there are
  1500 data rows, due to missing values

13
F Test Example
14
F Test Example

• The F statistic isF ( Between groups sums of
  squares / df1) / (Within groups sums of squares /
  df2 )
• For the example, F (14214/7) / (457058/1484)
  6.593
• The significance (Sig.) below 0.050 shows that
  the null hypothesis is rejected
• Hence there is a significant difference in means
• The critical F value is a function of df1 and df2

15
Between-Groups Sum of Squares

• For the mathematically curious
• Between-Groups Sum of Squares equals
• (the mean of each group minus the overall mean
  for all groups) squared, multiplied by the number
  of cases in the group summed over all
  groupsBGSS S (Ni(Xi-m)2) i

16
Within-Groups Sum of Squares

• Within-Groups Sum of Squares equals
• The group variance times (the number of cases in
  the group minus 1) summed over all groups WGSS
  S ( s2i(Ni-1) ) i
• Between groups variance measures the
  variables effect we want to study, whereas the
  within groups variance is the sampling error
  within the groups

17
F Test Example

• Conclusion since the null hypothesis was
  rejected, we conclude that there is a
  significant difference in political view by age
• Notice the recurring pattern for statistical
  tests if the Significance of the test result is
  below the critical value (e.g. 0.050), the null
  hypothesis is rejected
• Again, caution in this particular result since
  the Levene test had Sig. lt 0.050

18
Additional ANOVA Methods

• ANOVA can show that several variances were not
  equal - but tells us nothing more
• Now use additional ANOVA methods to determine
  which variances are unequal and look for trends

19
Types of ANOVA Output

• One-way ANOVA can produce two other major types
  of output
• Pairwise multiple comparisons examine subset
  pairs and determine if there is a significant
  difference between them
• Post hoc range tests look for groups of subsets
  which are statistically similar

20
Pairwise Multiple Comparisons

• Pairwise multiple comparisons identify which
  subsets are homogeneous - not different from each
  other
• Well try two common methods to determine if
  pairs of data are homogeneous

21
Pairwise Multiple Comparisons

• The Bonferroni method is better for a small
  number of comparisons (groups)
• The Tukey method is better for many comparisons
• Both Bonferroni and Tukey tests assume group
  variances are equal
• So check your Levene test results!

22
Pairwise Multiple Comparisons

• Start with the data set GSS91 political.sav
• Use Analyze / Compare Means / One-Way ANOVA... /
• Set age as the Dependent List
• Set partyid as the Factor...but wait!

23
Pairwise Multiple Comparisons

• Click on Post Hoc and under Equal Variances
  Assumed area, select the Bonferroni and Tukey
  tests
• Leave the significance level at 0.050
• Now generate the output
• The first three parts should look familiar
  descriptives, test homogeneity of variances, and
  the ANOVA results

24
Basic ANOVA Output

• Once again, the significance of F below 0.050
  means that we can reject the null hypothesis,
  which means that
• There is a difference in political views by age

25
Pairwise Multiple Comparisons

• Now look at the output from the Tukey and
  Bonferroni tests (see handout)
• Notice that every pair of views are tested, not
  just ones from adjacent political views
• And symmetric pairs are examined, like STRONG
  REPUBLICAN- STRONG DEMOCRAT and STRONG DEMOCRAT
  - STRONG REPUBLICAN
• Only difference is that the sign of the mean age
  difference, and sign of the confidence interval
  change

26
Pairwise Multiple Comparisons

• Look for test results which have Significance
  below the critical value (0.050) - means that the
  age difference is statistically significant
• Note that confidence interval does not include
  zero for significant results
• Tukey test is a.k.a. Tukeys honestly
  significant difference test, or Tukey HSD

27
Pairwise Multiple Comparisons

• Tukey results
• Notice that some very large average age
  differences (e.g. 9 years) are made insignificant
  by a large standard error
• Note that OTHER PARTY doesnt have any
  significant results, which is not surprising
  since its N is very low (14)

28
Pairwise Multiple Comparisons

• Bonferroni results
• Are very similar to Tukeys, but different
  Significance levels for one pair
• Could use results from multiple tests to see what
  conclusions they agree upon
• Either test shows what specific group means were
  significantly different from each other, and by
  how much

29
Post Hoc Range Tests

• The Tukey test also produces another output to
  identify Homogeneous Subsets - a post hoc range
  test
• Note that the groups are NOT listed in order by
  group they are listed in order of their
  ascending means (here, mean ages)
• Each column with Sig. lt 0.050 shows a subset of
  group means which do not differ significantly

30
Homogeneous Subsets
31
Post Hoc Range Tests

• Again apply the criterion of Sig. lt 0.050 to
  reject the null hypothesis of all equal means
• Only the second column is significant
• Shows that all of the views except IND,NEAR
  REP have statistically similar means
• Why have different results than the pairwise
  tests?

32
Pairwise vs. Post Hoc

• The pairwise multiple comparison test uses N
  based on the counts for the two groups being
  examined
• Whereas the post hoc range test uses N based on
  the harmonic mean of all eight groups
• The pairwise multiple comparison therefore uses a
  larger sample size, so its easier to detect
  significance

33
Harmonic Mean?

• The harmonic mean for N numbers Xi isH N / S
  (1/Xi) i
• So the harmonic mean of 3, 5, 8, and 9 isH 4 /
  (1/3 1/5 1/8 1/9) 5.20
• Our usual mean is the arithmetic mean, which
  for those values isX (3 5 8 9) / 4
  6.25
• Theres also a geometric mean, (PXi)1/N

34
Tukey B Test

• Another post hoc range test is the Tukey B test
  (also Tukeys-b in SPSS)
• This does not generate the detailed pairwise
  comparisons (yay!), and just presents a summary
  of like means, again organized by columns
• Note that this test is stricter than the plain
  Tukey test

35
Tukey B Test
36
Tukey B Test

• Notice that only patterns which meet the desired
  significance (Sig. lt 0.050) are presented
• Results are interpreted the same as Homogeneous
  Subsets
• Each column (1, 2, 3) is a set of groups which
  have similar means
• Now we have shown differences in ages among
  different political views

37
Contrasts Across Means

• Next, we might look for patterns across means,
  such as an increase or decrease in the measured
  variable across different groups
• For example, we could ask if people get more
  conservative linearly as they age

38
Contrasts Across Means

• Start with the data set GSS91 political.sav
• Use Analyze / Compare Means / One-Way ANOVA... /
• Set age as the Dependent List
• Set partyid as the Factor
• Click on the Contrasts button
• Check Polynomial and select Quadratic for
  Degree

39
Contrasts Across Means

• Notice that results are given for both a linear
  and a quadratic test across the means
• This test wont tell us what the relationship is
  mathematically (like Y 5X 12 for a linear
  relationship), but it will tell us if there is
  one

40
Contrasts Across Means
41
Contrasts Across Means

• The Weighted results are weighted by the number
  of cases in each group
• The Unweighted results consider each group
  equal
• Generally use the unweighted results, unless N
  varies among groups greatly
• Here, the Linear case is not significant
  (unweighted Sig. is over 0.050), but the
  Quadratic case is significant

42
Contrasts Across Means

• So we conclude from this test that there is a
  significant quadratic relationship across mean
  age by political view (something of the form Y
  aX2 bX c)
• But there is NOT a significant linear
  relationship (like Y bX c)

43
Determine Linearity

• Now see how to determine the linearity of a
  relationship
• Use Analyze / Compare Means / Means...
• And you thought wed still be under ANOVA
• Use age for Dependent variable
• Use partyid for Independent variable
• Under Options check Test for linearity and
  Anova table and eta

44
Determine Linearity
45
Determine Linearity

• The first output table is a case summary (not
  shown), and the second table is the Report on the
  next slide - just basic descriptive statistics
• The ANOVA Table (slide after Report) shows that
  the linear relationship is significant (Sig. lt
  0.050 for Linearity)

46
Determine Linearity
47
Determine Linearity
48
Determine Linearity

• The Measures of Association describe how well
  the linear relationship fits
• The R Squared is a correlation coefficient from
  linear regression - zero means no relationship,
  and unity (1) is a perfect fit
• This value of 0.006 is very weak, but not
  surprising given the wide spread of data

49
Determine Linearity

• The Eta Squared gives how much variance in
  age is explained by polviews - only 3.0
• This implies there are other factors to explain
  differences in political views
• Notice that some tests say there is a linear
  relationship in this case, but others disagree

50
Testing Options in SPSS

• SPSS has many additional tests for ANOVA we
  wont try to discuss them in detail
• See Marija Norusis books cited in the syllabus
• Pairwise multiple comparison and post hoc range
  tests
• Tukey
• Hochbergs GT2
• Gabriel
• Scheffe (see warning later)

51
Testing Options in SPSS

• Pairwise multiple comparison tests only
• Bonferroni
• Sidak
• Dunnett
• LSD (Least Significant Difference, not the other
  thing)

52
Testing Options in SPSS

• Pairwise multiple comparison tests which do not
  assume equal variances are
• Tamhanes T2
• For example, this also produces 7 pairs of
  parties that have pairwise significant
  differences in mean age, like the Tukey results
  seen earlier
• Dunnetts T3
• Games-Howell
• Dunnetts C

53
Testing Options in SPSS

• Post hoc range tests only
• Tukeys b
• S-N-K (Student-Newman-Keuls) (see warning later)
• Duncan (see warning later)
• R-E-G-W F (the Ryan-Einot-Gabriel-Welsch F-test)
• R-E-G-W Q (range test)
• Waller-Duncan

54
Warning!!

• SPSS offers some ANOVA methods which they dont
  recommend using they are
• Scheffe
• Student-Newman-Keuls (S-N-K)
• Duncan