One-Way Analysis of Variance (ANOVA)

1
One-Way Analysis of Variance (ANOVA)
2
Aside

• I dislike the ANOVA chapter (11) in the book with
  a passion if it were all written like this I
  wouldnt use the thing
• Therefore, dont worry if you dont understand
  it, use my notes Exam questions on ANOVA will
  come from my lectures and notes

3
One-Way ANOVA

• One-Way Analysis of Variance
• aka One-Way ANOVA
• Most widely used statistical technique in all of
  statistics
• One-Way refers to the fact that only one IV and
  one DV are being analyzed (like the t-test)
• i.e. An independent-samples t-test with treatment
  and control groups where the treatment (present
  in the tx grp and absent in the control grp) is
  the IV

4
One-Way ANOVA

• Unlike the t-test, the ANOVA can look at levels
  or subgroups of IVs
• The t-test can only test if an IV is there or
  not, not differences between subgroups of the IV
• I.e. our experiment is to test the effect of hair
  color (our IV) on intelligence
• One t-test can only test if brunettes are smarter
  than blondes, any other comparison would involve
  doing another t-test
• A one-way ANOVA can test many subgroups or levels
  of our IV hair color, for instance blondes,
  brunettes, and redheads are all subtypes of hair
  color, can so can be tested with one one-way ANOVA

5
One-Way ANOVA

• Other examples of subgroups
• If race is your IV, then caucasian,
  african-american, asian-american, hispanic (4)
  are all subgroups/levels
• If gender is your IV, than male and female (2)
  are your levels
• If treatment is your IV, then some treatment, a
  little treatment, and a lot of treatment (3) can
  be your levels

6
One-Way ANOVA

• OK, so why not just do a lot of t-tests and keep
  things simple?
• Many t-tests will inflate our Type I Error rate!
• This is an example of using many statistical
  tests to evaluate one hypothesis see the
  Bonferroni Correction
• It is less time consuming
• There is a simple way to do the same thing in
  ANOVA, they are called post-hoc tests, and we
  will go over them later on
• However, with only one DV and one IV (with only
  two levels), the ANOVA and t-test are
  mathematically identical, since they are
  essentially derived from the same source

7
One-Way ANOVA

• Therefore, the ANOVA and the t-test have similar
  assumptions
• Assumption of Normality
• Like the t-test you can place fast and loose with
  this one, especially with large enough sample
  size see the Central Limit Theorem
• Assumption of Homogeneity of Variance
  (Homoscedasticity)
• Like the t-test, this isnt problematic unless
  one levels variance is much larger than one the
  others (4 times as large) the one-way ANOVA is
  robust to small violations of this assumption, so
  long as group size is roughly equal

8
One-Way ANOVA

• Independence of Observations
• Like the t-test, the ANOVA is very sensitive to
  violations of this assumption if violated it is
  more appropriate to use a Repeated-Measures ANOVA

9
One-Way ANOVA

• Hypothesis testing in ANOVA
• Since ANOVA tests for differences between means
  for multiple groups or levels of our IV, then H1
  is that there is a difference somewhere between
  these group means
• H1 µ1 ? µ2 ? µ3 ? µ4, etc
• Ho µ1 µ2 µ3 µ4, etc
• These hypothesis are called omnibus hypothesis,
  and tests of these hypotheses omnibus tests

10
One-Way ANOVA

• However, our F-statistic does not tell us where
  this difference lies
• If we have 4 groups, group 1 could differ from
  groups 2-4, groups 2 and 4 could differ from
  groups 1 and 3, group 1 and 2 could differ from
  3, but not 4, etc.
• Since our hypothesis should be as precise as
  possible (presuming youre researching something
  that isnt completely new), you will want to
  determine the precise nature of these differences
• You can do this using multiple comparison
  techniques (more on this later)

11
One-Way ANOVA

• The basic logic behind the ANOVA
• The ANOVA yields and F-statistic (just like the
  t-test gave us a t-statistic)
• The basic form of the F-statistic is
  MStreatment/MSerror
• MS mean square or the mean of squares (why it
  is called this will be more obvious later)

12
One-Way ANOVA

• The basic logic behind the ANOVA
• MSbetween or MStreatment average variability
  (variance) between the levels of our IV/groups
• Ideally we want to maximize MStreatment, because
  were predicting that our IV will differentially
  effect our groups
• i.e. if our IV is treatment, and the levels are
  no treatment vs. a lot of treatment, we would
  expect the treatment group mean to be very
  different than the no treatment mean this
  results in lots of variability between these
  groups

13
One-Way ANOVA

• The basic logic behind the ANOVA
• MSwithin or MSerror average variance among
  subjects in the same group
• Ideally we want to minimize MSerror, because
  ideally our IV (treatment) influences everyone
  equally everyone improves, and does so at the
  same rate (i.e. variability is low)
• If F MStreatment/ MSerror, then making
  MStreatment large and MSerror small will result
  in a large value of F
• Like t, a large value corresponds to small
  p-values, which makes it more likely to reject Ho

14
One-Way ANOVA

• However, before we calculate MS, we need to
  calculate what are called sums of squares, or SS
• SS the sum of squared deviations around the
  mean
• Does this sound familiar? What does this sound
  like?
• Just like MS, we have SSerror and SStreatment
• Unlike MS, we also have SStotal SSerror
  SStreatment

15
One-Way ANOVA

• SStotal S(Xij - )2
• Its the formula for our old friend variance,
  minus the n-1 denominator!
• Note N the number of subjects in all of the
  groups added together

16
One-Way ANOVA

• SStreatment
• This means we
• Subtract the grand mean, or the mean of all of
  the individual data points, from each group mean
• Square these numbers
• Multiply them by the number of subjects from that
  particular group
• Sum them
• Note n number of subjects per group
• Hint The number of numbers that you sum should
  equal the number of groups

17
One-Way ANOVA

• That leaves us with SSerror SStotal
  SStreatment
• Remember SStotal SSerror SStreatment
• Degrees of freedom
• Just as we have SStotal,SSerror, and SStreatment,
  we also have dftotal, dferror, and dftreatment
• dftotal N 1 OR the total number of subjects
  in all groups minus 1
• dftreatment k 1 OR the number of levels of
  our IV (aka groups) minus 1
• dferror N k OR the total number of subjects
  minus the number of groups OR dftotal -
  dftreatment

18
One-Way ANOVA

• Now that we have our SS and df, we can calculate
  MS
• MStreatment SStreatment/dftreatment
• MSerror SSerror/dferror
• Remember
• MSbetween or MStreatment average variability
  (variance) between the levels of our IV/groups
• MSwithin or MSerror average variance among
  subjects in the same group

19
One-Way ANOVA

• We then use this to calculate our F-statistic
• F MStreatment/ MSerror
• The p-value associated with this F-statistic is a
  function of both F and your df
• Higher F and/or df ? Lower p
• Recall df n levels of your IV
• More Ss and/or fewer levels ? Higher df ? Lower
  p

20
One-Way ANOVA

• How can we change our experiment to increase the
  likelihood of a significant result/decrease p?
• Larger ES ? Higher F
• Increase the potency of the IV
• Higher df
• More Ss
• Fewer levels of your IV
• Collapse across groups Instead of looking at
  Kids vs. Young Adults vs. Adults, look at
  Children vs. Adults only
• Worst way to decrease p as this sacrifices how
  subtle-ly you can test your theory

21
One-Way ANOVA

• Example
• What effect does smoking have on performance?
  Spilich, June, and Renner (1992) asked nonsmokers
  (NS), smokers who had delayed smoking for three
  hours (DS), and smokers who were actively smoking
  (AS) to perform a pattern recognition task in
  which they had to locate a target on a screen.
  The data follow

22
One-Way ANOVA

• Example
• What is the IV, number of levels, and the DV?
• What is H1 and Ho?
• What is your dftotal, dfgroups, and dferror?

Non-Smokers Delayed Smokers Active Smokers
9 12 8
8 7 8
12 14 9
10 4 1
7 8 9
10 11 7
9 16 16
11 17 19
8 5 1
10 6 1
8 9 22
10 6 12
8 6 18
11 7 8
10 16 10
23
One-Way ANOVA
24
One-Way ANOVA

• Example
• Based on these results, would you reject or fail
  to reject Ho?
• What conclusion(s) would you reach about the
  effect of the IV on the DV?

25
One-Way ANOVA

• Assumptions of ANOVA
• Independence of Observations
• Homoscedasticity
• Normality
• Equal sample sizes not technically an
  assumption, but effects the other 3
• How do we know if we violate one (or more) of
  these? What do we do?

26
One-Way ANOVA

• Independence of Observations
• Identified methodologically
• Other than using repeated-measures tests (covered
  later), nothing you can do
• Equal Sample Sizes
• Add more Ss to the smaller group
• DONT delete Ss from the larger one

27
One-Way ANOVA

• Homoscedasticity
• Identified using Levenes Test or the Welch
  Procedure
• Again, dont sweat the book, SPSS will do it for
  you
• If detected (and group sizes very unequal), use
  appropriate transformation

28
One-Way ANOVA

• Homoscedasticity

29
One-Way ANOVA

• Normality
• Can identify with histograms of DVs (IVs are
  supposed to be non-normal)
• More appropriate to use skewness and kurtosis
  statistics
• If detected (and sample size very small), use
  appropriate transformation

30
One-Way ANOVA

• Normality
• Divide statistic by its standard error to get
  z-score
• Calculate p-value using z-score and df n

31
One-Way ANOVA

• Estimates of Effect Size in ANOVA
• ?2 (eta squared) SSgroup/SStotal
• Unfortunately, this is what most statistical
  computer packages give you, because it is simple
  to calculate, but seriously overestimates the
  size of effect
• ?2 (omega squared)
• Less biased than ?2, but still not ideal

32
One-Way ANOVA

• Estimates of Effect Size in ANOVA
• Cohens d
• Remember for d, .2 small effect, .5 medium,
  and .8 large

33
One-Way ANOVA

• Multiple Comparison Techniques
• Remember ANOVA tests for differences somewhere
  between group means, but doesnt say where
• H1 µ1 ? µ2 ? µ3 ? µ4, etc
• If significant, group 1 could be different from
  groups 2-4, groups 2 3 could be different from
  groups 1 and 4, etc.
• Multiple comparison techniques attempt to detect
  specifically where these differences lie

34
Multiple Comparison Techniques

• You could always run 2-sample t-tests on all
  possible 2-group combinations of your groups,
  although with our 4 group example this is 6
  different tests
• Running 6 tests _at_ (a .05) (a .05 x 6 .3)
  ?
• This would inflate what is called the familywise
  error rate in our previous example, all of the
  6 tests that we run are considered a family of
  tests, and the familywise error rate is the a for
  all 6 tests combined (.3) however, we want to
  keep this at .05

35
Multiple Comparison Techniques

• To perform multiple comparisons with a
  significant omnibus F, or not to
• Why would you look for a difference between two
  or more groups when your F said there isnt one?

36
Multiple Comparison Techniques

• Some say this is what is called statistical
  fishing and is very bad you should not be
  conducting statistical tests willy-nilly without
  just cause or a theoretical reason for doing so
• Think of someone fishing in a lake, you dont
  know if anything is there, but youll keep trying
  until you find something the idea is that if
  your hypothesis is true, you shouldnt have to
  look to hard to find it, because if you look for
  anything hard enough you tend to find it

37
Multiple Comparison Techniques

• However, others would say that the omnibus test
  is underpowered, particularly with a large number
  of groups and if only a few significant
  differences are predicted among them
• i.e. H1 µ1 ? µ2 ? µ3 ? µ4 µ10
• If you only predict groups 1 and 5 will differ,
  you are unlikely to get a significant omnibus F
  unless you have a ton of Ss
• I and most statisticians fall on this side of the
  argument i.e. its OK to ignore the omnibus
  test if you have a theoretical reason to predict
  specific differences among groups

38
A Priori Techniques

• A priori techniques
• Planned prior to data collection
• Involve specific hypotheses between group means
  (i.e. not just testing all group differences)
• Multiple t Tests
• As stated previously, not a good idea bc/ ?
  inflated ?
• Involves a different formula than a traditional t
  test
• Bonferroni t (Dunns Test)
• Simply using the Bonferroni Correction on a bunch
  of regular t tests
• For 6 comparisons ? .05/6 .0083
• Can combine the Bonferroni correction with the
  adjusted formula for t used above

39
A Priori Techniques

• Dunn-Šidák Test
• Almost identical to the Bonferroni t, but uses 1
  (1 - ?)1/c instead of ?/c, where c of
  comparisons
• Slightly less conservative
• For 6 comparisons 1-(1-.05)1/6 .0085

40
A Priori Techniques

• Holm Test
• Instead of assuming we evaluate hypotheses all at
  once, allows for ordering of when hypotheses are
  evaluated
• Uses most stringent ? for hypothesis w/ strongest
  support (largest t)
• Adjusts ? downward for next comparisons, taking
  into account that previous comparison was
  significant
• More powerful than Bonferroni t Dunn-Šidák Test
• Calculate t tests for all comparisons
• Arrange ts in decreasing order
• For 1st t test, use Dunn-Šidák method with normal
  c
• For 2nd t test, use c 1
• For 3rd t test, use c 2, etc.
• Continue until a nonsignificant result is obtained

41
A Priori Techniques

• Linear Contrasts
• What if, instead of comparing group x to group y,
  we want to compare group x, y, z to group a
  b?
• Coefficients how to tell mathematically which
  groups we are comparing
• Coefficients for the same groups have to be the
  same and all coefficients must add up to 0
• Comparing groups 1, 2, 3 to groups 4 5
• Groups 1 3 Coefficient 2
• Groups 4 5 Coefficient -3
• Use Bonferroni correction to adjust ? to of
  contrasts
• 4 contrasts ? use ? .05/4 .0125

42
A Priori Techniques

• Orthogonal Contrasts
• What if you want to compare groups within a
  contrast?
• I.e. Group 1 vs. Groups 2 3 and Group 2 vs.
  Group 3
• Assigning coefficients is the same, but
  calculations are different (dont worry about how
  different, just focus on the linear vs.
  orthogonal difference)

43
A Priori Techniques

• Both the Holm Test and linear/orthogonal
  contrasts sound good, which do I use?
• If making only a few contrasts Linear/Orthogonal
• If making many contrasts Holm Test
• More powerful and determining coefficients is
  confusing with multiple contrasts

44
Post Hoc Techniques

• Post hoc techniques
• Fishers LSD
• We replace in our 2-sample t-test formula
  with MSerror, and we get
• We then test this using a critical t, using our
  t-table and dferror as our df
• You can use either a one-tailed or two-tailed
  test, depending on whether or not you think one
  mean is higher or lower (one-tailed) or possibly
  either (two-tailed) than the other

45
Post Hoc Techniques

• Fishers LSD
• However, with more than 3 groups, using Fishers
  LSD results in an inflation of ? (i.e. with 4
  groups a .1)
• You could use the Bonferroni method to correct
  for this, but then why not just use it in the
  first place?
• This is why Fishers LSD is no longer widely used
  and other methods are preferred
• Newman-Keuls Test
• Like Fishers LSD, allows the familywise ? gt .05
• Pretty crappy test for that reason

46
Post Hoc Techniques

• Scheffés Test
• Fishers LSD Neuman-Keuls not conservative
  enough too easy to find significant results
• Scheffés Test too conservative result in a
  low degree of Type I Error but too high Type II
  Error (incorrectly rejects H1) too hard to find
  significant results
• Tukeys Honestly Significant Difference (HSD)
  test
• Very popular, but conservative

47
Post Hoc Techniques

• Ryan/Einot/Gabriel/Welsch (REGWQ) Procedure
• Like Tukeys HSD, but adjusts ? (like the
  Dunn-Šidák Test) to make the test less
  conservative
• Good compromise of Type I/Type II Error
• Dunnetts Test
• Specifically designed for comparing a control
  group with several treatment groups
• Most powerful test in this case

48
One-Way ANOVA

• Reporting and Interpreting Results in ANOVA
• We report our ANOVA as
• F(dfgroups, dftotal) x.xx, p .xx, d .xx
• i.e. for F(4, 299) 1.5, p .01, d .01 We
  have 5 groups, 300 subjects total in all of our
  groups put together We can reject Ho, however
  our small effect size statistic informs us that
  it may be our large sample size that resulted in
  us doing so rather than a large effect of our IV