Hypothesis Testing with Analysis of Variance

1
Hypothesis Testing with Analysis of Variance
Lecture 12B7/25/07
2
Lecture 12B Outline Analysis of Variance

• I. Computing ANOVA
• II. Post hoc comparisons
• III. Effect size (eta-squared)
• IV. Reporting results of ANOVA

3
Research problem

• Does the presence of others during an emergency
  affect helping behavior?
• Conduct an experiment with 3 conditions
• Wait alone
• Wait with 1 other person
• Wait with 2 other people
• IV Number of people present
• 3 levels (0, 1, 2)
• DV Time it takes (in seconds) to call for help

4
Data from Helping Study

• Seconds lapsed before calling for help

Are these 3 means significantly different from
each other?
M1 16
M2 24
M3 29
5
Hypothesis testing with ANOVA

• Step 1 Research question
• Does presence of others affect helping?
• Step 2 Statistical Hypotheses
• H0 ?1 ?2 ?3
• H1 At least one mean is different from another
• Step 3 Decision Rule
• Look up critical value of F in Table B.4

6
Hypothesis testing with ANOVA

• Step 4 Compute observed F-ratio
• Step 5 Make a Decision (Reject or Retain H0)
• Step 6 If H0 rejected, conduct post-hoc
  comparisons
• Step 7 Interpret and Report Findings

7
Computing ANOVA
F between-group variance
within-group variance
SS df

• Variance Mean Square (MS)

F MS between MS within
8
Computing ANOVA

• Steps in computing the ANOVA
• Compute SS
• Compute df
• Compute MS
• Compute F
• Keep track of your computations in an ANOVA
  Summary Table

9
Computing ANOVA

• The ANOVA Summary Table

10
Computing ANOVA

• STEP 1 Compute Sums of Squares (SS)

SSTotal

• Where
• X each value of X
• T treatment group total (?X)
• G grand total (?T)
• n sample size of each group
• N total sample size (?n)

SSBetween
SSWithin ?(SS for each group) or SSTotal
? SSBetween
11
Computing ANOVA

• STEP 2 Compute Degrees of Freedom (df)

• Where
• n sample size of each group
• N total sample size (?n)
• k number of groups

dfTotal N 1
dfBetween k 1
dfWithin N k or ?(n-1)
12
Computing ANOVA

• STEP 3 Compute Mean Squares (MS)

MSBetween
MSWithin
13
Computing ANOVA

• STEP 4 Compute the F-Ratio

F-Ratio
14
Computing ANOVA

• The ANOVA Summary Table

15
Computing ANOVA

• Sample ANOVA Summary Table

16
Computing ANOVA
17
Computing ANOVA
18
Computing ANOVA
19
Computing ANOVA
SSTotal
SSTotal
20
Computing ANOVA
SSBetween
SSBetween
21
Computing ANOVA
SSWithin SSTotal ? SSBetween
SSWithin 722 516 206
22
Computing ANOVA
SSWithin ?SS for each group
23
Computing ANOVA
SSWithin ?SS for each group
SSWithin ?SS 58 72 76 206
You will be given these values
24
Computing ANOVA

• Lets fill in our SS values

Notice 722 516 206 SST SSB SSW
25
Computing ANOVA

• Now compute degrees of freedom (df)

Where k 3 N 18
26
Computing ANOVA
Where k 3 N 18
27
Computing ANOVA
Notice 17 15 2 dfT dfB dfW
28
Computing ANOVA

• Now compute the Mean Squares (MS)

29
Computing ANOVA

• Now compute the Mean Squares (MS)

30
Computing ANOVA

• Now compute the F-Ratio

31
Computing ANOVA

• Now compute the F-Ratio

32
Computing ANOVA

• All of this work for the final F-ratio!

33
Finding the Critical Value

• Find Fcritical in Table B.4
• Need to know 3 things
• ? level
• dfnumerator dfbetween
• dfdenominator dfwithin
• If ? .05 and df 2,15, Fcritical 3.68

34
Critical values of F for df2,15
Critical region Reject H0
3.68
6.23
35
Make a Decision

• Does our observed F (18.79) exceed our critical
  value of F (3.68)?
• Yes!
• Reject H0

36
Interpret Findings

• At least two of the means are significantly
  different from each other
• But, which ones?
• Must conduct additional analyses to pinpoint
  specific mean differences
• Called post hoc tests

37
Post Hoc Tests

• Pinpoint specific group differences
• Conduct multiple comparisons, controlling for
  experimentwise Type I error rate
• Many types of post hoc tests
• Two common ones
• Tukey Honestly Significant Difference (HSD)
• Sheffe test

38
Tukey HSD Test

• Tukey Honestly Significant Difference (HSD)
• HSD minimum difference between means needed for
  statistical significance
• How big does the difference between two means
  have to be in order to conclude that they are
  significantly different from each other?
• Like a critical value, but a critical mean
  difference
• Assumes equal n

39
Tukey HSD Test

• Step 1 Find the value of q (Table B.5)
• Need to know 3 things
• ?
• dfW
• k
• Step 2 Compute HSD

HSD
Where n group sample size, assuming equal n in
each group
40
Tukey HSD Test

• Step 3 Compute difference between eachpair of
  means and compare to HSD
• M1 M2 ?
• M1 M3 ?
• M2 M3 ?
• Compare each mean difference to the HSD
• If the difference equals/exceeds the HSD,
  conclude that the means are significantly
  different from each other

41
Tukey HSD Test Example

• Step 1 Find the value of q (Table B.5)
• ? .05 dfW 15 k 3
• From Table B.5 q 3.67
• Step 2 Compute HSD

So, a pair of means must differ by at least 5.55
in order to be significantly different
HSD ? 5.55 seconds
42
Tukey HSD Test Example

• Step 3 Compute difference between eachpair of
  means and compare to HSD
• M1 M2 16 24 - 8
• M1 M3 16 29 -13
• M2 M3 24 29 -5

Exceeds 5.55
Exceeds 5.55
Does not exceed 5.55
43
Tukey HSD Test Example

• What do we conclude?
• M1 differs from M2 and M3
• People waiting alone helped significantly faster
  than people waiting with others
• M2 M3 do NOT differ from each other
• There was no difference in helping times for
  individuals waiting with 1 other person and
  individuals waiting with 2 other people

44
Measure of Effect Size

• Compute proportion of variance explained by the
  treatment effect
• Proportion of total variance accounted for by
  variability between groups
• In ANOVA, r2 typically called ?2 (pronounced
  eta squared)

r2
45
Measure of Effect Size Example

• 71 of the variance in helping behavior (number
  of second lapsed before seeking help) is
  explained by the number of people present

r2 ?2
46
Reporting Results of an ANOVA

• Formal description of findings
• There was a significant effect of the number of
  people present on the time it took (in seconds)
  for participants to seek help, F(2,15) 18.79,
  plt.05, ?2 .71.

47
Reporting an F-statistic

• A closer look
• F (2,15) 18.79, p lt .05, ?2 .71

Test statistic
effect size
alpha level
Pair of degrees of freedom
Observed value
48
Reporting Results of an ANOVA

• Formal description of findings
• Tukey post-hoc comparisons indicated that
  participants who were waiting alone helped
  significantly faster (M16, SD3.4) than
  participants who waited with one other person
  (M24, SD3.8) or with two other people (M29,
  SD3.9), p lt .05.

SD for each group SD
49
Additional considerations

• Assumptions of the ANOVA
• Please see your book
• Relationship between F and t
• Please see your book

50
Statistical Toolbox

• Descriptive Statistics
• Tables
• Graphs
• Measures of Central Tendency
• Measures of Variability
• Inferential statistics
• Standard Normal Curve
• Distribution of Sample Means
• z-test for one sample mean
• t-test for one sample mean
• t-test for two independent sample means
• t-test for dependent (related) samples
• Analysis of variance (ANOVA)