The paired t-test, non-parametric tests, and ANOVA July 13, 2004

1
The paired t-test, non-parametric tests, and
ANOVAJuly 13, 2004
2
Review the Experiment (note exact numbers have
been altered)

• Grade 3 at Oak School were given an IQ test at
  the beginning of the academic year (n90).
• Classroom teachers were given a list of names of
  students in their classes who had supposedly
  scored in the top 20 percent these students were
  identified as academic bloomers (n18).
• BUT the children on the teachers lists had
  actually been randomly assigned to the list.
• At the end of the year, the same I.Q. test was
  re-administered.

3
The results

• Children who had been randomly assigned to the
  top-20 percent list had mean I.Q. increase of
  12.2 points (sd2.0) vs. children in the control
  group only had an increase of 8.2 points (sd2.5)

4
Confidence interval (more information!!)

• 95 CI for the difference 4.01.99(.64) (2.7
  5.3)

5
The Paired T-test
6
The Paired T-test

• Paired data means youve measured the same person
  at different time points or measured pairs of
  people who are related (husbands and wives,
  siblings, controls pair-matched to cases, etc.
• For example, to evaluate whether an observed
  change in mean (before vs. after) represents a
  true improvement (or decrease)
• Null hypothesis difference (after-before)0

7
The differences are treated like a single random
variable
8
Example Data
Is there a significant increase in scores in this
group?   Average of differences 1   Sample
Variance 3.3 sample SD 1.82    T 12
1/(1.82/3.6) 1.98   data _null_ pval
1-probt(1.98, 12) put pval run 0.0355517436 Si
gnificant for a one-sided test borderline for
two-sided test
9
Example 2 Did the control group in the Oak
School experiment improveat all during the year?
p-value lt.0001
10
Confidence interval for annual change in IQ test
score

• 95 CI for the increase 8.22.0(.29) (7.6
  8.8)

11
Summary parametric tests
Equal variances are pooled
Unequal variances (unpooled)
12
Non-parametric tests
13
Non-parametric tests

• t-tests require your outcome variable to be
  normally distributed (or close enough).
• Non-parametric tests are based on RANKS instead
  of means and standard deviations (population
  parameters).

14
Example non-parametric tests
10 dieters following Atkins diet vs. 10 dieters
following Jenny Craig Hypothetical
RESULTS Atkins group loses an average of 34.5
lbs. J. Craig group loses an average of 18.5
lbs. Conclusion Atkins is better?
15
Example non-parametric tests
BUT, take a closer look at the individual
data Atkins, change in weight (lbs) 4, 3,
0, -3, -4, -5, -11, -14, -15, -300 J. Craig,
change in weight (lbs) -8, -10, -12, -16, -18,
-20, -21, -24, -26, -30
16
Enter data in SAS

• data nonparametric
• input loss diet
• datalines
• 4 atkins
• 3 atkins
• 0 atkins
• -3 atkins
• -4 atkins
• -5 atkins
• -11 atkins
• -14 atkins
• -15 atkins
• -300 atkins
• -8 jenny
• -10 jenny
• -12 jenny
• -16 jenny
• -18 jenny
• -20 jenny

17
Jenny Craig
30
25
20
P
e
r
c
15
e
n
t
10
5
0
-30
-25
-20
-15
-10
-5
0
5
10
15
20
Weight Change
18
Atkins
30
25
20
P
e
r
c
15
e
n
t
10
5
0
-300
-280
-260
-240
-220
-200
-180
-160
-140
-120
-100
-80
-60
-40
-20
0
20
Weight Change
19
t-test doesnt work

• Comparing the mean weight loss of the two groups
  is not appropriate here.
• The distributions do not appear to be normally
  distributed.
• Moreover, there is an extreme outlier (this
  outlier influences the mean a great deal).

20
Statistical tests to compare ranks

• Wilcoxon rank-sum test (equivalent to
  Mann-Whitney U test) is analogue of two-sample
  t-test.
• Wilcoxon signed-rank test is analogue of
  one-sample t-test, usually used for paired data

21
Wilcoxon rank-sum test

• RANK the values, 1 being the least weight loss
  and 20 being the most weight loss.
• Atkins
• 4, 3, 0, -3, -4, -5, -11, -14, -15, -300
•  1, 2, 3, 4, 5, 6, 9, 11, 12, 20
• J. Craig
• -8, -10, -12, -16, -18, -20, -21, -24, -26, -30
• 7, 8, 10, 13, 14, 15, 16, 17, 18,
  19

22
Wilcoxon rank-sum test

• Sum of Atkins ranks
•  1 2 3 4 5 6 9 11 12 2073
• Sum of Jenny Craigs ranks
• 7 8 10 13 14 1516 17 1819137
• Jenny Craig clearly ranked higher!
• P-value (from computer) .017
• from ttest, p-value.60

23
Tests in SAS

• /to get wilcoxon rank-sum test/
• proc npar1way wilcoxon datanonparametric
• class diet
• var loss
• run
• /To get ttest/
• proc ttest datanonparametric
• class diet
• var loss
• run

24
Wilcoxon signed-rank test

• H0 median weight loss in Atkins group 0
• Hamedian weight loss in Atkins not 0
• Atkins
• 4, 3, 0, -3, -4, -5, -11, -14, -15, -300
• Rank absolute values of differences (ignore
  zeroes)
• Ordered values 300, 15, 14, 11, 5, 4, 4, 3, 3, 0
• Ranks 1 2 3 4 5 6-7 8-9
  -
• Sum of negative ranks 123456.58.530
• Sum of positive ranks 6.58.515
• P-value(from computer).043 from paired
  t-test.27

25
Tests in SAS

• /to get one-sample tests (both students t and
  signed-rank/
• proc univariate datanonparametric
• var loss
• where diet"atkins"
• run

26
What if data were paired?

• e.g., one-to-one matching find pairs of study
  participants who have same age, gender,
  socioeconomic status, degree of overweight, etc.
• Atkins
• 4, 3, 0, -3, -4, -5, -11, -14, -15, -300
• J. Craig
• -8, -10, -12, -16, -18, -20, -21, -24, -26, -30

27
Enter data differently in SAS10 pairs, rather
than 20 individual observations

• data piared
• input lossa lossj
• difflossa-lossj
• datalines
• 4 -8
• 3 -10
• 0 -12
• -3 -16
• -4 -18
• -5 -20
• -11 -21
• -14 -24
• -15 -26
• -300 -30
• run

28
Tests in SAS

• /to get all paired tests/
• proc univariate datapaired
• var diff
• run
• /To get just paired ttest/
• proc ttest datapaired
• var diff
• run
• /To get paired ttest, alternatively/
• proc ttest datapaired
• paired lossalossj
• run

29
ANOVAfor comparing means between more than 2
groups
30
ANOVA (ANalysis Of VAriance)

• Idea For two or more groups, test difference
  between means, for quantitative normally
  distributed variables.
• Just an extension of the t-test (an ANOVA with
  only two groups is mathematically equivalent to a
  t-test).
• Like the t-test, ANOVA is parametric
  testassumes that the outcome variable is roughly
  normally distributed

31
The F-test
Is the difference in the means of the groups more
than background noise (variability within
groups)?
32
Spine bone density vs. menstrual regularity
1.2
1.1
1.0
S
P
I
N
E
0.9
0.8
0.7
amenorrheic
oligomenorrheic
eumenorrheic
33
Group means and standard deviations

• Amenorrheic group (n11)
• Mean spine BMD .92 g/cm2
• standard deviation .10 g/cm2
• Oligomenorrheic group (n11)
• Mean spine BMD .94 g/cm2
• standard deviation .08 g/cm2
• Eumenrroheic group (n11)
• Mean spine BMD 1.06 g/cm2
• standard deviation .11 g/cm2

34
The F-Test
35
The F-distribution

• The F-distribution is a continuous probability
  distribution that depends on two parameters n and
  m (numerator and denominator degrees of freedom,
  respectively)

36
The F-distribution

• A ratio of sample variances follows an
  F-distribution

• The F-test tests the hypothesis that two sample
  variances are equal.
• F will be close to 1 if sample variances are
  equal.

37
ANOVA Table
TSSSSB SSW
38
ANOVAt-test
39
ANOVA summary

• A statistically significant ANOVA (F-test) only
  tells you that at least two of the groups differ,
  but not which ones differ.
• Determining which groups differ (when its
  unclear) requires more sophisticated analyses to
  correct for the problem of multiple comparisons

40
Question Why not just do 3 pairwise ttests?

• Answer because, at an error rate of 5 each
  test, this means you have an overall chance of up
  to 1-(.95)3 14 of making a type-I error (if all
  3 comparisons were independent)
•  If you wanted to compare 6 groups, youd have to
  do 6C2 15 pairwise ttests which would give you
  a high chance of finding something significant
  just by chance (if all tests were independent
  with a type-I error rate of 5 each) probability
  of at least one type-I error 1-(.95)1554.

41
Multiple comparisons
With 18 independent comparisons, we have 60
chance of at least 1 false positive.
42
Multiple comparisons
With 18 independent comparisons, we expect about
1 false positive.
43
Correction for multiple comparisons

• How to correct for multiple comparisons post-hoc
• Bonferronis correction (adjusts p by most
  conservative amount, assuming all tests
  independent)
•    Holm/Hochberg (gives p-cutoff beyond which
  not significant)
• Tukeys (adjusts p)
• Scheffes (adjusts p)

44
Non-parametric ANOVA

• Kruskal-Wallis one-way ANOVA
• Extension of the Wilcoxon Sign-Rank test for 2
  groups based on ranks
•  
• Proc NPAR1WAY in SAS

45
Reading for this week

• Chapters 4-5, 12-13 (last week)
• Chapters 6-8, 10, 14 (this week)