Inference about Several Mean Vectors

1
Inferences about Several Mean Vectors

• Paired Comparisons
• Related Samples
• Have dependencies between measurements.
• Often used in pre-post situations (if used on two
  groups, group membership among the two elements
  is usually assigned randomly).
• Examples paired comparisons and repeated
  measures

2
To make a univariate inference on two matched
samples - Let X1jdenote the jth observation
from group 1 and X2j denote the jth observation
from group 2 - Create a new random variable Dj
- Given that Dj N(d, sd), we have that
where
3
This result leads directly to confidence
intervals and hypothesis tests for the mean
difference d. Let D1,,Dn be a random sample
from a N(d, sd) distribution. The hypothesis H0
d d0 ( 0 usually) is rejected in favor
of H1 d ? d0 at a level of significance a, if
4
We can also build a confidence interval for the
mean difference d. Let D1,,Dn be a random
sample from a N(d, sd) distribution. Then the
100(1 a) confidence interval for d is given by
Now we extend this procedure to p-dimensions!
5
We will let Xlji represent the value of the ith
variable taken from the jth observation of the
lth group. For g 2 groups, we can then create p
new variables Dji
Let
and assume that
6
If the D1,,Dn Np(d, Sd) and are independent
random vectors, then


where
and we know that
7
Let d1,,dn be observed difference vectors from
a Np(d, Sd) distribution. The hypothesis H0 d
0 is rejected in favor of H1 d ? 0 at a
level of significance a, if







8
A 100(1 a) confidence region for the mean
difference vector d consists of all d such that


100(1 a) simultaneous confidence intervals
for the individual mean differences di are given
by
Bonferroni 100(1 a) confidence intervals for
the individual mean differences di are given by
9
Example suppose we had the following fifteen
sample observations from two groups on some
random variables X1 and X2
At a significance level of a 0.05, do these
data support the assertion that they were drawn
from a population with similar means? In other
words, test the null hypothesis
10
First we need to calculate the difference
vectors d1j and d2j
These are the two variables of interest!
11
The scatter plot of pairs (d1j, d2j).
sample centroid (d1, d2)
_ _
hypothesized centroid (d1, d2)0
12
Our critical value is
So we have Decision rule do not reject H0 if
T2 ? 8.20 otherwise reject H0
13
- Calculate the Test Statistic
so
14
T2 0.0971 ? 8.20 so do not reject H0. The
sample evidence does not refute the claim that
the mean difference vector differs from
15
Univariate Repeated Measures Design
Observations are made for a single response on
the same set of elements, usually at uniform time
intervals. Here the jth observation is
q not p!
Here we have measurements on a single response on
each element at q points in time.
16
We usually are concerned with contrasts of
(i.e., group or time period means) or sometimes
(i.e., respondent means)
17
A common contrast is
or
2
q-1
either of which allows us to test m1 ? mq.
18
Consider an Nq(m, S) population, and let C be a
contrast matrix. The hypothesis H0 Cm 0 is
rejected in favor of H1 Cm ? 0 at a level of
significance a, if




19
Under these circumstances, a 100(1 a)
confidence region for contrasts Cm is given by

Similarly, simultaneous 100(1 a) confidence
intervals for single contrasts cm are given by


20

• Example
• Suppose we asked fifteen consumers to rate the
  overall palatability for different yogurt blends
  on a 100 point scale.
• We have systematically varied the sugar content
  and acidity of the blends each can be either
  low or high, so we have the following
  experimental design

1
2
Low
Well call these the treatment labels
Sugar Content
3
4
High
Low
High
Acidity
There are three common contrasts of interest in
this design two main effects (sugar content and
acidity) and one interaction.
21
For our design
1
2
Low
Sugar Content
3
4
High
Low
High
Acidity
The three particular contrasts of interest
are (m1 m2) - (m3 m4) sugar content main
effect (m1 m3) - (m2 m4) acidity main
effect (m1 m4) - (m2 m3) sugar content x
acidity interaction We can now construct a
contrast matrix C.

22
With mean vector
our contrast matrix could be
with each vector representing an individual
contrast of interest.
23
For the sample data provided below we have the
following summary data
and we wish to test the null hypothesis H0 Cm
0

24
A scatterplot and overlaid interaction plot
could look like this
The Green Points are means at low sugar contents
Yellow Points are means at high sugar contents.
This plot suggests main effects and an
interaction.
25
At a level of significance a 0.01, the null
hypothesis H0 Cm 0 is rejected in favor
of H1 Cm ? 0 if


where
26
Thus our decision rule is do not reject H0
if T2 ? 20.834 otherwise reject H0 And
the calculated value of the test statistic is
27
So by our decision rule T2 175.702 ?
20.834 we reject H0. The sample evidence
refutes the claim that the three contrasts of the
mean vector (m1 m2) - (m3 m4) sugar
content main effect (m1 m3) - (m2 m4)
acidity main effect (m1 m4) - (m2 m3)
sugar content x acidity interaction are jointly
equal to zero.
28
Note that we could convert the T2 to an F as
commonly done in ANOVA.
and find the corresponding pvalue
(0.000000459). - if either of our factors
(sugar content or acidity) had more than two
levels, this could easily be accommodated by
expanding the number of columns of the contrast
matrix C. - this analysis could also be
accomplished using a two factor ANOVA with
interaction
29
Comparing Mean Vectors From Two
Populations
If independent samples are drawn from two
populations, they are compared on a summary
level. -Inference on the difference between two
mean vectors instead of on the mean difference
vector Consider random samples of size n1 from
population 1 and n2 from population 2.
30
We may wish to make inferences about the
difference between the group means (usually m1 -
m2 0). To do so we will need to make the
following assumptions - -
- This structure is sufficient for making
inferences about the scalar m1 - m2 when the
sample sizes n1 and n2 are large the result is
that
31

• This result leads directly to large sample
  confidence intervals and hypothesis tests for the
  difference between means m1 and m2.
• If the variances are equal, the pooled variance
  t-test may be used.
• For the unequal variances, the Satterthwaite
  approximation may be used.
• Usually the proceeded by the F-test for comparing
  two variances.

32
Suppose we wish to make inferences about the
difference between the group mean vectors
(usually m1 m2). To do so we will need to
make the following assumptions - -
- This structure is sufficient for making
inferences about the p x 1 vector m1 - m2 when
the sample sizes n1 and n2 are large relative to
p.


33
Let us make the following assumptions -
- - This structure is sufficient for
making inferences about the p x 1 vector m1 - m2
when the sample sizes n1 and n2 are small
relative to p.

34
We know that if S1 S2 S, then

is an estimate of (n1 - 1)S and

is an estimate of (n1 - 1)S. Consequently, we
can combine (or pool) the information in both
samples

35
Now to test the hypothesis H0 m1 - m2
d0 we consider the squared distance from the
sample estimate x1 x2 from the hypothesized
difference d0. Since

_ _


independence of the samples implies
36
and since Spooled estimates S, we have that


is an estimator of
as a result
37
Comparing Mean Vectors m From More Than Two
Populations

If independent samples are drawn from three (or
more) populations, they are compared in a manner
similar to Analysis of Variance or
ANOVA. Consider random samples of from g
populations arranged as
We use the multivariate analog of ANOVA (MANOVA
or Multivariate Analysis of Variance) to assess
the assertion that g groups have equal mean
vectors.
38
common variance
Recall that in ANOVA we assumed -
- These random samples are taken independently
from the g populations Thus we can decompose
any observation Xlt in the data set as
and its estimate as
This, of course, leads to the reparameterization
39
So our null hypothesis of equal means across the
g groups becomes H0 t1 t2 ? tg Note
also that, if the assumptions are met, then e
N(0, s2) and that we must impose a restriction
such as
or perhaps t1 0 to achieve unique
least-squares estimates.
40
The decomposition of the jth observation of the
lth group
suggests a similar sample decomposition
estimate of m
estimate of tl
estimate of el
41
If we then sum both sides over the subscript l,
the result is the classic partitioning of the
sums of squares
SSres (residual or error or within samples SS)
SStr (treatment or model or between samples SS)
SStot or SScor (total corrected SS)
which can be restated as
SSres (residual or error or within samples SS)
SStr (treatment or model or between samples SS)
SSobs (total observation SS)
SSmean (mean SS)
42
Note that the MStr and the MSres are independent
estimates of the pooled population variance s2.
Thus if the treatment means are equal,
and
so we have
43
and our test of the null hypothesis of equal
means across the g groups H0 t1 t2 ?
tg is rejected at level of significance a is
44
The results of an Analysis of Variance are
usually summarized and displayed in an ANOVA
Table
Treatments
Error
Total
45

• Example consider an expanded experiment based
  on the previous situation
• A sample of ninety respondents is taken
• Each is randomly assigned each to one of six
  groups based on the sugar content (3 levels) and
  acidity (two levels) of the yogurt blends.
• Respondents are asked to rate the overall
  palatability and purchase intent for their
  assigned yogurt blends on a 100 point scale.

46
For our design
The (previously identified) contrasts of
interest are still sugar content main
effect acidity main effect sugar content x
acidity interaction for either variable
(palatability or purchase intent), but are now
much more complex.
47
We have the following raw data on our two
variables
What model is appropriate?
48
common covariance
In MANOVA we must assume - - These
random samples are taken independently from the g
populations Thus we can decompose any
observation Xlj in the data set as

and its estimate as
This, of course, leads to the reparameterization
49
So our null hypothesis of equal means across the
g groups becomes H0 t1 t2 ? tg Note
also that, if the assumptions are met, then el
Np(0, S) and that we must impose a restriction
such as






to achieve unique least-squares estimates (this
is the most common of such restrictions).
50
The decomposition of the jth observation of the
lth group
suggests a similar sample decomposition
estimate of m
estimate of tl
estimate of el



51
_
If we subtract the overall sample mean x from
both sides and square the results
then sum both sides over the subscripts j and l,
the result is the multivariate generalization of
the classic partitioning of the sums of squares
SSres (residual or error or within samples SS)
SStr (treatment or model or between samples SS)
SStot or SScor (total corrected SS)
52
The resulting MANOVA Table looks very similar to
an ANOVA Table
Treatments
Error
Total
Sometimes called sums of squares and
crossproducts (SSCPs or SSPs)
Note the lack of Mean Squares - their
distributions are much more complex in
p-dimensions.
53
One approach (suggested by Wilks 1932) to
testing the null hypothesis of equal mean vectors
across the g groups H0 t1 t2 ? tg is
the ratio of generalized variances



This is related to the likelihood ratio
criterion.
54
Under certain conditions, the exact distribution
of this Wilks lambda (L) is known
?2
1
2
?2
?1
2
?1
3
55
Bartlett 1938 showed that
so we can reject the null hypothesis of equal
mean vectors across the g groups H0 t1 t2 ?
tg at an a level of significance if



56
Recall the yogurt blend experiment. By asking
respondents to rate the overall palatability and
purchase intent, p2.
This is the resulting experimental design.
57
Hypothesis testing to assess the potential
validity of our assertion at a 0.01. -
State the Null and Alternative Hypotheses
- Select the Appropriate Test Statistic we
would like to do a MANOVA, but are the covariance
matrices equal?
58
Lets look at the sample covariance matrices
SLow Sugar SModerate Sugar and SHigh Sugar

• These do look reasonably similar.
• The Johnson Wichern suggested criteria for
  assuming equality of covariance matrices is that
  entries range such that one entry no more than 4
  times the other.
• These ratios appear to be in the 1.5 1.8 range.

59
- Since the data appear relatively normal, the
population covariance matrices appear to be
(relatively) equal, and we have p 2 variables
and g 3 groups, well use an exact L test
where
60
- State the Desired Level of Significance a,
Find the Critical Value(s) and State the Decision
Rule We have a 0.01 and p 2, nLow Sugar
nModerate Sugar nHigh Sugar p - 2 86
degrees of freedom, so f4,86 (0.01) 3.545
1-a0.99
a0.01
99 Do Not Reject Region
Reject Region
3.545
So we have Decision rule do not reject H0 if F
? 3.545 otherwise reject H0
61
- Calculate the Test Statistic First we
calculate the sums of squares
and use these results to calculate the value of
the test statistic Wilks lambda (L)
62
and the calculated value of the test statistic
is
0.44066
- Use the Decision Rule to Evaluate the Test
Statistic and Decide Whether to Reject or Not
Reject the Null Hypothesis 0.44066 ? 3.545 so
do not reject H0. The sample evidence does not
the refute the claim that no simultaneous
difference exists between mean palatability and
purchase intent ratings for the low, moderate,
and high sugar content yogurts.
63
Note also that we can accommodate many
experimental design (including multiple factors,
covariates, and blocks interactions, nonlinear
effects, nesting, etc.) in MANOVA. For example,
consider a Balanced Two-Factor MANOVA the model
(including interaction) is
where elkr Np(0, S) and independent.



64
Note again that we must impose a restriction
such as
to achieve unique least-squares estimates (this
is the most common set of such restrictions).
65
So our null hypothesis of equal means across the
g groups of factor 1 is H0 t1 t2 ?
tg our null hypothesis of equal means across
the b groups of factor 2 is H0 b1 b2 ?
bb and our null hypothesis of no interaction
is H0 g11 g12 ? ggb 0 As always, we
test the higher order terms (interactions first).









66
In a manner analogous to the univariate
two-factor model, we can partition the variation
in responses as follows
SSPfac1 (Factor 1 SS)
SSPtot or SSPcor (total corrected SS)
SSPfac2 (Factor 2 SS)
SSPint (Interaction SS)
SSPres (residual or error or within samples SS)
67
The resulting MANOVA Table looks very similar to
an ANOVA Table
Factor 1
Factor 2
Interaction
Error
Total
Again note the lack of Mean Squares.
68
For the likelihood ratio test of the null
hypothesis of no interaction H0 g11 g12 ?
ggb 0 we reject the null hypothesis at the a
significance level if the value of the test
statistics



where
69
Similarly, for the likelihood ratio test of the
null hypothesis of no main effect associated with
Factor 1 H0 t1 t2 ? tg we reject the
null hypothesis at the a significance level if
the value of the test statistics



where
70
and for the likelihood ratio test of the null
hypothesis of no main effect associated with
Factor 2 H0 b1 b2 ? bb we reject the
null hypothesis at the a significance level if
the value of the test statistics



where
71
Example consider again the yogurt blends
experiment but now considering the two factor
model.
We will look at both sugar content and acidity
main effects and the interaction on both
variables (Palatability and Purchase Intent)
jointly.
72
Considering the two factors (sugar content and
acidity) with a potential interaction, the
appropriate sums of squares and crossproducts
matrices are
73
For the likelihood ratio test of the null
hypothesis of no interaction H0 g11 g12 ?
ggb 0 we have a Wilks lambda of



so the calculated value of the test statistic is
74
The critical value is
and 56.076 ? 13.277 so we reject H0
evidence interaction effect for the two variables
(palatability and purchase intent ratings) across
the six combinations of sugar content acidity.
75
Similarly, for the likelihood ratio test of the
null hypothesis of no main effect associated with
Factor 1 H0 t1 t2 ? tg we have a Wilks
lambda of



so the calculated value of the test statistic is
76
The critical value is
and 5.246 ? 13.277 so we do not reject H0
no evidence of a difference in the two variables
(palatability and purchase intent ratings) across
the three levels of sugar content (low, moderate,
and high).
77
and for the likelihood ratio test of the null
hypothesis of no main effect associated with
Factor 2 H0 b1 b2 ? bb we have a Wilks
lambda of



so the calculated value of the test statistic is
78
The critical value is
and 150.3442 ? 9.2104 so we reject H0 the
sample results refute the claim that there is no
difference in the two variables (palatability and
purchase intent ratings) across the two levels of
acidity (low and high).
79
Special Applications of MANOVA

• Suppose a series of p treatments are administered
  to two or more groups of subjects. We could use
  MANOVA to directly test the null hypothesis
• H0 m1 m2 ? mg
• We gain little understanding of where differences
  between the mean vectors exist if we reject this
  hypothesis.
• One approach to discerning where differences
  between the mean vectors exist is Profile
  Analysis.

80

• The mean vectors could actually differ because
• Differences between successive means are
  different across groups

Differences between successive means are
different across groups
Differences between successive means are similar
across groups
Here we are actually considering whether the
group profiles are parallel.
81

• If the group profiles are parallel, we consider
  whether equality of the group profiles
• Means are different across groups

Means are different across groups
Means are similar across groups
Here we are actually considering whether the
group profiles are coincident.
82

• If the group profiles are coincident, we can
  consider equality of individual components of
  group profiles
• Means are different within groups

Means are different within groups
Means are similar within groups
Here we are actually considering whether the
group profiles are level or constant.
83

• This suggests a stepwise approach to profile
  analysis
• Test for Parallel Profiles (equality of
  differences between successive means across
  groups)
• Here the null hypothesis is
• H01 Cm1 Cm2

When comparing two groups, this is rejected at
the a significance level if
Where the contrast matrix C is of dimension (p
1) x p

84

• If we conclude the profiles are parallel, we
  then test to see if they are coincident
• Test for Coincident Profiles (equality of means
  across groups)
• Here the null hypothesis is
• H02 1m1 1m2

Because parallel profiles will have equal sums
iff their components (individual means) are
equal.
85

• If we conclude the profiles are also coincident,
  we then test to see if they are level
• Test for Level Profiles (equality of combined
  means)

Here we are concerned with the mean vector over
all groups m. The null hypothesis is H03 Cm
0 We pool the grouped data here because parallel
and coincident profiles will have equal
components (individual means). When comparing two
groups, this is rejected at the a significance
level if