Inference about a Mean Vector

1
Inferences about a Mean Vector

• Basic Inference about a Single Mean m
• Hypothesis Testing
• Scientific method-based
• Evaluates conjectures about a population.
• Null Hypothesis
• Statement about the parameter that includes (but
  is not necessarily limited to) equality.
• Usually denoted
• H0 parameter (? ?) hypothesized value
• Indicates perceived truth or status quo.
• Believe differences are due to random or chance
  variation.

2

• Alternative Hypothesis
• Contradicts the null.
• Never includes equality (and so includes the lt
  and/or gt relationship between conjectured value
  and the tested parameter.
• Usually denoted
• H1 parameter (lt ? gt) hypothesized value.
• Believe the difference between the observations
  and the hypothesized value are systematic (i.e.,
  due to something other than random or chance
  variation.

3

• Critical Region
• Area containing all possible estimated values of
  the parameter that will result in rejection of
  the null hypothesis.
• Critical Value(s)
• The value(s) that divide the critical region(s)
  from the do not reject region
• Test Statistic Sample
• Data-based value that will be compared to the
  critical region(s). The generic form of the test
  statistic is

4

• Decision Rule
• Statement that specifies values of the test
  statistic that will result in rejection or
  non-rejection of the null hypothesis
• Conclusion
• Statement of whether the null hypothesis was
  rejected or not.

Note that our conclusion is stated with regards
to the null hypothesis, which can either be i)
rejected or ii) not rejected that is, we never
accept the null hypothesis.
5
Errors

• Type I Error
• Rejection of a true null hypothesis. The
  probability is denoted as ?.
•  
• Type II Error
• Non-rejection of a false null hypothesis. The
  probability is denoted as ?.

6
Risks

• Level of Significance
• The probability of rejecting the null when it is
  actually true. P(Type I Error) a.
• Usually determine to be the maximum desired risk
  of committing a type one error.
• Observed Level of Significance or P-Value
• Estimated probability selecting a sample at least
  as contradictory to the null as the current
  sample. Observed risk of incorrectly rejecting
  the null.
• Conclusions
• Reject the null if p-value lt a.

7
Steps in Hypothesis Testing
Translate the Practical Problem into a
Statistical Problem - State the Null
Alternative Hypotheses - Select the
Appropriate Test Statistic - State the Desired
Level of Significance a, Find the Critical
Value(s) and State the Decision Rule
- Calculate the Test Statistic - Evaluate the
Test Statistic and Decide Whether to Reject or
Not Reject the Null Hypothesis. Translate the
Statistical conclusion into a Practical Solution.
8
Inference about a Mean Vector m

A natural generalization of the squared
univariate distance t is the multivariate analog
Hotellings T2
where
_
_
_
note that here n-1S is the estimated covariance
matrix of X.


9
This gives us a framework for testing hypotheses
about a mean vector, where the null and
alternative hypotheses are H0 ? ?0 H1 ? ?
?0 The T2 statistic can be rewritten as


multivariate normal Np(m,S) random vector
multivariate normal Np(m,S) random vector
(Wishart Wp,n 1(S) random matrix/df)-1



10
So when the null hypothesis is true, the T2
statistic can be written as the product of two
multivariate normal Np(m,S) and a Wishart Wp,n
1(S) distribution this is a complete
generalization of the univariate case, where we
could write the squared test statistic t2 as


univariate normal N(m,s) random variable
univariate normal N(m,s) random variable
(Chi Square random variable/df)-1
T2 is distributed as a multiple of the
F-distribution.
What happens to this in the univariate case?
11
Example
At a significance level of a 0.10, do these
data support the assertion that they were drawn
from a population with a centroid (4.0,
-1.5)? In other words, test the null hypothesis
12
The scatter plot of pairs (x1, x2), sample
centroid (x1, x2), and hypothesized centroid (m1,
m2) is provided below
hypothesized centroid (m1, m2)0
sample centroid (x1, x2)
_ _
Do these data appear to support our null
hypothesis?
13
The steps of hypothesis testing produce -
State the Null and Alternative Hypotheses
- Select the Appropriate Test Statistic n p
15 2 13 is not very large, but the data
appear relatively bivariate normal, so use
14
a 0.10 and n1 p 2, n2 n p 13 degrees
of freedom, we have f2,13 (0.10) 2.76
0.5-a0.90
a0.10
90 Do Not Reject Region
Reject Region
But we dont yet have a decision rule since
15

• State the Desired Level of Significance a,
• Find the Critical Value(s) and State the Decision
  Rule

Thus our critical value is
So we have Decision rule do not reject H0 if
T2 ? 5.95 otherwise reject H0
16
- Calculate the Test Statistic We have
so
17
- Use the Decision Rule to Evaluate the Test
Statistic and Decide Whether to Reject or Not
Reject the Null Hypothesis T2 5.970 ?
5.95 so reject H0. The sample evidence supports
the claim that the mean vector differs from
18
The results of the univariate tests (on each
variable) also reject their respective null
hypotheses. Why do these and bivariate test act
consistent?
hypothesized centroid (m1, m2)0
sample centroid (x1, x2)
_ _
What do these data suggest about the
relationship between X1 and X2?
19
Suppose the same values for variables X1 and X2
are used but their pairings are changed.
Of course, the univariate test results would not
change but at a significance level of a 0.10,
do these data now support the assertion that they
were drawn from a population with a centroid
(4.0, -1.5)? In other words, retest the null
hypothesis
20
We will have to recalculate the T2 test statistic
(why?)
so
21
Use the Decision Rule to Evaluate the Test
Statistic and Decide Whether to Reject or Not
Reject the Null Hypothesis T2 3.411 ?
5.95 so do not reject H0. The sample evidence
does not support the claim that the mean vector
differs from
22
Why are the results different?
hypothesized centroid (m1, m2)0
sample centroid (x1, x2)
_ _
23
Why is our decision so radically different?
Again, look at the data!
hypothesized centroid (m1, m2)0
sample centroid (x1, x2)
_ _
What do these data suggest about the
relationship between X1 and X2?
24
Hotellings T2 and Likelihood Ratio Tests

• The T2 test was introduced as the generalization
  of the univariate likelihood ratio test.
• The likelihood ratio method may be applied
  directly to the multivariate case.
• The maximum of the multivariate likelihood (over
  possible values of m and S) is given by

See text for remaining details on the equivalence
of the likelihood ratio statistic and T2.
25
Confidence Regions and Simultaneous Comparisons
of Component Means
To extend the concept of a univariate 100(1 - a)
confidence interval for the mean
to a p-dimensional multivariate space, let q be
a vector of unknown population. The 100(1 a)
Confidence Region determined by data X X1, X2
,,Xn is denoted R(X), is the region
satisfying PR(X) will contain the true q 1 -
a





26
For example, a univariate 100(1 - a) confidence
interval for the mean may look like this
A corresponding confidence region in
p-dimensional multivariate space is given by
Equivalently,
provided nS-1 is the appropriate measure of
distance.

27
More formally, a 100(1 - a) confidence region
for the mean vector m of a p-dimensional normal
distribution is the ellipsoid determined by all
possible points m that satisfy


where
28
Example Use our original bivariate sample to
construct a two-dimensional 90 confidence region
and determine if the point (4.0, -1.5) lies in
this region.

We have already calculated the summary
statistics
and
29
So a 100(1 - a) confidence region for the mean
vector m is the ellipsoid determined by all
possible points m that satisfy


30
We can draw the axes of the ellipsoid that
represents the 100(1 - a) confidence region for
the mean vector m the ellipsoid determined by
all possible points m that satisfy


For our ongoing problem at the 90 level of
confidence, this is equivalent to
31
Recall that the directions and relative lengths
of axes of this confidence interval ellipsoid are
determined by going
units along the corresponding eigenvectors
ei. Beginning at the centroid x, the axes of
the confidence interval ellipsoid are

_

32
Example We can draw the 100(1 - a) confidence
region for the mean vector m the eigenvalue-
eigenvector pairs li, ei for the sample
covariance matrix S are



so the half-lengths of the major and minor axes
are given by
33
The axes lie along the corresponding
eigenvectors ei

when these vectors are plotted with the sample
centroid x as the origin
_

34
sample centroid (x1, x2)
_ _
35
sample centroid (x1, x2)
_ _
Now we move ? 3.73 units along the vector e1

and ? 2.793 units along the vector e2.

36
shadow of the 90 confidence region on the X1 axis
shadow of the 90 confidence region on the X2 axis
We can also project the ellipse onto the axes to
form simultaneous confidence intervals.
37
Simultaneous Confidence Statements The joint
confidence region

• for some constant c correctly assesses the
  plausibility of various value of m. However
• We may desire confidence statements about
  individual components of the mean vector m .
• Such intervals are referred to as Simultaneous
  Confidence Intervals.
• One approach to constructing these intervals is
  to relate them to the T2-based confidence region.

38
Suppose X Np(m, S) and that they form the
linear combination


From earlier results we have that
The pertinent sample statistics are
and
39
Simultaneous confidence intervals for choices of
a will produce a 100(1 a) confidence interval
for mz am.


which leads to the statement
40
Considering values of a for which t2 ? c2,

with the maximum occurring where a ? S-1(x m).
41
Thus, for a sample X1,,Xn from the Np(m, S)
population with positive definite S, we have
simultaneously for all a that



will contain am with probability 1 a. These
are often referred to as T2-intervals.

42
Note that successive choices of a 1 0 0 .
0, a 0 1 0 . 0,, a 0 0 0 . 1 for
the T2-intervals allow us to conclude that



will all hold simultaneously with probability 1
a.
43
or that a contrast construction of a (such as
a 1 -1 0 . 0) for the T2-intervals allows
us to test


which again will hold simultaneously with
probability 1 a.
44
shadow of the 90 confidence region on the X1 axis
3.6
6.9
-0.9
shadow of the 90 confidence region on the X2 axis
-5.3
Note that the projections the ellipse onto the
axes do form the simultaneous confidence
intervals.
45
shadow of the 90 confidence region on the X1 axis
univariate 90 confidence interval for X1
4.0
6.5
-1.5
univariate 90 confidence interval for X2
-4.7
shadow of the 90 confidence region on the X2 axis
Note that the univariate intervals are shorter
they do not consider covariance between X1 and X2!
46
If we make a Bonferroni-type adjustment to the
one-at-a-time (univariate) confidence intervals

• p univariate confidence intervals will hold with
  an overall confidence level greater than or equal
  to 1-a.
• The resulting intervals will be shorter than the
  corresponding T2 simultaneous intervals for each
  mi.

47
Large Sample Inferences About a Population Mean
Vector
When the sample is large (n gt gt p) we dont need
the multivariate normality assumption. Recall,
for X MVN we know that

.
so we can say that
48
Let X1,Xn be a random sample from a population
with mean m, positive definite covariance matrix
S, and some arbitrary distribution. When n p is
large, the hypothesis H0 m m0 is rejected
in favor of H1 m ? m0 at a level of
significance a, if




49
The difference between small sample test and the
large sample test is the critical value of the
distribution
In fact, as n p grows, the distribution for
the normal theory (small sample) test almost
surely approaches the distribution of the large
sample test
50
Multivariate Control Charts
For univariate control charts, the centerline and
limits for the x chart chart are given by

• Samples means are plotted on this chart over
  time.
• The chart acts as a hypothesis test for every
  sample.
• Rejection implies a process is out of control.

51
T2-Chart
For a historically stable process, the jth point
is plotted as
The LCL is set at zero, with UCL given by