Psychology 516

1
Psychology 516 Applied Multivariate Statistics
2

• Major Goals
• Expand your repertoire of analytical options.
• Understand enough theory to appreciate
  appropriate and inappropriate application.
• Be able to use and interpret available software.
• Know where to go for additional help.

3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
The basic starting point for any statistical
analysis is a matrix of data. For most
applications in the social sciences, this matrix
will be a People x Variables array. But, the
objects of measurement need not be peoplethey
could be animals, work groups, cities, etc.
7
The variables (V) can be continuous measures,
categories represented by numbers,
transformations, products or combinations of
other variables.
8
Nearly all statistical proceduresunivariate and
multivariateare based on linear combinations.
Understanding that basic fact has far-reaching
implications for using statistical procedures to
their fullest advantage. A linear combination
(LC) for a particular person (i) is nothing more
than a weighted (W) sum of variables (V) LCi
W1Vi,1 W2Vi,2 . . . WKVi,K
9
LCi W1Vi,1 W2Vi,2 . . . WKVi,K A very
simple example is the total score on a
questionnaire. The individual items on the
questionnaire are the variables V1, V2, V3, etc.
The weights are all set to a value of 1 (i.e., W1
W2 . . . Wk 1).
10
The items combined in a linear combination need
not be variables. In statistics, the items
combined are often people (P). LCj W1P1,j
W2P2,j . . . WNPN,j A good example is the
sample mean. In this case the weights are set to
the reciprocal of the sample size (i.e., W1 W2
. . . Wk 1/N).
11
Different statistical procedures derive the
weights (W) in a linear combination to either
maximize some desirable property (e.g., a
correlation) or to minimize some undesirable
property (e.g., error). The weights are sometimes
empirically determined and sometimes they are
dictated by theory (e.g., dummy, effect, and
contrast codes) to produce linear combinations of
particular interest.
12
The simplest possible inferential statistic-the
bivariate correlation-involves just two variables.
r
13
Continuous
Continuous
In its usual form, the correlation is calculated
on variables that are continuous.
r
14
Continuous
Categorical
When one of the variables is categorical, the
calculation produces a point-biserial correlation.
r
15
Categorical
Categorical
When both variables are categorical, the
calculation produces a phi coefficient.
r
16
All forms of these correlations, however, can be
recast as a linear combination V2 BV1 A

r
17
B and A can be chosen so that the sum of the
squared deviations between V2 and V2 are
minimized. This is the ordinary least squares
(OLS) rulean error minimization procedure.
Solving for B and A using this rule also produces
the maximum possible correlation between V2 and
V2.


r
18

If we standardize the variables, then V2 bV1
and r b.
r
19
The problem can be easily expanded to include
more than one predictor. This is a multiple
regression problem, easily cast as a linear
combination V4 B1V1 B2V2 B3V3 A
Continuous
Continuous

R
20
The values for B1, B2, B3, and A are found by the
least squares rule minimize the sum of the
squared differences between V4 and V4. This also
produces the maximum possible correlation between
V4 and V4.

Continuous
Continuous

R
21
V1, V2, and V3 could be categorical contrast
variables, perhaps coding the two main effects
and the interaction from an experimental design.
In that case, the multiple regression produces an
analysis of variance.
Continuous
Categorical
R
22
Although not obvious here, dummy codes, effect
codes, and contrast codes produce linear
combinations of people.
Continuous
Categorical
R
23
Or V2 might be the square of V1 and V3 might be
the cube of V1. Then the multiple regression
examines the polynomial trends.
Continuous
Continuous
R
24
V4 might be a linear combination of other
variables. For example, if V5 is a pretest and V6
is a posttest, we might define V4 as the
difference between V6 and V5. V4 W6V6 W5V5 ,
where W6 1 and W5 -1. In this case, the
weights are theoretical, not derived empirically.
R
25
The analysis now becomes a repeated measures
multiple regression. If V1, V2, and V3 are
categorical, it is a repeated measures analysis
of variance.
R
26
If the outcome variable is categorical, the
basic nature of the analysis does not change. We
still seek an optimal linear combination of V1,
V2, and V3.
Categorical
R
27
When the outcome variable is unstructured, two
approaches are common discriminant analysis and
logistic regression. In this case, the nature of
the categories has not been imposed by the
researcher.
Categorical
R
28
If the outcome categories have been imposed by
the researcher, as would be true if the groups
were levels of an experimental variable, then the
problem becomes a multivariate analysis of
variance, although discriminant analysis and
logistic regression could be used as well.
Categorical
R
29
The basic multiple regression problem can be
generalized to situations that involve more than
one outcome variable.
R
30
Set A Now we seek a linear combination from each
set of variables, with weights derived so that
the correlation between the linear combinations
is maximized.
Set B
R
31
Set B LCB W7V7 W8V8 W9V9
W10V10 W11V11 W12V12
Set A LCA W1V1 W2V2 W3V3 W4V4
W5V5 W6V6
We seek weights in each linear combinations that
maximize the correlation between the linear
combinations. This is known as a canonical
correlation.
RAB
32
Sometimes we are not interested in relations
between sets of variables but instead focus on a
single set and seek a linear combination that has
desirable properties.
33
For example, we might seek a linear combination
of V1 through V12 that captures most of the key
information in those variables. If such a linear
combination exists, we could replace 12 variables
with 1 new variable, simplifying other analyses.
34
Or we might wonder how many dimensions underlie
the 12 variables. These multiple dimensions also
would be represented by linear combinations,
perhaps constrained to be uncorrelated. These
questions are addressed in principal components
analysis and factor analysis.
35
When multiple dimensions or latent variables
underlie a collection of measures, the relations
among those latent variables are also often of
interest.
A
B
C
D
36
Structural equation models examine the relations
among latent variables. Two kinds of linear
combinations are needed.
A
D
B
The relations among the latent variables can also
be represented as a linear combination, similar
to multiple regression D WAA WBB WCC
Each latent variable is a linear combination A
W1V1 W2V2 W3V3
C
37
Sometimes we might shift the status of people
and variables in our analysis. Our interest
might be in whether a smaller number of
dimensions or clusters might underlie the larger
collection of people.
38
Approaches such as multidimensional scaling and
cluster analysis can address such questions.
These are conceptually similar to principal
components analysis, but on a transposed matrix.
39
The key idea is that the original data matrix can
be transformed using linear combinations to
provide useful ways to summarize the data and to
test hypotheses about how the data are
structured. Sometimes the linear combinations are
of variables and sometimes they are of people.
40
Once a linear combination is created, we also
need to know something about its variability.
Everything we need to know about the variability
of a linear combination is contained in the
variance-covariance matrix of the original
variables (S). The weights that are applied to
create a linear combination can also be applied
to S to get the variance of that LC.
s12r12 _______ (s21s22)½
41
Next up . . . Matrix Algebra Statistical
formulas, especially multivariate formulations,
are most conveniently expressed in matrix form
and manipulated using matrix algebra.