Multivariate Analysis and Data Reduction

1
Multivariate Analysis and Data Reduction
2
Multivariate Analysis

• Multivariate analysis tries to find patterns and
  relationships among multiple dependent variables.
• Situations where multiple dependent variables are
  common
• Test validation and development of scales.
• Multi-measure paradigms (e.g., physiological
  psychology).
• Complex sociological studies, including surveys
  and archival research.

3
Extending Correlation

• Correlation examines the type and extent of
  relationship between two variables
• Positive (direct), negative (inverse), no
  relation.
• A correlation matrix shows relationships among
  multiple pairs of variables (average
  interrcorrelation can be calculated for the
  matrix).
• What if latent causes exist that affect many of
  the variables in a study?
• What if several different causes have different
  impacts on different of the variables?

4
Factor Analysis

• Factors are underlying constructs (causes) for
  the variability in a set of multiple variables.
• Factor analysis applies matrix algebra to a
  correlation matrix in order to extract a set of
  common factors.
• The relationship between each variable and the
  extracted factors is quantified by factor
  loadings.
• Eigenvalues show how much of the variance is
  explained by each factor.

5
Goals of Factor Analysis

• The goal of factor analysis is to make sense of
  the overlap and relationships among multiple
  dependent variables (measures).
• The procedure seeks a more economical explanation
  of the observed behavior.
• The number of important factors should be less
  than the number of variables input.
• It is up to the investigator to make sense out of
  the factors identified by the analysis.

6
Rotation

• Factor analysis is an iterative process.
• It begins by trying to find the factor that
  produces the highest correlations among a set of
  variables.
• It then locates the factor that produces the
  second highest correlations, and so forth.
• You decide how many factors are relevant.
• This original factor structure may be difficult
  to interpret, so factors are rotated to find a
  meaningful interpretation.

7
Interpretation of Results

• Orthogonal factors are at right angles to each
  other.
• Orthogonal factors are assumed to be independent
  of each other.
• Varimax rotation produces orthogonal rotations.
• Ideally, a variable should load high on one
  factor and low or not at all on other factors.
• In reality, factors may be complex to interpret.
• The emergent structure depends on the input.

8
Two Different Rotations of the Same Intelligence
Data

• Spearmans G
• The main factor that emerges when minimum
  residual factor analysis is applied to multiple
  measures of intelligence (e.g., on an IQ test).
• Gardners Multiple Intelligences
• A factor structure consisting of the maximum
  orthogonal factors emerging from the same
  analysis of multiple measures of intelligence
  with Varimax rotation.
• Both are valid solutions for the same data.

9
Cluster Analysis

• A method of reducing multiple input variables (or
  cases/subjects) into larger groups based on
  similarity of scores.
• Cluster analysis is used when you want to group
  cases or variables but dont already know which
  belong together.
• Similarity of features, not shared variance is
  the basis for clustering.

10
Interpretation of Cluster Analysis Results

• Two approaches putting items together
  (agglomorative) or dividing large groups into
  smaller ones (divisive).
• The technique provides the solution but it must
  be interpreted by the investigator.
• The structure of the solution is determined by
  the number and type of input variables or cases
  (subjects).
• Systematic or valid sampling is essential.

11
Discriminant Analysis

• Discriminant analysis uses the characteristics of
  variables to predict membership in defined
  groups.
• Used when the group membership is already known.
• The relationship between the groups and the
  variables is analyzed, resulting in factor
  loadings for discriminant functions.
• Input variables can be changed to test different
  models for predicting groups.

12
Multidimensional Scaling

• Similarities and differences among items are used
  to create a plot showing relationships among
  them.
• Input is similarity judgments, distances, or
  correlation matrix for multiple variables.
• Similarities are converted to distances for
  plotting.
• An iterative calculation figures out the plot
  that best retains the relative distances among
  all items.
• Dimensions can be inferred from the plot.

13
Important Measures of Model Fit

• Frequently, more than two dimensions are needed
  to accurately display relationships among cases.
• Stress shows how well the locations of the items
  fit within the number of dimensions requested.
• The more items, the greater the stress.
• The more dimensions, the lower the stress.
• Higher dimensionality is difficult to interpret.
• The highest dimensions are noise (error).

14
Individual Differences Scaling

• Some methods take into account that different
  individuals emphasize different dimensions when
  making judgments about items.
• The relative weights for each dimension, for each
  subject, can be analyzed.