Data Analysis Techniques III:

1

• Data Analysis Techniques III
• Multiple Discriminant Analysis
• Factor Analysis

2
Discriminant Analysis
3
Discriminant Analysis

• Used to classify individuals into one of two or
  more alternative groups on the basis of a set of
  measurements
• Used to identify which variable contribute to
  making classification
• Major Uses
• Prediction
• Description

4
Objectives

• Determining linear composites of the predictor
  variables to separate groups by measuring between
  groups variation relative to within-groups
  variation
• Developing procedures for assigning new objects,
  firms, or individuals, whose profiles, but not
  group identity, are known, to one of the two
  groups
• Testing whether significant differences exist
  between the two groups based on the group
  centroids
• Determining which variables account most in
  explaining inter-group differences

5
Discriminant Function

• zi b1 xi1 b2 xi2 b3 xi3 ... bn xin
• Where
• z discriminant score
• b discriminant weights
• x predictor (independent) variables
• In a particular group, each individual has a
  discriminant score (zi)
• S zi centroid (group mean)
• Where i individual
• Centroid
• Indicates most typical location of an individual
  from a particular group

6
Cut-off Score

• Criterion against which each individuals
  discriminant score is judged to determine into
  which group the individual should be classified
• For equal group sizes
• Z cut-off ZA ZB
• 2
• For unequal group size
• Z cut-off NB ZA NA ZB
• Na nb

7
Determination of Significance

• Null Hypothesis
• In the population, the means of all discriminant
  function in all groups are equal
• Ho µA µB
• If Ho is rejected, interpret results

8
Classification and Validation

• Holdout Method
• Use part of sample to construct classification
  rule
• Other subsample used for validation
• Uses classification matrix and hit ratio to
  evaluate groups classification
• Uses discriminant weights to generate
  discriminant scores for cases in subsample

9
Classification and Validation

• Cross-validation Method (leave one out)
• Focus on validation accuracy
• Uses sample minus 1 case (n 1) to estimate the
  discriminant function and to construct the
  classification table function is fitted
  repeatedly until each case has been excluded
• Each case excluded is classified by the function
  derived from the n 1 sample

10
Classification accuracy

• Proportional Chance Criterion
• Cpro p2 (1- p)2
• p proportion in group 1
• 1 p proportion in group 2

11
Interpretation

• Discriminant weights
• standardized canonical discriminant function
  coefficients larger weights contribute more to
  the discriminating power of the function
• Discriminant loadings
• structure matrix measure the linear
  correlation between each independent variable and
  the D function

12
Interpretation

• Steps for interpretation
• 1 group means, standard deviations
• 2 test for equality of group means
• 3 significance level of function
• (Cancorr2 variance explained)
• 4 validation of function classification matrix
• 5 classification accuracy (vs. Cpro)

13
Interpretation

• Generally, predictors with relatively large
  standardized coefficients contribute more to the
  discriminating power of the function
• Canonical or discriminant loadings show the
  variance that the predictor shares with the
  function

14
Factor Analysis
15
Factor Analysis

• Technique
• serves to combine questions or variables to
  create new factors
• Purpose
• identify underlying constructs in the data
• reduce the number of variables to a more
  manageable set

16
Factor Analysis

• Methodology
• Two commonly employed factor analytic procedures
• Principal Component Analysis
• Used when the need is to summarize information in
  a larger set of variables to a smaller set of
  factors
• Common Factor Analysis
• Used to uncover underlying dimensions surrounding
  the original variables

17
Principal Component Analysis

• The objective of factor analysis is to represent
  each of these variables as a linear combination
  of a smaller set of factors
• This can be represented as
• X1 I11F1 I12F2 e1
• X2 I21F1 I22F2 e2
• .
• .
• Xn in1f1 in2f2 en
• Where X1, ... xn represent standardized scores
• F1,F2 are the two standardized factor scores
• I11, i12,....I52 are factor loadings
• E1,...E5 are error variances

18
Factor Analysis Terms

• Factor
• variable or construct that is not directly
  observable but needs to be inferred from the
  input variables
• Factor Scores
• Values of each factor underlying the variables
• Factor Loadings
• Correlations between the factors and the original
  variables

19
Factor Analysis Terms

• Eigenvalue
• Represents the amount of variance in the original
  variables that is associated with a factor (sum
  of the square of factor loadings of each variable
  on a factor represents the eigenvalue)
• Scree Plot
• A plot of the eigenvalues against the number of
  factors, in order of extraction.

20
Factor Analysis Terms

• Communality
• The amount of the variable variance that is
  explained by the factor
• Factor Rotation
• Factor analysis can generate several solutions
  for any data set. Each solution is a particular
  factor rotation, generated by a particular factor
  rotation procedure

21
Factor Analysis Terms

• Percentage of Variance
• Number of factors extracted is determined when
  the cumulative percentage of variance extracted
  reaches a satisfactory level
• Significance Test
• Statistical significance of the separate
  eigenvalues is determined, and only those factors
  that are statistically significant are retained

22
Factor Analysis

• Rule of Thumb
• All included factors (prior to rotation) must
  explain at least as much variance as an "average
  variable (I.e. 1.0)
• Eigenvalues Criteria
• Only factors with eigenvalues greater than 1.0
  are retained

23
Cluster Analysis
24
Cluster Analysis

• Technique that serves to combine objects to
  create new groups
• Used to group variables, objects or people
• The input is any valid measure of correlations
  between objects, such as
• Correlations
• Distance measures (Euclidean distance)
• Association coefficients
• Also, the number of clusters or the level of
  clustering can be input

25
Cluster Analysis Methods

• Hierarchical Clustering
• Can start with all objects in one cluster and
  divide and subdivide them until all objects are
  in their own single-object cluster
• Non-hierarchical Approach
• Permits objects to leave one cluster and join
  another as clusters are being formed

26
Hierarchical Clustering

• Single Linkage
• Clustering criterion based on the shortest
  distance
• Complete Linkage
• Clustering criterion based on the longest
  distance
• Average Linkage
• Clustering criterion based on the average
  distance

27
Hierarchical Clustering

• Ward's Method
• Based on the loss of information resulting from
  grouping of the objects into clusters (minimize
  within cluster variation)
• Centroid Method
• Based on the distance between the group centroids
  (i.e. is the point whose coordinates are the
  means of all the observations in the cluster)

28
Non-hierarchical Clustering

• Sequential Threshold
• Cluster center is selected and all objects within
  a prespecified threshold grouped
• Parallel Threshold
• Several cluster centers are selected and objects
  within threshold level are assigned to the
  nearest center
• Optimizing
• Modifies the other two methods in that the
  objects can be later reassigned to clusters on
  the basis of optimizing some overall criterion
  measure

29
How many clusters?

• Four Ways
• Analyst specifies number in advance
• Analyst specifies levels of clustering in advance
• Determine number of clusters from the pattern of
  clusters generated in the program
• Plot the ratio of within-group variance and the
  between-group variance against the number of
  clusters. The point at which a sharp bend occurs
  indicates the number of clusters

30
Summary

• Discriminant analysis classifies objects or
  respondents into two or more groups
• A very useful technique to explain which
  independent variables best explain group
  membership
• The critical part is how well a significant
  function classifies individuals into dependent
  groups

31
Summary

• Factor analysis widely used method for
  dimensionalizing unstructured data
• Key points to remember
• - cases at least 4-5 times the of variables
• - if factors equals variables, then each
  variable is unique
• - if only one factor, then all variables explain
  one construct
• - look for highest loading across factors, then
  for highest within
• PC summarized most of the variance in fewest
  factors
• Varimax search for best set of loadings close to
  0 or 1 to improve interpretability
• Cluster Analysis combines objects to create new
  groups