Canonical Correlation Analysis

1
Canonical Correlation Analysis

• Shyh-Kang Jeng
• Department of Electrical Engineering/
• Graduate Institute of Communication/
• Graduate Institute of Networking and Multimedia

2
Canonical Correlation Analysis

• Seeks to identify and quantify the association
  between two sets of variables
• Examples
• Relating arithmetic speed and arithmetic power to
  reading speed and reading power
• Relating government policy variables with
  economic goal variables
• Relating college performance variables with
  precollege achievement variables

3
Canonical Correlation Analysis

• Focuses on the correlation between a linear
  combination of the variables in one set and a
  linear combination of the variables in another
  set
• First to determine the pair of linear
  combinations having the largest correlation
• Next to determine the pair of linear combinations
  having the largest correlation among all pairs
  uncorrelated with the initially selected pair,
  and so on

4
Canonical Correlation Analysis

• Canonical variables
• Pairs of linear combinations used in canonical
  correlation analysis
• Canonical correlations
• Correlations between the canonical variables
• Measures the strength of association between the
  two sets of variables
• Maximization aspect
• Attempt to concentrate a high-dimensional
  relationship between two sets of variables into a
  few pairs of canonical variables

5
Example 10.5 Job Satisfaction
6
Example 10.5 Job Satisfaction
7
Canonical Variables and Canonical Correlations
8
Canonical Variables and Canonical Correlations

• Covariances between pairs of variables from
  different sets are contained in S12 or,
  equivalently S21
• When p and q are relatively large, interpreting
  the elements of S12 collectively is very
  difficult
• Canonical correlation analysis can summarize the
  associations between two sets in terms of a few
  carefully chosen covariances rather than the pq
  covariances in S12

9
Canonical Variables and Canonical Correlations
10
Canonical Variables and Canonical Correlations

• First pair of canonical variables
• Pair of linear combinations U1, V1 having unit
  variances, which maximize the correlation
• kth pair of canonical variables
• Pair of linear combinations Uk, Vk having unit
  variances having unit variances, which maximize
  the correlation among all choices uncorrelated
  with the previous k-1 canonical variable pairs

11
Result 10.1
12
Result 10.1
13
Result 10.1
14
Proof of Result 10.1
15
Proof of Result 10.1
16
Proof of Result 10.1
17
Proof of Result 10.1
18
Proof of Result 10.1
19
Canonical Variates
20
Comment
21
Comment
22
Example 10.1
23
Example 10.1
24
Example 10.1
25
Alternative Approach
26
Identifying Canonical Variablesby Correlation
27
Example 10.2
28
Canonical Correlations vs. Other Correlation
Coefficients
29
Example 10.3
30
Sample Canonical Variates and Sample Canonical
Correlations
31
Result 10.2
32
Matrix Forms
33
Sample Canonical Variates for Standardized
Observations
34
Example 10.4
35
Example 10.5 Job Satisfaction
36
Example 10.5 Job Satisfaction
37
Example 10.5 Sample Correlation Matrix Based on
784 Responses
38
Example 10.5 Canonical Variate Coefficients
39
Example 10.5 Sample Correlations between
Original and Canonical Variables
40
Matrices of Errors of Approximations
41
Matrices of Errors of Approximations
42
Matrices of Errors of Approximations
43
Example 10.6
44
Example 10.6
45
Example 10.6
46
Sample Correlation Matrices between Canonical and
Component Variables
47
Proportion of Sample Variances Explained by the
Canonical Variables
48
Proportion of Sample Variances Explained by the
Canonical Variables
49
Example 10.7
50
Result 10.3
51
Bartletts Modification
52
Test of Significance of Individual Canonical
Correlations
53
Example 10.8
54
Example 10.8