Principal Components

1
Principal Components

• Shyh-Kang Jeng
• Department of Electrical Engineering/
• Graduate Institute of Communication/
• Graduate Institute of Networking and Multimedia

2
Concept of Principal Components
x2
x1
3
Principal Component Analysis

• Explain the variance-covariance structure of a
  set of variables through a few linear
  combinations of these variables
• Objectives
• Data reduction
• Interpretation
• Does not need normality assumption in general

4
Principal Components
5
Result 8.1
6
Proof of Result 8.1
7
Result 8.2
8
Proof of Result 8.2
9
Proportion of Total Variance due to the kth
Principal Component
10
Result 8.3
11
Proof of Result 8.3
12
Example 8.1
13
Example 8.1
14
Example 8.1
15
Geometrical Interpretation
16
Geometric Interpretation
17
Standardized Variables
18
Result 8.4
19
Proportion of Total Variance due to the kth
Principal Component
20
Example 8.2
21
Example 8.2
22
Principal Components for Diagonal Covariance
Matrix
23
Principal Components for a Special Covariance
Matrix
24
Principal Components for a Special Covariance
Matrix
25
Sample Principal Components
26
Sample Principal Components
27
Example 8.3
28
Example 8.3
29
Scree Plot to Determine Number of Principal
Components
30
Example 8.4 Pained Turtles
31
Example 8.4
32
Example 8.4 Scree Plot
33
Example 8.4 Principal Component

• One dominant principal component
• Explains 96 of the total variance
• Interpretation

34
Geometric Interpretation
35
Standardized Variables
36
Principal Components
37
Proportion of Total Variance due to the kth
Principal Component
38
Example 8.5 Stocks Data

• Weekly rates of return for five stocks
• X1 Allied Chemical
• X2 du Pont
• X3 Union Carbide
• X4 Exxon
• X5 Texaco

39
Example 8.5
40
Example 8.5
41
Example 8.6

• Body weight (in grams) for n150 female mice were
  obtained after the birth of their first 4 litters

42
Example 8.6
43
Comment

• An unusually small value for the last eigenvalue
  from either the sample covariance or correlation
  matrix can indicate an unnoticed linear
  dependency of the data set
• One or more of the variables is redundant and
  should be deleted
• Example x4 x1 x2 x3

44
Check Normality and Suspect Observations

• Construct scatter diagram for pairs of the first
  few principal components
• Make Q-Q plots from the sample values generated
  by each principal component
• Construct scatter diagram and Q-Q plots for the
  last few principal components

45
Example 8.7 Turtle Data
46
Example 8.7
47
Large Sample Distribution for Eigenvalues and
Eigenvectors
48
Confidence Interval for li
49
Approximate Distribution of Estimated Eigenvectors
50
Example 8.8
51
Testing for Equal Correlation
52
Example 8.9
53
Monitoring Stable Process Part 1
54
Example 8.10Police Department Data
First two sample cmponents explain 82 of the
total variance
55
Example 8.10Principal Components
56
Example 8.1095 Control Ellipse
57
Monitoring Stable Process Part 2
58
Example 8.11T2 Chart for Unexplained Data
59
Example 8.12 Control Ellipse for Future Values
Example 8.10 data after dropping out-of-control
case
60
Example 8.12 99 Prediction Ellipse
61
Avoiding Computation with Small Eigenvalues