Unsupervised Learning - PCA

1
Unsupervised Learning - PCA

• The neural approach-gtPCA SVD kernel PCA
• Hertz chapter 8
• Presentation based on Touretzky
• various additions

2
(No Transcript)
3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
Use a set of pictures of faces to construct a PCA
space. Shown are first 25 principal components.
(C. DeCoro)
29
(No Transcript)
30
Variance as function of number of PC
31
Each iteration, from left to right, corresponds
to addition of 8 principal components
32
Matching general images to faces using eigenfaces
space
33
(No Transcript)
34
Singular Value Decomposition
SVD involves expanding an mxn matrix X of rank
kmin(m,n) into a sum of k unitary matrices of
rank 1, in the following way
This can be rewritten in the matrix representation
where ?is a (non-square) diagonal matrix, and U,V
are orthogonal matrices. Ordering the non-zero
elements of ? in descending order, we can get an
approximation of lower rank r of the matrix X by
choosing 0 for jgtr leading to
35
Singular Value Decomposition continued
This is the best approximation of rank r to X,
i.e. it leads to the minimal sum of square
deviations
36
Relation between SVD and PCA
SVD uses the same unitary transformations as PCA
performed on the rows or columns of X (using the
columns or rows as feature spaces). The singular
values of SVD are the square toots of the
eigenvalues of PCA.
37
SVD Singular Value Decomposition
38
Applications of SVD

• Dimensionality reduction, compression
• Noise reduction
• Pattern search, clustering

Example microarray of expression data, DNA
chips
39
Affymetrix GeneChip probe array
Image courtesy of Affymetrix
40
Hybridization of tagged probes
Image courtesy of Affymetrix
41
Microarray Experiment Result
42
SVD example gene expressions of nine rats
43
SVD Singular Value Decomposition
44
Alter et al 2000identify eigengenes that are
responsible for yeast cell cycle
45
Processing SVD
46
Processing SVD 1 dimension
47
Processing SVD 10 dimensions
48
Processing SVD 100 dimension
49
Kernel PCA
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
(No Transcript)