Subspace Clustering

1
Subspace Clustering

• Ali Sekmen
• Computer Science
• College of Engineering
• Tennessee State University

1st Annual Workshop on Data Sciences
2
Outline

• Subspace Segmentation Problem
• Motion Segmentation
• Principal Component Analysis
• Dimensionality Reduction
• Spectral Clustering
• Presenter
• Dr. Ali Sekmen

3
Subspace Segmentation

• In many engineering and mathematics applications,
  data lives in a union of low dimensional
  subspaces
• Motion segmentation
• Facial images of a person with the same
  expression under different illumination
  approximately lie on the same subspace

4
Face Recognition
5
Problem Statement
6
Problem Statement
7
Problem Statement
8
What are we trying to solve?
9
Example Motion Segmentation
10
Motion Segmentation
Motion segmentation problem can simply be defined
as identifying independently moving rigid objects
in a video.
11
Motion Segmentation
12
Motion Segmentation
Z
Y
X
13
Motion Segmentation
14
Motion Segmentation
15
Motion Segmentation
16
Motion Segmentation
Y
X
17
Motion Segmentation
Motion Segmentation
18
Motion Segmentation
19
Motion Segmentation
20
Principal Component Analysis

• The goal is to reduce dimension of dataset with
  minimal loss of information
• We project a feature space onto a smaller
  subspace that represent data well
• Search for a subspace which maximizes the
  variance of projected points
• This is equivalent to linear least square fitting
• Minimize the sum of squared distances between
  points and subspace
• We find directions (components) that maximizes
  variance in dataset
• PCA can be done by
• Eigenvalue decomposition of a data covariance
  matrix
• Or SVD of a data matrix

21
Least Square Approximation
22
Principal Component Analysis
23
Principal Component Analysis
24
PCA with SVD
Coordinates w.r.t. new basis
25
Principal Component Analysis
inch
cm
26
Principal Component Analysis
inch cm
10 28
12 19
15 40
20 47
23 56
26 69
27
Solution with SVD
28
PCA Pre-Processing
inch cm
10 28
12 19
15 40
20 47
23 56
26 69
29
PCA Optimization
30
PCA Reduce Dimensionality
31
PCA Reduce Dimensionality
32
General PCA
33
Spectral Clustering

• A very powerful clustering algorithm
• Easy to implement
• Outperforms traditional clustering algorithms
• Example k-means
• It is not easy to understand why it works
• Given a set of data points and some similarity
  measure between all pairs of data points, we
  divide data into groups
• Points in the same group are similar
• Points in different groups are dissimilar

34
Spectral Clustering

• Most of subspace clustering algorithms employ
  spectral clustering as the last step

35
Similarity
36
Spectral Clustering
37
Spectral Clustering
38
Spectral Clustering
39
Spectral Clustering Example
From Lecture Notes of Ulrike von Luxburg
40
Spectral Clustering Example
From Lecture Notes of Ulrike von Luxburg
41
Spectral Clustering Example
From Lecture Notes of Ulrike von Luxburg
42
Spectral Clustering Example
From Lecture Notes of Ulrike von Luxburg
43
Spectral Clustering Example
From Lecture Notes of Ulrike von Luxburg
44
Spectral Clustering Example
45
Spectral Clustering Example
46
Spectral Clustering Example
47
(No Transcript)