Combined Central and Subspace Clustering for Computer Vision Applications

1
Combined Central and Subspace Clustering
forComputer Vision Applications

• Le Lu René Vidal
• Computer Science Department Center for
  Imaging Science
• Johns Hopkins University Johns
  Hopkins University

2
Motivation

• Central and subspace clustering methods are at
  the core of many segmentation problems in
  computer vision.
• However, both methods fail to give the correct
  segmentation in many practical scenarios, e.g.
  when data points are close to the intersection of
  two subspaces or when two cluster centers in
  different subspaces are spatially close.
• A natural metric of considering both types of
  constraints?

3
Contributions

• We address the problem of clustering a set of
  points lying in a union of subspaces and
  distributed around multiple cluster centers
  inside each subspace.
• We propose a generalization of Kmeans and
  Ksubspaces that clusters the data by minimizing a
  cost function that combines both central and
  subspace distances.

4
Two Toy Examples (1)

• A set of points in R3 drawn from 4 clusters
  labeled as A1, A2, B1, B2. Clusters B1 and B2 lie
  in the X-Y plane and clusters A1 and A2 lie in
  the Y-Z plane. Note that some points in A2 and B2
  are drawn from the intersection of the two planes
  (Y-axis).

5
Two Toy Examples (1)

• Left Subspace clustering by GPCA assigns all the
  points in the Y-axis to the Y-Z plane, thus it
  misclassifies some points in B2.
• Right Subspace clustering using GPCA followed by
  central clustering inside each plane using Kmeans
  misclassifies some points in B2.

6
Two Toy Examples (2)

• A set of points in R3 distributed around 4
  clusters labeled as A1, A2 B1, B2. Clusters B1
  and B2 lie in the X-Y plane and clusters A1 and
  A2 lie in the Y-Z plane. Note that cluster B2 (in
  blue) is spatially close to cluster A2 (in red).

7
Two Toy Examples (2)

• Left Central clustering by Kmeans assigns some
  points in A2 to B2.
• Right Subspace clustering using GPCA followed
  by central clustering inside each subspace using
  Kmeans gives the correct clustering into four
  groups.

8
Problem Statement

• Combined Central-Subspace Clustering

9
Objective Functions

• Central Clustering (K-Means)
• Subspace Clustering (K-Subspace)

10
Objective Functions

• Joint Central Subspace Clustering

11
Objective Functions

• By using Lagrange multipliers

12
Algorithm

1. Initialization Obtain an initial estimate of
   the normal vectors bjj1n and the cluster
   centers µjkj1n k1mj using GPCA followed by
   Kmeans in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

13
Computing the memberships

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

14
Computing the cluster centers

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

15
Computing the normal vectors

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

16
Remark 1

• Extension from hyperplanes to subspaces
• Bj Null Space of

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

17
Remark 2

• Maximum Likelihood Solution

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

18
Remark 2

• Covariances (variances inside and orthogonal to
  the hyperplanes)
• Variances

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

19
Clustering Performance (simulated)

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

20
Clustering Performance (simulated)

• Top Clustering error as a function of noise in
  the data.
• Bottom Error in the estimation of the normal
  vectors (degrees) as a function of the level of
  noise in the data.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

21
Illumination-Invariant Face Clustering

• YALE Face Database B (64 images per subject with
  fixed pose and changing illumination)

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

22
Illumination-Invariant Face Clustering

• GPCA (1)

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

23
Illumination-Invariant Face Clustering

• Mixture of PPCA (2)

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

24
Illumination-Invariant Face Clustering

• GPCA-KMeans Joint Optimization (3)

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

25
Video Shot Segmentation

• http//www.open-video.org
• Mountain Sequence Drama Sequence

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

26
Video Shot Segmentation

• GPCA (1) for Mountain Sequences

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

27
Video Shot Segmentation

• Kmeans (2) for Mountain Sequences

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

28
Video Shot Segmentation

• GPCA-KMeans Joint Optimization (3) for Mountain
  Sequences

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

29
Video Shot Segmentation

• GPCA (1) for Drama Sequences

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

30
Video Shot Segmentation

• GPCA-KMeans Joint Optimization (2) for Drama
  Sequences

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

1. Initialization Obtain an initial estimate of the
   normal vectors bjj1n and the cluster centers
   µjkj1n k1mj using GPCA followed by Kmeans
   in each subspace.
2. Computing the memberships Given the normal
   vectors bjj1n and the cluster centers
   µjkj1nk1mj, compute the memberships
   wijk.
3. Computing the cluster centers Given the
   memberships wijk and the normal vectors
   bjj1n, compute the cluster centers
   µjkj1nk1mj.
4. Computing the normal vectors Given the
   memberships wijk and the cluster centers
   µjkj1nk1mj, compute the normal vectors
   bjj1n.
5. Iterate Repeat steps 2,3,4 until convergence of
   the memberships.

31
Conclusion and Discussion

• A simple, geometrically intuitive clustering
  method of combining central and subspace
  constraints to solve computer vision problems.
• Insights to solve the intrinsic subspace
  clustering ambiguity of subspace intersections.
• Model Selection Duality to mixture of factor
  analysers and variational bayesian approach
  Ghahramani, Beal, 2000?
• Interesting Related Work Hyper-Graph Clustering
  Agarwal, Belongie, et al. 2005, 2006