Segmentation and Clustering

1
Segmentation and Clustering

2
Déjà Vu?

• Q Havent we already seen this with snakes?
• A no, that was about boundaries, this is about
  interiors
• Want regions of similar contents

3
Segmentation via Clustering

• SegmentationDivide imageinto regionsof
  similar contents

• ClusteringAggregate pixelsinto regionsof
  similar contents

4
But Wait!

• We speak of segmenting foregroundfrom background
• Segmenting out skin colors
• Segmenting out the moving person
• How do these relate to similar regions?

5
Segmentation and Clustering

• Defining regions
• Should they be compact? Smooth boundary?
• Defining similarity
• Color, texture, motion,
• Defining similarity of regions
• Minimum distance, mean, maximum

6
Segmentation and Clustering Applications
Foreground /background segmentation
Finding the moving objects
Findingskin-colored regions
Intelligentscissors
Finding thecars in avideo sequence
Semantics
7
Themes

• Energy minimization
• Snakes discretize, greedy minimization
• Snakes, shape from shading differential eqn
• Stereo with smoothness min-cost graph cuts
• Today k-means clustering
• Statistics
• Templates

Hot topic in vision research combining these
8
Segmentation and Clustering Applications
Foreground /background segmentation
Finding the moving objects
Findingskin-colored regions
Intelligentscissors
Finding thecars in avideo sequence
Statistics
Templates
9
Clustering Based on Color

• Lets make a few concrete choices
• Arbitrary regions
• Similarity based on color only
• Similarity of regions distance between mean
  colors

10
Simple Agglomerative Clustering

• Start with each pixel in its own cluster
• Iterate
• Find pair of clusters with smallestinter-cluster
  distance
• Merge
• Stopping threshold

11
Simple Divisive Clustering

• Start with whole image in one cluster
• Iterate
• Find cluster with largest intra-cluster variation
• Split into two pieces that yield largest
  inter-cluster distance
• Stopping threshold

12
Difficulties with Simple Clustering

• Many possibilities at each iteration
• Computing distance between clusters or optimal
  split expensive
• Heuristics to speed this up
• For agglomerative clustering, approximate each
  cluster by average for distance computations
• For divisive clustering, use summary (histogram)
  of a region to compute split

13
k-means Clustering

• Instead of merging or splitting, start out with
  the clusters and move them around

• Pick number of clusters k
• Randomly scatter k cluster centers in color
  space
• Repeat
• Assign each data point to its closest cluster
  center
• Move each cluster center to the mean of the
  points assigned to it

14
k-means Clustering
15
k-means Clustering
16
k-means Clustering
17
k-means Clustering
18
k-means Clustering
19
k-means Clustering
20
k-means Clustering
21
k-means Clustering
22
k-means Clustering

• This process always converges (to something)
• Not necessarily globally-best assignment
• Informal proof look at energy minimization
• Reclassifying points reduces (or maintains)
  energy
• Recomputing centers reduces (or maintains) energy
• Cant reduce energy forever

23
Results of Clustering
Original Image
k-means, k5
k-means, k11
24
Results of Clustering
Sample clusters with k-means clusteringbased on
color
25
Other Distance Measures

• Suppose we want to have compact regions
• New feature space 5D(2 spatial coordinates, 3
  color components)
• Points close in this space are close both in
  color and in actual proximity

26
Results of Clustering
Sample clusters with k-means clusteringbased on
color and distance
27
Other Distance Measures

• Problem with simple Euclidean distancewhat if
  coordinates range from 0-1000 but colors only
  range from 0-255?
• Depending on how things are scaled, gives
  different weight to different kinds of data
• Weighted Euclidean distance adjust weights to
  emphasize different dimensions

28
Mahalanobis Distance

• Automatically assign weights based on actual
  variation in the datawhere C is covariance
  matrix of all points
• Gives each dimension equal weight
• Also accounts for correlations between different
  dimensions