Segmentation as Clustering

1
Segmentation as Clustering

• Cluster together tokens (pixels) that belong
  together
• Often referred to as grouping
• Agglomerative clustering
• attach next token to cluster it is closest to
• repeat
• Divisive clustering
• split existing cluster along best boundary
• Repeat
• Split-and-merge
• combines agglomeration and division

2
Issues in Clustering

• Cluster (or Point) to Cluster Distance
• A metric in some space
• Euclidean (but suppose were in
  color-texture-position space, then?)
• Exponentially weighted
• Normalized by cluster variance(s)
• Distance between closest elements (single link)
• Maximum distance between elements (complete link)
• Average distance between elements (group average)
• Number of clusters
• Intrinsically difficult how to know?
• Dendrogram captures cluster hierarchy use this

3
From Points to a Dendrogram
Dendrology The study of trees (of the biological
sort).
4
K-Means Clustering

• Elementary, but popular (greedy, effective)
• Goal is to find k mean vectors (gt1 d space)
  corresponding to cluster centers
• Especially good for Gaussian mixtures
• A form of stochastic hillclimbing in a
  log-likelihood function

5
K-Means Concept

• Choose a fixed number of clusters (how?)
• Choose cluster centers and point-cluster
  allocations to minimize objective function
• Search impractical too many possibilties.
• x could be any set of features for which we can
  compute a distance (caution scale)

6
K-Means in Words

• Fix cluster centers, assign each point to the
  cluster with the closest center.
• With this assignment, recompute the centers.
• Loop until convergence.
• Big Assumption We know what k should be.
• Could we loop over k in some cases? Would need
  an objective measure of overall clustering
  effectiveness.
• Could we do this hierarchically? Why? How?

7
K-Means Algorithm

• Begin
• Initialize n, k, m1, m2, , mk
• Do
• classify n samples according to nearest mi
• Recompute mi
• Until no change in mi
• Return m1, m2, , mk
• End

8
K-Means Clustering Results
Original Image
Clusters on intensity alone
Clusters on color alone
Clustering on intensity, color alone - no
proximity.
9
K-means, color alone, 11 segments
Image
Clusters on color
10
K-Means, Color Alone, 1st 4 of 11 Segments
Not necessarily connected. No texture
measure. Some segments meaningless.
11
K-Means, Color Position, 20 Segments
Large regions broken up Still no texture
measure
12
Graph Theoretical Clustering

• Represent tokens as nodes in a graph
• Attributed
• Represent relations (associations) as arcs in
  this graph
• Weighted, attributed
• Then we can
• Partition into strongly connected subgraphs
• Use the graph to generate grouping hypotheses

13
Graph Terminology

• G V,E With V the vertices E edges
• E in V X V. Each edge connects two vertices
• Directed graph, edges (a,b) and (b,a) distinct
• Undirected graph, (a,b) (b,a)
• Weighted graph a weight assigned to each edge
• Connected vertices There is a path of edges and
  vertices leading from one to the other
• Connected graph All vertices connected
• Connected component Maximal connected subgraph

14
Adjacency Matrix
15
Weight (Adjacency) Matrix
Graph
Weighted adjacency matrix
Reasonable cut?
Encode the edge weights according to some measure
of association or similarity.
16
Partition
Because of the weak (0.1) edges, the four nodes
on the left and the five on the right seem to
form two natural groups, or clusters.
17
One Idea Nodes PixelsWeights Affinities
Intensity
Distance
Color
18
Texture Affinity
Adopt a set of filters (e.g. Gabor filters).
Gabor filters are Gaussian-weighted
sinusoids - Range of scales (spatial
frequencies) - Range of orientations - Sine and
Cosine forms Histogram of filter outputs in a
neighborhood - vector h aff(x,y)
exp -h(x) - h(y)2 / 2s2
19
Another Idea Gestalt Graphs

• Each Gestalt graph is based on a salient Gestalt
  organizational principle
• Nodes denote tokens (from the previous level, if
  there is one) (piecewise circular arcs from the
  edge contours, for example)
• Arcs denote the existence of a Gestalt
  relationship
• Complex organizations may be derived via various
  graph operations on this basis set

20
Gestalt Graph Examples

• Proximity Tokens are near and similarly oriented
• Endpoint Proximity End points are near
• Continuity Possible continuations based on
  orientation/intercept or curvature/center
• Similarity Similar photometric or geometric
  properties
• Common Region Tokens share a common region

21
Constructing Gestalt Graphs using Voting Methods

• Problem Find all pairs of tokens satisfying some
  compatibility relation R defining the links of
  the graph
• The compatibility relation can be expressed in
  the following form as a function of the token
  attributes

22
Compatibility Volumes
With this form for R, we construct an n
dimensional space spanned by the attributes.
Each token is a point in this space. In this
space, let each token vote for all points with
which it is compatible. (hypercuboid)
Intersecting compatibility volumes define the
links in the Gestalt graph for that relation.
23
Example
Attribute Space
Gestalt Graph
Now, analyze graph for connected components,
cliques, cycles, spanning trees, etc.