Shape Classification Using the Inner-Distance

1
Shape Classification Using the Inner-Distance

• Haibin Ling
• David W. Jacobs
• IEEE TRANSACTION ON PATTERN ANAYSIS AND MACHINE
  INTELLIGENCEFEBRUARY 2007

2
Outline

• Introduction
• Related work
• Inner-Distance
• Articulation invariant signatures
• Inner-distance shape context
• Shortest path texture context
• Experiments

3
Introduction

• We use the inner-distance to build shape
  descriptors that are robust to articulation and
  capture part structure.
• Inner-distance is defined as the length of the
  shortest path between landmark points within the
  shape boundary.

4
Related work

• Three categories to handle parts classification
• Statistical methods to describe the articulation
  between parts and often require a learning
  process to find the model parameters.
• To measures the similarity between shapes between
  shapes via part-to-part matching and junction
  parameter distribution.
• To capture the part structure by considering the
  interior of shape boundary.
• Our method belongs to.
• Skeleton-based approaches

5
The inner-distance The definition

• Define a shape as a connected and closed
  subset of R2. Given a shape and two points
  ,the inner-distance between denoted
  as ,is defined as the length of
  the shortest path connecting and within
  .

Note (1) In rare case where there are multiple
shortest paths, we arbitrarily choose one. (2)
Shapes are defined by their boundary, hence only
boundary points are used as landmark points.
6
The inner-distance Computation

• Shortest path algorithms
• (1) Build a graph with the sample points. For
  each pair of sample points p1 and p2, if the line
  segment connecting p1 and p2 fall entirely within
  the object, Let an edge between p1 and p2 is
  added to the graph with its weight equal to the
  Euclidean distance p1 - p2 .

• Note
• Neighboring boundary points are always connected.
• (2) The inner-distance reflects the existence of
  holes without using samples points from hole
  boundary.

7
The inner-distance Computation

• (2) find the inner-distance between all pairs of
  points according to the graph.
• The whole computation takes .
• It takes time to checi whether a line
  segment between two points is inside the given
  shape.
• The complixity of graph construction is of
  .

8
The inner-distanceA model of articulate objects

Articulated objects. (a) An articulated shape.
(b) Overlapping junstions. (c) Ideal articulation.
9
The inner-distanceA model of articulate objects

• is constant and very small compared to
  the size of the articulated parts.
• An articulated to another articulated object
  is one-to-one continuous mapping .

10
The inner-distance articulation insensitivity

• Changes of the inner-distance are due to junction
  deformations. That means change is very small
  compared to the size of parts.

11
The inner-distance articulation insensitivity

• Theorem
• Proof Is decomposed into
  segments.

12
The inner-distance articulation insensitivity
13
The inner-distance articulation insensitivity

• Example

14
The inner-distance ability to capture structures

• It is hard to prove because no clear part
  decomposition.
• Show how the inner-distance capture part
  structure with examples

15
The inner-distance ability to capture structures
With about the same number of sample points, the
four shapes are virtually indistinguishable using
distribution of Euclidean distance. However,
their distributions of the inner-distance are
quite different except for the first two shapes.
Note more sample points will not affect the
above statement.
16
Articulation Invariant Signatures

• The inner-distance is used to build articulation
  invariant signatures for 2D shapes using
  multidimensional scaling (MDS).
• Given sample points on the shape
  O.the inner-distance .MDS finds
  the transformed points such that
  the Euclidean distance
  minimize the stress S(Q)

17
Articulation Invariant Signatures

• Example
• MDSSCDP
• Use MDS to get articulation invariance
  signatures.
• Build the shape context on the signatures.
• Use dynamic programming for shape context matching

(a) and (c) show two shapes related by
articulation. (b) and (d) show their signatures.
18
Related workShape Contexts for 2D shape

• The shape context was introduced by Belongie et
  al.
• Due to its simplicity and discriminability, the
  shape context has become quite popular recently
  in shape matching tasks.
• It describes the relative spatial distribution.

19
Related workShape Contexts for 2D shape

• Given n sample on shape.The
  shape context at points is defined as a
  histogram of the relative coordinates of
  the remaining n-1 points.
• Where the bins uniformly divide the log-polar
  space.
• The shape context uses the Euclidean distance to
  measure the spatial relation between landmark
  points. This causes less discriminability for
  complex shapes with articulations.

20
Inner-Distance Shape Context (IDSC)

• To extend the shape context, Euclidean distance
  is directly replaced by the inner-distance.

21
Inner-Distance Shape Context (IDSC)

• The angle between the contour tangent at p and
  the direction of at p is
  insensitive to articulation, called inner-angle,
  denoted .
• Inner-angle is used for the orientation bins.
• Noise may reduces the stability of the
  inner-angle, smoothing contour before computing
  it.

22
Inner-Distance Shape Context (IDSC)

• Example

In the histogram, the x axis denotes the
orientation bins and the y axis denotes log
distance bins.
23
Shape matching through Dynamic programming

• Given two shapes A and B, points sequences on
  their contour , say, for A and
  for B, assume .
• A matching from A to B is a mapping.
• is matched to if , and
  otherwise left unmatched.
• should minimize the match cost.

24

Shape matching through Dynamic programming

• is the penalty for leaving
  unmatched, and for , is
  the cost of matching to .
• and are the shape context histogram
  of and . K is the number of
  histogram bins.

25
Shape matching through Dynamic programming

• DP is used to solve the matching problem since it
  uses the ordering information provided by shape
  contours.
• By default, assumes the two contours are already
  aligned at their start and end points.
• Without this assumption, one simple solution is
  to try different alignments at all points on the
  first contour and choose the best one.

26
Shape matching through Dynamic programming

• Because shapes can be first rotated according to
  their moments, it is sufficient to try aligning a
  fixed number of points, say k points.
• Usually, k is much smaller than m and n.

27
Shape distance

• The matching cost is used to measured the
  similarity between shapes.
• IDSCDP is better than SCDP
• Better performance
• Only two parameters to tune
• The penalty for a point with no matching,
  usually set 0.3.
• The number of start points k for different
  alignments, usually set 4-8.
• Easy to implement since it does not require the
  appearance and transformation model.

28
Shortest path texture context

• The combination of texture and shape information,
  because
• Shapes from different classes sometimes are more
  similar than those from the same class.
• Shapes are often damaged due to occlusion and
  self-overlapping.

29
Shortest path texture context

• The texture information along these paths
  provides a natural articulation insensitive
  texture description.
• The angles between intensity gradient directions
  and shortest path directions are used, called
  relative orientations.

30
Shortest path texture context

• The SPTC for each is a three dimensional
  histogram .
• The inner-distance
• The inner-angle
• The (weighted) relative orientation
• The relative orientations are weighted by
  gradient magnitudes.

31
Shortest path texture context
32
Experiment

• the number of inner-distance bins or
  the number of inner-angle binsthe number of
  relative orientation bins
• The number of different starting points for
  alignment
• The penalty for one occlusion

33
Experiment

(a) Articulated database. (b) MDS of the
articulated database using the inner-distance.
34
Experiment
Retrieval result on the articulate data set
SCDP
IDSCDP
35
Experiment

• MPEG7 CE-Shape-1 shape database is widely tested,
  which consists of 1400 silhouette images from 70
  classes. Each class has 20 different shapes.
• Bullseye test for every image in the database,
  it is matched with all other images and the top
  40 most similar candidates are counted.

36
Experiment

• The score of the test is the ratio of the number
  of correct hits of all images to the highest
  possible number of hits (which is 20x1400).

37
Experiment

• The Kimia Database
• Data set 1 25 instance from six categories

38
Experiment

• Data set 2 99 instances from nine categories

39
Experiment

• The ETH-80 image set

• This data set contains 80 objects from eight
  classes, with 41 images of each object obtained
  from different viewpoints.

40
ExperimentFoliage image retrieval

• Swedish leaf data set

41
ExperimentFoliage image retrieval

• Smithsonian data set343 leaf images from 93
  species.

42
ExperimentFoliage image retrieval
43
ExperimentFoliage image retrieval
44
ExperimentHuman body matching