Recognizing Objects in Range Data Using Regional Point Descriptors

1
Recognizing Objects in Range Data Using Regional
Point Descriptors
a.k.a. 3D Shape Contexts

• A. Frome, D. Huber, R. Kolluri, T. Bulow, and J.
  Malik. Proceedings of the European Conference on
  Computer Vision, May, 2004.

Talk prepared by Nat Duca, duca_at_jhu.edu
2
Motivation

• Find instances of known shapes in 2.5D range
  scans

Image source Frome04
3
2D Shape Contexts

• Take a random point on the shape

Image source Belongie02
4
2D Shape Contexts

• Compute the offset vectors to all other samples

5
2D Shape Contexts

• Histogram the vectors against sectors and shells
• Perform this for a large sampling of points

6
Extension to 3D

• Step 1 pick random points on surface

Image source Koertgen03
7
Extension to 3D

• For each point, compute and histogram offsets

Image source Koertgen03
8
Extension to 3D

• For each point, compute offsets

Image source Koertgen03
9
Extension to 3D

• Now we histogram the offset vectors.
• The 3D histogram of looks like

Image source Frome04
10
Extension to 3D

• Shells are spaced logarithmically apart
• Histogram votes are weighted by the volume of the
  bin
• Some Ln difference of the histogram vector can be
  used to compare two contexts

Image source Frome04, Koertgen03
11
Challenges

• How do we orient the histogram spheres
• How do we compute distance between a model and
  one of its subsets?
• Speed

12
Initial histogram orientation

• Align the objects north-pole to the surface
  normal
• Problems
• One degree of freedom remains
• Histogram values depend on the precision of the
  surface normals
• The paper solves both problems using
• Brute force rotation
• spherical harmonics

13
Harmonic shape context

• Each shells histogram is a spherical function
• Convert each shell to a harmonic representation
  and store the amplitude coefficients only
• Initial histogram placement doesnt matter,
• Noise in surface normals doesnt affect descriptor

Image source Weisstein04
14
the big picture Partial Shape Matching

• For a query shape Sq and a stored model Si, their
  nearness is defined as

A shape context placed randomly on the query
surface Sq
A precomputed shape context for model Si query
surface
15
Experiment 1 resilience to noise

• (a) model with 5cm gaussian noise
• (b) model with 10cm gaussian noise
• (c) reference (databased) model

Image source Frome04
16
Experiment 2 partial matching

• Input
• 
  or
• Output

Image source Frome04
17
Evaluating the results

• Where does the blame lie
• Spherical histogram
• Harmonics representation
• Point choice
• Representative descriptor approach
• Is their presentation fair?

18
Results for noise

• Where does the blame lie
• Spherical histogram
• Harmonics representation
• Point choice
• Representative descriptor
• Is their presentation fair?
• Comments
• Recognition rate across 100 trials, how many
  times did we get the correct answer back the
  first time?
• All three techniques are equivalent in absence of
  noise

Results for 5cm noise
Image source Frome04
19
Results for noise

• Where does the blame lie
• Spherical histogram
• Harmonics representation
• Point choice
• Representative descriptor
• Is their presentation fair?
• Comments
• Why is the harmonic approach doing worse? We
  expect it to be doing as well or better than the
  basic approach

10cm noise, 55cm normal window
Image source Frome04
20
Results for noise

• Where does the blame lie
• Spherical histogram
• Harmonics representation
• Point choice
• Representative descriptor
• Is their presentation fair?
• Comments
• Notice how, when the normals are better filtered,
  the harmonics do better! How can this be so?

10cm noise, 105cm normal window
Image source Frome04
21
Results for partial matching

• Where does the blame lie
• Spherical histogram
• Harmonics representation
• Point choice
• Representative descriptor
• Is their presentation fair?
• Comments
• Rank depth of R means that the correct answer
  appeared in the top R results.
• Clearly, the harmonics are throwing away too much
• Or is the fact that the shells are rotationally
  independent to blame?

View 1
Image source Frome04
22
Results for partial matching

• Where does the blame lie
• Spherical histogram
• Harmonics representation
• Point choice
• Representative descriptor
• Is their presentation fair?
• Comments
• Rank depth of R means that the correct answer
  appeared in the top R results.
• The authors claim that the ground is setting off
  the match

View 2
Image source Frome04
23
Speed considerations

• We use a spherical hash with J sectors, and KxL
  latitudinal and longitudinal divisions
• The basic vector is (roughly) J x K x L in size
• The harmonic representation is roughly the same
  size
• Without harmonics, they must store L extra
  rotations in order J x K x L2
• They use Locality Sensitive Hashing to reduce the
  amount of effort required here

24
Speed considerations LSH results
Without hashing
Image source Frome04
25
Summary

• What was introduced
• 3D histogram extension of 2D shape contexts
• A poorly-performing spherical harmonic
  decomposition of the 3D histogram
• The representative decriptor method works pretty
  well
• What would have been nice
• Precision of query when the shells are
  logarithmically or linearly separated
• Is the representative descriptor approach the
  limiting factor? We need more data to confirm or
  deny!

26
Image sources

• Frome04 A. Frome, D. Huber, R. Kolluri, T.
  Bulow, and J. Malik. Proceedings of the European
  Conference on Computer Vision, May, 2004
• Belongie02 S. Belongie et al. Shape matching and
  object recognition using shape contexts. IEEE
  Trans on Pattern Analysis and Machine
  Intelligence. 24(4)509-522, April 2002.
• Koertgen03 M. Körtgen, G.-J. Park, M. Novotni,
  R. Klein "3D Shape Matching with 3D Shape
  Contexts", in proceedings of The 7th Central
  European Seminar on Computer Graphics, April
  2003
• Weisstein Eric W. Weisstein. "Spherical
  Harmonic." From MathWorld--A Wolfram Web
  Resource. http//mathworld.wolfram.com/SphericalHa
  rmonic.html