Alignment%20and%20Matching

1
Alignment and Matching

• Thomas Funkhouser and Michael Kazhdan
• Princeton University

2
Challenge

• The shape of model does not changewhen the model
  is translated, scaled,or rotated

3
Outline

• Matching
• Alignment
• Exhaustive Search
• Invariance
• Normalization
• Part vs. Whole
• Conclusion

4
Exhaustive Search

• Search for the best aligning transformation
• Compare at all alignments
• Match at the alignment for which models are
  closest

Exhaustive search for optimal rotation
5
Exhaustive Search

• Search for the best aligning transformation
• Compare at all alignments
• Match at the alignment for which models are
  closest

6
Exhaustive Search

• Search for the best aligning transformation
• Use signal processing for efficient correlation
• Represent model at many different transformations
• Properties
• Gives the correct answer
• Is hard to do efficiently

7
Outline

• Matching
• Alignment
• Exhaustive Search
• Invariance
• Normalization
• Part vs. Whole
• Conclusion

8
Invariance

• Represent a model with information that is
  independent of the transformation
• Extended Gaussian Image, Horn Translation
  invariant
• Shells Histograms, Ankerst Rotation invariant
• D2 Shape Distributions, Osada Translation/Rotatio
  n invariant

Shells Histogram
D2 Distribution
EGI
9
Invariance

• Represent a model with information that is
  independent of the transformation
• Power spectrum representation
• Fourier Transform for translation and 2D
  rotations
• Spherical Harmonic Transform for 3D rotations

Energy
Energy
Frequency
Frequency
Circular Power Spectrum
Spherical Power Spectrum
10
Translation Invariance
1D Function
11
Translation Invariance






1D Function
Cosine/Sine Decomposition
12
Translation Invariance






1D Function

Constant
Frequency Decomposition
13
Translation Invariance







1D Function


Constant
1st Order
Frequency Decomposition
14
Translation Invariance







1D Function



Constant
1st Order
2nd Order
Frequency Decomposition
15
Translation Invariance







1D Function






Constant
1st Order
2nd Order
3rd Order
Frequency Decomposition
16
Translation Invariance
Amplitudes invariantto translation







1D Function






Constant
1st Order
2nd Order
3rd Order
Frequency Decomposition
17
Rotation Invariance

• Represent each spherical function as a sum of
  harmonic frequencies (orders)

18
Rotation Invariance

• Store how much (L2-norm) of the shape resides
  in each frequency to get a rotation invariant
  representation

Constant
1st Order
2nd Order
3rd Order




19
Power Spectrum

• Translation-invariance
• Represent the model in a Cartesian coordinate
  system
• Compute the 3D Fourier transform
• Store the amplitudes of the frequency components

Cartesian Coordinates
y
Translation Invariant Representation
z
x
20
Power Spectrum

• Single axis rotation-invariance
• Represent the model in a cylindrical coordinate
  system
• Compute the Fourier transform in the angular
  direction
• Store the amplitudes of the frequency components

q
Cylindrical Coordinates
r
h
Rotation Invariant Representation
21
Power Spectrum

• Full rotation-invariance
• Represent the model in a spherical coordinate
  system
• Compute the spherical harmonic transform
• Store the amplitudes of the frequency components

q
Spherical Coordinates
f
r
Rotation Invariant Representation
22
Power Spectrum

• Power spectrum representations
• Are invariant to transformations
• Give a lower bound for the best match
• Tend to discard too much information
• Translation invariant n3 data -gt n3/2
  data
• Single-axis rotation invariant n3 data -gt n3/2
  data
• Full rotation invariant n3 data -gt n2
  data

23
Power Spectrum
Method Translation Rotation
EGI Constant Order
Crease Histograms Constant Order Spherical Constant Order
D2 Square Spherical Constant Order
Shells Spherical Constant Order
Spherical Extent Cylindrical Full
Harmonic Descriptor Spherical Full
24
Outline

• Matching
• Alignment
• Exhaustive Search
• Invariance
• Normalization
• Part vs. Whole
• Conclusion

25
Normalization

• Place a model into a canonical coordinate frame
  by normalizing for
• Translation
• Scale
• Rotation

26
Normalization
Horn et al., 1988

• Place a model into a canonical coordinate frame
  by normalizing for
• Translation Center of mass
• Scale
• Rotation

Initial Models
Translation-Aligned Models
27
Normalization
Horn et al., 1988

• Place a model into a canonical coordinate frame
  by normalizing for
• Translation
• Scale Mean variance
• Rotation

Translation-Aligned Models
Translation- and Scale-Aligned Models
28
Normalization

• Place a model into a canonical coordinate frame
  by normalizing for
• Translation
• Scale
• Rotation PCA alignment

PCA Alignment
Translation- and Scale-Aligned Models
Fully Aligned Models
29
Rotation

• Properties
• Translation and rotation normalization is
  guaranteed to give the best alignment
• Rotation normalization is ambiguous

PCA Alignment
Directions of the axes are ambiguous
30
Normalization (PCA)

• PCA defines a coordinate frame up to reflection
  in the coordinate axes.
• Make descriptor invariant to axial-reflections
• Reflections fix the cosine term
• Reflections multiply the sine term by -1

y
Translation Invariant Representation
z
x
31
Retrieval Results (Rotation)
Size
Gaussian EDT
Method Floats
Exhaustive Search 8192
PCA Flip Invariance 8192
PCA 8192
Cylindrical PS 4352
Spherical PS 512
Precision
Time
Method Secs.
Exhaustive Search 20.59
PCA Flip Invariance .67
PCA .67
Cylindrical PS .32
Spherical PS .03
Recall
32
Alignment

• Exhaustive search
• Best results
• Inefficient to match
• Normalization
• Provably optimal for translation and scale
• Works well for rotation if models have well
  defined principal axes and the directional
  ambiguity is resolved
• Invariance
• Compact
• Efficient
• Often less discriminating

33
Outline

• Matching
• Alignment
• Exhaustive Search
• Invariance
• Normalization
• Part vs. Whole
• Conclusion

34
Partial Shape Matching

• Cannot use global normalization methods that
  depend on whole model information
• Center of mass for translation
• Mean variance for scale
• Principal axes for rotation

Normalized Whole
Normalized Part
(Mis-)Aligned Models
35
Partial Shape Matching

• Cannot use global normalization methods that
  depend on whole model information
• Exhaustively search for best alignment
• Normalize using local shape information
• Use transformation invariant representations

Normalized Whole
Normalized Part
(Mis-)Aligned Models
36
Spin Images Shape Contexts

• Translation (Exhaustive Search)
• Represent each database model by many descriptors
  centered at different points on the surface.

Model
Multi-Centered Descriptors
37
Spin Images Shape Contexts

• Translation (Exhaustive Search)
• To match, center at a random point on the query
  and compare against the different descriptors of
  the target

Randomly-Centered Descriptor
Query Part
Best Match
Target Descriptor
38
Spin Images Shape Contexts

• Rotation (Normalization)
• For each center, represent in cylindrical
  coordinates about the normal vector
• Spin Images Store energy
• in each ring
• Harmonic Shape Contexts
• Store power spectrum of
• each ring
• 3D Shape Contexts Search
• over all rotations about the
• normal for best match

n
n
n
39
Spin Images Shape Contexts
Image courtesy of Frome et al, 2003

• Spin images and shape contexts allow for
  part-in-whole searches by exhaustively searching
  for translation and using the normal for rotation
  alignment
• Spin Images Store energy
• in each ring
• Harmonic Shape Contexts
• Store power spectrum of
• each ring
• 3D Shape Contexts Search
• over all rotations about the
• normal for best match

40
Conclusion

• Aligning Models
• Exhaustive Search
• Normalization
• Invariance
• Partial Object Matching
• Cant use global normalization techniques
• Translation Exhaustive Search
• Rotation Normal Exhaustive/Invariant

41
Conclusion

• Shape Descriptors and Model Matching
• Decoupling representation from registration
• Can design and evaluate descriptors without
  having to solve the alignment problem
• Can develop methods for alignment without
  considering specific shape descriptors