Selecting%20Distinctive%203D%20Shape%20Descriptors%20for%20Similarity%20Retrieval

1
Selecting Distinctive 3D Shape Descriptors for
Similarity Retrieval

• Philip Shilane and Thomas Funkhouser

2
Large Databases of 3D Shapes
3
Shape Retrieval
3D Model
BestMatches
Model Database
4
Local Matches for Retrieval
3D Model
BestMatches
Model Database
5
Local Matches for Retrieval
3D Model
BestMatches
Model Database
Cost Function
6
Local Matches for Retrieval
Using many local descriptors is slow.
3D Model
BestMatches
Model Database
Cost Function
7
Local Matches for Retrieval
Using many local descriptors is slow. Many
descriptors do not represent distinguishing parts.
3D Model
BestMatches
Model Database
Cost Function
8
Local Matches for Retrieval
Focusing on the distinctive regions improves
retrieval time and accuracy.
3D Model
BestMatches
Model Database
Cost Function
9
Related Work

• Selecting Local Descriptors
• RandomMori 2001Frome 2004

10
Related Work

• Selecting Local Descriptors
• Random
• SalientGal 2005Lee 2005Frintrop 2004

11
Related Work

• Selecting Local Descriptors
• Random
• Salient
• Likelihood Johnson 2000 Shan 2004

12
Distinction Retrieval Performance
The distinction of each local descriptor is based
on how well it retrieves shapes of the correct
class.
QueryDescriptors
Retrieval Results
13
Distinction Retrieval Performance
The distinct descriptors that distinguish between
classes are classification dependent.
QueryDescriptors
Retrieval Results
14
Approach
We want a predicted distinction score for each
descriptor on the model.
Descriptors
Distinction
15
Approach
We map descriptors into a 1D space where we learn
distinction from a training set.
Distinction
Distinction
Descriptors
1D Parameterization
16
Approach
Likelihood Parameterization
Likelihood of shape descriptors is a 1D function
that groups descriptors with similar distinction.
Distinction
Descriptors
17
System Overview
Training
Shape DB
Likelihood
Local Descriptors
Descriptor DB
Distinction Function
Retrieval Evaluation
Classification
Query
Likelihood
Evaluate Distinction
Local Descriptors
Match
SelectDescriptors
Shape
RetrievalList
18
System Overview
Training
Shape DB
Likelihood
Local Descriptors
Descriptor DB
Distinction Function
Retrieval Evaluation
Classification
Query
Likelihood
Evaluate Distinction
Local Descriptors
Match
SelectDescriptors
Shape
RetrievalList
19
System Overview
Training
Shape DB
Likelihood
Local Descriptors
Descriptor DB
Distinction Function
Retrieval Evaluation
Classification
Query
Likelihood
Evaluate Distinction
Local Descriptors
Match
SelectDescriptors
Shape
RetrievalList
20
System Overview
Training
Shape DB
Likelihood
Local Descriptors
Descriptor DB
Distinction Function
Retrieval Evaluation
Classification
Query
Likelihood
Evaluate Distinction
Local Descriptors
Match
SelectDescriptors
Shape
RetrievalList
21
Likelihood of Descriptors
Multi-dimensional normal density Johnson 2000
22
Likelihood of Descriptors
The likelihood function is proportional to the
descriptor density.
23
Map from Descriptors to Likelihood
Flat regions are the most common while wing tips
and the cockpit area are rarer.
Less Likely
More Likely
24
System Overview
Training
Shape DB
Likelihood
Local Descriptors
Descriptor DB
Distinction Function
Retrieval Evaluation
Classification
Query
Likelihood
Evaluate Distinction
Local Descriptors
Match
SelectDescriptors
Shape
RetrievalList
25
Measuring Distinction
Evaluate the retrieval performance of every query
descriptor.
QueryDescriptors
Evaluation Metric
Retrieval Results

• 0.33

26
Measuring Distinction
Some descriptors are better for retrieval than
others.
QueryDescriptors
Evaluation Metric
Retrieval Results

• 0.33
• 1.0

27
System Overview
Training
Shape DB
Likelihood
Local Descriptors
Descriptor DB
Distinction Function
Retrieval Evaluation
Classification
Query
Likelihood
Evaluate Distinction
Local Descriptors
Match
SelectDescriptors
Shape
RetrievalList
28
Build Distinction Function
Measure likelihood and retrieval performance of
each descriptor.
29
Build Distinction Function
Measure likelihood and retrieval performance of
each descriptor.
30
Build Distinction Function
Measure likelihood and retrieval performance of
each descriptor.
31
Build Distinction Function
Retrieval performance is averaged within each
likelihood bin.
32
Descriptor Distinction
A likelihood mapping separates descriptors with
different retrieval performance.
Less Likely
More Likely
33
Descriptor Distinction
The most common features are the worst for
retrieval.
Less Likely
More Likely
34
Predicting Distinction
The likelihood mapping predicts descriptor
distinction.
Descriptors
Distinction
Distinction Function
35
System Overview
Training
Shape DB
Likelihood
Local Descriptors
Descriptor DB
Distinction Function
Retrieval Evaluation
Classification
Query
Likelihood
Evaluate Distinction
Local Descriptors
Match
SelectDescriptors
Shape
RetrievalList
36
Selecting Distinctive Descriptors
The k most distinctive descriptors with a minimum
distance constraint are selected.
Mesh
Descriptors
DistinctionScores
3 SelectedDescriptors
37
Matching with Selected Descriptors
3D Model
BestMatches
Model Database
38
Results

• Examples of Distinctive Descriptors
• Evaluation for Retrieval

39
Distinctive Descriptor Examples
Descriptors on the head and neck represent
consistent regions of the models.
40
Distinctive Descriptor Examples
Descriptors on the front of the jet are
consistent as opposed to on the wings.
41
Challenge
The wheels are consistent features for cars.
42
Shape Database

• 100 Models in 10 Classes from the Princeton
  Shape Benchmark
• Models come from different branchesof the
  hierarchical classification

43
Shape Descriptors

• Mass per Shell Shape Histogram (SHELLS) Ankerst
  1999
• Spherical Harmonics of the Gaussian Euclidean
  Distance Transform (SHD) Kazhdan 2003

44
Local vs. Global Descriptors
Using local descriptors improves retrieval
relative to global descriptors.
45
Focus on Distinctive Descriptors
Using a small number of distinct descriptors
maintains retrieval performance while improving
retrieval time.
46
Alternative Selection Techniques
47
Alternative Selection Techniques
48
Alternative Selection Techniques
Distinction improves retrieval more than other
techniques.
49
Conclusion

• Method to select distinctive descriptors
• Distinctive descriptors can improve retrieval
• Mapping descriptors through likelihood and
  learned retrieval performance to distinction is
  better than other alternatives
• Distinction is independent of type of descriptor

50
Future Work

• Explore other definitions of likelihood including
  mixture models

51
Future Work

• Explore other definitions of likelihood including
  mixture models
• Consider non-likelihood parameterizations

52
Future Work

• Explore other definitions of likelihood including
  mixture models
• Consider non-likelihood parameterizations
• Combine descriptors while accounting for
  deformation Funkhouser and Shilane, SGP

53
Acknowledgements

• Szymon Rusinkiewicz
• Joshua Podolak
• Princeton Graphics Group
• Funding Sources
• National Science Foundation Grant CCR-0093343 and
  Grant 11S-0121446
• Air Force Research Laboratory Grant
  FA8650-04-1-1718

54

• The End