The Pyramid Match Kernel and Its Improvement

1
The Pyramid Match Kernel and Its Improvement

• Guo LiJun

2

• PMK Kristen Grauman,Trevor Darrell ICCV05, Oral
• Projects that use LIBPMK
• Multiple Kernel Learning from Sets of Partially
  Matching Image Features, UKACC Control 2008.
• Distributed Image Search in Camera Sensor
  Networks, ACM SenSys 2008.
• Automated Annotation of Drosophila Gene
  Expression Patterns Using a Controlled
  Vocabulary, Bioinformatics 2008.
• Photo-based Question Answering, ACM Multimedia
  2008.
• Unsupervised Feature Selection via Distributed
  Coding for Multi-view Object Recognition, CVPR
  2008.
• Combining Brain Computer Interfaces with Vision
  for Object Categorization, CVPR 2008.
• Scalable Classifiers for Internet Vision Tasks,
  CVPR Internet Vision Workshop 2008.
• Object Category Recognition Using Probabilistic
  Fusion of Speech and Image Classifiers, MLMI
  2007.
• Envisioning Sketch Recognition A Local Feature
  Based Approach to Recognizing Informal Sketches,
  Ph.D. thesis (2007).
• Adaptive Vocabulary Forests for Dynamic Indexing
  and Categry Learning, ICCV 2007.

3

• Related Work
• Local Features
• Kernel Methods
• Earth Movers Distance
• Multiresolution Histograms

4
Sets of features
5
Sets of features
6
Problem

• How to build a discriminative classifier using
  the set representation?

• Kernel-based methods (e.g. SVM) are appealing for
  efficiency and generalization power
• But what is an appropriate kernel?
• Each instance is unordered set of vectors
• Varying number of vectors per instance

7
Other solution

• Measuring Similarity Between Sets of Features
• 1)Voting
• 2)Bags of Prototypical Features
• 3)Computing Correspondences

8
Existing set kernels

• Fit (parametric) model to each set, compare with
  distance over models
• Kondor Jebara, Moreno et al., Lafferty
  Lebanon, Cuturi Vert,
• Wolf Shashua

Restrictive assumptions
9

• Similarity Measures for Unordered Features

10
Partial matching for sets of features

• Compare sets by computing a partial matching
  between their features.

Robust to clutter, segmentation errors, occlusion
11
Pyramid match
12
Pyramid match overview
Pyramid match kernel measures similarity of a
partial matching between two sets

• Place multi-dimensional, multi-resolution grid
  over point sets
• Consider points matched at finest resolution
  where they fall into same grid cell
• Approximate similarity between matched points
  with worst case similarity at given level

No explicit search for matches!
13
Pyramid match kernel
Approximate partial match similarity
14
Feature extraction
15
Counting matches
Histogram intersection
16
Counting new matches
Histogram intersection
17
Pyramid match kernel

• Weights inversely proportional to bin size
• Normalize kernel values to avoid favoring large
  sets

18
Efficiency

• For sets with m features of dimension d, and
  pyramids with L levels, computational complexity
  of
• Pyramid match kernel
• Existing set kernel approaches
• or

19
Example pyramid match
Level 0
20
Example pyramid match
Level 1
21
Example pyramid match
Level 2
22
Example pyramid match
pyramid match
optimal match
23
(No Transcript)
24
Building a classifier

• Train SVM by computing kernel values between all
  labeled training examples
• Classify novel examples by computing kernel
  values against support vectors
• One-versus-all for multi-class classification

Convergence is guaranteed since pyramid match
kernel is positive-definite.
25
Object recognition results

• ETH-80 database 8 object classes
• Features
• Harris detector
• PCA-SIFT descriptor, d10

Kernel Complexity Recognition rate
Match Wallraven et al. 84
Bhattacharyya affinity Kondor Jebara 85
Pyramid match 84
Eichhorn and Chapelle 2004
26
Object recognition results

• Caltech objects database 101 object classes
• Features
• SIFT detector
• PCA-SIFT descriptor, d10
• 30 training images / class
• 43 recognition rate
• (1 chance performance)
• 0.002 seconds per match

27
Localization

• Inspect intersections to obtain correspondences
  between features
• Higher confidence correspondences at finer
  resolution levels

observation
target
28
Pyramid match regression

• Pose estimation from contour features
• Train SVR with CG data
• Features shape context
  histograms

29
Summary Pyramid match kernel
optimal partial matching between sets of features
number of new matches at level i
difficulty of a match at level i
30
Summary Pyramid match kernel

• A new similarity measure based on implicit
  correspondences that approximates the optimal
  partial matching
• linear time complexity
• no independence assumption
• model-free
• insensitive to clutter
• positive-definite function
• fast, effective object recognition

31
Disadvantage

• Disregard all information about the spatial
  layout of the features

Improvement
Spatial Pyramid Matching
32
Spatial Pyramid Matching
Learn Method PMK or EMD-NN