Separating learning and recognition in the constellation model

1
Separating learning and recognition in the
constellation model
Rob Fergus Pietro Perona Andrew Zisserman
Oxford University California Institute of
Technology
2
Overview of talk

• Quick review of Constellation Model
• Design issues with the Constellation Model
• - Intensity or gradient based appearance
  representation
• - Shape vs Appearance
• 3. Alternative mechanisms for recognition
• - Different graphical model for shape
• - Efficient exhaustive search
• - Parts acting as feature detectors

3
Goal

• Recognition of object categories
• Unassisted learning

4
Motorbikes example

• Kadir Brady saliency region detector

5
Motorbikes
Samples from appearance model
6
Generative probabilistic model
Foreground model
Gaussian part appearance pdf
Gaussian shape pdf
Clutter model
Gaussian background appearance pdf
Uniform shape pdf
Poission pdf on detections
7
Detection Representation of Regions

• Find regions within image using
  interest operators
• - Kadir Brady
• Curves
• Multi-scale Harris
• etc.

Intensity-based Appearance
Projection onto PCA basis
K x K patch
Normalize
Gradient-based Appearance (PCA-Sift Sukthankar
CVPR 04)
Projection onto PCA basis
K x K patch
Normalised
Gradients
8
Different representations for appearance

• 4 different categories
• Intensity based PCA vs Gradient based PCA
• 4 different patches size (7x7,11x11,15x15,21x21)
• 15 vs 25 PCA dimensions

Airplanes
Faces
Spotted Cats
Motorbikes
9
Varying the relative importance of shape and
appearance
Different dimensionality of terms
For a 5 part model Shape 8 dims joint
App 75 (515) dims independent
Introduce scaling term a
a
10
Different weighting Different models
Appearance bias
Default
Shape bias
11
Region-based learning recognition

• Pros
• Given no prior knowledge about class, gives set
  of potentially distinctive regions
• Massive reduction in data O(106) pixels ? O(102)
  regions
• Boost in signal/noise
• Enables us to infer shape, appearance etc. using
  EM-based scheme.
• Cons
• Very sparse representation of data
• Interest operator may not fire on
• distinctive points of object
• Misalignments in location and scale

12
A typical face model
13
The Bush Problem
14
Exhaustive recognition

• Use appearance densities of each part as
  interest operator
• Run over every location in image
• Impose star-structured shape model using
  Huttenlocher et. al.s scheme (CVPR 2000)
• - Allows us to find global maxima over all
  locations in image, not just subset of regions
  identified by crude interest operator

15
1. Convolve image with PCA basis
- Generic basis only needs to be done once,
regardless of categories

16
2. Compute likelihood ratio maps
More than template matching learnt variability
of part is used
................
17
3. Apply Huttenlocher distance transform
18
Face collages
Matching constraint
19
Motorbike collages
20
(No Transcript)
21
Leopard collages
22
(No Transcript)
23
Cars from Side (Agarwal-Roth)
Old EER 11.5 New EER 7.8
24
(No Transcript)
25
(No Transcript)
26
Barometer model
27
Annotating Fawlty Towers Video

• Low resolution, poor quality images
• 1463 frames total (every 30th)
• 189 frames (12.9) contain a barometer
• No normalisation/pre-processing

28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
(No Transcript)
32
Grandfather Clock model
33
Grandfather Clock results
34
Conclusions

• Shape Appearance is better than just
  appearance
• Exhaustive search seems promising
• More than just template matching
• Using learnt variability
• More efficient in scaling with categories
• Requires a good model to be effective
• Improve models
• - More parts
• - Use different features types