Agenda

1
Agenda

• Introduction
• Bag-of-words models
• Visual words with spatial location
• Part-based models
• Discriminative methods
• Segmentation and recognition
• Recognition-based image retrieval
• Datasets Conclusions

2
(No Transcript)
3
(No Transcript)
4
Aim

• Given an image and object category, to segment
  the object

Object Category Model
Segmentation
Cow Image
Segmented Cow

• Segmentation should (ideally) be
• shaped like the object e.g. cow-like
• obtained efficiently in an unsupervised manner
• able to handle self-occlusion

Slide from Kumar 05
5
Examples of bottom-up segmentation

• Example Normalized Cuts, Shi Malik, 1997
• Difficult without top-down cues

Borenstein and Ullman, ECCV 2002
6
Random Fields for segmentation
I Image pixels (observed) h
foreground/background labels (hidden) one label
per pixel ? Parameters
Likelihood
Posterior
Joint
Prior

• Generative approach models joint
• ? Markov random field (MRF)
• 2. Discriminative approach models posterior
  directly
• ? Conditional random field (CRF)

7
Generative Markov Random Field
i
Prior has no dependency on I
j
I (pixels)
Image Plane
8
Conditional Random Field
Lafferty, McCallum and Pereira 2001
Discriminative approach
Pairwise
Unary

• Dependency on I allows introduction of pairwise
  terms that make use of image.
• For example, neighboring labels should be
  similar only if pixel colors are similar ?
  Contrast term

e.g Kumar and Hebert 2003
9
OBJCUT
Kumar, Torr Zisserman 2005
Pairwise
Unary
Label smoothness
Distance from O
Color Likelihood
Contrast
O (shape parameter)

• O is a shape prior on the labels from a Layered
  Pictorial Structure (LPS) model
• Segmentation by
• - Match LPS model to image (get number of
  samples, each with a different pose
• Marginalize over the samples using a single
  graph cut
• Boykov Jolly, 2001

10
OBJCUTShape prior - O - Layered Pictorial
Structures (LPS)

• Generative model
• Composition of parts spatial layout

Layer 2
Spatial Layout (Pairwise Configuration)
Layer 1
Parts in Layer 2 can occlude parts in Layer 1
Kumar, et al. 2004, 2005
11
OBJCUT Results
Using LPS Model for Cow
In the absence of a clear boundary between object
and background
Segmentation
Image
12
Layout Consistent Random Field
Winn and Shotton 2006

• Variant of conditional random field

I Image pixels (observed) h
foreground/background labels (hidden) one label
per pixel ? Parameters
13
Layout CRF Part detector
Winn and Shotton 2006
14
Layout consistency
Winn and Shotton 2006
Neighboring pixels
(p,q)
?
(p,q1)
(p,q)
(p1,q1)
(p-1,q1)
Layoutconsistent
15
Stability of part labelling
Part color key
16
Other recognition segmentation papers
Figure from Borenstein and Ullman, ECCV 2002
Object-Specific Figure-Ground Segregation
Stella X. Yu and Jianbo Shi, 2002
Image parsing Tu, Zhu and Yuille 2003
Implicit Shape Model - Liebe and Schiele, 2003
LOCUS model See Jons talk tomorrowKannan,
Jojic and Frey 2004 Winn and Jojic, 2005
Todorovic and Ahuja, CVPR 2006
3D Layout CRF, Hoiem et al. CVPR 2007
See CVPR 2007 course slides for more details
17
Summary

• Strength
• Explains every pixel of the image
• Useful for image editing, layering, etc.
• Issues
• Invariance issues
• (especially) scale, view-point variations
• Inference difficulties