Stanford CS223B Computer Vision, Winter 2005 Lecture 16: Recognition

1
Stanford CS223B Computer Vision, Winter
2005Lecture 16 Recognition

• Sebastian Thrun, Stanford
• Rick Szeliski, Microsoft
• Hendrik Dahlkamp and Dan Morris, Stanford
• slide credit Alexander Roth and Kevin Murphy

2
Example Segmentation

• Keith Primdahl, Oren Feinstein, Itai Katz, Yi
  Lang Mok

3
Rectification

• Keith Primdahl, Oren Feinstein, Itai Katz, Yi
  Lang Mok

4
Rectification

• Keith Primdahl, Oren Feinstein, Itai Katz, Yi
  Lang Mok

5
Conferences

• Field and Service Robotics (March 6)
  www.fsr2005.org
• International Conference on Computer Vision
  (March 10/14), research.miscrosoft.com/iccv2005
• Neural Information Processing Systems (early
  June), nips.cc

6
Object Recognition

• Local Features
• Global Model
• What else?

Inside the object (intrinsic features)
Object size
Pixels
Global appearance
Parts
Agarwal Roth, (02), Moghaddam, Pentland (97),
Turk, Pentland (91),Vidal-Naquet, Ullman,
(03) Heisele, et al, (01), Agarwal Roth, (02),
Kremp, Geman, Amit (02), Dorko, Schmid,
(03) Fergus, Perona, Zisserman (03), Fei Fei,
Fergus, Perona, (03), Schneiderman, Kanade (00),
Lowe (99) Etc.
7
Local Features

• Idea Mark a region of a target object
• Extract statistic (e.g., average brighness,
  average brightness gradient, normalized template)
• Match to new images

8
Gaussian Model
gradient magnitude
brightness
9
Example Application
Learning trees
Road finding
David Lieb and Andrew Lookingbill
10
Road Finding
David Lieb and Andrew Lookingbill
11
Global Models Eigenfaces

• Same idea just applied to very large templates
• Developed in 1991 by M.Turk and A. Pentland
• Based on PCA
• Relatively simple
• Fast
• Robust

slide credit Alexander Roth
12
Eigenfaces

• PCA seeks directions that are efficient for
  representing the data

not efficient
efficient
Class A
Class A
Class B
Class B
slide credit Alexander Roth
13
Eigenfaces

• PCA maximizes the total scatter

scatter
Class A
Class B
slide credit Alexander Roth
14
Eigenfaces

• PCA reduces the dimension of the data
• Speeds up the computational time

slide credit Alexander Roth
15
Eigenfaces, the algorithm

• Assumptions
• Square images with WHN
• M is the number of images in the database
• P is the number of persons in the database

slide credit Alexander Roth
16
Eigenfaces, the algorithm

• The database

slide credit Alexander Roth
17
Eigenfaces, the algorithm

• We compute the average face

slide credit Alexander Roth
18
Eigenfaces, the algorithm

• Then subtract it from the training faces

slide credit Alexander Roth
19
Eigenfaces, the algorithm

• Now we build the matrix which is N2 by M
• The covariance matrix which is N2 by N2

slide credit Alexander Roth
20
Eigenfaces, the algorithm

• Find eigenvalues of the covariance matrix
• The matrix is very large
• The computational effort is very big
• We are interested in at most M eigenvalues
• We can reduce the dimension of the matrix

slide credit Alexander Roth
21
Eigenvalue Theorem

• Define
• dimension N2 by N2
• dimension M by M (e.g., 8 by 8)
• Let be an eigenvector of
• Then is eigenvector of
• Proof

slide credit Alexander Roth
22
Eigenfaces, the algorithm

• Compute another matrix which is M by M
• Find the M eigenvalues and eigenvectors
• Eigenvectors of C and L are equivalent
• Build matrix V from the eigenvectors of L

slide credit Alexander Roth
23
Eigenfaces, the algorithm

• Eigenvectors of C are linear combination of image
  space with the eigenvectors of L
• Eigenvectors represent the variation in the faces

slide credit Alexander Roth
24
Example Set

• Photobook, MIT

25
Eigenfaces
26
Normalized Eigenfaces
27
Eigenfaces, the algorithm

• Compute for each face its projection onto the
  face space
• Compute the between-class threshold

slide credit Alexander Roth
28
Eigenfaces, the algorithm

• To recognize a face
• Subtract the average face from it

slide credit Alexander Roth
29
Eigenfaces, the algorithm

• Compute its projection onto the face space
• Compute the distance in the face space between
  the face and all known faces

slide credit Alexander Roth
30
Eigenfaces, the algorithm

• Distinguish between
• If its not a face
• If
  its a new face
• If its a
  known face

slide credit Alexander Roth
31
Eigenfaces, the algorithm

• Reconstruct the face from eigenfaces
• Compute the distance between the face and its
  reconstruction

slide credit Alexander Roth
32
Eigenface Reconstruction
q1
q2
q4
q8
Original Image
q16
q32
q64
q100
33
More Eigenfaces
34
Eigenfaces, the algorithm

• Problems with eigenfaces
• Different illumination
• Different head pose
• Different alignment
• Different facial expression

slide credit Alexander Roth
35
Fisherfaces

• Developed in 1997 by P.Belhumeur et al.
• Based on Fishers LDA
• Faster than eigenfaces, in some cases
• Has lower error rates
• Works well even if different illumination
• Works well even if different facial express.

slide credit Alexander Roth
36
Fisherfaces

• LDA seeks directions that are efficient for
  discrimination between the data
• LDA maximizes the between-class scatter
• LDA minimizes the within-class scatter

Class A
Class B
slide credit Alexander Roth
37
Comparison

• FERET database
• best ID rate eigenfaces 80.0, fisherfaces
  93.2

slide credit Alexander Roth
38
References

• 1 M. Turk, A. Pentland, Face Recognition Using
  Eigenfaces
• 2 J. Ashbourn, Avanti, V. Bruce, A. Young,
  Face Recognition Based on Symmetrization and
  Eigenfaces
• 3 http//www.markus-hofmann.de/eigen.html
• 4 P. Belhumeur, J. Hespanha, D. Kriegman,
  Eigenfaces vs Fisherfaces Recognition using
  Class Specific Linear Projection
• 5 R. Duda, P. Hart, D. Stork, Pattern
  Classification, ISBN 0-471-05669-3, pp. 121-124
• 6 F. Perronin, J.-L. Dugelay, Deformable Face
  Mapping For Person Identification, ICIP 2003,
  Barcelona

slide credit Alexander Roth
39
Recognition In Context

• Torralba, Murphy, Freeman Contextua Object
  Detection using Boosted Random Fields (NIPS 2004)

slide credit Kevin Murphy
40
Conventional Object Detection
Inside the object (intrinsic features)
Object size
Pixels
Global appearance
Parts
Agarwal Roth, (02), Moghaddam, Pentland (97),
Turk, Pentland (91),Vidal-Naquet, Ullman,
(03) Heisele, et al, (01), Agarwal Roth, (02),
Kremp, Geman, Amit (02), Dorko, Schmid,
(03) Fergus, Perona, Zisserman (03), Fei Fei,
Fergus, Perona, (03), Schneiderman, Kanade (00),
Lowe (99) Etc.
41
Object might be hard to find in clutter
For each object
- Need to search over locationsand scales
scale
- Error prone (classifier must have very low
false positive rate)
y
- Slow (many patches to examine)
x
slide credit Kevin Murphy
42
Local ambiguity What is this?
slide credit Kevin Murphy
43
A car on the street?
slide credit Kevin Murphy
44
An ashtray on the table?
slide credit Kevin Murphy
45
Global scene context affects interpretation of
local patches
slide credit Kevin Murphy
Source Torralba, IJCV 2003
46
The multiple personalities of a blob
slide credit Kevin Murphy
47
The multiple personalities of a blob
slide credit Kevin Murphy
48
Isolated object may not be recognizable
Information
Contextual features
Local features
Distance
slide credit Kevin Murphy
49
Symptom of only using local features
Some false alarms occur in image regions in which
is impossible for the target to be present given
the context.
slide credit Kevin Murphy
50
The type of scene informs us about the types of
objects and their locations
We know there is a keyboard present in this scene
even if we cannot see it clearly.
51
Looking outside the box
Outside the object (contextual features)
Inside the object (intrinsic features)
Object size
Pixels
Parts
Global appearance
Local context
Global context
Kruppa Shiele, (03), Fink Perona
(03) Carbonetto, Freitas, Barnard (03), Kumar,
Hebert, (03) He, Zemel, Carreira-Perpinan (04),
Moore, Essa, Monson, Hayes (99) Strat Fischler
(91), Murphy, Torralba Freeman (03)
Agarwal Roth, (02), Moghaddam, Pentland (97),
Turk, Pentland (91),Vidal-Naquet, Ullman,
(03) Heisele, et al, (01), Agarwal Roth, (02),
Kremp, Geman, Amit (02), Dorko, Schmid,
(03) Fergus, Perona, Zisserman (03), Fei Fei,
Fergus, Perona, (03), Schneiderman, Kanade (00),
Lowe (99) Etc.
slide credit Kevin Murphy
52
What is context?

• Scenes
• Other objects

slide credit Kevin Murphy
53
Scene classification
Office
Corridor
Street
slide credit Kevin Murphy
54
Scene predicts object presence and location
(top-down)
Pkbd
Pcar
Scene
vg
slide credit Kevin Murphy
55
Integrate top-down priming with bottom-up
detection
slide credit Kevin Murphy
Murphy, Torralba, Freeman, NIPS 2003
56
What is context?

• Scenes
• Other objects

slide credit Kevin Murphy
57
Combined object detection and image segmentation

• Large image regions can provide good contextual
  cues for small objects

Inter-objectcontext
slide credit Kevin Murphy
58
Iterative pixel labeling
59
Car/ road/ building
slide credit Kevin Murphy
60
Screen/keyboard/mouse
slide credit Kevin Murphy
61
Summary Object Detection/Recognition

• Global methods, e.g., Eigenfaces
• Local methods, e.g., pixel classification plus
  segmentation/smoothing
• Context methods (other objects, scene type)