Good Features to Track Jianbo Shi and Carlo Tomasi

1
Good Features to TrackJianbo Shi and Carlo Tomasi

• 5980 Paper Presentation
• February 10, 2003
• Monica LaPoint

2
Agenda

• Background
• Problem Statement
• Related Work
• Proposed Solution
• Experiments and Analysis
• Critical Review
• Summary
• Questions

3
Background

• Vision can be used as another sensor
• Robot motion is ascertained by following the
  movement of features within the image
• Provides input into position estimation
• Goal provide accurate input into estimation
  from visual information

4
Problem Statement

• Need to choose features that are best suited for
  tracking (feature selection)
• Need to verify that features are still trackable
  and have not drifted away from original targets
  (feature quality)
• Need to reliably identify features from image to
  image (feature tracking)

5
Which features are good?
6
Related Work Feature Selection

• Select based on texturedness or cornerness by a
  variety of measures
• High standard deviations in spatial intensity
  profile Moravec 1980
• Laplacian image analysis Mar et all 1979,
  Kitchen, Rosenfield 1980 , Dreschler, Nagel
  1981
• Issue These measures do not directly relate to
  how well a feature can be tracked given a
  tracking algorithm

7
Proposed SolutionFeature Selection

• Choose features that can be tracked well
• Features with high eigenvalues will be above the
  image noise level
• Features with similar eigenvalues are well
  conditioned
• NOTE Does not guarantee good trackability, just
  a starting point

8
Which features are good?
9
Related Work Feature Quality

• Feature retention is based on quality measure
• Given a feature in two frames, rms residue
  measures the change in feature appearance
• Issue Occlusions can go undetected and features
  can unknowingly be lost since measure

10
Proposed SolutionFeature Quality

• Monitor quality of features during tracking with
  new dissimilarity measure that includes a
  deformation matrix that represents the affine
  motion model as well as translation of features
  within the frame.

11
Proposed Solution Feature Qaulity

• Affine motion model
• Motion in terms of translation movement of
  complete window
• Motion of individual pixels within window is
  different

Motion from front to side cause corners to move
differently in a 2D frame
12
Related Work Feature Tracking

• Features are tracked from frame to frame using
  feature matching by
• Sum of squared differences Burt 1982 and
  Anandan 1989
• Optimized translation of features from frame to
  frame within a window Cafforio, Rooca 1976,
  Connor, Limb 1974, others

13
Proposed SolutionFeature Tracking

• Feature tracking continues to use the pure
  translation model Lucas,Kanade 1981
• Best for determining small displacements
• Deformation matrix is replaced with 2x2 identify
  matrix

14
Requisite Math...

• Point motion is described by
• where
• J is the current image
• I is the 1st image
• A 1D (1 is 2x2 identity matrix and D is the
  deformation matrix)
• d is the translation
• In the pure translational model, D is set 0

15
Requisite Math...

• Dissimilarity is measured by
• w(x) can be 1 or Gaussian function to emphasize
  center of window
• Differences are squared and summed
• Solved using Newton Raphson method
• Yields 6x6 system of linear equations
• Smaller system available for tracking only

16
Experiments Affine motion model

• Targets from a sequence of images are processed
  via the computed deformation matrix and
  translations
• Each of the images appear similarly showing that
  the matrix encapsulates the actions correctly

17
Experiments Affine motion model
18
Experiments-Calculating deformation matrix
Original Image
Target transformed image
Iterations of deformation calculation (4,8, and
19)
19
Experiments-Feature Quality
3 At edge boundarynew method correctly
identifies as lower quality feature
20
Experiments-Feature Quality
21 Good feature identified by all methods
21
Experiments-Feature Quality
58 poor feature identified by all methods
22
Experiments-Feature Quality
60-Feature identified as poor feature with new
method
23
Experiments-Feature Quality
78-New method identifies problem due to occlusion
earlier
24
Experiments-Feature Quality
89-Both methods identify as poor feature
25
Summary - Contributions

• Discusses concept of using two models of motion
  during tracking
• Pure translational model for tracking small
  changes from frame to frame
• Added affine motion model for a better measure of
  changes over many frames
• Introduces new dissimilarity measure to determine
  quality of selected features during tracking

26
Summary - Contributions

• Proposes algorithm for calculating the
  dissimilarity measure
• Provides unbiased rules for feature selection
  that promote good feature tracking

27
Critical Review

• Interested in algorithm behavior for different
  kinds of motion
• Cites forward and side motion only
• Complete calculation for algorithm not included
  and not readily available
• No analysis of time to compute or computational
  complexity

28
Critical Review

• Algorithm comparison and trade offs not discussed
• Did not expound on two models of motion idea or
  provide a case why it would be better

29
Questions????

• Other Resources
• Tomasi tracker http//vision.stanford.edu/birch/k
  lt/
• Visual tracking with deformation models Rehg,
  J.M. Witkin, A.P. (1991)
• Detection and Tracking of Point Features Carlo
  Tomasi, Takeo Kanade (1991)
•  http//citeseer.nj.nec.com/tomasi91detection.ht
  ml