Advanced Tracking: KLT Tracker

1
Advanced Tracking KLT Tracker

• Introduction
• Pixel-based tracking
• KLT tracker
• What to remember

2
Advanced tracking

• Simple tracking blob, feature vector, predictor
  (Kalman Filter)
• Advanced tracking cluttered background and
  multiple obj.
• Two types of advanced tracking algorithms
• Pixel-based (low-level)
• (Geometric) model-based (high-level)
  Condensation ASM

3
Pixel-based tracking

• Complex objects and background gt
• hard to extract a model of the object
• Solution model texture template of object
• Pixel-based tracking Template matching
• Three problems associated with TM
• How to define a good template
• TM is variant to geometric transformations
• The template update paradox

4
TM and geometric transforms (2)

• Pixel relationship y Dx d

Transformation
Equation
Invariant

Translation
D I
-
Rotation
Scaling
-
Affine
-
5
The template update paradox (3)

• Change in illumination non-affine motion
• The template should adapt to these changes
• Model template template(t-1)
• Problem End up tracking the background
• Paradox Adaptive update without tracking the
  background

6
KLT-tracker

• Kanada-Lucas-Tomasi (CMU) (1990-1994)
• KLT uses TM but handles all 3 problems
• Other good algorithms, but
• Survived 10 years
• Good SW available, e.g. demo in OpenCV
• Describe KLT wrt the three problems
• Based on the paper (CVPR94)

7
KLT-tracker Problem 1

• Define a template
• Measure the amount of texture in a window
• Calculate gradients (edges) for each pixel
  (gx,gy)

8
KLT-tracker Problem 1

• Calculate Covar(gx,gy)
• Calculate Eigen-vectors and values (l1,l2)
• If both eigen-values are small gt no texture
• Accept template if min(l1,l2) gt TH
• Automatic method. Only have to define size and TH

9
KLT-tracker Problem 2

• 2 Sensitive to geometric transformations
• Assumptions
• A pixel intensity is constant over time
• Only affine motion is present
• Together we have
• J(Dxdx) I(x) ltgt J(Axd) I(x), where A
  ID
• D 0 gt normal template matching
• KLT invariant to affine transformation (P2)
• Similarity measure SSD
• Same as for standard TM except 6 vs. 2 par.

10
KLT-tracker Problem 2

• SSD gt e SSW J(Axd) - I(x) 2
• Match min arg e
• 6 parameters gt brute force no good!
• Instead
• Partial derivatives equal to zero
• Taylor expansion to simplify matters gt
• Tz a (6x6) z D11, D12, D21,
  D22,dx,dyT
• Newton-Raphson iteration

A,d
11
KLT-tracker Problem 3

• High frame-rate during tracking gt
• D 0 and zd e (2x2) Newton-Raphson
• Use the template from last frame as model gt
• adaptive (good) tracking
• Monitoring the templates. Stop tracking
• when low similarity (e gt TH)

t 0
t
t1
time
Match ( d )
Model
Monitoring (affine A,d )
12
What to remember TM KLT

• 3 problems associated with template matching
• How to define a template
• Variant to geometric transformations
• The template updating paradox
• KLT
• 1 Automatic method based on edges
• 2 Apply affine motion model
• 3 Adaptive update AND evaluation (monitor)
• A general purpose low-level tracker (few
  parameters)
• No high-level reasoning, e.g. during occlusion
• Demo