Pyramidal Implementation of Lucas Kanade Feature Tracker

1
Pyramidal Implementation of Lucas Kanade Feature
Tracker

• Jia Huang
• Xiaoyan Liu
• Han Xin
• Yizhen Tan

2
Abstract

• Introduction
• Tracking algorithm
• Lucas-Kanade algorithm
• Iterative implementation
• Tracking features analysis
• Feature lost
• Feature selection

3
Objective

• For a given point u in image A, find its
  corresponding location v u d in image B.

d
Image B
Image A
4
Residual function and Window size
To find the location ? Minimize residual
function
Integration window size
Nature tradeoff
Small integration window
Higher accuracy
Larger integration window
Higher robustness
5
Pyramid Implementation of LK algorithm

• Calculate a set of pyramid representations of
  original image
• Apply traditional tracking algorithm for each
  level
• Results of current iteration is propagated to
  next iteration
• Key point the same window size is used for each
  level

Top View
Side View
6
Lucas-Kanade algorithm(1)

• At the level L, we define images A and B

7
Lucas-Kanade algorithm(2)

• At the optimum, the first derivative of
• After first order Taylor expansion
• Components in the equation above

8
Lucas-Kanade algorithm(3)

• Two derivative images are expressed
• With these notation, we can get
• The optimum optical flow vector is

9
Iterative scheme of LK algorithm(1)

• Pyramidal diagram
• Inner loop K-level
• K initialized to 1, assume that the previous
  computations from iterations 1,2,...,k-1 provide
  an initial guess
• The new translated image according to

10
Iterative scheme of LK algorithm(2)

• The goal to compute the residual pixel motion
  vector , that minimizes the
  error function
• Image mismatch vector , where the image
  difference delta I k defined as
• New pixel displacement guess is computed for the
  next iteration step k1

11
Iterative scheme of LK algorithm(3)

• On average, 5 iterations are enough
• At the 1st iteration (k1), the initial guess is
  set to zero
• The final solution for the optical flow vector is
• Outer loop L-level
• The vector d is propagated to the next level L-1
  and overall procedure is repeated L-1, L-2, , 0

12
Declaring a Feature Lost

• Several cases of lost feature
• the point falls outside of the image
• image patch around the tracked point varies
• too much between image A and image B
• too large displacement
• How to solve it
• combine a traditional tracking approach with
• an affine image matching

13
Feature Lost Example(1)
Image A
Image B
14
Feature Lost Example(2)
Image A
Image B
15
Feature Selection

• Intuitive
• To select the point u on image A good to
  track.
• Process steps
• Compute the G matrix and ?m
• Call ?max the maximum value of ?m
• Retain the pixels that have a ?m value larger
  than a
• percentage of ?max
• Retain the local max. pixels
• Keep the subset of those pixels so that the
  minimum
• distance between pixels is larger than a
  threshold

16
Example of LK Feature Tracking
Image A
Image B
17
More Examples
Image B
Image A
18
Summary

• Lucas-Kanade Feature Tracker is one of the most
  popular versions of two-frame differential
  methods for motion estimation
• Iterative implementation of the Lucas-Kanade
  optical flow computation provides sufficient
  local tracking accuracy.

19
Thanks for your attention

• Any question?