Tracking Features with Large Motion

1
Tracking Features with Large Motion
2
Abstract

• Problem When frame-to-frame motion is too large,
  KLT feature tracker does not work.
• Solution Estimate the motion at the deepest
  pyramid level by matching the characteristic
  curves of the consecutive images.

3
Introduction

• Feature tracking is an important issue in many
  computer vision applications.
• In order to allow the tracker to handle large
  motion, people usually use a pyramidal
  implementation of the KLT feature tracker.

4

• We propose a method to extend the pyramidal KLT
  feature tracker to deal with image I and J are
  taken from widely different viewpoint.
• I and J are two consecutive images in an image
  sequence.

5
Sum of square differences (SSD)

• Given a feature point on I, the goal is to find
  the correspondence on J.
• Where I(x) and J(x) are intensity with image
  point .d displacement vectorW a small
  integration window centered at the feature point.

6
KLT feature tracker

• Let , we can use the linear system to find d
  

7

• Automatically selecting good features is also an
  important issue.
• Let if ,then the image point is a good
  feature.

8
Pyramidal implementation of KLT feature tracker

• Let be original image of I and J.
  downsampling of downsampling of height
  of the image pyramid .
• Similarly, we can obtain images for image J.
• After constructing the image pyramid of image I
  and J, we apply the pyramidal KLT feature
  tracker.

9

• First step the displacement vector at
  the deepest level.
• Second steps downsampling factor.
  repeat second step until estimate .

10
Accommodating very large motion

• There are two problems
• In practice, is a small number otherwise the
  image size of will be too small to carry
  enough details for each feature.where
  d true displacement vector for a
  feature point

11

• The feature point dissolve when the height of the
  image pyramid is too large.

12

• For those cases, our method provides a solution
  by computing the motion estimates and at the
  deepest pyramid level.
• The effect of computing and is to provide
  a coarse motion at deepest level which makes the
  residual motion small enough to satisfy the
  assumption.

13
Characteristic curves

• Define as the characteristic curve for x-axis
  computed from image I.
• for y-axis from image I.

14
(No Transcript)
15

• After the four curves have been computed, we
  compute by matching the two characteristic
  curves .
• can be computed in the same manner.

16
Motion estimation at the deepest pyramid level

• The goal of the motion estimator is to find the
  best labeling that assigns a label to each
  element .

17

• domain of the curve ( )
  displacement to element ( ) when
  element is considered to be occluded.
  the ordered set of element where
• The penalty is use to penalize the situations
  where discontinuity is occurred.
• The penalty serves as a threshold that affects
  whether the motion estimator should assign a
  label to the element .

18

• After finding the optimal labeling, the motion
  estimator computes by
• For those element in , we compute their
  motion estimates by interpolating the
  displacements of the elements in the
  neighborhood.

19
Feature tracking with pre-checking

• Consider a feature point .
• x is a lost feature when one of the following
  conditions is satisfied
• Therefore, no computational power is wasted.

20
Results and comparisons

• Two image sequences are tested here.
• Show 71-frames 320 x 240 pixels
• Building 73-frames 480 x 320 pixels

21
Birchfields implementation
22
Comparison between the number of the tracked
features
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
Comparison between the running time