Tracking Hands with Distance Transforms

1
Tracking Hands with Distance Transforms

• Dave Bargeron
• Noah Snavely

2
The problem

• Input A video with a (rigid) hand
• Output A sequence of hand locations and
  orientations

3
Approach

• Generate hand templates for all possible
  orientations
• Find the edges in each input image
• For each edge image, find the template and
  location which minimizes the chamfer distance

4
Step 1 Generate Templates

• Create 3D hand model
• Render in a set of orientations
• Use depth buffer to find silhouette and contours

5
Steps 2 3 Find the hand

• Compute the distance transform of the edge image
• Slide each template over the distance transform,
  compute the chamfer distance
• Pick the template with the minimum chamfer
  distance

6
Problems

• Large number of templates
• (3211 rotations) x (3072 translations) x (5
  scales) 49,320,960 templates
• In a cluttered image
• Chamfer distance has many local optima
• Global optimum may not be correct
• Solving each frame separately not a good idea

7
Solution Part 1 Template Tree

• Coarse-to-fine search in parameter space

8
Solution Part 2 Tracking

• Detect the hand in frame 0
• For each frame k gt 0
• Compute the most likely transition from state in
  frame k-1
• Use chamfer distance as a likelihood
• Use transition probability (assumed Gaussian) as
  a prior
• Use transition probabilities to prune branches of
  the search tree

9
Results Hand Detection
Input
Edge image
Distance transform
Output
10
Results Video
11
Results Video
Edge Images
Distance Transforms
12
Results Video
13
Extensions

• Better tracking
• Use color in addition to shape
• Use edge orientations
• More templates allow for on-line generation for
  refinement
• More flexible tracking
• Track deformable hand
• Automatically determine hand parameters (e.g.
  finger length)

14
References

• Björn Stengers Ph.D thesis
• http//mi.eng.cam.ac.uk/bdrs2/papers/stenger04_th
  esis.pdf
• B. Stenger, et. al.
• Filtering Using a Tree-Based Estimator. ICCV
  2003.
• Pedro Felzenszwalb and Dan Huttenlocher.
• Distance Trasforms of Sampled Functions.