Nalin Pradeep Senthamil Masters Student, ECE Dept.

1
Nalin Pradeep Senthamil Masters Student, ECE
Dept.

• Advisor,
• Dr Stan Birchfield
• Committee Members,
• Dr Adam Hoover, Dr Brian Dean

2
Accurate Tracking of Non-Rigid Objects using
Level Sets

• Clemson University, Clemson, SC USA
• Accepted in ICCV, 2009

3
Outline

• Tracking Overview
• Literature
• Proposed Approach
• Object Fragmentation
• Region Growing Mechanism
• GMM modeling (feature-spatial)
• Level Set Framework
• Fragment Motion using Joint-KLT
• Results
• Conclusion

4
Tracking Overview

• Idea Obtain Trajectories over time to locate
  object
• Three Main Categories
• Point Tracking Kalman, Particle filters
• Kernel Tracking Collins et al (linear RGB),
  Comaniciu (Mean-Shift)
• Contour Tracking Shah et al, Cremers et al
• Applied to Surveillance Vessel, human, vehicle
  etc
• Why not internet videos ? 65,000 videos get
  uploaded in YouTube everyday (rich market)

5
Literature

• Linear RGB Collins et al. 2003
• Ada-boost classifier Avidan 2005
• Fragments based fixed size Adam et al. 2006
• Key-point Feature learning Grabner et al. 2007
• Shape priors Cremers et al. 2006
• Contour tracking using texture Shah et al 2005
• Limitations
• Ignore secondary cues such as multimodality
• Lack in determining accurate object shape
• Usually non-contour based techniques drift during
  occlusion
• Often ignore spatial arrangement of pixels

6
Algorithm Block Diagram
Update made at each frame
7
Object Fragmentation

• Region Growing Mechanism
• Random pixel selected from mask fragment (f)
• Neighboring pixels added to (f) within G (std
  deviation)
• Gaussian Model of (f) updated
• Each (f) represents a Gaussian ellipsoid
• Both Object and background are fragmented

8
Object Modeling (GMM)
Joint feature-spatial space,
9
Strength Map
ve for FGND -ve for BKGND
10
Level Set Framework

• Level Set is numerical technique for fitting
  contour
• Level Set on 2D image is viewed as 3D function
• Contour in level set identified at zero level

11
Level Set for strength map

• In general, Level set evolution defined by
• Gradient Descent Iteration

Strength Image
Contour (zero level set)
Strength Image
Divergence operator
12
Level-Set Evolution

• Iterations using Elmo strength map
• Curve can grow inward and outward
• Figure shows for first frame as example
• Curve evolves from previous contours in
  subsequent tracking

13
Fragment Motion

• Joint-KLT Combines algorithms of KLT and HS
• Hence,
• Used to align coordinate system of object and
  model fragments
• Increases accuracy of strength map

data term
smoothness term
14
Fragment Motion (contd.)

• N features tracked in each fragment are
  averaged
• Motion of each fragment gives prior information
  before computing strength map
• Drastic motion can be addressed

KLT
Joint-KLT
15
Results - Videos
16
Shape Matching

• Hausdorff metric is mathematical measure to
  compare two sets of points
• Application in Occlusion Handling and Shape
  recognition

a and b are two point sets
17
Occlusion Handling

• Rate of decrease in object size determines
  occlusion
• Contour shapes learnt online is used to
  hallucinate during occlusion
• Best shape is identified using Hausdorff distance
  metric
• Previously learnt subsequent shapes are
  hallucinated during occlusion

18
Results Occlusion Videos
19
Results More Comparison Videos
20
Quantitative Comparison
Walk Behind
Elmo Doll
Girl Circle
Average Normalized error obtained against
ground-truth of sequences at every 5 frames.
21
Conclusion

• Tracking algorithm based on modeling object and
  background with mixture of Gaussians
• Simple and efficient region growing mechanism to
  achieve fast computation
• Embedding strength map into Level-Set Framework
• Joint KLT introduced in the framework to improve
  accuracy
• Future Work
• Robust shape prior learning and matching
• Self-occlusion handling for unknown fragments

22
Alternative Tracking Framework (outline)

• Overview
• Proposed Approach
• Vessel Detection
• Saliency Map
• Thresholding
• Vessel Tracking
• Strength Map using Linear RGB
• ML Framework for Search
• Results

23
Object Detection Using Saliency Map

• Saliency Property of objects standing out
  relative to their neighbors.
• There is a statistical relationship between
  backgrounds of all natural images similar to
  pre-attentive search done by human visual system.
• Zhang et al (CVPR 2007) observed redundancies in
  log Fourier spectra of natural images. Hence, any
  statistical singularities in the spectrum can be
  treated as anomalies.

24
Saliency Map Computation

• Algorithm
• Let be the image.
• Real part of Fourier Spectrum
• Phase
• Log Spectrum
• Spectral Residual
• Saliency Map
  , jsqrt(-1)

25
Sample Saliency Map detections
26
Object Tracking

• Objects detected through saliency used as FGND
• Immediate surrounding used as BKGND
• Strength Model Computed similar to Collins Linear
  RGB
• 49 features selected from linear combination used
  to identify strength map
• Maximum Likelihood Framework based search used to
  localize objects in each frame
• Region search was identified based on object
  velocity

27
Object Tracking Strength Model

• 49 features of RGB are normalized into 0-255 and
  discretized into 0-32 histogram bins
• For each feature,
• Variance Ratio of Log-likelihood is identified
  that best discriminates object from background

28
Strength Model - Outputs
29
Object Tracking ML Framework

• Objective was to recover tight bound around
  object
• ML Framework is like EM algorithm
• Search objective is to maximize the function
  (Mean, Covariance)

30
Object Tracking ML Framework

• To maximize the function, Mean and Covariance are
  computed iteratively
• E-Step
• M-Step

Iterated for 2-3 times to get optimal values
31
Conclusion

• Algorithm was real time and supported around
  25-30 fps in speed
• Saliency map based detection was introduced
• Concept of strength map from adaptive-fragmentat
  ion is applied here
• Depends only on color (linearRGB), and
  combination with KLT features would add
  robustness to the system. Good way to combine is
  explored.

32
Thank you !