Tracking Using A Highly Deformable Object Model

1
Tracking Using A Highly Deformable Object Model

• Nilanjan Ray
• Department of Computing Science
• University of Alberta

2
Overview of Presentation

• Tracking deformable objects
• Motivations desirable properties of a deformable
  object model
• An example application (mouse heart tracking)
• Some technical background
• Level set function and its application in image
  processing
• Non-parametric probability density function (pdf)
  estimation
• Similarity/dissimilarity measures for pdfs
• Proposed tracking technique
• Results, comparisons and demos
• Ongoing investigations
• Incorporating color cues, and other features
• Adding constraints on object shape
• Application in morphing (?)
• Incorporating object motion information (??)
• Summary
• Acknowledgements

3
Tracking Deformable Objects

• Desirable properties of deformable models
• Adapt with deformations (sometimes drastic
  deformations, depending on applications)
• Ability to learn object and background
• Ability to separate foreground and background
• Ability to recognize object from one image frame
  to the next, in an image sequence

Show cine MRI video
4
Some Existing Deformable Models

• Deformable models
• Highly deformable
• Examples snake or active contour, B-spline
  snakes,
• Good deformation, but poor recognition (learning)
  ability
• Not-so-deformable
• Examples
• Active shape and appearance models
• G-snake
• Good recognition (learning) capability, but of
  course poor deformation ability

So, how about good deformation and good
recognition capabilities?
5
Technical Background Level Set Function

• A level set function represents a contour or a
  front geometrically
• Consider a single-valued function ?(x, y) over
  the image domain intersection of the x-y plane
  and ? represents a contour
• (X(x, y), Y(x, y)) is the point on the curve
  that is closest to the (x, y) point
• Matlab demo (lev_demo.m)

6
Applications of Level Set Image Segmentation

• Matlab segmentation demo (yezzi.m)
• Vessel segmentation
• Brain reconstruction
• Virtual endoscopy
• Trachea fly through
• tons out there

Show videos
7
Level Set Applications Image Denoising

• Two example videos

Show video
8
Level Set Applications Robotics

• Finding shortest path

Show video
9
Level Set Applications Computer Graphics

• Morphing
• Simulation
• Animation
• .

http//www.sci.utah.edu/stories/2004/fall_levelset
.html
Go to http//graphics.stanford.edu/fedkiw/ for
amazing videos
10
More Applications of Level Set Methods

• Go to http//math.berkeley.edu/sethian/2006/Appli
  cations/Menu_Expanded_Applications.html

11
Technical Background Non-Parametric Density
Estimation
Normalized image intensity histogram
I(x, y) is the image intensity at (x, y) ?i is
the standard deviation of the Gaussian kernel C
is a normalization factor that forces H(i) to
integrate to unity
12
Technical Background Similarity and
Dissimilarity Measures for PDFs
Kullback-Leibler (KL) divergence (a dissimilarity
measure)
Bhattacharya coefficient (a similarity measure)
P(z) and Q(z) are two PDFs being compared
13
Proposed Method Tracking Deformable Object

• Deformable Object model (due to Leventon 1)
• From the first frame learn the joint pdf of level
  set function and image intensity (image feature)
• Tracking
• From second frame onward search for similar joint
  pdf

1 M. Leventon, Statistical Models for Medical
Image Analysis, Ph.D. Thesis, MIT, 2000.
14
Deformable Object Model

• Joint probability density estimation with
  Gaussian kernels

Level set function value l Image intensity i
J(x, y) is the image intensity at (x, y) point on
the first image frame ?(x, y) is the value of
level set function at (x, y) on the first image
frame C is a normalization factor
We learn Q on the first video frame given the
object contour (represented by the level set
function)
15
Proposed Object Tracking

• On the second (or subsequent) frame compute the
  density
• Match the densities P and Q by KL-divergence
• Minimize KL-divergence by varying the level set
  function ?(x, y)

Note that here only P is a function of ?(x, y)
I(x, y) is the image intensity at (x, y) on the
second/subsequent frame ?(x, y) is the level set
function at on the second/subsequent frame
16
Minimizing KL-divergence

• In order to minimize KL-divergence we use
  Calculus of variations
• After applying Calculus of variations the rule of
  update (gradient descent rule) for the level set
  function becomes

t iteration number ?t timestep size
17
Minimizing KL-divergence Implementation

• There is a compact way of expressing the update
  rule

convolution
is a function defined simply as
Where g1 is a convolution kernel
18
Minimizing KL-divergence A Stable Implementation

• The previous implementation is called explicit
  scheme and is unstable for large time steps if
  small time step is used then the convergence will
  be extremely slow
• One remedy is a semi-implicit scheme of numerical
  implementation

Where g is a convolution kernel
is a function defined simply as
In this numerical scheme ?t can be large and
still the solution will be convergent So very
quick convergence is achieved in this scheme
19
Results Tracking Cardiac Motion
A few cine MRI frames and delineated boundaries
on them
Show videos

20
Numerical Results and Comparison
Sequence with slow heart motion
Sequence with rapid heart motion
Comparison of mean performance measures
21
Extensions Tracking Objects in Color Video

• If we want to learn joint distribution of level
  set function and color channels (say, r, g, b),
  then non-parametric density estimation suffers
  from
• Slowness
• Curse of dimensionality
• Another important theme is combine edge
  information and region information of objects
• One remedy sometimes is to take a linear
  combination of r, g, and b channels
• Fishers linear discriminant can be used to learn
  the coefficients of linear combination
• A demo

22
Extensions Adding Object Shape Constraint

• Can we constrain the object shape in this
  computational framework?

Minimize
where
23
Application in Computer Graphics Morphing
Initial object Shape and intensity/texture
Final object Shape and intensity/texture
(J1, ?1)
(I2, ?2)
(I1, ?1),
(I2, ?2),
..
Morphing generate realistic intermediate tuples
(It, ?t)
24
Morphing Formulation

• Generate intermediate shapes, i.e., level set
  function ?t (say, via interpolation)
• Next, generate intermediate intensity It by
  maximizing
• Once again we get a similar PDE for It

25
Morphing Preliminary Results
Show videos
26
Summary

• Highly deformable object tracking Variational
  minimization of KL-divergence leading to fast and
  stable partial differential equations
• Several exciting extensions
• Application in morphing

27
Acknowledgements

• Baidyanath Saha
• CIMS lab and Prof. Hong Zhang
• Prof. Dipti P. Mukherjee, Indian Statistical
  Institute
• Department of Computing Science, UofA