JetStream: Probabilistic Contour Extraction with Particles

1
JetStream Probabilistic Contour Extraction with
Particles

• Patrick Perez, Andrew Blake, and Michel Gangnet,
  Microsoft Research, St George House, 1 Guildhall
  Street, Cambridge, CB2 3NH, UK
• http//research.microsoft.com/vision
• Presented by
• Vladan Radosavljevic

2
Outline

• Introduction
• Image Gradient
• Related Work
• Probabilistic contour tracking
• Tracking framework
• Dynamics
• Measurement
• Iterative computation of posterior - particles
• Model Ingredients
• Likelihood ratio
• Dynamics
• Proposal Sampling Function
• Experimental Results
• Conclusion

3
Motivation

• Contour extraction
• from segmenting images with closed contours
• to the extraction of linear structures of
  particular interest such as roads

4
Introduction - image gradient

• The gradient of an image
• The gradient points in the direction of most
  rapid change in intensity

5
Introduction

• Most approaches to contour extraction rely on
  some minimal cost principle
• k is the curvature, s is the arc-length
• y(r(s)) scalar or vector derived at location r(s)
  from the raw image I
• Often y(r(s)) is the gradient norm
• This function captures some kind of regularity on
  candidate curves, for example - rewarding, by a
  lower cost, the presence of large gradients along
  the contour

6
Introduction related work

• First approach - dynamic programming
• optimal curve in the form of chain pixels
• unless optimality is abandoned, and huge storage
  resources are available, there are tight
  restrictions on the form of cost function
• Second approach - growing a contour from the seed
  point according to cost function
• given the current contour, a new segment is
  appended to it according both to a shape prior
  (mainly smoothness) and to the evidence provided
  by the data at the location under concern
• deterministic complete discontinuous curves
  provided by edge detectors

7
Introduction related work

• A probabilistic point of view
• the contours are seen as the paths of a
  stochastic process
• tracking problem
• We are interesting in the method that is able
• to avoid spurious distracting contours
• to track the multiple off-springs starting at
  branching contours
• to interpolate over transient evidence gaps
• JetStream a method with particle filtering

8
Probabilistic contour tracking

• Tracking contours in still images is an
  unconventional tracking problem because of the
  absence of a real notion of time
• The time is only associated with the growing of
  the contour in the image plane consider a
  spatial chain as a temporal chain
• Contrary to standard tracking problems where data
  arrive one bit after another as time passes by,
  the whole set of data y is standing there at once
• There is no straightforward way of tuning the
  speed, or equivalently the length of successive
  moves

9
Tracking framework - dynamics

• xi points in the plane R2
• (x0, x1,...,xn) - curve in some standard way,
  e.g., the xis are the vertices of a polyline,
  starting from x0 then moving along the contour to
  xn
• dynamics
• assuming second order dynamics with some kernel
  q
• then a priori density is

10
Tracking framework - measurement

• measurement y (observed image) conditioned on
  x0n is considered as independent spatial process
  which is not true in reality but it is a
  reasonable approximation
• where ? is a discrete set of measurements
  locations in the image plane
• the conditional distributions
  depend only on whether or not point u is
  contained in contour x0n
• in particular, for any two points u along the
  contour the corresponding conditional
  distributions are identical, similarly for any
  two points in the background

11
Tracking framework - measurement

• Therefore, each is either
  pon if u belongs to the contour x0n, or poff if
  not
• Finally
• where

12
Iterative computation of posterior

• Function
  can be considered as a cost function
• Expressed as the minimization of this function,
  the contour extraction problem then amounts to
  seeking the maximum a posteriori (MAP) estimate
  
• Posterior densities can be computed recursively
• but there is no closed form of the pi
• pi can be approximated by a finite set
  of M sample paths (the particles)
• Best path at step i can be
  which is a Monte Carlo approximation of
  posterior expectation

13
Iterative computation of posterior

• Prediction each path is grown one step
  by sampling from the proposal density
  function f
• If the paths are samples from pi then
  the extended paths are samples
  from fpi
• Since we want samples from distribution pi1,
  extended paths are weighted according to ratio
• The resulting weighted path set now provides an
  approximation of the target distribution pi1
• M paths are drawn with replacement from the
  weighted set
• The weights are

14
Likelihood ratio l

• Measurement the norm of the luminance (or color)
  gradient
• poff pon
• poff exponential distribution
• pon - complex mixture, better keep as less
  informative as possible

15
Likelihood ratio l

• The direction of the gradient also retains
  precious information that a data model based only
  on gradient norm neglects the distribution of ?
  is symmetric, and it becomes tighter as the norm
  of the gradient increases

16
Likelihood ratio l

• However, at corners, the norm of the gradient is
  usually large but its direction cannot be
  accurately measured
• Using a standard corner detector, each pixel u is
  associated with a label c(u) 1 if a corner is
  detected, and 0 otherwise
• Where a corner has been detected assumption is
  that distribution is uniform

17
Dynamics q

• Because of the absence of natural time, it is
  better to consider a dynamics with fixed step
  length d. The definition of second order dynamics
  then amounts to specifying an a priori
  probability distribution on direction change Ti
• Finally
• To allow for abrupt direction changes at the
  locations where corners have been detected, the
  normal distribution is mixed with a small
  proportion of uniform distribution

18
Proposal sampling function f

• With choice f q, corners will be mostly ignored
  since the expected number of particles
  undertaking drastic direction changes is vM,
  where typically ? 0.01 and M 100
• At locations where no corners are detected, the
  proposal density is the normal component of the
  dynamics.
• If location lies on a detected corner, the next
  proposed location is obtained by turning of an
  angle picked uniformly
• Therefore

19
Proposal sampling function f

20
JetStream - iteration

21
Experimental Results Interactive cut-out

• The extraction of a region of interest from one
  image
• In practice, JetStream is run for a fixed number
  n of steps (100 in our experiments) from initial
  conditions x01 chosen by the user.
• If the result is satisfactory, n more steps are
  undertaken.
• If not, a restart region within the particle
  flow, and an associated restart direction, can be
  chosen by the user

22
Experimental Results Interactive cut-out

• More sophisticated user interaction
• Provide the user with the facility to place one
  or more dams, defined as regions Rk where

23
Experimental Results Interactive cut-out

24
Experimental Results Road extraction

• In the specific context of road extraction, one
  is in fact interested in recovering ribbons
• Using JetStream as defined previously results in
  paths jumping from one side of the ribbon to the
  other

25
Experimental Results Road extraction

• The state space is extended to include a width
  variable mi, which indicates the distance at step
  i between the two sides of the ribbon
• It expresses that xi and xi- defined as
• are on a contour while xi is not

26
Experimental Results Road extraction

• A simple first order dynamics is chosen for m

27
Experimental Results Road extraction

28
Conclusion

• JetStream is applicable if there are not sharp
  corners in the images
• If sharp corners exist or the images intensity
  distribution is complex JetStream doesnt provide
  good results

29
References

• 1 M. Pérez, A. Blake, and M. Gangnet.
  JetStream Probabilistic contour extraction
  with particles. In Int. Conf. on Computer
  Vision, ICCV 2001, Vancouver, Canada, July 2001.