Geometric Active Contours

1
Geometric Active Contours
Computer Science Department
Technion-Israel Institute of Technology

• Ron Kimmel
• www.cs.technion.ac.il/ron

Geometric Image Processing Lab
2
Edge Detection

• Edge Detection
• The process of labeling the locations in the
  image where the gray levels rate of change is
  high.
• OUTPUT edgels locations,
• direction, strength
• Edge Integration
• The process of combining local and perhaps
  sparse and non-contiguous edgel-data into
  meaningful, long edge curves (or closed contours)
  for segmentation
• OUTPUT edges/curves consistent with the local
  data

3
The Classics

• Edge detection
• Sobel, Prewitt, Other gradient estimators
• Marr Hildreth
• zero crossings of
• Haralick/Canny/Deriche et al.
• optimal directional local max of derivative
• Edge Integration
• tensor voting (Rom, Medioni, Williams, )
• dynamic programming (Shashua Ullman)
• generalized grouping processes (Lindenbaum et
  al.)

4
The New-Wave

• Snakes
• Geodesic Active Contours
• Model Driven Edge Detection

5
Geodesic Active Contours

• Snakes Terzopoulos-Witkin-Kass 88
• Linear functional efficient implementation
• non-geometric depends on parameterization
• Open geometric scaling invariant, Fua-Leclerc 90
• Non-variational geometric flow
• Caselles et al. 93, Malladi et al. 93
• Geometric, yet does not minimize any functional
• Geodesic active contours Caselles-Kimmel-Sapiro
  95
• derived from geometric functional
• non-linear inefficient implementations
• Explicit Euler schemes limit numerical step for
  stability
• Level set method Ohta-Jansow-Karasaki 82,
  Osher-Sethian 88
• automatically handles contour topology
• Fast geodesic active contours Goldenberg-Kimmel-Ri
  vlin-Rudzsky 99
• no limitation on the time step
• efficient computations in a narrow band

6
Laplacian Active Contours

• Closed contours on vector fields
• Non-variational models Xu-Prince 98, Paragios et
  al. 01
• A variational model Vasilevskiy-Siddiqi 01
• Laplacian active contours open/closed/robust
• 
  Kimmel-Bruckstein 01

Most recent variational measures for good old
operators Kimmel-Bruckstein 03
7
Segmentation
8
Segmentation

• Ultrasound images

Caselles,Kimmel, Sapiro ICCV95
9
Segmentation

• Pintos

10
Woodland Encounter Bev Doolittle 1985

• With a good prior who needs the data

11
Segmentation
Caselles,Kimmel, Sapiro ICCV95
12
Prior knowledge
13
Prior knowledge
14
Segmentation
15
Segmentation
16
Segmentation
Caselles,Kimmel, Sapiro ICCV95
17
Segmentation

• With a good prior who needs the data

18
Wrong Prior???
19
Wrong Prior???
20
Wrong Prior???
21
Curves in the Plane

• C(p)x(p),y(p), p 0,1

C(0.1)
C(0.2)
C(0.7)
C(0)
C(0.4)
C(0.8)
C(0.95)
y
C(0.9)
x
22
Arc-length and Curvature

• s(p) dp

C
23
Calculus of Variations

• Find C for which is an
  extremum
• Euler-Lagrange

24
Calculus of Variations

• Important Example
• Euler-Lagrange , setting
• Curvature flow

25
Potential Functions (g)
I(x,y)
I(x)
Image
x
x
g(x)
g(x,y)
Edges
x
x
26
Snakes Geodesic Active Contours

• Snake model
• Terzopoulos-Witkin-Kass 88
• Euler Lagrange as a gradient descent
• Geodesic active contour model
• Caselles-Kimmel-Sapiro 95
• Euler Lagrange gradient descent

27
Maupertuis Principle of Least Action
p
1

• Snake Geodesic active contour
• up to some , i.e
• Snakes depend on parameterization.
• Different initial parameterizations
• yield solutions for different
• geometric functionals

y
0
x
Caselles Kimmel Sapiro, IJCV 97
28
Geodesic Active Contours in 1D
I(x)

• Geodesic active contours are
• reparameterization invariant

x
g(x)
x
29
Geodesic Active Contours in 2D
G I
s
g(x)
30
Controlling -max
Smoothness
g
I
Cohen Kimmel, IJCV 97
31
Fermats Principle

• In an isotropic medium, the paths taken by light
  rays are extremal geodesics w.r.t.
• i.e.,

Cohen Kimmel, IJCV 97
32
Experiments - Color Segmentation
Goldenberg, Kimmel, Rivlin, Rudzsky, IEEE T-IP
2001
33
Tumor in 3D MRI
Caselles,Kimmel, Sapiro, Sbert, IEEE T-PAMI 97
34
Segmentation in 4D
Malladi, Kimmel, Adalsteinsson,  Caselles,
Sapiro, Sethian SIAM Biomedical workshop 96
35
Tracking in Color Movies
Goldenberg, Kimmel, Rivlin, Rudzsky, IEEE T-IP
2001
36
Tracking in Color Movies
Goldenberg, Kimmel, Rivlin, Rudzsky, IEEE T-IP
2001
37
Edge Gradient Estimators
Xu-Prince 98, Paragios et al. 01,
Vasilevskiy-Siddiqi 01, Kimmel-Bruckstein 01
38
Edge Gradient Estimators

• We want a curve with large points and
  small s so
• Consider the functional
• Where is a scalar function, e.g.
  .

39
The Classic Connection

• Suppose and we consider a closed
  contour for C(s).
• We have
• and by Greens Theorem we have

40
The Classic Connection

• Therefore
• Hence curves that maximize are
  curves that enclose all regions where
  is positive!
• We have that the optimal curves in this case are
• The Zero Crossings of the Laplacian
• isnt this familiar?

41
The Classic Connection

• It is pedagogically nice, but the MARR-HILDRETH
  edge detector is a bit too sensitive.
• So we do not propose a grand return to MH but a
  rethinking of the functionals used in active
  contours in view of this.
• INDEED, why should we ignore the gradient
  directions (estimates) and have every edge
  integrator controlled by the local gradient
  intensity alone?

42
Our Proposal

• Consider functional of the form
• These functionals yield regularized curves that
  combine the good properties of LZCs where
  precise border following is needed, with the good
  properties of the GAC over noisy regions!

43
Implementation Details

• We implement curve evolution that do gradient
  descent w.r.t. the functional
• Here the Euler Lagrange Equations provide the
  explicit formulae.
• For closed contours we compute the evolved curve
  via the Osher-Sethian miracle numeric level set
  formulation.

44
Closed contours

• EL eq.

GAC
LZC
LZC
GAC
Kimmel-Bruckstein IVCNZ01
45
Closed contours

• EL eq.

GAC
LZC
LZCeGAC
Kimmel-Bruckstein IVCNZ01
46
Open contours

• Along the curve
• b.c. at C(0) and C(L)

Kimmel-Bruckstein IVCNZ01
47
Open contours
Kimmel-Bruckstein IVCNZ01
48
Geometric Measures

• Weighted arc-length
• Weighted area
• Alignment
• Robust-alignment
• e.g.

Variational meaning for Marr-Hildreth edge
detector Kimmel-Bruckstein IVCNZ01
49
Geometric Measures

• Minimal variance
• Chan-Vese, Mumford-Shah,
• Max-Lloyd, Threshold,

50
Geometric Measures

• Robust minimal deviation

51
Haralick/Canny-like Edge Detector

• Haralick suggested as edge
  detector

Laplace
52
Haralick/Canny Edge Detector

• Haralick

co-area
h
Thus, indicates optimal alignment
topological homogeneity
53
Closed Contours Level Set Method
y

• implicit
  representation of C
• Then,
• Geodesic active contour level set formulation
• Including weighted (by g) area minimization

x
C(t)
y
C(t) level set
x
54
Operator Splitting Schemes

• Additive operator splitting (AOS) Lu et al. 90,
  Weickert, et al. 98
• unconditionally stable for non-linear diffusion
• Given the evolution
• 
  
• write
• Consider the operator
• Explicit scheme
• , the time step, is upper bounded for
  stability

55
Operator Splitting Schemes

• Implicit scheme
• inverting large bandwidth matrix
• First order, semi-implicit, additive operator
  splitting (AOS), or
• locally one-dimensional (LOD) multiplicative
  schemes are stable and efficient given by linear
  tridiagonal systems of equations
• that can be solved for by Thomas algorithm

LOD
AOS
56
Operator Splitting Schemes

• We used the following relation (AOS)
• Locally One-Dimensional scheme (LOD)
• Decoupling the axes and the implicit formulation
  leads to computational efficiency
• The 1st order splitting idea is based on the
  operator expansion

57
Example Geodesic Active Contour
y

• The geodesic active contour model
• Where I is the image and f the implicit
  representation of the curve
• If f is a distance, then ,
• and the short time evolution is
• Note that and thus
• can be computed once for the whole image

x
C(t)
y
x
Goldenberg, Kimmel, Rivlin, Rudzsky, IEEE T-IP
2001
58
Example Geodesic Active Contour

• f is restricted to be a distance map
• Re-initialization by Sethians
• fast marching method every iteration in
• O(n).
• Computations are performed in a narrow band
  around the zero set
• Multi-scale approach
• process a Gaussian pyramid of the image

y
x
C(t)
y
x
59
Tracking Objects in Movies

• Movie volume as a spatial-temporal 3D hybrid
  space
• The AOS scheme is
• Edge function derived by the
• Beltrami framework Sochen Kimmel Malladi 98
• Contour in frame n is the initial condition for
  frame n1.

y
t
x
y
t
x
60
Experiments - Curvature Flow
61
Experiments - Curvature Flow CPU Time
62
Tracking
Goldenberg, Kimmel, Rivlin, Rudzsky, ECCV 2002
63
Tracking
Goldenberg, Kimmel, Rivlin, Rudzsky, ECCV 2002
64
Tracking
Goldenberg, Kimmel, Rivlin, Rudzsky, ECCV 2002
65
Information extraction
Goldenberg, Kimmel, Rivlin, Rudzsky, ECCV 2002
66
Thin Structures
Holzman-Gazit, Goldshier, Kimmel 2003
67
Segmentation in 3D
Change in topology
Caselles,Kimmel, Sapiro, Sbert, IEEE T-PAMI 97
68
Gray Matter Segmentation

• Coupled surfaces
• EL equations

Goldenberg Kimmel Rivlin Rudzsky, VLSM 2001
69
Gray Matter Segmentation
Goldenberg Kimmel Rivlin Rudzsky, VLSM 2001
70
Gray Matter Segmentation
Goldenberg Kimmel Rivlin Rudzsky, VLSM 2001
71
(No Transcript)
72
Futurism

• Recognition from
• periodic motion

Dynamism of a Dog on a Leash Giacomo Balla, 1912
Eadweard Muybridge, Animals in Motion, 1887
Goldenberg, Kimmel, Rivlin, Rudzsky, ECCV 2002
73
Classification (dogs cats)
walk
run
gallop
cat...
Goldenberg, Kimmel, Rivlin, Rudzsky, ECCV 2002
74
Classification (dogs cats)
Goldenberg, Kimmel, Rivlin, Rudzsky, ECCV 2002
75
(No Transcript)
76
Classification (people)
walk
run
run45
Goldenberg, Kimmel, Rivlin, Rudzsky, ECCV 2002
77
Classification (people)
Goldenberg, Kimmel, Rivlin, Rudzsky, ECCV 2002
78
(No Transcript)
79
Conclusions

• Geometric-Variational method for segmentation and
  tracking in finite dimensions based on prior
  knowledge (more accurately, good initial
  conditions).
• Using the directional information for edge
  integration.
• Geometric-variational meaning for the
  Marr-Hildreth and the Haralick (Canny) edge
  detectors, leads to ways to design improved ones.
• Efficient numerical implementation for active
  contours.
• Various medical and more general applications.

• www.cs.technion.ac.il/ron

80
Gray Matter Segmentation
Goldenberg Kimmel Rivlin Rudzsky, VLSM 2001
81
Edge Indicator Function for Color

• Beltrami framework Color image 2D surface
• in
  space
• The induced metric tensor for the image surface
• Edge indicator largest eigenvalue of the
  structure tensor metric. It represents the
  direction of maximal
  change in

I
Y
X
82
AOS

• Proof
• The whole low order splitting idea is based on
  the operator expansion