General Purpose Image Segmentation with Random Walks

1
General Purpose Image Segmentation with Random
Walks

• Leo Grady
• Department of Imaging and Visualization
• Siemens Corporate Research

2
Outline

• Overview of Siemens Corporate Research (SCR)
• General purpose segmentation
• Random walker algorithm
• Concept
• Properties
• Theory
• Numerics
• Results
• New
• Conclusion

3
Overview of SCR

• About 200 full time research staff
• 75 people working on medical imaging
• Basic research ?? clinical products
• 1/3 mid/long term research - 2/3 applied
  projects

Princeton, USA
4
Overview of SCR
Clinical Imaging

• Goals of clinical application software
• Measures something that could not be
  measured practically before
• Makes diagnosis more accurate or treatment
  more effective
• Enables therapy that was not possible before
• Increases patient control
• Saves time
• Reduces cost

5
Overview of SCR
Core interests Segmentation, registration,
visualization
6
Overview of SCR
Offline Online Intervention

• So far diagnostic radiology offline problem
• Interventional imaging online problem
• Continuous imaging, constant human input
• Rich source of new problems

7
Outline

• Overview of Siemens Corporate Research (SCR)
• General purpose segmentation
• Random walker algorithm
• Concept
• Properties
• Theory
• Numerics
• Results
• New
• Conclusion

8
General Purpose Segmentation
Goal Input an image and output the desired
segmentation
Problem Two users might want different objects
from same image
9
General Purpose Segmentation
Requires user interaction
10
General Purpose Segmentation
11
General Purpose Segmentation
Popular seeding algorithms
12
General Purpose Segmentation
Popular seeding algorithms
Graph cuts Max-flow/min-cut found between seeds

• Fast
• Probabilistic interpretation
• Requires lots of seeds to avoid small cut
  problem
• Metrication artifacts
• True minimum only for two objects (i.e.,
  foreground/background)

13

Outline

• Overview of Siemens Corporate Research (SCR)
• General purpose segmentation
• Random walker algorithm
• Concept
• Properties
• Theory
• Numerics
• Results
• New
• Conclusion

14

Random Walker - Concept
Given labeled voxels, for each voxel ask What is
the probability that a random walker starting
from this voxel first reaches each set of labels?
Do not despair Can be computed analytically!
15
Random Walker - Concept
Partially labeled image
Segmented image
Probabilities
16
Random Walker - Concept
17

Outline

• Overview of Siemens Corporate Research (SCR)
• General purpose segmentation
• Random walker algorithm
• Concept
• Properties
• Theory
• Numerics
• Results
• New
• Conclusion

18
Random Walker - Properties
Naturally respects weak object boundaries
19
Random Walker - Properties
Naturally respects weak object boundaries
20
Random Walker - Properties
21
Random Walker - Properties

• Segmented regions are connected to a seed
• The probabilities for a blank image (e.g., all
  black) yield a Voronoi-like segmentation
• The expected segmentation for an image of pure
  noise (identical r.v.s) is equal to the
  Voronoi-like segmentation obtained from a blank
  image

22
Random Walker - Properties
Graph cuts
Random walker
23
Random Walker - Properties
24

Outline

• Overview of Siemens Corporate Research (SCR)
• General purpose segmentation
• Random walker algorithm
• Concept
• Properties
• Theory
• Numerics
• Results
• New
• Conclusion

25
Random Walker - Theory
How to compute?
Solution to random walk problem equivalent to
minimization of the Dirichlet integral
with appropriate boundary conditions.
The solution is given by a harmonic function,
i.e., a function satisfying
26
Random Walker - Theory
Discrete or continuous space?
27
Random Walker - Theory
Attractive numerical properties of a harmonic
function

• Mean value theorem
• Maximum/Minimum principle

28
Random Walker - Theory
Need to represent Laplacian on a graph In the
notation of algebraic topology, the Laplacian is
given by
0-coboundary operator (since we operate on nodes)
is the incidence matrix
With the constituitive matrix Ceij eijwij
playing the role of the metric tensor, the
combinatorial Laplace-Beltrami operator is given
as
29
Random Walker - Theory
Energy functional
Subject to boundary conditions at seed locations
Euler-Lagrange
30
Random Walker - Theory
Laplacian matrix defined by graph as
Decompose Laplacian matrix into labeled (marked)
and unlabeled blocks and define an indicator
vector for the marked nodes
Must solve a sparse, SPD, system of linear
equations for probabilities
Since probabilities must sum to unity, for K
labels, only K-1 systems must be solved
31
Random Walker - Concept
Random walk formulated on a lattice (graph) that
represents the image
32
Random Walker - Theory
Therefore, we can formulate a combinatorial
Dirichlet integral
Represents minimum power distribution of an
electrical circuit
We can analytically solve the equivalent circuit
problem for the random walker probabilities
33
Random Walker - Theory
Situation exactly analogous to DC circuit
steady-state
Labels Unit voltage sources or
grounds Weights Branch
conductances Probabilities Steady-state
potentials
34
Random Walker - Theory
Algorithm summary

• Generate weights based on image intensities
• Build Laplacian matrix
• Solve system of equations for each label
• Assign pixel (voxel) to label for which it has
  the highest probability

35
Random Walker - Theory
Equally valid interpretations of algorithm

• What is the steady-state temperature distribution
  in the inhomogeneous domain, given fixed
  temperatures at the seeds?
• What is the probability that a random walker
  leaving this node first reaches a label of each
  color?
• What is the electrical potential at this node
  when the labeled nodes are fixed to unity voltage
  (w.r.t. ground)?
• What is the (normalized) effective resistance
  between this node and the labeled nodes?

36
Random Walker - Theory
Equally valid interpretations of algorithm
5. If a 2-tree (tree with a missing edge) is
drawn randomly, what is the probability that this
node is connected to each label?
Interpretation used to prove noise robustness
37

Outline

• Overview of Siemens Corporate Research (SCR)
• General purpose segmentation
• Random walker algorithm
• Concept
• Properties
• Theory
• Numerics
• Results
• New
• Conclusion

38
Random Walker - Numerics
Main computational burden is solving the system
of linear equations
Fortunately, system is sparse, symmetric,
positive definite For a lattice (or any regular
graph), the sparsity structure of the matrix is
circulant
39
Random Walker - Numerics
Advantages of a GPU implementation

• Structure of the Laplacian matrix allows for
  efficient storage and operations Off diagonals
  may be packed into RGBA
• Progressive visualization of solution possible
• Z-buffer allows masking out of seeds

40

Outline

• Overview of Siemens Corporate Research (SCR)
• General purpose segmentation
• Random walker algorithm
• Concept
• Properties
• Theory
• Numerics
• Results
• New
• Conclusion

41
Random Walker - Results
42
Random Walker - Results
43
Random Walker - Results
44
Random Walker - Results
45
Random Walker - Results
Cardiac segmentation across modalities
46
Random Walker - Results
Segmentation of objects with varying size, shape
and texture
47

Outline

• Overview of Siemens Corporate Research (SCR)
• General purpose segmentation
• Random walker algorithm
• Concept
• Properties
• Theory
• Numerics
• Results
• New
• Conclusion

48
Random Walker - New
Possible to incorporate other terms Intensity
priors
Useful for multiple, disconnected objects
49
Random Walker - New
Systematic study of weighting function
Gaussian weighting
Reciprocal weighting
Run on 62 CT datasets with seeds and manual
segmentations
50
Random Walker - New
Systematic study of edge topology
6-connected
10-connected
26-connected
51
Random Walker - New
Random Walker - New
Formulate as special case of general segmentation
approach - Compare with other instances of
algorithm
52
Random Walker - New
Precomputation

• Precompute eigenvectors of Laplacian
• Input seeds
• Instant result (approximation)

53

Outline

• Overview of Siemens Corporate Research (SCR)
• General purpose segmentation
• Random walker algorithm
• Concept
• Properties
• Theory
• Numerics
• Results
• New
• Conclusion

54
Conclusion
Random walker algorithm is

• General-purpose
• Robust to noise and weak boundaries
• Has a single parameter (not adjusted for these
  results)
• Stable
• Accurate
• Available

55
Conclusion More Information
Writings and code
My webpage http//cns.bu.edu/lgrady
Random walkers paper http//cns.bu.edu/lgrady/gr
ady2006random.pdf
Random walkers MATLAB code http//cns.bu.edu/lgr
ady/random_walker_matlab_code.zip
Random walker demo page http//cns.bu.edu/lgrady
/Random_Walker_Image_Segmentation.html
MATLAB toolbox for graph theoretic image
processing at http//eslab.bu.edu/software/grapha
nalysis/
CVPR Short Course Fundamentals linking discrete
and continuous approaches to computer vision - A
topological view http//cns.bu.edu/lgrady/Short_C
ourse.html