Graph Cuts and Stereo

1
Graph Cuts and Stereo

• Jim Rehg
• Many slides from Yuri Boykov

2
MRF framework in the context of stereo

• image pixels (vertices)

• neighborhood relationships (n-links)

MRF defining property
Hammersley-Clifford Theorem
3
MAP estimation of MRF configuration
4
Energy minimization
5
Generalized Potts model
6
Graph cuts for spatially coherent stereo on 2D
grids
We will globally minimize the same energy of
disparities for pixels p on a grid G
spatial coherence
photo consistency
7
Pixel interactions Vconvex vs.
discontinuity-preserving
Robust discontinuity preserving Interactions V
Convex Interactions V
V(dL)
dLLp-Lq
8
Pixel interactionsconvex vs.
discontinuity-preserving
9
a-expansionsexamples of metric interactions
10
Graph Cuts Basics (see Cormens book)Simple 2D
example
Goal divide the graph into two parts separating
red and blue nodes
A graph with two terminals S and T

• Cut cost is a sum of severed edge weights
• Minimum cost s-t cut can be found in polynomial
  time

11
Minimum s-t cuts algorithms

• Augmenting paths Ford Fulkerson, 1962
• Push-relabel Goldberg-Tarjan, 1986

12
Augmenting Paths

• Find a path from S to T along non-saturated edges

• Increase flow along this path until some edge
  saturates

13
Augmenting Paths

• Find a path from S to T along non-saturated edges

• Increase flow along this path until some edge
  saturates

• Find next path

• Increase flow

14
Augmenting Paths

• Find a path from S to T along non-saturated edges

• Increase flow along this path until some edge
  saturates

Iterate until all paths
from S to T have at least one saturated edge
MAX FLOW
15
Lemma on Flow-Cut
T
S
16
Minimization of E(f) via graph cuts
p-vertices (pixels)
17
Multiway cut
vertices V pixels terminals
Remove a subset of edges C
edges E n-links t-links

• C is a multiway cut if terminals are separated
  in G(C)

18
Main Result (generalized Potts model)

• Under some technical conditions on
  the multiway min-cut C on G gives___
  that minimizes E( f ) - the posterior energy
  function for the generalized Potts model.

• Multiway cut Problem find minimum cost
  multiway cut C graph G

19
Solving multiway cut problem

• Case of two terminals
• max-flow algorithm (Ford, Fulkerson 1964)
• polinomial time (almost linear in practice).
• NP-complete if the number of labels gt2
• (Dahlhaus et al., 1992)
• Efficient approximation algorithms that are
  optimal within a factor of 2

20
a-expansion algorithm

• Start with any initial solution
• For each label a in any (e.g. random) order
• Compute optimal a-expansion move (s-t graph cuts)
• Decline the move if there is no energy decrease
• Stop when no expansion move would decrease energy

21
a-expansion move
Basic idea
break multi-way cut computation into a sequence
of binary s-t cuts
22
a-expansion moves
In each a-expansion a given label a grabs
space from other labels
For each move we choose expansion that gives the
largest decrease in the energy binary
optimization problem
23
Multi-scan-line stereo with s-t graph cuts
(RoyCox98)
24
Multi-scan-line stereo with s-t graph cuts
(RoyCox98)
25
IshikawaGeiger 98 What energy do we minimize
this way?
Concentrate on one pair of neighboring pixels
26
IshikawaGeiger 98 What energy do we minimize
this way?
Concentrate on one pair of neighboring pixels
27
IshikawaGeiger 98 What energy do we minimize
this way?
The combined energy over the entire grid G is
(photo consistency) cost of vertical edges
cost of horizontal edges (spatial consistency)
28
Scan-line stereo vs. Multi-scan-line stereo
s-t Graph Cuts (multi-scan-line optimization)
Dynamic Programming (single scan line
optimization)
29
Some results from RoyCox
multi scan line stereo (graph cuts)
single scan-line stereo (DP)
30
Some results from RoyCox
multi scan line stereo (graph cuts)
single scan-line stereo (DP)