Guest Lecture : Comp 256 Graph Cuts for Discrete Optimization in Computer Vision

1
Guest Lecture Comp 256Graph Cuts for Discrete
Optimizationin Computer Vision

• Sudipta N. Sinha
• April 6, 2005
• ( some slides taken from ECCV04 Tutorial
  Discrete Optimization
• Methods in Computer Vision Boykov, Torr and
  Zabih )

2
Outline

• Labeling Problem Energy Minimization
• Energy Minimization using Graph Cuts
• a - expansion algorithm.
• Graph Cuts Network Flow
• Graph Cut Formulation for Computer Vision
  problems
• Image Restoration
• Stereo
• Segmentation
• Volumetric Reconstruction

3
The Labeling Problem

• Common idea behind many Computer Vision problems
• Assign labels to pixels based on noisy
  measurements (input images)
• In the presence of uncertainties, find the best
  Labeling !
• A Combinatorial Optimization problem !
• (Stereo, 3D Reconstruction, Segmentation, Image
  Restoration)

4
The Labeling Problem

• Common visual constraints encourages specific
  type of labeling
• Spatially Constant With Discontinuities
• Spatially Smooth With Discontinuities.
• Examples ..

5
Energy Minimization

• Optimizing the labeling problem can be thought of
  as minimizing some energy function.
• measure of image discrepancy
• measure of
  smoothness or
• other visual
  constraints

6
Energy Minimization

• Justification Bayesian Estimation of MRF.
• Markov Random Field (MRF)
• Set of Sites S
• Set of Labels
• Neighborhood System
• Set of Random Variables
• that take on labels from L.
• Markov Property
• At the MAP Estimate of certain type of MRFs,
• the energy function E( f ) is minimized.

( Details Boykov Veksler Zabih CVPR 98 )
7
Choice for Interaction Term V( fp , fq )
Pott Interaction Model Linear Interaction Model

8
Energy Minimization via graph cuts

• Simple Example 2-label case
• p vertices, t vertices,
• n links, t links.

9
What Energy Functions can be minimized via graph
cuts ?

General- purpose
Special- purpose
Convex V Global min
Potts V 2-approximation
Regular V Strong local min
Arbitrary V Local min
Expansion move algorithm BVZ PAMI 01
10
a - expansion algorithm

• 2label minimization computed by solving max-flow
  (s-t cut) exactly in polynomial time.
• Multi-way cut is NP-Hard for 3 labels.
• Boykov et. al (PAMI01) proposes an approximation
  algorithm that uses a
• cycle of a - expansion moves.
• Each a - expansion move is computed by solving
  max-flow once on a different graph.

11
a - expansion algorithm

• Start with arbitrary labeling
• Perform Optimization Cycles (till convergence)
• In Each Cycle,
• Do a expansion move once for each label,
• ( try to find a better labeling f than current
  f )
• When no f is found in a cycle, convergence !
• See Boykov ( PAMI01 ) for details of the Graph
  Construction
• for computing each a expansion move by solving
  max-flow.

12
Network Flows
Def Flow Network Directed graph G (V, E ) Edge
capacity c( u, v ) gt 0 Special vertices
source s and sink t Path s?v??t exists.

• Def Flow - A function f V V ? R
  satisfying
• Capacity constraint u, v ? V f (u, v ) lt
  c ( u, v )
• Skew symmetry u, v ? V f (u, v )
  f ( v, u )
• Flow conservation
• Value of Flow f

13
The Maximum Flow Problem

• Find a flow that has the maximum value.

• Maximum Flow Algorithms
• Augmenting paths Ford Fulkerson, 1962
• Push-relabel Goldberg-Tarjan, 1986

14
Graph Cuts Network Flow
Partition the graph into two parts separating
red and blue nodes
A graph with two terminals S and T

• Cut cost is a sum of severed edge weights

15
Graph Cuts Network Flow

• The Max-flow Min-Cut Theorem
• If f is a maximum flow,
• f c (S,T ) for some cut (S,T ) of G
• The cost of the minimum s-t cut the maximum
  flow.
• Thus, we will find minimum s-t cuts in graphs by
• solving for max-flow.

16
Graph Cut based Image Restoration
Sites Pixels , Labels Intensities,
Neighborhood System 8 pixel
neighborhood Visual Constraint Intensities
vary smoothly, Discontinuities on
intensity boundaries. Pixel Interaction Models
Pott Energy Model , Linear Energy Model
17
Graph Cut based Stereo
Disparity Image Smoothly varying with
discontinuities How to deal with occlusions ?
18
Graph Cut based Stereo
Sites Pair of Pixels Labels (0,1) where
19
Graph Cut based Segmentation
User Guided Segmentation Specifies hard
constraints.
20
Graph Cut based Voxel Occupancy
( Snow, Viola, Zabih CVPR00)
Visual Hull Reconstruction from noisy
silhouettes. Sites voxels , Labels 0
(foreground) 1 (background) Neighborhood System
6 voxel neighborhood Labeling Constraint
foreground objects are smooth, labeling is
expected to be piece-wise constant. Pixel
Interaction Models Pott Energy Model. Exact
Energy Minimization Possible !
21
Graph Cut based Voxel Occupancy
( Snow, Viola, Zabih CVPR00)
22
Direct Max-flow formulation of N-view stereo
( Roy, Cox IJCV 99)

• Goal To solve the n-view stereo correspondence
  problem by treating all camera views uniformly
• Approach a global optimization approach that
  converts the stereo problem into the maximum flow
  problem on a graph.
• No Energy Minimization here !
• Cost Functional based on
• photo-consistency ?

23
Direct Max-flow formulation of N-view stereo
( Roy, Cox IJCV 99)
Enforcing Smoothness.
Graph Construction
24
Direct Max-flow formulation of N-view stereo
( Roy, Cox IJCV 99)
Simplification in this formulation Occlusion is
not modeled
25
Conclusions

• Labeling Problems posed as Energy Minimization
  Problems
• Graph-cut based discrete optimization for some
  Energy Functions. ( See Kolmogorov ECCV02 for
  detail)
• s-t Graph cuts / Binary energy minimization
• Multiway cuts / Multi-label energy minimization
• a -expansion algorithm
• Energy Minimization framework for the problems of
  
• Image Restoration, Stereo, Segmentation,
  Volumetric Reconstruction
• Reference ECCV Tutorial
• Discrete Optimization Methods in Computer Vision
• http//wwwcms.brookes.ac.uk/philiptorr/eccv_tutor
  ial_2004.htm
• http//www.cs.cornell.edu/rdz/graphcuts.html