Self-Validated Labeling of MRFs for Image Segmentation

1
Self-Validated Labeling of MRFs for Image
Segmentation
Accepted by IEEE TPAMI

• Wei Feng 1,2, Jiaya Jia 2 and Zhi-Qiang Liu 1
• 1. School of Creative Media, City University of
  Hong Kong
• 2. Dept. of CSE, The Chinese University of Hong
  Kong

2
Outline

• Motivation
• Graph formulation of MRF labeling
• Graduated graph cuts
• Experimental results
• Conclusion

3
Outline

• Motivation
• Graph formulation of MRF labeling
• Graduated graph cuts
• Experimental results
• Conclusion

4
Self-Validated Labeling

• Common problem segmentation, stereo etc.
• Self-validated labeling two parts
• Labeling quality accuracy (i.e., likelihood) and
  spatial coherence
• Labeling cost (i.e., the number of labels)
• Bayesian framework to minimize the Gibbs energy
  (equivalent form of MAP)

5
Motivation

• Computational complexity remains a major weakness
  of the MRF/MAP scheme
• Robustness to noise
• Preservation of soft boundaries
• Insensitive to initialization

6
Motivation

• Self-validation How to determine the number of
  clusters?
• To segment a large number of images
• Global optimization based methods are robust, but
  most are not self-validated
• Split-and-merge methods are self-validated, but
  vulnerable to noise

7
Motivation

• For a noisy image consisting of 5 segments
• Lets see the performance of the state-of-the art
  methods

8
Motivation

• Normalized cut (NCut) 1
• Unself-validated segmentation (i.e., the user
  needs to indicated the number of segments, bad)
• Robust to noise (good)
• Average time 11.38s (fast, good)
• NCut is unable to return satisfying result when
  feeded by the right number of segments 5 it can
  produce all right boundaries, mixed with many
  wrong boundaries, only when feeded by a much
  larger number of segments 20.

1 J. Shi and J. Malik, Normalized cuts and
image segmentation, PAMI 2000.
9
Motivation

• Bottom-up methods
• E.g., Mean shift 2
• E.g., GBS 3
• Self-validated (good)
• Very fast (lt 1s, good)
• But, sensitive to noise (bad)

2 D. Comaniciu and P. Meer. Mean shift A
robust approach towards feature space analysis,
PAMI 2002. 3 P. F. Felzenszwalb and D. P.
Huttenlocher. Efficient graph based image
segmentation, IJCV 2004.
10
Motivation

• Data-driven MCMC4
• Self-validated (good)
• Robust to noise (good)
• But, very slow (bad)

4 Z. Tu and S.-C. Zhu, Image segmentation by
data-driven Markov chain Monte Carlo, PAMI 2002.
11
Motivation

• As a result, we need a self-validated
  segmentation method, which is fast and robust to
  noise.
• Our method graduated graph mincut
• Tree-structured graph cuts (TSGC)
• Net-structured graph cuts (NSGC)
• Hierarchical graph cuts (HGC)

Time Seg
TSGC 2.96s 5
NSGC 5.7s 5
HGC 2.01s 6
12
Motivation
5
5 C. DElia, G. Poggi, and G. Scarpa, A
tree-structured Markov random field model for
Bayesian image segmentation, IEEE Trans.
Image Processing, vol. 12, no. 10, pp. 12501264,
2003.
13
Outline

• Motivation
• Graph formulation of MRF labeling
• Graduated graph cuts
• Experimental results
• Conclusion

14
Graph Formulation of MRFs

• Graph formulation of MRFs (with second order
  neighborhood system N2) (a) graph G ltV,Egt with
  K segments L1, L2 . . . LK and observation Y
  (b) final labeling corresponds to a multiway cut
  of the graph G.

15
Graph Formulation of MRFs

• Property Gibbs energy of segmentation Seg(I) can
  be defined as
• MRF-based segmentation ? multiway (K-way) graph
  mincut problem (NP-complete, K2 solvable)

16
Outline

• Motivation
• Graph formulation of MRF labeling
• Graduated graph cuts
• Experimental results
• Conclusion

17
Graduated Graph Mincut

• Main idea
• To gradually adjust the optimal labeling
  according to the Gibbs energy minimization
  principle.
• A vertical extension of binary graph mincut (in
  constrast to horizontal extension, a-expansion
  and a-ß swap)

18
Graduated Graph Mincut
19
Binary Labeling of MRFs
20
Binary Labeling of MRFs
21
Tree-structured Graph Cuts
22
Tree-structured Graph Cuts
23
Tree-structured Graph Cuts
(over-segmentation)
24
Net-structured Graph Cuts
25
Net-structured Graph Cuts
26
Net-structured Graph Cuts
27
Hierarchical Graph Cuts
28
Hierarchical Graph Cuts
29
Graduated Graph Cuts

• Summary
• An effective tool for self-validated labeling
  problems in low level vision.
• An efficient energy minimization scheme by graph
  cuts.
• Converting the K-class clustering into a sequence
  of K-1 much simpler binary clustering.
• Independent to initialization
• Very close good local minima obtained by
  a-expansion and a-ß swap

30
Segmentation Evolution
Iter 1
Iter 2
Iter 3
Iter 4
Mean image
31
Outline

• Motivation
• Graph formulation of MRF labeling
• Graduated graph cuts
• Experimental results
• Conclusion

32
Comparative Results
Comparative Experiments
33
Robustness to Noise
Robust to noise
34
Preservation of Soft Boundary
35
Consistency to Ground Truth
36
Coarse-to-Fine Segmentation
37
Performance Summary
38
Outline

• Motivation
• Graph formulation of MRF labeling
• Graduated graph cuts
• Experimental results
• Conclusion

39
Conclusion

• An efficient self-validated labeling method that
  is very close to good local minima and guarantees
  stepwise global optimum
• Provides a vertical extension to binary graph cut
  that is independent to initialization
• Ready to apply to a wide range of clustering
  problems in low-level vision

40

• Thanks!