Improved Moves for Truncated Convex Models

1
Improved Moves for Truncated Convex Models

• M. Pawan Kumar
• Philip Torr

2
Aim
Efficient, accurate MAP for truncated convex
models
V1
V2

















Vn
Random Variables V V1, V2, , Vn
Edges E define neighbourhood
3
Aim
Accurate, efficient MAP for truncated convex
models
lk
?abik
??abik wab min d(i-k), M
li
wab is non-negative
d(.) is convex
Vb
?bk
Va
?ai
Truncated Linear
Truncated Quadratic
??abik
??abik
i-k
i-k
4
Motivation
Low-level Vision
min i-k, M

• Smoothly varying regions

Boykov, Veksler Zabih 1998

• Sharp edges between regions

Well-researched !!
5
Things We Know

• NP-hard problem - Can only get approximation

• Best possible integrality gap - LP relaxation

Manokaran et al., 2008

• Solve using TRW-S, DD, PP

Slower than graph-cuts

• Use Range Move - Veksler, 2007

None of the guarantees of LP
6
Real Motivation
Gaps in Move-Making Literature
Chekuri et al., 2001
2
2 v2
O(vM)
Multiplicative Bounds
7
Real Motivation
Gaps in Move-Making Literature
Boykov, Veksler and Zabih, 1999
2
2
2 v2
2M
O(vM)
-
Multiplicative Bounds
8
Real Motivation
Gaps in Move-Making Literature
Gupta and Tardos, 2000
2
2
2 v2
4
O(vM)
-
Multiplicative Bounds
9
Real Motivation
Gaps in Move-Making Literature
Komodakis and Tziritas, 2005
2
2
2 v2
4
O(vM)
2M
Multiplicative Bounds
10
Real Motivation
Gaps in Move-Making Literature
2
2
2 v2
2 v2
O(vM)
O(vM)
Multiplicative Bounds
11
Outline

• Move Space
• Graph Construction
• Sketch of the Analysis
• Results

12
Move Space

• Initialize the labelling

• Choose interval I of L labels

• Each variable can

• Retain old label

• Choose a label from I

• Choose best labelling

Va
Vb
Iterate over intervals
13
Outline

• Move Space
• Graph Construction
• Sketch of the Analysis
• Results

14
Two Problems

• Choose interval I of L labels

• Each variable can

• Retain old label

• Choose a label from I

• Choose best labelling

Large L gt Non-submodular
Non-submodular
Va
Vb
15
First Problem
Submodular problem
Va
Vb
Ishikawa, 2003 Veksler, 2007
16
First Problem
Non-submodular Problem
Va
Vb
17
First Problem
Submodular problem
Va
Vb
Veksler, 2007
18
First Problem
am1
bm1
am2
bm2
am2
bm2
an
bn
t
Va
Vb
19
First Problem
am1
bm1
am2
bm2
am2
bm2
an
bn
t
Va
Vb
20
First Problem
am1
bm1
am2
bm2
am2
bm2
an
bn
t
Va
Vb
21
First Problem
am1
bm1
am2
bm2
am2
bm2
an
bn
t
Va
Vb
22
First Problem
am1
bm1
am2
bm2
am2
bm2
an
bn
t
Va
Vb
Model unary potentials exactly
23
First Problem
am1
bm1
am2
bm2
am2
bm2
an
bn
t
Va
Vb
Similarly for Vb
24
First Problem
am1
bm1
am2
bm2
am2
bm2
an
bn
t
Va
Vb
Model convex pairwise costs
25
First Problem
Wanted to model ?abik wab min d(i-k), M
For all li, lk ? I
Have modelled ?abik wab d(i-k) For all li,
lk ? I
Va
Vb
Overestimated pairwise potentials
26
Second Problem

• Choose interval I of L labels

• Each variable can

• Retain old label

• Choose a label from I

• Choose best labelling

Non-submodular problem !!
Va
Vb
27
Second Problem - Case 1
s
8
8
am1
bm1
am2
bm2
an
bn
t
Va
Vb
Both previous labels lie in interval
28
Second Problem - Case 1
s
8
8
am1
bm1
am2
bm2
an
bn
t
Va
Vb
wab d(i-k)
29
Second Problem - Case 2
s
8
ub
am1
bm1
am2
bm2
an
bn
t
Va
Vb
Only previous label of Va lies in interval
30
Second Problem - Case 2
s
8
ub
M
am1
bm1
am2
bm2
an
bn
t
Va
Vb
ub unary potential of previous label of Vb
31
Second Problem - Case 2
s
8
ub
M
am1
bm1
am2
bm2
an
bn
t
Va
Vb
wab d(i-k)
32
Second Problem - Case 2
s
8
ub
M
am1
bm1
am2
bm2
an
bn
t
Va
Vb
wab ( d(i-m-1) M )
33
Second Problem - Case 3
am1
bm1
am2
bm2
an
bn
t
Va
Vb
Only previous label of Vb lies in interval
34
Second Problem - Case 3
s
8
ua
M
am1
bm1
am2
bm2
an
bn
t
Va
Vb
ua unary potential of previous label of Va
35
Second Problem - Case 4
am1
bm1
am2
bm2
an
bn
t
Va
Vb
Both previous labels do not lie in interval
36
Second Problem - Case 4
s
ua
ub
Pab
M
M
am1
bm1
ab
am2
bm2
an
bn
t
Va
Vb
Pab pairwise potential for previous labels
37
Second Problem - Case 4
s
ua
ub
Pab
M
M
am1
bm1
ab
am2
bm2
an
bn
t
Va
Vb
wab d(i-k)
38
Second Problem - Case 4
s
ua
ub
Pab
M
M
am1
bm1
ab
am2
bm2
an
bn
t
Va
Vb
wab ( d(i-m-1) M )
39
Second Problem - Case 4
s
ua
ub
Pab
M
M
am1
bm1
ab
am2
bm2
an
bn
t
Va
Vb
Pab
40
Graph Construction
Find st-MINCUT. Retain old labelling if energy
increases.
am1
bm1
am2
bm2
an
bn
t
Va
Vb
ITERATE
41
Outline

• Move Space
• Graph Construction
• Sketch of the Analysis
• Results

42
Analysis
Va
Vb
Current labelling f(.)
QC QC
QP
43
Analysis
Va
Vb
Va
Vb
Current labelling f(.)
Partially Optimal f(.)
Q0

QC QC
44
Analysis
Va
Vb
Va
Vb
Current labelling f(.)
Partially Optimal f(.)
QP- Q0

QP - QC
45
Analysis
Va
Vb
Va
Vb
Va
Vb
Current labelling f(.)
Partially Optimal f(.)
Local Optimal f(.)
QP- Q0
0
0
QP - QC
46
Analysis
Va
Vb
Va
Vb
Va
Vb
Current labelling f(.)
Partially Optimal f(.)
Local Optimal f(.)
QP- Q0
0
Take expectation over all intervals
47
Analysis
Truncated Linear
L M
4
Gupta and Tardos, 2000
L v2M
2 v2
Truncated Quadratic
L vM
48
Outline

• Move Space
• Graph Construction
• Sketch of the Analysis
• Results

49
Synthetic Data - Truncated Linear
Energy
Time (sec)
Faster than TRW-S
Comparable to Range Moves
With LP Relaxation guarantees
50
Synthetic Data - Truncated Quadratic
Energy
Time (sec)
Faster than TRW-S
Comparable to Range Moves
With LP Relaxation guarantees
51
Stereo Correspondence
Disparity Map
Unary Potential Similarity of pixel colour
Pairwise Potential Truncated convex
52
Stereo Correspondence
Teddy
53
Stereo Correspondence
Teddy
54
Stereo Correspondence
Tsukuba
55
Summary

• Moves that give LP guarantees
• Similar results to TRW-S
• Faster than TRW-S because of graph cuts

56
Questions Not Yet Answered

• Move-making gives LP guarantees
• True for all MAP estimation problems?
• Huber function? Parallel Imaging Problem?
• Primal-dual method?
• Solving more complex relaxations?

57
Questions?
Improved Moves for Truncated Convex Models
Kumar and Torr, NIPS 2008
http//www.robots.ox.ac.uk/pawan/