Efficiently Solving Convex Relaxations

1
Efficiently Solving Convex Relaxations
for MAP Estimation

• M. Pawan Kumar
• University of Oxford

Philip Torr Oxford Brookes University
2
Aim

• To solve convex relaxations of MAP estimation

6
3
0
1
2
4
0
Label 1
1
2
4
1
1
3
Label 0
1
0
5
3
7
0
2
b
c
d
a
Random Variables V a, b, c, d
Edges E (a, b), (b, c), (c, d)
Label Set L 0, 1
Labelling m 1, 0, 0, 1
3
Aim

• To solve convex relaxations of MAP estimation

6
3
0
1
2
4
0
Label 1
1
2
4
1
1
3
Label 0
1
0
5
3
7
0
2
b
c
d
a
Cost(m) 2 1 2 1 3 1 3 13
Minimum Cost Labelling? NP-hard problem
Approximate using Convex Relaxations
4
Aim

• To solve convex relaxations of MAP estimation

6
3
0
1
2
4
0
Label 1
1
2
4
1
1
3
Label 0
1
0
5
3
7
0
2
b
c
d
a
Objectives

• Solve tighter convex relaxations LP and SOCP
• Handle large number of random variables, e.g.
  image pixels

5
Outline

• Integer Programming Formulation
• Linear Programming Relaxation
• Additional Constraints
• Solving the Convex Relaxations
• Results and Conclusions

6
Integer Programming Formulation
2
4
0
Unary Cost
Label 1
1
3
Label 0
5
0
2
b
a
Labelling m 1 , 0
2 4
2
Unary Cost Vector u 5
7
Integer Programming Formulation
2
4
0
Unary Cost
Label 1
1
3
Label 0
5
0
2
b
a
Labelling m 1 , 0
2 4 T
2
Unary Cost Vector u 5
Label vector x -1
1
1 -1 T
Recall that the aim is to find the optimal x
8
Integer Programming Formulation
2
4
0
Unary Cost
Label 1
1
3
Label 0
5
0
2
b
a
Labelling m 1 , 0
2 4 T
2
Unary Cost Vector u 5
Label vector x -1
1
1 -1 T
1
Sum of Unary Costs
?i ui (1 xi)
2
9
Integer Programming Formulation
2
4
0
Pairwise Cost
Label 1
1
3
Label 0
5
0
2
b
a
Labelling m 1 , 0
Pairwise Cost of a and a
3
0
10
Integer Programming Formulation
2
4
0
Pairwise Cost
Label 1
1
3
Label 0
5
0
2
b
a
Labelling m 1 , 0
Sum of Pairwise Costs
1
?ij Pij (1 xi)(1xj)
0
3
0
4
11
Integer Programming Formulation
2
4
0
Pairwise Cost
Label 1
1
3
Label 0
5
0
2
b
a
Labelling m 1 , 0
Sum of Pairwise Costs
1
?ij Pij (1 xi xj xixj)
0
3
0
4
X x xT
Xij xi xj
12
Integer Programming Formulation
Constraints

• Integer Constraints

xi ?-1,1
X x xT
13
Integer Programming Formulation
1
1
? Pij (1 xi xj Xij)
x argmin
? ui (1 xi)

4
2
Convex
xi ?-1,1
X x xT
14
Outline

• Integer Programming Formulation
• Linear Programming Relaxation
• Additional Constraints
• Solving the Convex Relaxations
• Results and Conclusions

15
Linear Programming Relaxation
Schlesinger, 1976
Retain Convex Part
1
1
? Pij (1 xi xj Xij)
x argmin
? ui (1 xi)

4
2
xi ?-1,1
X x xT
16
Linear Programming Relaxation
Schlesinger, 1976
Retain Convex Part
1
1
? Pij (1 xi xj Xij)
x argmin
? ui (1 xi)

4
2
xi ?-1,1
Xij ?-1,1
1 xi xj Xij 0
17
Dual of the LP Relaxation
Wainwright et al., 2001
?1
a
b
c
?1
a
b
c
?2
d
e
f
?2
d
e
f
g
h
i
?3
?3
g
h
i
?4
?5
?6
? (u, P)
a
b
c
d
e
f
g
h
i
? ?i?i ? ?
?4
?5
?6
18
Dual of the LP Relaxation
Wainwright et al., 2001
?1
Q(?1)
a
b
c
a
b
c
d
e
f
?2
Q(?2)
d
e
f
g
h
i
?3
Q(?3)
g
h
i
Q(?4)
Q(?5)
Q(?6)
? (u, P)
a
b
c
Dual of LP
d
e
f
max ? ?i Q(?i)
g
h
i
? ?i?i ? ?
?4
?5
?6
19
Tree-Reweighted Message Passing
Kolmogorov, 2005
?4
?5
?6
a
b
c
a
b
c
?1
Pick a variable
a
?2
d
e
f
d
e
f
g
h
i
g
h
i
?3
u2
u4
u1
u3
c
b
a
a
d
g
Reparameterize such that ui are min-marginals
Only one pass of belief propagation
20
Tree-Reweighted Message Passing
Kolmogorov, 2005
?4
?5
?6
a
b
c
a
b
c
?1
Pick a variable
a
?2
d
e
f
d
e
f
g
h
i
g
h
i
?3
(u2u4)/2
(u2u4)/2
(u1u3)/2
(u1u3)/2
c
b
a
a
d
g
Average the unary costs
TRW-S
Repeat for all variables
21
Outline

• Integer Programming Formulation
• Linear Programming Relaxation
• Additional Constraints
• Solving the Convex Relaxations
• Results and Conclusions

22
Cycle Inequalities
Chopra and Rao, 1991
a
b
c
e
f
d
At least two of them have the same sign
xi
xixj xjxk xkxi
xj
xk
Xij Xjk Xki
X xxT
At least one of them is 1
Xij Xjk Xki ? -1
23
Cycle Inequalities
Chopra and Rao, 1991
a
b
c
e
f
d
Xij Xjk Xkl - Xli ? -2
Generalizes to all cycles
LP-C
24
Second-Order Cone Constraints
Kumar et al., 2007
a
b
c
e
f
d
Xc xcxcT
1 (Xc - xcxcT) ? 0
SOCP-C
(xixjxk)2 3 Xij Xjk Xki
25
Second-Order Cone Constraints
Kumar et al., 2007
a
b
c
e
f
d
SOCP-Q
1 (Xc - xcxcT) ? 0
26
Outline

• Integer Programming Formulation
• Linear Programming Relaxation
• Additional Constraints
• Solving the Convex Relaxations
• Results and Conclusions

27
Modifying the Dual
a
b
c
d
e
f
g
h
i
? ?j sj
? ?j sj
28
Modifying TRW-S
a
b
b
c
a
b
c
a
d
g
d
e
e
f
d
e
f
b
e
h
d
e
e
f
g
h
i
c
f
i
g
h
h
i
Pick a variable --- a
Pick a cycle/clique with a
REPEAT
max ? ?i Q(?i)
?j sj
? ?i?i ? ?
Can be solved efficiently
?j sj
Run TRW-S for trees with a
29
Properties of the Algorithm
Algorithm satisfies the reparametrization
constraint
Value of dual never decreases
CONVERGENCE
Solution satisfies Weak Tree Agreement (WTA)
WTA not sufficient for convergence
More accurate results than TRW-S
30
Outline

• Integer Programming Formulation
• Linear Programming Relaxation
• Additional Constraints
• Solving the Convex Relaxations
• Results and Conclusions

31
4-Neighbourhood MRF
Test SOCP-C
Test LP-C
50 binary MRFs of size 30x30
u N (0,1)
P N (0,s2)
32
4-Neighbourhood MRF
s 5
LP-C dominates SOCP-C
33
8-Neighbourhood MRF
Test SOCP-Q
50 binary MRFs of size 30x30
u N (0,1)
P N (0,s2)
34
8-Neighbourhood MRF
s 5 /?2
SOCP-Q dominates LP-C
35
Conclusions

• Modified LP dual to include more constraints
• Extended TRW-S to solve tighter dual
• Experiments show improvement
• More results in the poster

36
Future Work

• More efficient subroutines for solving
  cycles/cliques
• Using more accurate LP solvers - proximal
  projections
• Analysis of SOCP-C vs. LP-C

37
Questions?
38
Timings
Method Time/Iteration
BP 0.0027
TRW-S 0.0027
LP-C 7.7778
SOCP-C 8.8091
SOCP-Q 9.1170
Linear in the number of variables!!
39
Video Segmentation
Keyframe
User Segmentation
Segment remaining video .
40
Video Segmentation
Input
Belief Propagation
8175
25620
18314
41
Video Segmentation
Input
??-swap
1187
1368
1289
42
Video Segmentation
Input
?-expansion
2453
1266
1225
43
Video Segmentation
Input
TRW-S
6425
1309
297
44
Video Segmentation
Input
LP-C
719
264
294
45
Video Segmentation
Input
SOCP-Q
0
0
0
46
4-Neighbourhood MRF
s 1
47
4-Neighbourhood MRF
s 2.5
48
8-Neighbourhood MRF
s 1/?2
49
8-Neighbourhood MRF
s 2.5 /?2