Strong LP Formulations

1
Strong LP Formulations Primal-Dual
Approximation Algorithms

• David Shmoys
• (joint work Tim Carnes Maurice Cheung)

June 23, 2011
TexPoint fonts used in EMF. Read the TexPoint
manual before you delete this box. AAAAAAAAAAAA
2
Introduction

• The standard approach to solve combinatorial
  integer programs in practice start with a
  simple formulation add valid inequalities
• Our agenda show that same approach can be
  theoretically justified by approximation
  algorithms
• An -approximation algorithm produces a solution
  of cost within a factor of of the optimal
  solution in polynomial time

3
Introduction

• Primal-dual method
• a leading approach in the design of
  approximation algorithms for NP-hard problems
• Consider several capacitated covering problems
  - covering knapsack problem
• - single-machine scheduling problems
• Give constant approximation algorithms based on
  strong linear programming (LP) formulations

4
Approximation Algorithms and LP

• Use LP to design approximation algorithms
• Optimal value for LP gives bound on optimal
  integer programming (IP) value
• Want to find feasible IP solution of value within
  a factor of of optimal LP solution
• Key is to start with right LP relaxation
• LP-based approximation algorithm produces
  additional performance guarantee on each problem
  instance
• Empirical success of IP cutting plane methods
  suggests stronger formulations - needs theory!

5
Primal-Dual Approximation Algorithms

• Do not even need to solve LP!!
• Each min LP has a dual max LP
• of equal optimal value
• Goal Construct feasible integer solution S along
  with feasible solution D to dual of LP relaxation
  such that
• cost(S) cost(D) LP-OPT OPT
• ) -approximation algorithm

6
Adding Valid Inequalities to LP

• LP formulation can be too weak if there is
  big integrality gap - OPT/LP-OPT is often
  unbounded
• Fixed by adding additional inequalities to
  formulation
• Restricts set of feasible LP solutions
• Satisfied by all integer solutions, hence valid
• Key technique in practical codes to solve integer
  programs

7
Knapsack-Cover Inequalities

• Carr, Fleischer, Leung and Phillips (2000)
  developed valid knapsack-cover inequalities and
  LP-rounding algorithms for several capacitated
  covering problems
• Requires solving LP with ellipsoid method
• Further complicated since inequalities are not
  known to be separable
• GOAL develop a primal-dual analog!

8
Highlights of Our Results

• For each of the following problems, we have a
  primal-dual algorithm that achieves a performance
  guarantee of 2
• Min-Cost (Covering) Knapsack
• Single-Demand Capacitated
• Facility Location
• Single-Item Capacitated Lot-Sizing
• We extend the knapsack-cover inequalities to
  handle this more general setting
• Single-Machine Minimum-Weight Late Jobs 1? wj
  Uj
• Single-Machine General Minimum-Sum Scheduling
  1? fj

Used valid knapsack-cover inequalities developed
by Carr, Fleischer, Leung and Phillips as LP
formulation
9
Highlights of Our Results

• For each of the following problems, we have a
  primal-dual algorithm that achieves a performance
  guarantee of 2
• Min-Cost (Covering) Knapsack
• Single-Demand Capacitated
• Facility Location
• Single-Item Capacitated Lot-Sizing
• We extend the knapsack-cover inequalities to
  handle this more general setting
• Single-Machine Minimum-Weight Late Jobs 1? wj
  Uj
• Single-Machine General Minimum-Sum Scheduling
  1? fj

Used valid knapsack-cover inequalities developed
by Carr, Fleischer, Leung and Phillips as LP
formulation
10
Min-Sum 1-Machine Scheduling 1? fj

• Each job j has a cost function fj(Cj) that is
  non-negative non-decreasing function of its
  completion time Cj
• Goal minimize ?j fj(Cj)
• What is known? Bansal Pruhs (FOCS 10) gave
  first constant-factor algorithm
• Main result of Bansal-Pruhs adds release dates,
  and permits preemption result is
  O(loglog(nP))-approximation algorithm
• OPEN QUESTIONS Is a constant-factor doable?

11
10 Open Problems

• Better constant factors?
• Any constant factor?
• Primal-dual when
• rounding is known?
• But nothing of the type
• good constant factor
• is known, but is a
• factor of 1? possible
• for any ? gt0?

12
Min-Sum 1-Machine Scheduling 1? fj

• Each job j has a cost function fj(Cj) that is
  non-negative non-decreasing function of its
  completion time Cj
• Goal minimize ?j fj(Cj)
• What is known? Bansal Pruhs (FOCS 10) gave
  first constant-factor algorithm
• Main result of Bansal-Pruhs adds release dates,
  and permits preemption result is
  O(loglog(nP))-approximation algorithm
• OPEN QUESTIONS Is a constant-factor doable?
• - Can 1² be achieved w/o release dates?

13
Primal-Dual for Covering Problems

• Early primal-dual algorithms
• Bar-Yehuda and Even (1981) weighted vertex
  cover
• Chvátal (1979) weighted set cover
• Agrawal, Klein and Ravi (1995)
• Goemans and Williamson (1995)
• generalized Steiner (cut covering) problems
• Bertismas Teo (1998)
• Jain Vazirani (1999)
• uncapacitated facility location problem
• Inventory problems
• Levi, Roundy and Shmoys (2006)

14
Minimum (Covering) Knapsack Problem

• Given a set of items F each with a cost ci and a
  value ui
• Want to find a subset of items with minimum cost
  such that the total value is at least D

minimize ?i ? F ci xi subject to ?i ? F ui xi ?
D xi ? 0,1 for each i ? F
15
Bad Integrality Gap

• Consider the min knapsack problem with the
  following two items
• Integer solution must take item 1 and incurs a
  cost of 1
• LP solution can take all of item 2 and just 1/D
  fraction of item 1, incurring a cost of 1/D

c1 1 u1 D
c2 0 u2 D-1
16
Knapsack-Cover Inequalities
D

• Proposed by Carr, Fleischer, Leung and Phillips
  (2000)
• Consider a subset A of items in F
• If we were to take all items in A, then we still
  need to take enough items to meet leftover demand

D u(A)
A 1,2,3
1
3
2
?i 2 F n A ui xi D-u(A)
u(A) ?i 2 A ui
17
Knapsack-Cover Inequalities
D
6
?i 2 FnA ui xi D-u(A)

• This inequality adds nothing new, but we can now
  restrict the values of the items
• where
• since these inequalities only need to be valid
  for integer solutions

4
D u(A)
5
7
3
?i 2 FnA ui(A) xi D-u(A)
2
ui(A) min ui , D-u(A)
A 1,2,3
1
18
Knapsack-Cover Inequalities on Bad Example
c1 1 u1 D
c2 0 u2 D-1

• Before Integer solution picks item 1 for cost
  1
• LP solution picks item 2 and 1/D of item 1
  for cost 1/D
• Now Consider knapsack-cover ineq with A 2
• Then D u(A) 1 and ui(A) 1 so
• Thus LP must take all of item 1 for cost 1

19
Strengthened Min Knapsack LP

• When A the knapsack-cover inequality becomes
• which is the original min knapsack inequality
• New strengthened LP is

?i 2 FnA ui(A) xi D-u(A)
) ? i 2 F ui xi D
Minimize ?i 2 F ci xi subject to ?i 2 FnA ui(A)
xi D- u(A), for each subset A xi 0,
for each i 2 F
20
Dual Linear Program
optDual max ?A µ F (D-u(A))v(A) subject to
?A µ F i ? A ui(A) v(A) ci , for each i 2
F v(A) 0, for each A µ F

• Dual of LP formed by knapsack-cover inequalities

21
Primal-Dual
D 5
1.25
2
0.25
2
0.75
Dual Variables Initially all zero
Dual Variables v() 0.25
A
A 3
D - u(A) 5 Increase v(A)
D - u(A) 3 Increase v(A)
22
Primal-Dual
D 5
1.25
1
0.25
2
0.75
Dual Variables v() 0.25
A 3
D - u(A) 3 Increase v(A)
23
Primal-Dual
D 5
1.25
1
0.25
2
0.75
Dual Variables v() 0.25
A 3
D - u(A) 3 Increase v(A)
24
Primal-Dual
D 5
1
1.75
0
2.92
0.5
Dual Variables v() 0.25
Dual Variables v() 0.25 v(3) 0.5
A 3
A 3,5
D - u(A) 3 Increase v(A)
D - u(A) 1 Increase v(A)
25
Primal-Dual
D 5
1
1.75
0
2.92
0.5
Dual Variables v() 0.25 v(3) 0.5
A 3
A 3,5
D - u(A) 1 Increase v(A)
26
Primal-Dual
D 5
1
1.25
0
7.25
0
Dual Variables v() 0.25 v(3) 0.5
Dual Variables v() 0.25 v(3) 0.5 v(3,5)
1
A 3
A 3,5
A 3,5,1
D - u(A) 1 Increase v(A)
D - u(A) -1 Increase v(A)
Stop!
27
Primal-Dual
Primal-Dual Cost 4.5
c1 2.5 u1 2
c2 2 u2 1
c3 0.5 u3 2
c4 10 u4 5
c5 1.5 u5 2
Opt. Integer Cost 4
c1 2.5 u1 2
c2 2 u2 1
c3 0.5 u3 2
c4 10 u4 5
c5 1.5 u5 2
28
Primal-Dual Summary

• Start with all variables set to zero and solution
  A as the empty set
• Increase variable v(A) until a dual constraint
  becomes tight for some item i
• Add item i to solution A and repeat
• Stop once solution A has large enough value to
  meet demand D
• Call final solution S and set xi 1 for all i 2 S

29
Analysis

• Let l be last item added to solution S
• If we increased dual variable v(A) then l was
  not in A
• Thus if v(A) gt 0 then A µ ( S \ l )
• Since u( S\ l ) lt D
• then u((S\ l )\A) lt D u(A)

30
Analysis (continued)

• We have u((S\ l )\A) lt D u(A) if v(A) gt 0
• Cost of solution is

Dual LP
31
Primal-Dual Theorem

• For the min-cost covering knapsack problem, the
  LP relaxation with knapsack-cover inequalities
  can be used to derive a (simple) primal-dual
  2-approximation algorithm.

32
Knapsack-Cover Inequalities Everywhere

• Bansal, Buchbinder, Naor (2008) Randomized
  competitive algorithms for generalized caching
  (and weighted paging)
• Bansal, Gupta, Krishnaswamy (2010)
  485-approximation algorithm for min-sum set cover
• Bansal Pruhs (2010) O(log log
  nP)-approximation algorithm for general
  1-machine preemptive scheduling O(1) with
  identical deadlines

33
Minimum-Weight Late Jobs on 1 Machine

• Each job j has processing time pj , deadline dj,
  weight wj
• Choose a subset L of jobs of minimum-weight to be
  late - not scheduled to complete by deadline
• This problem is (weakly) NP-hard can be
  solved in O( n ?j pj ) time Lawler Moore,
  (1²)-approximation in O(n3/²) time Sahni
• If there also are release dates that constrain
  when a job may start, no approximation result is
  possible - focus on max-weight set of jobs
  scheduled on time Bar-Noy, Bar-Yehuda,
  Freund, Naor, Schieber - allow preemption
  Bansal Pruhs

34
What if all deadlines are the same?

• Total processing time is ?j pj ! P
• WLOG assume schedule runs through 0,P
• Deadline D ) at least P-D units of processing are
  done after D
• So just select set of total processing at least
  P-D of minimum total weight
• , minimum-cost covering knapsack problem

35
Same Idea for General Deadlines

• Total processing time is ?j pj ? P
• WLOG assume schedule runs through 0,P
• Assume d1 d2 dn
• Deadline di ) among all jobs with deadlines di
  , ? P(i)-di units of processing are done after
  di where S(i) j dj ? di and P(i) ?j ?
  S(i) pj

• Minimize ? wj yj
• subject to ?j ? S(i) pj yj P(i)-di,
  i1,,n yj 0, j1,,n

36
Strengthened LP Knapsack Covers

• Minimize ? wj yj
• subject to ?j ? S(L,i) pj (L,i) yj ? D(L,i),
  for each L,i
• where S(L,i) j dj ? di , j ? L
• D(L,i) max ?j ? S(L,i) pj - di , 0
• pj (L,i) min pj , D(L,i)
• Dual
• Maximize ? D(L,i) v(L,i)
• subject to ?(L,i) j 2 S(L,i) pj(L,i) v(L,i)
  wj for each j
• v(L,i) 0 for each L,i

37
Primal-Dual Summary

• Start with all dual variables set to 0 and
  solution A as the empty set
• Increase variable v(A,i) with largest D(A,i)
  until a dual constraint becomes tight for some
  item i
• Add item i to solution A and repeat
• Stop once solution A is sufficient so remaining
  jobs N-A can be scheduled on time
• Examine each item j in A in reverse order and
  delete j if reduced late set is still feasible
• Call final solution L and set yj 1 for all j 2
  L

38
Highlights of the Analysis

• Lemma. Suppose current iteration increases
  v(L,i), and let L(i) be jobs put in final late
  set L afterwards. Then 9 job k ? L(i) so that
  L-k is not feasible.
• Note in previous case, since all deadlines were
  equal, the last job l added satisfies this
  property.
• Here, the reverse delete process is set exactly
  to ensure that the Lemma holds.

39
Highlights of the Analysis

• Lemma. Suppose current iteration increases
  v(L,i), and let L(i) be jobs put in final late
  set L afterwards. Then 9 job k ? L(i) so that
  L-k is not feasible.
• Lemma. ?j j ? i, j ? L(i) k pj(L,i) lt
  D(L,i) if v(L,i)gt0.

40
Highlights of the Analysis

• Lemma. Suppose current iteration increases
  v(L,i), and let L(i) be jobs put in final late
  set L afterwards. Then 9 job k ? L(i) so that
  L-k is not feasible.
• Lemma. ?j j ? i, j ? L(i) k pj(L,i) lt
  D(L,i) if v(L,i)gt0.
• Fact. pk(L,i) ? D(L,i) (by definition
  of pk(L,i) )

41
Highlights of the Analysis

• Lemma. Suppose current iteration increases
  v(L,i), and let L(i) be jobs put in final late
  set L afterwards. Then 9 job k ? L(i) so that
  L-k is not feasible.
• Lemma. ?j j ? i, j ? L(i) k pj(L,i) lt
  D(L,i) if v(L,i)gt0.
• Fact. pk(L,i) ? D(L,i) (by definition
  of pk(L,i) )
• Corollary. ?j j ? i, j ? L(i) pj(L,i) lt 2D(L,i)
  if v(L,i)gt0.

42
Previous Analysis (flashback)

• We have u((S\ l )\A) lt D u(A) if v(A) gt 0
• Cost of solution is

Dual LP
43
Highlights of the Analysis

• Lemma. Suppose current iteration increases
  v(L,i), and let L(i) be jobs put in final late
  set L afterwards. Then 9 job k ? L(i) so that
  L-k is not feasible.
• Lemma. ?j j ? i, j ? L(i) k pj(L,i) lt
  D(L,i) if v(L,i)gt0.
• Fact. pk(L,i) ? D(L,i) (by definition
  of pk(L,i) )
• Corollary. ?j j ? i, j ? L(i) pj(L,i) lt 2D(L,i)
  if v(L,i)gt0.

44
Highlights of the Analysis

• Corollary ?j j ? i, j ? L(i) pj(L,i) lt 2D(L,i)
  if v(L,i)gt0.
• Same trick here
• ?j ? L wj ?j ? L ?(L,i) j 2 S(L,i) pj(L,i)
  v(L,i)
• ?(L,i) v(L,i) ?j j ? i, j ? L(i)
  pj(L,i)
• ?(L,i) 2D(L,i) v(L,i)
• 2 OPT

45
Primal-Dual Theorem

• For the 1-machine min-weight late jobs scheduling
  problem with a common deadline, the LP relaxation
  with knapsack-cover inequalities can be used to
  derive a (simple) primal-dual 2-approximation
  algorithm.

46
General 1-Machine Min-Cost Scheduling

• Each job j has its own nondecreasing cost
  function fj (Cj ) where Cj denotes completion
  time of job j
• Assume that all processing times are integer
• Goal construct schedule to minimize total cost
  incurred
• LP variables xjt 1 means job j has Cj t
• Knapsack cover constraint for each t and L,
• require that total processing time of jobs
  finishing at time t or later is sufficiently large

47
Primal-Dual Theorem(s)

• For 1-machine min-cost scheduling, LP relaxation
  with knapsack-cover inequalities can be used to
  derive a (simple) primal-dual pseudo-polynomial
  2-approximation algorithm.
• For 1-machine min-cost scheduling, LP relaxation
  with knapsack-cover inequalities can be used to
  derive a (simple) primal-dual (2²)-approximation
  algorithm.

48
Weak LP Relaxation

• Total processing time is ?j pj ? P
• WLOG assume schedule runs through 0,P
• xjt 1 means job j completes at time t

Minimize ? fj(t) xjt subject to ?t ?
1,,P xjt 1, j1,,n ?j ?
1,n ?s ? t,,P pj xjs ? D(t)
t1,,P xjt ? 0 j1,,n
t1,,P where D(t) P-t1.
49
Strong LP Relaxation

• Total processing time is ?j pj ? P
• WLOG assume schedule runs through 0,P
• xjt 1 means job j completes at time t

Minimize ? fj(t) xjt subject to ?t
?1,,P xjt 1, for all j ?j ?
L ?s ?t,,P pj (L,t) xjs D(L,t) for
all L,t xjt 0, for all
j,t where D(t) P-t1, D(L,t) max0,
D(t)-?j ? L pj , and pj(L,t) minpj,
D(L,t).
50
Primal and Dual LP

• Minimize ? fj(t) xjt
• subject to ?t ?1,,P xjt 1,
  for all j
• ?j ? L ?st,,P pj (L,t) xjs D(L,t)
  for all L,t xjt 0,
  for all j,t
• D(t) P-t1 D(L,t) max0, ?j ?
  L pj t1
• pj(L,t) minpj, D(L,t)
• Maximize ?L ?t D(L,t) v(L,t)
• subject to ?L j ? L?t1,,s pj(L,t) v(L,t) ?
  fj(s) for all j,s
• v(L,t) ? 0 for all L,t

51
Primal-Dual Summary

• Start w/ all dual variables set to 0 and each At
  
• Increase variable v(At,t) with largest D(At,t)
  until a dual constraint becomes tight for some
  item i (break ties by selecting latest time)
• Add item i to solution As for all s t and
  repeat
• Stop once solution A is sufficient so remaining
  jobs N-A satisfy all demand constraints
• Focus on pairs (j,t) where t is latest job j is
  in At and perform a reverse delete
• Set dj t for job j by remaining pairs (j,t)
• Schedule in Earliest Due Date order

52
Primal-Dual Theorem

• For 1-machine min-cost scheduling, LP relaxation
  with knapsack-cover inequalities can be used to
  derive a (simple) primal-dual pseudo-polynomial
  2-approximation algorithm.

53
Removing the Pseudo with a (1²) Loss

• This requires only standard techniques
• For each job j, partition the potential job
  completion times 1,,P into blocks so that
  within block the cost for j increases by 1²
• Consider finest partition based on all n jobs
• Now consider variables xjt that assign job j to
  finish in block t of this partition.
• All other details remain basically the same.

Fringe Benefit more general models, such as
possible periods of machine non-availability
54
Primal-Dual Theorem

• For 1-machine min-cost scheduling, LP relaxation
  with knapsack-cover inequalities can be used to
  derive a (simple) primal-dual (2²)-approximation
  algorithm.

55
Some Open Problems

• Give a constant approximation algorithm for
  1-machine min-sum scheduling with release
  dates allowing preemption
• Give a (1²)-approximation algorithm for
  1-machine min-sum scheduling, for arbitrarily
  small ² gt 0
• Give an LP-based constant approximation algorithm
  for capacitated facility location
• Use configuration LP to find an approximation
  algorithm for bin-packing problem that uses at
  most ONE bin more than optimal

56
Thank you!

• Any questions?