Combinatorial Optimization

1
Combinatorial Optimization

• Chapter 3
• Duality

2
LP in general form

• min c x
• s.t. aI x bi i e M
• aI x bi i e M
• xj 0 j e N
• xj free j e N
• Introduce surplus variables for inequality
  constraints
• Replace free variables by two nonnegative
  variables.

3
In standard form

• min c x
• s.t. A x b
• x 0
• Where A has extra columns for xs in N, and for
  slack variables,
• x has extra variables for xs in N and for
  slack variables,
• c has extra elements for xs in N.

4
Starting the dual

• The simplex method gives an optimal solution x0
  to the LP in standard form, with a basis
  Q,satisfying
• c - (cB B-1)A 0.
• Thus, p cB B-1 is a feasible solution to
  the problem
• p A c where p e Rm.

5
The dual

• p Aj cj, j e N,
• p Aj cj, j e N, and p Aj - cj, j e N,
  and hence p Aj cj, j e N.
• - pi 0, i e M, or pi 0, i e M.
• Objective max p b.

6
Definition of the Dual

• Definition 1.3 Given an LP in general form,
  called the primal, the dual is defined as follows

7
Strong duality

• Theorem 3.1 If an LP has an optimal solution, so
  does its dual, and at optimality their costs are
  equal.
• Proof. Let x and p be optimal solutions to the
  primal and the dual resp., then
• cx p Ax p b. ()

From the primal
From the dual
8
Strong duality proof

• Thus the primal has cost at least as high as the
  dual. Then, if the primal has a feasible
  solution, the cost of the dual cannot be
  unbounded.
• It has feasible solution p , and since it is
  not unbounded, it must have an optimum. The cost
  of p is
• p b cB B-1 b cB x0
• which is the optimal solution of the primal.
• Together with () this suffices to prove the
  Theorem.

9
The dual of the dual

• Theorem 3.2. The dual of the dual is the primal.
• Proof. Left as an exercise.
• Theorem 3.3 Given a primal-dual pair, either of
  the following holds
• Both have a finite optimum
• Both are infeasible
• One is unbounded the other is infeasible
• Proof. Skip.

10
Complementary slackness

• Theorem 3.4 A pair, x, p, respectively feasible
  in a primal-dual pair is optimal if and only if
• ui pi (aix - bi) 0 for all i (1)
• vj (cj - pj Aj) xj 0 for all j. (2)

11
Complementary slackness (cont.)

• Proof. Since pi 0 if aix bi, and aix bi
  otherwise, ui 0 for all i. Likewise, vj 0 for
  all j. Thus u0 if and only if (1) holds and
  v0, if and only if (2) holds.
• Now, notice that uv cx pb. Thus, if (1)
  and (2) hold, uv0, and cx pb.
• This implies that x and p have the same objective
  value, and hence are both optimal.

12
Complementary slackness

• Consequences
• If an inequality constraint in the dual is not
  binding, then the corresponding primal variable
  must have value zero and vice versa. Likewise, if
  a nonnegative variable has strictly positive
  value, then the corresponding constraint must be
  binding.

13
Farkas Lemma

• Skip 3.3

14
The shortest path problem and its dual

• Definition 3.3 Given a directed graph G(V,E)
  and a nonnegative weight cj 0 associated with
  each arc ej e E, an instance of the shortest path
  problem (SP) is the problem of finding a directed
  path from a distinguished source node s to a
  distinguished terminal node t, with the minimum
  total weight.

15
Shorest path problem

• Feasible set
• F sequences P (ej1,..,ejk) P is a directd
  path from s to t in G .
• Cost c(P) Si1k ejI .
• Formulation as LP uses node-arc incidence matrix
  A
• 1 if arc j leaves node i
• aij -1 -f arc j enters node i
• 0 otherwise

16
SP Example
a
e1
3
1
e4
2
s
t
e3
e5
2
1
e2
b
17
LP formulation

• Min e1 2e2 2e3 3e4 e5
• s.t.
• e1 e2 -1
• - e4 - e5 1
• - e1 e3 e4 0
• - e2 - e3 e5 0
• 0 ei 1, i 15

18
Redundancy

• Claim Any solution that satisfies 3 of the four
  constraints, satisfies all 4 of them.
• Easily checked.
• This we omit the constrant for the sink t.
• Next we consider the tableau of a feasible
  solution

19
Tableau
20
Tableau of feasible bfs
Add row 1 to row 2
Basis e1, e4, e5 e5 has value 0, this bfs is
degenerate
21
Pivotting
Bring e2 into the basis
Basis e1, e2, e5 e1 has value 0, this bfs is
degenerate
22
The dual of SP

• Max ps pt
• s.t.
• pA c.
• p free.
• In case of SP the constraints of the dual are
  pi pj cij.

23
Complementary slackness for SP

• A path f as defined by primal variables, and an
  assignment of dual variables p are jointly
  optimal if and only if
• A primal variable with positive value corresponds
  to a dual constraint that is satisfied at
  equality.
• A dual constraint not satisfied at equality
  corresponds to a primal variable with value zero.

24
Graphical representation
a
3
e1
3
1
e4
2
t
s
0
e3
e5
3
2
1
e2
b
1
Compare the dual variables with the Dijkstra
algorithm labels when searching a path from t to
s!
25
Rest of Chapter

• You can skip 3.5, 3.6, 3.7.

26
Exercises

• Slide 9
• 3
• 11
• 12
• 13
• 17