Chapter 5. The Duality Theorem

1
Chapter 5. The Duality Theorem

• Given a LP, we define another LP derived from the
  same data, but with different structure. This LP
  is called the dual problem (????).
• The main purpose to consider dual is to obtain an
  upper bound (estimate) on the optimal objective
  value of the given LP without solving it to
  optimality. Also dual problem provides
  optimality conditions of a solution x for a LP
  and help to understand the behavior of the
  simplex method.
• Very important concept to understand the
  properties of the LP and the simplex method.

2

• Approach to find optimal value of an opt. problem
  (max form)
• Find lower bound and upper bound so
  that for optimal value , we have
• Lower bound usually obtained from a feasible
  solution.
• If x is a feasible solution to LP, cx
  provides a lower bound.
• Upper bound usually obtained by solving a
  relaxation of the problem or by finding a
  feasible solution to a dual problem.
• ex) linear programming relaxation of an integer
  program
• Given IP, max cx, Ax ? b, x ? 0 and integer,
  consider the linear programming relaxation, max
  cx, Ax ? b, x ? 0.
• Let z be the optimal value of IP and z be the
  optimal value of LP relaxation. Then z ? z.
  ( Let x be the optimal solution of IP. Then x
  is a feasible solution of LP relaxation. So there
  may exist a feasible solution to LP which
  provides better objective value)

3

• (continued)
• Dual of the LP problem Let y be a feasible
  solution to the dual problem. Then the dual
  objective value provided by y gives an upper
  bound on the optimal LP objective value. (study
  in Chapter 5.)
• If the lower and upper bound are the same, we
  know that z is optimal value. We may need to
  find the optimal solution additionally, but the
  optimal value is found.
• Although the lower and upper bound may not be the
  same, from the gap ( upper bound lower
  bound), we can estimate the quality of the
  solution we have.

4

• Taking nonnegative linear combination of
  inequality constraints Consider two constraints
  and .
  (1)
• In vector notation
• If we multiply scalars y1 ? 0 to the 1st
  constraint and y2 ? 0 to the 2nd constraint and
  add the l.h.s. and r.h.s. respectively,
• we get
  .. (2)
• In vector notation,
• Any vector that satisfies (1) also satisfies
  (2), but converse is not true.
• Moreover, the coefficient vector in l.h.s. of
  (2) is obtained by taking the nonnegative linear
  combination of the coefficient vectors in (1)

5

x2
2x1x2?4
y1(1, 2)y2(2, 1), y1, y2 ? 0
x12x2?3
(1, 2)
(2, 1)
(5/3, 2/3)
(y1a1y2a2)x ? (3y14y2)
x1
(0, 0)
6

• ex)
• Lower bound consider feasible solution (0, 0,
  1, 0) ? z ? 5
• (3, 0, 2, 0) ? z ? 22
• Upper bound consider inequality obtained by
  multiplying 0 to the 1st , 1 to the 2nd, and 1 to
  the 3rd constraints and add the l.h.s. and r.h.s.
  respectively
• 
  (1)

7

• Since we multiplied nonnegative numbers, any
  vector that satisfies the three constraints
  (including feasible solutions to the LP) also
  satisfies (1). Hence any feasible solution to
  LP, which satisfies the three constraints and
  nonnegativity, also satisfies (1).
• Note that all the points satisfying 4x1 3x2
  6x3 3x4 ? 58 do not necessarily satisfy the
  three constraints in LP.
• Further, any feasible solution to the LP must
  satisfy
• ( ? 58)
• since any feasible solution must satisfy
  nonnegativity constraints on the variables and
  the coefficients in the second expression is
  greater than or equal to the corresponding
  coefficients in the first expression.
• So 58 is an upper bound on the optimal value of
  the LP.

8

• Now, we may use nonnegative weights yi for each
  constraint.
• In vector notation,
• Objective function of the LP is

9

• Hence as long as the nonnegative weights yi
  satisfy
• we can use as an upper
  bound on optimal value.
• To find more accurate upper bound (smallest upper
  bound), we want to solve
• Dual problem obtained. Note that the objective
  value of any feasible solution to the dual
  problem provides an upper bound on the optimal
  value of the given LP.

10

• General form

subject to
(P)
(D)
subject to
11

• Thm (Weak duality relation)
• Suppose (x1, , xn) is a feasible solution to
  the primal problem (P) and (y1, , yn) is a
  feasible solution to the dual problem,
• then
• pf) ?
• Cor If we can find a feasible x to (P) and a
  feasible y to (D) such that , then x
  is an optimal solution to (P) and y is an
  optimal solution to (D).
• pf) For all feasible solution x to (P), we have
• Similarly, for all feasible y to (D), we have
• ?

12

• Strong Duality Theorem If (P) has an optimal
  solution (x1, , xn ), then (D) also has an
  optimal solution, say (y1, , ym ), and
• (i.e., no duality gap, dual optimal value
  primal optimal value 0)
• Note that strong duality theorem says that if
  (P) has an optimal solution, the dual (D) is
  neither unbounded nor infeasible, but always has
  an optimal solution.

13

• Idea of proof Read the optimal solution of the
  dual problem from the coefficients of the slack
  variables in the z-row from the optimal
  dictionary (tableau).
• ex)

Note that the dual variables y1, y2, y3 matches
naturally with slack variables x5, x6, x7. For
example x5 is slack variable for the first
constraint and y1 is dual variable for the first
constraint, and so on.
14

• At optimality, the tableau looks

In the z-row of the dictionary, the coefficients
of the slack variables are 11 for x5, 0 for
x6, -6 for x7. Assigning these values with
reversed signs to the corresponding dual
variables, we obtain desired optimal solution of
the dual y1 11, y2 0, y3 6.
15

• Idea of the proof
• Note that the coefficients of x5, x6, and x7 in
  the z-row show what numbers we multiplied to the
  corresponding equation and add them to the z-row
  in the elementary row operations (net effect of
  many row operations)
• ex)
• Suppose we performed row operations
• (row2) ? 2(row 1) (row 2), then ( z-row) ?
  3(row 2) (z-row).
• The net effect in z-row is adding 6(row 1)
  3(row 2) to the z-row and the scalar we
  multiplied can be read from the coefficients of
  x5 and x6 in the z-row.

16

• (ex-continued)

(row 1)?2 (row 2)
17

• (ex-continued)

(row 2) ? 3 z-row
18

• Example Initial tableau

Optimal tableau
19

• Let yi be the scalar we multiplied to the i-th
  row and add to the z-row in the net effect.
• Then the coefficient of slack variables in the
  z-row represent the yi values we multiplied to
  the i-th row for i 1, ,m.
• Also the coefficients of structural variables in
  the z-row are given as
• Now in the optimal tableau, all the coefficients
  in the z-row are ? 0, which implies
• If we take yi as a dual solution, they are
  dual feasible.

20

• Also the constant term in the z-row gives the
  value
• So it is the negative of the dual objective
  value of the dual solution (-yi ), i 1, ,,, , m
• Note that the constant term in the z-row also
  gives the negative of the objective value of the
  current primal feasible solution.
• So we have found a feasible dual solution which
  gives the same dual objective value as the
  current primal feasible solution.
• From previous Corollary, the dual solution and
  the primal solution are optimal to the dual and
  the primal problem, respectively.
• It is the idea of the proof.

21

• pf of strong duality theorem)
• Suppose we introduce slack variables
• and solve the LP by simplex and obtain optimal
  dictionary with
• Let
• We claim that yi, i 1, , m is an optimal
  dual solution, i.e. it satisfies dual
  constraints and

This equation must be satisfied by any x that
satisfies the dictionary (excluding the
nonnegativity constraints) since the set of
feasible solutions does not change for any
dictionary.
22

• since any feasible solution to dictionary
  should satisfy this.

Now this equation should hold for all feasible
solutions to the dictionary. From the initial
dictionary, we know that any feasible solution to
the dictionary can be obtained by assigning
arbitrary values to x1, , xn and setting xni
bi - ?j1n aijxj , ( i 1, , m). Use these
solutions. Note that, in the above equation, the
variables xni do not appear. So it must hold
for any choices of xj , j 1, , n.
23

• Equality must hold for all choices of x1, , xn.
  
• Hence
• Hence y is dual feasible.
• Also we have that
• Since
• yi, i 1, , m is an optimal dual solution
  and
  
• ?