ESI 6448 Discrete Optimization Theory

1
ESI 6448Discrete Optimization Theory

• Lecture 28

2
Last class

• Lagrangian relaxation
• application to STSP
• Strength of Lagrangian dual
• 2 viewpoints for Lagrangian relaxation
• wLD maxcx Dx ? d, x ? conv(X)

3
Lagrangian relaxation

• z maxcx Ax ? b, Dx ? d, x ? Zn
• if Dx ? d are complicating constraints
• drop Dx ? d and get relaxation
• provides (weak) bounds
• Not just dropping but incorporating those into
  objective function (Lagrangian relaxation)
• Transform IP z maxcx Dx ? d, x ? X
  intoIP(u) z(u) maxcx u(d Dx) x ? X for
  any value ofu (u1, , um) ? 0

4
Lagrangian dual problem

• Problem IP(u) is a relaxation problem for IP for
  all u ? 0.
• z(u) ? z (upper bound on the optimal value of IP)
• To find the tightest upper bound over u, solve
  the Lagrangian dual problem wLD minz(u) u
  ? 0
• If the constraints are Dx d, wLD min z(u)

5
Optimality

• If u ? 0, and(i) x(u) is an optimal solution,
  and(ii) Dx(u) ? d, and(iii) (Dx(u))i di
  whenever ui gt 0,then x(u) is optimal in IP.

6
Application to UFL

7
Application to STSP

1-tree constraints
8
2 viewpoints for Lagrangian relaxation

• X x ? Zn Ax ? b
• conv(X) x ? Rn Ax ? b
• z maxcx Dx ? d, x ? X
• z(u) maxcx u(d Dx) x ? conv(X)
• 2 viewpoints for z(u, x) cx u(d Dx)
• affine function of x for u fixed
• affine function of u for x fixed
• wLD minz(u) u ? 0

9
Example

x2
X
x1
z(u, xi)
u
10
Strength of Lagrangian dual

• wLD maxcx Dx ? d, x ? conv(X)
• LD can be viewed as the problem of minimizing the
  piecewise linear convex, but nondifferentiable
  function z(u)
• If X x ? Zn Ax ? b andconv(X) x ? Rn
  Ax ? b, thenwLD maxcx Ax ? b, Dx ? d, x
  ? Rn

z(u)
u
11
Solving LD

• Large number of constraints
• constraints (or cutting plane) generation
  approach is required
• use separation problem
• Alternative approach
• use subgradient to solve

12
Gradient

• When f Rm?R is differentiable at u, the
  gradient vector ? (u) is ?f (u) (?f /?u1, , ?f
  /?um).
• The gradient vector is the local direction of
  maximum increase of f (u), and ?f (v) 0 solves
  minf (u) u ? Rm.
• The classical steepest descent method for
  minimizing f (u) is given by the sequence of
  iterations
• method for finding local minimum of a function
  when the gradient of the function can be computed
• With appropriate assumption on the sequence of
  step sizes ? t, the iterates ut converges to
  a minimizing point.

13
Subgradient

• A subgradient at u of a convex function f Rm?R
  is a vector ? (u)?Rm s.t. f (v) ? f (u) ?
  (u)T(v u) for all v?Rm.
• generalization of gradient
• ? (u) can be viewed as the slope of the
  hyperplane that supports the set (v, w)?Rm1 w
  ? f (v) at (v, w) (u, f (u))

f (v)
w f (u) ? (u)T(v u)
(u, f (u))
u
14
Subgradient algorithm

• At each iteration one takes a step from the
  present point uk in the direction opposite to a
  subgradient d Dx(uk).
• The difficulty is in choosing the step lengths
  ?kk1,

15
Step lengths

• (a) guarantees convergence, but slow
• (b) or (c) lead to faster convergence, but have
  difficulties
• (b) needs sufficiently large ?0 and ?
• (c) requires a dual upper bound (unknown).
• use primal lower bound and if not converge,
  increase it

16
Stopping criteria

• Ideally, the subgradient algorithm can be stopped
  when we find a subgradient 0.
• Typically, the stopping rule is either
• to stop after a fixed number of iterations or
• to stop if the function has not increased by at
  least a certain amount within a given number of
  iterations
• Without convergence, we can stop at iteration t
  if
• st 0
• if data are integral, then z(ut) z lt 1
• after a specific number of subgradient iterations
  has occurred, i.e. t ? tmax

17
UFL

18
Example

19
STSP

• wLD max z(u)
• Step direction
• minimizing
• Follow the directions of subgradients.
• dualized constraints are the equality constraints
• Dual variable u is unbounded.
• Step size (using rule (c))

20
Example

• Suppose we found (1, 2, 3, 4, 5, 1) of length 148
  by a heuristic, but we have no lower bounds.
• Update the dual variables using rule (c)
• ?k 1 and use w 148 instead of w.
• ?k (148 z(uk)) / ?i?V (2 ?e??(i) xe(uk))2

30
1
2
24
40
3
5
24
30
4
21
Example (Iteration 1)

• z(u1) 130 2?i u1i 130 ? z
• (2 ?e??(i) xe(u1)) (0, 0, -2, 1, 1)
• ?1 (148 z(u1)) / ?i?V (2 ?e??(i) xe(u1))2
  18 / 6 3
• u2 u1 ?1 (2 ?e??(i) xe(u1)) 3(0, 0,
  -2, 1, 1) (0, 0, -6, 3, 3)

30
1
2
24
26
26
3
5
24
4
22
Example (Iteration 2)

• z(u2) 143 2?i u2i 143 ? z
• (2 ?e??(i) xe(u2)) (0, 0, -1, 0, 1)
• ?2 (148 z(u2)) / ?i?V (2 ?e??(i) xe(u2))2
  5 / 2
• u3 u2 ?2(2 ?e??(i) xe(u2)) (0, 0, -6,
  3, 3) 5/2 (0, 0, -1, 0, 1) (0, 0, -17/2,
  3, 11/2)

30
1
2
30
32
3
5
24
27
4
23
Example (Iteration 3)

• z(u3) 147.5 2?i u3i 147.5 ? z
• z ? 148, hence 148 is the optimal value.

30
1
2
32.5
34.5
29
3
5
21.5
4
24
Today

• Subgradient algorithm
• simple and easy to implement
• difficult to choose step lengths
• theoretic or practical stopping criteria
• example (STSP)