ESI 6448 Discrete Optimization Theory

1
ESI 6448Discrete Optimization Theory

• Lecture 34

2
Last class

• Improving local search
• Tabu search
• Simulated annealing
• Genetic algorithms
• Worst-case analysis of heuristics
• Greedy heuristic for IKP
• Tree(/matching) heuristics for STSP
• based on triangle inequality

3
Improving local search heuristics

• Tabu search
• move to the best solution in the neighborhood
  (worse solution can be selected)
• cycling may occur ? maintain tabu list
• Simulated annealing
• choose a neighbor randomly, but based on
  probability related to the value of solutions
• in the long run, should converge to a good local
  minimum
• Genetic algorithms
• evolve the population (set of solutions) from
  one generation to the next generation
• mimic biological operations (crossover, mutation)

4
Tabu search

• 1. Initialize an empty tabu list.
• 2. Get an initial solution S.
• 3. While the stopping criterion is not
  satisfied3.1. Choose a subset Q(S) ? Q(S) of
  non-tabu solutions.3.2. Let S arg min f (T)
  T ? Q(S).3.3. Replace S by S and update
  the tabu list.
• 4. On termination, the best solution found is the
  heuristic solution.

5
Simulated annealing

• 1. Get an initial solution S.
• 2. Get an initial temperature T and a reduction
  factor r with 0 lt r lt 1.
• 3. While not yet frozen, do the following3.1.
  Perform the following loop L times 3.1.1
  Pick a random neighbor S of S. 3.1.2
  Let ? f (S) f (S). 3.1.3 If ? ? 0,
  set S S. 3.1.4 If ? gt 0, set S S
  with probability e-?/T.3.2 Set T ? rT. (reduce
  the temperature)
• 4. Return the best solution found.

6
Concepts for local search

• Communication
• neighborhood structure
• possible to get from any solution S to any other
  solution S in a small number of moves
• Diversification
• facilitating movement between very different
  areas of the search space
• high initial temperature in SA, long tabu list in
  TS, using random restarts
• Intensification
• increasing the search effort in promising areas
  of the search space
• choosing optimally in the neighborhood, enlarging
  Q(S) temporarily

7
Genetic algorithms

• For each iteration (generation),
• Evaluation evaluate the fitness of the
  individuals.
• Parent Selection select certain pairs of
  solutions (parents) based on their fitness.
• Crossover combine each pair of parents to
  produce one or two new solutions (offspring).
• Mutation modify some of the offspring randomly.
• Population Selection Based on the fitness, a
  new population is selected replacing some or all
  of the original population by an identical number
  of offspring.

8
Worst-case analysis of heuristics

• Integer knapsack problem (IKP)
• STSP
• triangle inequality
• if ei ? E for i 1, 2, 3 are three sides of a
  triangle, then cei cej ? cek for i ? j ? k, i,
  j, k ? 1, 2, 3

i2
i1
j 1
j
i
ci,j ? ci,i1 cj-1,j
9
Worst-case analysis of heuristics

• STSP (contd)
• Eulerian graph a graph where the degree of each
  node is even
• For a connected Eulerian graph and an arbitrary
  node v, it is possible to construct a walk
  starting and ending at v in which each edge is
  traversed exactly once.
• Given a complete graph H on node set V with edge
  lengths satisfying the triangle inequality, let G
  (V, E) be a connected Eulerian subgraph of H.
  Then the original graph contains a Hamiltonian
  (STSP) tour of length at most ?e?M ce.
• conversion of the walk into a tour by selecting
  distinct nodes plus an edge between the last node
  and the first node

10
Tree heuristic for STSP

• In the complete graph, find a minimum-length
  spanning tree with edges ET and length zT ?e?ET
  ce.
• Double each edge of ET to form a connected
  Eulerian graph.
• Convert the Eulerian graph into a tour of length
  zH.
• The tree algorithm
• zH ? 2z

11
Tree/matching heuristic for STSP

• In the complete graph, find a minimum-length
  spanning tree with edges ET and length zT ?e?ET
  ce.
• Let V be the set of nodes of odd degree in (V,
  ET). Find a perfect matching M of minimum length
  zM in the complete subgraph containing nodes in
  V only.(V, ET ? M) is a connected Eulerian
  graph.
• Convert the Eulerian graph into a tour of length
  zC.
• Christofides algorithm
• zC ? 3/2 z

12
MIP-based heuristics

• Dive-and-fix heuristic
• take the LP sol at any node of the
  branch-and-bound tree and dive down to find a
  feasible sol
• Relax-and-fix heuristic
• relax less important variables and fix more
  important variables first
• Cut-and-fix heuristic
• using an effective strong cutting plane
  algorithm, find some integer variables to have
  values close to integer

13
Dive-and-fix

• For a mixed 0-1 problem,

14
Relax-and-fix

• Consider the problem
• Relax
• Fix
• Heuristic sol

15
Cut-and-fix

• Consider the problem
• Cut
• Fix (or bound)
• Heuristic sol

16
Today

• Improving local search
• Tabu search
• Simulate annealing
• Genetic algorithms
• Worst-case analysis of heuristics
• Greedy heuristic for IKP
• Tree(/matching) heuristics for STSP
• based on triangle inequality