TSP: an introduction

1
TSP an introduction

• Vittorio Maniezzo
• University of Bologna

2
Travelling Salesman Problem (TSP)

• n locations given (home of salesperson and n-1
  customers).
• dij distance matrix given, contains distances
  (Km, time, ) between each pair of locations.
• The TSP asks for the shortest tour starting home,
  visiting each customer once and returning home
• Symmetric / asymmetric versions.

3
Introduction

• Which is the shortest tour?
• Difficult to say

4
TSP history

• The general form of the TSP appears to have been
  first studied by mathematicians starting in the
  1930s by Karl Menger in Vienna.
• Mathematical problems related to the traveling
  salesman problem were treated in the 1800s by the
  Irish mathematician Sir William Rowan Hamilton.
• The picture shows Hamilton's Icosian Game that
  requires players to complete tours through the 20
  points using only the specified connections.
  (http//www.math.princeton.edu/tsp/index.html)

5
Past practice
6
Current practice

• Actual tours of persons or vehicles (deliveries
  to customers)
• Given Graph of streets, where some nodes
  represent customers.
• Find
• shortest tour among customers.
• Precondition
• Solve shortest path problems ? Distance matrix

7
Application PCB

• Minimization of time for drilling holes in PCBs
  (printed circuit boards)

8
Application setup costs

• Minimization of total setup costs (or setup
  times) when setup costs are sequence dependent
• All product types must be produced and the setup
  must return to initial configuration

i \ j 1 2 3 4 5
1 0 18 17 20 21
2 18 0 21 16 18
3 16 98 0 19 20
4 17 16 20 0 17
5 18 18 19 16 0
9
Application paint shop

• Minimization of total setup costs (or setup
  times) when setup costs are sequence dependent
• All colors must be processed

i \ j White Pink Red Blue
White 0 5 7 7
Pink 10 0 2 5
Red 15 7 0 5
Blue 15 10 5 0
10
Solution Methods

• n nodes ? (n-1)! possible tours
• Some possible solution method
• Complete enumeration
• IP Branch and Bound (and Cut, and Price, )
• Dynamic programming
• Approximation algorithms
• Heuristics
• Metaheuristics

n 5 10 15
20 n! 120 3,628.800 1,311012 2,431018
11
Standard Example

• Example with 5 customers (1 home, 4 customers)
• Zimmermann, W., O.R., Oldenbourg, 1990
• e.g. cost of 1-2-3-4-5 1821191718 93
• First simplify by reducing costs

12
Lower bound

• Subtract minimum from each row column
• Reduction constant 1716161616 1 82
• all tours have costs which are 82 greater,
• e.g. cost of 1-2-3-4-5 15302 11 93 -
  82
• Optimal tour does not change (see Toth, Vigo,
  02).

13
Complete Enumeration

• 5 customers ? 4! 2?3?4 24 tours

14
A wider scope

• A combinatorial optimization problem is defined
  over a set C c1 , , cn of basic components.
  
• A solution of the problem is a subset S ? C
• F ? 2C is the subset of feasible solutions, (a
  solution S is feasible iff S?F).
• z 2C ? ? is the cost function,
• the objective is to find a minimum cost feasible
  solution S, i.e., to find S? F such that z(S)
  ? z(S), ?S ? F.
• Failing this, the algorithm anyway returns the
  best feasible solution found, S ? F.

15
TSP as a CO example

• TSP is defined over a weighted undirected
  complete graph G(V,E,D), where V is the set of
  vertices, E is the set of edges and D is the set
  of edge weights.
• The component set corresponds to E (CE),
• F corresponds to the set of Hamiltonian cycles
  in G
• z(S) is the sum of the weights associated to
  the edges belonging to the solution S.

16
LP-Formulation
LP formulation is a binary LP (IP) which is
almost identical to the assignment
problem plus conditions to avoid short
cycles
17
Example for short cycles

• Optimal solution of LP without conditions to
  avoid short cycles solution of assignment
  problem
• short cycles 1-3-1 and 2-5-4-2
• ? conditions to avoid short cycles are needed

18
Conditions to avoid short cycles

• Several formulations in the literature
• e.g. Dantzig-Fulkerson-Johnson
• exponentially many!

19
Example avoid short cycles

• Above example with n 5 cities
• must consider
  subsets Q
• 1, 2 1, 3 1, 4 1, 5 2, 3 2, 4
  2, 4 3, 4 3, 5 4, 5
• constraint for subset Q 1, 2 and V-Q 3, 4,
  5 x13 x14 x15 x23 x24 x25 ? 1
• n 6 ?
  subsets (constraints)
• exponentially many!

20
Heuristics for the TSP

• Vittorio Maniezzo
• University of Bologna

21
Computational issues

• The size of the instances met in real world
  applications rules out the possibility of solving
  them to optimality in an acceptable time.
• Nevertheless, these instances must be solved.
• Thus the need to look for suboptimal solutions,
  provided that they are of acceptable quality
  and that they can be found in acceptable time.

22
Heuristic algorithms

• How to cope with NP-completeness
• small instances
• polynomial special cases
• approximation algorithms guarantee to find a
  solution within a known gap from optimality
• probabilistic algorithms guarantee that for
  instances big enough, the probability of getting
  a bad solution is very small
• heuristic algorithms no guarantee, but
  historically, on the average, these algorithms
  have the best quality/time trade off for the
  problem of interest.

23
heuristics focus on solution structure

• Simplest heuristics exploit structural properties
  of feasible solutions in order to quickly come to
  a good one.
• They belong to one of two main classes
  constructive heuristics or local search
  heuristics.

24
Constructive heuristics

• 1. Sort the components in C by increasing costs.
• 2. Set S? and i1.
• 3. Repeat
• If (S ? ci is a partial feasible solution)
  then S S?ci.
• ii1.
• Until S? F.

25
Constructive heuristics

• A constructive approach can yield optimal
  solutions for certain problems, eg. the MST.
• In other cases it could be unable to construct a
  feasible solution.
• TSP order all edges by increasing edge cost,
  take the least cost one and add to it increasing
  cost edges, provided they do not close subtours,
  until a Hamiltonian circuit is completed.
• More involved constructive strategies give rise
  to well-known TSP heuristics, like the Farthest
  Insertion, the Nearest Neighbor, the Best
  Insertion or the Sweep algorithms.

26
Nearest Neighbor Heuristic

• 0. Start from any city e.g. the first one
• 1. Find nearest neighbor (to the last city) not
  already visited
• 2. repeat 1 until all cities are visited.
• Then connect last city with starting point
• Ties can be broken arbitrarily.
• Starting from different starting points gives
  different solutions! Some will be bad some will
  be good.
• GREEDY ? the last connections tend to be long and
  improvement heuristics should be used afterwards

27
Best Insertion Heuristic

• 0. Select 2 cities A, B and start with short
  cycle A-B-A
• 1. Insert next city (not yet inserted) in the
  best position in the short cycle
• 2. Repeat 1. until all cities are visited
• Using different starting cycles and different
  choices of the next city different solutions are
  obtained
• Often when symmetric starting cycle 2 cities
  with maximum distancenext city maximizes min
  distance to inserted cities

28
An approximation algorithm

• In some cases, it is possible to construct
  solutions which are guaranteed to be not too
  bad.
• Approximation algorithms provide a guarantee on
  worst possible value for the ratio (zh z)/z.
• TSP hypothesis it is always cheaper to go
  straight from a node u to another node v instead
  of traversing an intermediate node w (triangular
  inequality).

29
Approx-TSP

• An approx. Algorithm for the TSP with triangular
  inequality is
• Approx-TSP-Tour(G,w)
• 1 select a root vertex r?V
• 2 construct a MST T for G rooted in r
• 3 let L be the list of vertices visited in a
  preorder tree walk of T
• 4 return the hamiltonian cycle H which visits
  all vertices in the order L

30
Approx-TSP example
a
d
e
b
f
g
c
h
31
Approx-TSP example
a
d
e
b
f
g
c
h
32
Approx-TSP example
a
d
e
b
f
g
c
h
33
Approx-TSP example
a
d
e
b
f
g
c
h
34
Approx-TSP performance

• Theorem
• Approx-TSP-Tour is an approximation algorithm
  with a ratio bound of 2 for the traveling
  salesman problem with triangle inequality.
• Proof
• Because the optimal tour solution must have one
  more edge than the MST of T, so c(MST) ? c(T).
• c(Tour) 2c(MST)
• Because of the assumption of triangle inequality,
  c(H) ? c(Tour).
• Therefore, c(H) ? 2c(T).
• Since the input is a complete graph, so the
  implementation of Prims algorithm runs in O(V2).
  Therefore the total time complexity of
• Approx-TSP-Tour also is O(V2).

35
Local Search neighborhoods

• The neighborhood of a solution S, N(S), is a
  subset of 2C defined by a neighborhood function
  N 2C ? 22c.
• Often only feasible solutions considered, thus
  neighborhood functions N F ? 2F.
• The specific function used has a deep impact on
  the algorithm performance and its choice is left
  to the algorithm designer.

36
Local search

• 1. Generate an initial feasible solution S.
• 2. Find S'?N(S), such that z(S')min z(S), ?S?
  N(S).
• 3. If z(S') lt z(S) then SS'
• go to step 2.
• 4. S S.
• The update of a solution in step 3 is called a
  move from S to S'.
• It could be made to the first improving solution
  found.

37
Local search

• There are problems for which a local search
  approach guarantees to find an optimal solution
  (ex. the simplex algorithm).
• For TSP, two LS are 2-opt and 3-opt, which take a
  solution (a list of n vertices) and exhaustively
  swap the elements of all pairs or triplets of
  vertices in it.
• More sophisticated neighborhoods give rise to
  more effective heuristics, among which Lin and
  Kernighan LK73.

38
Local search heuristics

• 2 opt (arcs) try all inversions of some
  subsequence
• if improvement ? start again from beginning

• 3 opt (arcs) try all shifts of some
  subsequence to different positions
• if improvement ?
• start again from
• beginning

3-opt (nodes) swap every triplet of nodes in the
permutation representing the solution. if
improvement ? start again from beginning
2-opt (nodes) swap every pair of nodes in the
permutation representing the solution. if
improvement ? start again from beginning
39
Metaheuristcs for the TSP(some of them)

• Vittorio Maniezzo
• University of Bologna, Italy

40
Metaheuristics focus on heuristic guidance

• Simple heuristics can perform very well, but can
  get trapped in poor quality solutions (local
  optima).
• New approaches have been presented starting from
  the mid '70ies.
• They are algorithms which manage the trade-off
  between search diversification, when search is
  going on in bad regions of the search space, and
  intensification, aimed at finding the best
  solutions within the region being analyzed.
• These algorithms have been named metaheuristics.

41
Metaheuristics

• Metaheuristics include
• Simulated Annealing
• Tabu Search
• GRASP
• Genetic Algorithms
• Variable Neighborhood Search
• ACO
• Particle Swarm Optimization (PSO)

42
Simulated Annealing

• Simulated Annealing (SA) AK89 modifies local
  search in order to accept, in probability,
  worsening moves.
• 1. Generate an initial feasible solution S,
• initialize S S and temperature parameter T.
• 2. Generate S?N(S).
• 3. If z(S') lt z(S) then SS', if (z(S) gt z(S))
  S S
• else accept to set SS' with probability p
  e-(z(S')-z(S))/kT.
• 4. If (annealing condition) decrease T.
• 5. If not(end condition) go to step 2.

43
SA example ulysses 16

• Simulated annealing trace

44
Tabu search

• Tabu Search (TS) GL97 escapes from local minima
  by moving onto the best solution of the
  neighborhood at each iteration, even though it is
  worse than the current one.
• A memory structure called tabu list, TL, forbids
  to return to already explored solutions.
• 1. Generate an initial feasible solution S,
• set S S and initialize TL?.
• 2. Find S' ?N(S), such that
• z(S')min z(S), ?S? N(S), S? TL.
• 3. SS', TLTL ? S, if (z(S) gt z(S)) set S
  S.
• 4. If not(end condition) go to step 2.

45
TS example ulysses 16

• Tabu Search trace

46
GRASP

• GRASP (Greedy Randomized Adaptive Search
  Procedure) FR95 restarts search from another
  promising region of the search space as soon as a
  local optimum is reached.
• GRASP consists in a multistart approach with a
  suggestion on how to construct the initial
  solutions.
• 1. Build a solution S (S in the first
  iteration) by a constructive greedy randomized
  procedure based on candidate lists.
• 2. Apply a local search procedure starting from S
  and producing S'.
• 3. If z(S')ltz(S) then SS'.
• 4. If not(end condition) go to step 2.

47
Genetic algorithms

• Genetic Algorithms (GA) G89 do not rely on
  construction or local search but on the parallel
  update of a set R of solutions.
• This is achieved by recombining subsets of 2
  elements in R, the parent sets, to obtain new
  solutions.
• To be effective on CO problems they are often
  hybridized with SA, TS or local search.

48
Genetic algorithms

• 1. Initialize a set R of solutions.
• 2. Construct a set Rc of solutions by recombining
  pairs of randomly chosen elements in R.
• 3. Construct a set Rm of solutions by randomly
  modifying elements in Rc.
• 4. Construct the new set R by extracting elements
  from Rm, following a montecarlo strategy with
  repetitions.
• 5. If not(end condition) go to step 2.

49
Components of a GA

• A problem to solve, and ...
• Encoding technique (gene, chromosome)
• Initialization procedure (creation)
• Evaluation function (fitness)
• Selection of parents (reproduction)
• Genetic operators (mutation, recombination)
• Parameter settings (practice and art)

50
GA jargon

• Procedure GA
• initialize population
• evaluate population
• while not(End_condition)
• select parents for reproduction
• perform recombination and mutation
• evaluate population

51
Populations

• Solutions are called chromosomes.
• Chromosomes could be
• Bit strings (0101 ... 1100)
• Real numbers (43.2 -33.1 ... 0.0 89.2)
• Permutations of element (E11 E3 E7 ... E1 E15)
• Lists of rules (R1 R2 R3 ... R22 R23)
• Program elements (genetic programming)
• ... any data structure ...

52
Reproduction
Parents are selected at random with selection
chances biased in relation to chromosome
evaluations. Only parents will reproduce.
Selection is with repetition. Every pair of
parents will generate two children.
53
Chromosome Modification

• Modifications are stochastically triggered
• Operator types are
• Mutation
• Crossover (recombination)

54
Crossover Recombination

• P1 (0 1 1 0 1 0 0 0) (0 1 0 1 1 0 0 0)
  C1
• P2 (1 1 0 1 1 0 1 0) (1 1 1 0 1 0 1 0)
  C2
• One (or two) cutting points are randomly chosen.
• The code between the points is swapped in the two
  parents.

55
Mutation Local Modification
Before (1 0 1 1 0 1 1 0) After (0
1 1 0 0 1 1 0) Before (1.38 -69.4
326.44 0.1) After (1.38 -67.5 326.44
0.1)

• Causes movement in the search space.

56
Evaluation

• The evaluator decodes a chromosome and assigns it
  a fitness measure.
• The fitness measures how good a solution is it
  is directly related to the objective function in
  maximization problems and inversely in
  minimization problems.
• The evaluator is the only link between a GA and
  the problem it is solving.

57
An Abstract Example
Fitness
Distribution of individuals in generation 0
Fitness
Distribution of individuals in generation n
58
TSP Representation

• Representation is an ordered list of city
• numbers known as an order-based GA.
• 1) Amsterdam 2) Berlin 3) Bologna 4)
  London
• 5) Madrid 6) Paris 7) Prague
  8) Vienna
• CityList1 (3 5 7 2 1 6 4 8)
• CityList2 (2 5 7 6 8 1 3 4)

59
Crossover

• Crossover can be defined in different ways.
• One example
• Parent1 (3 5 7 2 1 6 4 8)
• Parent2 (2 5 7 6 8 1 3 4)
• Child (8 5 7 2 1 6 3 4)
• This operator is called the Order1 crossover.

60
Mutation

• Mutation can be implemented as a 2-swap
• Before (5 8 7 2 1 6 3 4)
• After (5 8 6 2 1 7 3 4)

61
TSP Example 30 Cities
62
Solutioni (Distance 941)
63
Solutionj (Distance 800)
64
Solutionn (Distance 652)
65
Best Solution (Distance 420)