Simulated Annealing

1
Simulated Annealing Boltzmann Machines

• ???

??????? ?????????
2
Content

• Overview
• Simulated Annealing
• Deterministic Annealing
• Boltzmann Machines

3
Simulated Annealing Boltzmann Machines

• Overview

??????? ?????????
4
Hill Climbing
E
E cost (energy)
5
The Problem with Hill Climbing

• Gets stuck at local minima
• Gradient decent approach
• Hopfield neural networks
• Possible solutions
• Try different initial states
• Increase the size of the neighborhood
• (e.g. in TSP try 3-opt rather than 2-opt)

6
Stochastic Approaches
Goal escape from local-minima.

• Stochastic optimization refers to the
  minimization (or maximization) of a function in
  the presence of randomness in the optimization
  process.
• The randomness may be present as either noise in
  measurements or Monte Carlo randomness in the
  search procedure, or both.

7
Two Important Methods

• Simulated Annealing (SA)
• Motivated by the physical annealing process
• Evolution from a single solution
• Genetic Algorithms (GA)
• Motivated by the evolution process of biology
• Evolution from multiple solutions

8
Two Important Methods

• Simulated Annealing (SA)
• Motivated by the physical annealing process
• Evolution from a single solution
• Genetic Algorithms (GA)
• Motivated by the evolution process of biology
• Evolution from multiple solutions

Kirkpatrick, S , Gelatt, C.D., Vecchi, M.P. 1983.
Optimization by Simulated Annealing. Science,
vol 220, No. 4598, pp 671-680.
J. Holland, Adaptation in Natural and Artificial
Systems, University of Michigan Press, 1975.
9
Simulated Annealing Boltzmann Machines

• Simulated Annealing

??????? ?????????
10
Global Optimization
11
Statistical Mechanics in a Nutshell
T

• Statistical mechanics is the study of the
  behavior of very large systems of interacting
  components in thermal equilibrium at a
  temperature, say T.

12
Boltzmann Factor
T
kB Boltzmann constant
Z(T) Boltzmann partition function
13
Boltzmann Factor

• Raising temperature
• the system becomes more active
• the average energy becomes higher

T
T1 lt T2 lt T3
E
14
SimulationMetropolis Acceptance Criterion
E
E cost (energy)
15
SimulationMetropolis Acceptance Criterion
T1 lt T2 lt T3
16
Simulated Annealing Algorithm

• Create initial solution S
• Initialize temperature T
• repeat
• for k 1 to iteration-length do
• Generate a random transition from S to S
• Let ?E E(S) ? E(S)
• if ?E lt 0 then S S
• else if exp??E/T gt rand(0,1) then S S
• Reduce temperature T
• until no change in E(S)
• Return S

17
Simulated Annealing Algorithm
Hill Climbing

• Create initial solution S
• Initialize temperature T
• repeat
• for k 1 to iteration-length do
• Generate a random transition from S to S
• Let ?E E(S) ? E(S)
• if ?E lt 0 then S S
• else if exp??E/T gt rand(0,1) then S S
• Reduce temperature T
• until no change in E(S)
• Return S

18
Main Components of SA

• Solution representation
• Appropriate for computing energy (cost)
• Transition mechanism between solutions
• Incremental changes of solutions
• Cooling schedule
• Initial system temperature
• Temperature decrement function
• Number of iterations between temperature change
• Acceptance criteria
• Stop criteria

19
Example
Given n-city locations specified in a
two-dimensional space, find the minimum tour
length. The salesman must visit each and every
city only once and should return to the starting
city forming a closed path.
Traveling Salesman Problem
20
Example
Traveling Salesman Problem
21
Example
Traveling Salesman Problem
22
Example
Traveling Salesman Problem
23
Example
Traveling Salesman Problem
24
Example
Traveling Salesman Problem
25
Example
Traveling Salesman Problem
26
Example
Traveling Salesman Problem
27
Solution Representation (TSP)
Assume cities are fully connected with symmetric
distance.
28
Solution Representation (TSP)
1
2
3
4
6
5
7
9
11
8
10
gt
29
Energy (Cost) Computation (TSP)
d10,1
1
2
3
4
6
5
7
9
11
8
10
d23
d12
d12
d23
d34
d46
d65
d57
d79
d9.11
d11,8
d8,10
gt
d34
d46
d10,1
d9,11
d65
d57
d11,8
d79
d8,10
30
State Transition (TSP)
1. Randomly select two edges
31
State Transition (TSP)
1
2
3
4
5
6
7
8
9
10
d12
d23
d45
d56
d67
d78
d9,10
1. Randomly select two edges
2. Swap the path
32
State Transition (TSP)
1
2
3
4
5
6
7
8
9
10
8
7
6
5
4
d12
d23
d9,10
1
2
10
1. Randomly select two edges
3
9
2. Swap the path
4
8
7
5
6
33
Cooling Schedules
Geometric Schedule
Empirical evidence shows that typically 0.8 ? ? ?
0.99 yields successful applications (fairly slow
cooling schedules).
34
Simulation
100 cities are randomly chosen from 10?10 square.
100-city TSP
35
Simulation
100 cities are randomly chosen from 10?10 square.
100-city TSP
1000?N iterations are made for each test.
Total Length Direct Search Simulated Annealing (T0 is starting temperature) Simulated Annealing (T0 is starting temperature) Simulated Annealing (T0 is starting temperature) Simulated Annealing (T0 is starting temperature) Simulated Annealing (T0 is starting temperature) Simulated Annealing (T0 is starting temperature) Simulated Annealing (T0 is starting temperature)
Total Length Direct Search T01 T02 T05 T010 T025 T050 T0100
Test1 89.211 82.757 82.732 79.113 81.792 82.701 79.405 79.528
Test2 89.755 81.325 81.334 80.532 83.166 80.461 79.25 82.549
Test3 81.44 82.063 84.296 80.629 81.658 80.35 79.21 79.933
Test4 85.038 82.449 82.388 80.996 79.764 82.688 82.131 84.17
Test5 87.256 80.658 80.814 82.261 80.82 81.338 80.406 79.869
Test6 88.989 82.92 81.284 82.607 80.599 83.526 81.43 80.894
Test7 85.895 84.479 80.807 80.402 80.837 79.504 81.052 82.201
Test8 85.654 80.549 81.251 80.195 82.878 80.169 80.604 80.125
Test9 88.246 80.461 79.832 79.123 80.617 81.676 80.741 81.306
Test10 84.586 82.446 82.855 81.249 82.885 79.68 80.532 81.665
Average 86.607 82.011 81.759 80.711 81.501 81.21 80.476 81.224
Each temperature T is hold for 100?N
reconfigurations or 10?N successful
reconfigurations, whichever comes first. T is
reduced by 10 each time.
36
Simulation
100 cities are randomly chosen from 10?10 square.
100-city TSP
Total Length Direct Search Simulated Annealing (T0 is starting temperature) Simulated Annealing (T0 is starting temperature) Simulated Annealing (T0 is starting temperature) Simulated Annealing (T0 is starting temperature) Simulated Annealing (T0 is starting temperature) Simulated Annealing (T0 is starting temperature) Simulated Annealing (T0 is starting temperature)
Total Length Direct Search T01 T02 T05 T010 T025 T050 T0100
Test1 89.211 82.757 82.732 79.113 81.792 82.701 79.405 79.528
Test2 89.755 81.325 81.334 80.532 83.166 80.461 79.25 82.549
Test3 81.44 82.063 84.296 80.629 81.658 80.35 79.21 79.933
Test4 85.038 82.449 82.388 80.996 79.764 82.688 82.131 84.17
Test5 87.256 80.658 80.814 82.261 80.82 81.338 80.406 79.869
Test6 88.989 82.92 81.284 82.607 80.599 83.526 81.43 80.894
Test7 85.895 84.479 80.807 80.402 80.837 79.504 81.052 82.201
Test8 85.654 80.549 81.251 80.195 82.878 80.169 80.604 80.125
Test9 88.246 80.461 79.832 79.123 80.617 81.676 80.741 81.306
Test10 84.586 82.446 82.855 81.249 82.885 79.68 80.532 81.665
Average 86.607 82.011 81.759 80.711 81.501 81.21 80.476 81.224
37
Simulated Annealing Boltzmann Machines

• Deterministic Annealing

??????? ?????????
38
The Problems of SA

• SA techniques are inherently slow because of
  their randomized local search strategy.
• Converge to global optimum in probability one
  sense only if the cooling schedule is in the
  order of

39
The Problems of SA
Geman, S. Geman, D. (1984) Stochastic
relaxation, Gibbs distributions and the Bayesian
restoration of images, IEEE Trans. on Pattern
Analysis and Machine Intelligence 6, 721-741.

• SA techniques are inherently slow because of
  their randomized local search strategy.
• Converge to global optimum in probability one
  sense only if the cooling schedule is in the
  order of

Geman and Geman 1984
40
Review Simulated Annealing Algorithm

• Create initial solution S?0, 1n
• Initialize temperature T
• repeat
• for k 1 to iteration-length do
• Generate a random transition from S to S by
  inverting a random bit si
• Let ?E E(S) ? E(S)
• if ?E lt 0 then S S
• else if exp??E/T gt rand(0,1) then S S
• Reduce temperature T
• until no change in E(S)
• Return S

41
Review Simulated Annealing Algorithm

• Create initial solution S?0, 1n
• Initialize temperature T
• repeat
• for k 1 to iteration-length do
• Generate a random transition from S to S by
  inverting a random bit si
• Let ?E E(S) ? E(S)
• if ?E lt 0 then S S
• else if exp??E/T gt rand(0,1) then S S
• Reduce temperature T
• until no change in E(S)
• Return S

Stochastic nature
42
Deterministic Annealing (DA)
Also called mean-field annealing.

• Create initial solution S?0, 1n
• Initialize temperature T
• repeat
• for k 1 to iteration-length do
• Choose a random bit si
• Reduce temperature T
• until convergence criterion met
• Return S

43
Simulated Annealing Boltzmann Machines

• Boltzmann Machine

??????? ?????????
44
Boltzmann Machines
Discrete Hopfield NN
Boltzmann Machine

Simulated Annealing
45
Update Rules

• Discrete Hopfield NN
• Boltzmann Machine

Unipolar neuron
46
Update Rules
Cooling schedule is required.

• Discrete Hopfield NN
• Boltzmann Machine

Unipolar neuron
47
Exercises

• Computer Simulations on the same TSP problem
  demonstrated previously using
• Simulated Annealing
• Deterministic Annealing, and
• Boltzmann Machine.
• Perform some analyses on your results.