Music-Inspired Optimization Algorithm Harmony Search

1
Music-Inspired Optimization AlgorithmHarmony
Search
Zong Woo Geem
2
What is Optimization?

• Procedure to make a system or design as
  effective, especially the mathematical techniques
  involved. (? Meta-Heuristics)
• Finding Best Solution
• Minimal Cost (Design)
• Minimal Error (Parameter Calibration)
• Maximal Profit (Management)
• Maximal Utility (Economics)

3
Types of Optimization Algorithms

• Mathematical Algorithms
• Simplex (LP), BFGS (NLP), BB (DP)
• Drawbacks of Mathematical Algorithms
• LP Too Ideal (All Linear Functions)
• NLP Not for Discrete Var. or Complex Fn.,
  Feasible Initial Vector, Local Optima
• DP Exhaustive Enumeration, Wrong Direction
• Meta-Heuristic Algorithms
• GA, SA, TS, ACO, PSO,

4
Existing Nature-Inspired Algorithms
5
Existing Meta-Heuristic Algorithms

• Definition Synonym
• Evolutionary, Soft computing, Stochastic
• Evolutionary Algorithm (Evolution)
• Simulated Annealing (Metal Annealing)
• Tabu Search (Animals Brain)
• Ant Algorithm (Ants Behavior)
• Particle Swarm (Flock Migration)
• Mimicking Natural or Behavioral Phenomena ? Music
  Performance

6
Algorithm from Music Phenomenon
7
Algorithm from Jazz Improvisation
Click Below
8
Analogy
Mi, Fa, Sol
Do, Re, Mi
Sol, La, Si
Do
Mi
Sol
f (100, 300, 500)
100mm 200mm 300mm
500mm 600mm 700mm
300mm 400mm 500mm
100mm
300mm
500mm
9
Comparison Factors

• Musical Inst. ? Decision Var.
• Pitch Range ? Value Range
• Harmony ? Solution Vector
• Aesthetics ? Objective Function
• Practice ? Iteration
• Experience ? Memory Matrix

10
Good Harmony Bad Harmony
?
An Algorithm which Keeps Better Harmonies!
11
Procedures of Harmony Search

• Step 0. Prepare a Harmony Memory.
• Step 1. Improvise a new Harmony with Experience
  (HM) or Randomness (rather than Gradient).
• Step 2. If the new Harmony is better, include it
  in Harmony Memory.
• Step 3. Repeat Step 1 and Step 2.

12
HS Operators

1. Random Playing
2. Memory Considering
3. Pitch Adjusting
4. Ensemble Considering
5. Dissonance Considering

13
Random Playing
x ? Playable Range E3, F3, G3, A3, B3, C4, D4,
E4, F4, G4, A4, B4, C5, D6, E6, F6, G6, A6, B6,
C7
14
Memory Considering
x ? Preferred Note C4, E4, C4, G4, C4
15
Pitch Adjusting
x or x-, x ? Preferred Note
16
Ensemble Considering
17
Rule Violation (Parallel 5th)
18
Example of Harmony Search
19
Initial Harmony Memory
20
Next Harmony Memory
21
With Three Operators
1, 2, 3, 4, 5
1
f
6
1
4
2
22
HS Applications forBenchmark Problems
23
Six-Hump Camel Back Function
f(-0.08983, 0.7126) -1.0316285 (Exact) f
(-0.08975, 0.7127) -1.0316285 (HS)
24
Multi-Modal Function
25
Artificial Neural Network - XOR
                 
T T F
T F T
F T T
F F F
Bias
Sum of Errors in BP 0.010 Sum of Errors in HS
0.003
26
HS Applications forReal-World Problems
27
Sudoku Puzzle
28
Music Composition Medieval Organum
Interval Rank Interval Rank
Fourth 1 Fifth 2
Unison 3 Octave 3
Third 4 Sixth 4
Second 5 Seventh 5
29
Project Scheduling (TCTP)
30
University Timetabling
31
Internet Routing
32
Web-Based Parameter Calibration
RMSE 1.305 (Powell), 0.969 (GA), 0.948 (HS)
33
Truss Structure Design
GA 546.01, HS 484.85
34
School Bus Routing Problem
Min C1 ( of Buses) C2 (Travel Time) s.t. Time
Window Bus Capacity
GA 409,597, HS 399,870
35
Generalized Orienteering Problem
Max. Multi-Objectives 1. Natural Beauty 2.
Historical Significance 3. Cultural Attraction 4.
Business Opportunity
Case1 Case2 Case3 Case4 Case5
ANN 12.38 13.05 12.51 12.78 12.36
HS 12.38 13.08 12.56 12.78 12.40
36
Water Distribution Network Design

• MP 78.09M
• GA 38.64M (800,000)
• SA 38.80M (Unknown)
• TS 37.13M (Unknown)
• Ant 38.64M (7,014)
• SFLA 38.80M (21,569)
• CE 38.64M (70,000)
• HS 38.64M (3,373)
• 5 times out of 20 runs

37
Large-Scale Water Network Design

• Huge Variables
• (454 Pipes)
• GA 2.3M Euro
• HS 1.9M Euro

38
Multiple Dam Operation
Max. Benefit (Power, Irrigation)
GA 400.5, HS 401.3 (GO)
39
Hydrologic Parameter Calibration
Mathematical 143.60, GA 38.23, HS 36.78
40
Ecological Conservation
With 24 Sites, SA 425, HS 426
41
Satellite Heat Pipe Design
42
Satellite Heat Pipe Design
BFGS
HS
Minimize Mass
Maximize Conductance
BFGS Mass 25.9 kg, Conductance 0.3808 W/K HS
Mass 25.8 kg, Conductance 0.3945 W/K
43
Oceanic Oil Structure Mooring
44
RNA Structure Prediction
45
Medical Imaging
46
Radiation Oncology
47
Astronomical Data Analysis
48
All that Jazz

• Robotics
• Visual Tracking
• Internet Searching
• Management Science
• Et Cetera

49
Paradigm Shifta change in basic assumptions
within the ruling theory of science
50
Stochastic Partial Derivative of HS
51
Stochastic Co-Derivative of HS
52
Parameter-Setting-Free HS

• Overcoming Existing Drawbacks
• Suitable for Discrete Variables
• No Need for Gradient Information
• No Need for Feasible Initial Vector
• Better Chance to Find Global Optimum
• Drawbacks of Meta-Heuristic Algorithms
• Requirement of Algorithm Parameters

53
(No Transcript)
54
Wikipedia (Web Encyclopedia)
55
Books on Harmony Search
56
Visitor Clustering (As of Nov. 2010)
57
Citations in Major Literaturein tantum ut si
priora tua fuerint parva,et novissima tua
multiplicentur nimis.Iob 87
58
What is Your Contribution?
59
Question for Harmony Search?

• Visit the Website
• HarmonySearch.info