Applications of PSO

1
5 October 2009 CITS7212 Computational
Intelligence School of Computer Science and
Software Engineering The University of Western
Australia Maria Bravo Rojas Evgeni Sergeev
Applications of PSO
2
Original PSO paper

• Highlight the results!
• Problem weights of a 2,3,1 (3 neurons in hidden
  layer) ANN for the XOR problem
• 2x3 3x1 weights 3 1 biases 13
  dimensions.
• Fitness function average sum-squared error.
• 20 particles took 30.7 PSO iterations on average
  to achieve error lt 0.05.
• ANN weights to classify the Fisher Iris Data Set
• 284 PSO iterations on average.
• ANN for electroencephalogram spike waveforms
• Backpropagation 89 correct. PSO 92 correct.
  (On test set.)
• Kennedy and Eberhart, 1995.

3
Original PSO paper (cont.)

• PSO vs GAs
• PSO optimises the Schaffer f6 function
  successfully. (Finds the global minimum.)
• Kennedy and Eberhart, 1995.

4
PSO with constraints

• In addition to the fitness function, a problem
  may have a set of constraints of the form f(x1,
  x2, , x3) lt C where x1, x2, , x3 are the
  dimensions of the search space.
• The standard PSO algorithm is modified by
• Initialising particles only in the feasible
  region.
• Pbest and gbest/lbest are recorded only in the
  feasible region.
• Results comparable to those of GAs on a range of
  test cases (functions defined by closed
  formulae).
• Hu and Eberhart 2002.

5
Multi-objective optimisation

• Several fitness functions f1, f2, , fn.
• Searching for Pareto-optimal solutions a set of
  points s.t. no point is dominated by any other
  point. (A dominates B iff it achieves better
  fitness in at least one fi, and not worse in all
  others.)

fitness2

• PSO the fitness function is a weighted average
  of f1, f2, , fn.
• For a given set of weights, optimisation
  techniques tend to find some Pareto-optimal
  solutions. So weights are varied
• Once each run, or by
• Bang-Bang Weighted Aggregation
• Dynamic Weighted Aggregation
• Parsopoulos and Vrahatis, 2002.

fitness1
where t is the iteration count
6
A problem in finance

• Construction of optimal risky portfolios
• Distribute weights (which add up to 1.0) among a
  number of assets.
• An asset i has an expected return Ei and variance
  si.
• Maximise Ep/sp, where the rules are

• PSO produces a slight improvement over the
  Excel's built-in Solver.
• Kendall and Su 2005.

7
A problem in scheduling

• Two-stage flow shop problem. There are n jobs,
  each consisting of m (stage 1) tasks which can be
  run in parallel on m machines, but take varying
  amounts of time for each job-task combination.
  There is one other (stage 2) task for each job,
  which must run after all of the stage 1 tasks
  have been completed, and its run time also varies
  between jobs. Each job has a due date. Objective
  minimise the maximum lateness over all jobs.
• Application online reports consisting of
  multiple mutually-independent queries to a
  distributed database.
• Earlier result showed that only permutations need
  to be considered (jobs are ordered and all the
  tasks on a machine are run in that same
  sequence).
• Binary PSO was used in a space of all
  permutations of n.
• Movement towards pbest/gbest is defined as
  setting the same task at the same position in the
  sequence as the pbest/gbest, with probability
  given by the velocity. Sequence is then adjusted
  to be a valid permutation.
• Allahverdi and Al-Anzi 2006.

8
A problem in power systems

• Economic Dispatch Problem
• Minimise the sum of
• Fi(Pi)aiPi2biPiciei Sin(fi(Pimin-Pi))
• This is a model of the relationship between power
  produced and the amount of fuel consumed with
  different parameters for different generators i.
• The Pi terms are constrained Pimin lt Pi lt Pimax.
• The sinusoid term is the valve point effect.
• Another constraint is the power balance. Between
  each pair of generators, there is a loss term.
• PSO applied together with a gradient-based
  technique PSO searches for solutions, SQP is
  used to fine-tune them.
• Victoire and Jeyakumar 2004.

9
Other

• Antenna shape optimisation (optimise parameters
  provided to a simulation program). Robinson et
  al. 2002.

10
TORCS project

• Overcame problems with fitness evaluation.
• Using median of 5 trials to confirm a change in
  pbest.
• Basic PSO algorithm
• With 2, 5, 9, 16 particles. Clique and loop
  topologies.
• Experiment with stochastic repelling from
  clamping bounds against fast convergence at
  those bounds.
• Elite improvement experiments
• Take a good solution from a pool. Randomise 1, 2,
  or 3 of its dimensions. Run PSO until
  convergence.
• Same, except slightly perturb all dimensions.
• Experiment introduce turbulence against
  unnecessary convergence along some dimensions.

11
The Particle in a n-dimensional space
a
b
-d
c
-c
d
-b
-a
12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
References

• Hu, X. and Eberhart, R. C. Solving constrained
  nonlinear optimization problems with particle
  swarm optimization. Proceedings of the Sixth
  World Multiconference on Systemics, Cybernetics
  and Informatics 2002 (SCI 2002), Orlando, USA.
  2002
• Parsopoulos, K. E. and Vrahatis, M. N. Particle
  swarm optimization method in multiobjective
  problems. Proceedings of the ACM Symposium on
  Applied Computing 2002 (SAC 2002), pp. 603-607,
  2002
• G. Kendall and Y. Su, "A particle swarm
  optimisation approach in the construction of
  optimal risky portfolios," in Proceedings of the
  IASTED International Conference on Artificial
  Intelligence and Applications, part of the 23rd
  Multi-Conference on Applied Informatics, pp.
  140-145, Innsbruck, Austria, February 2005.

19
References 2

• Ali Allahverdi, Fawaz S. Al-Anzi A PSO and a
  Tabu search heuristics for the assembly
  scheduling problem of the two-stage distributed
  database application. Computers Operational
  Research 33 1056-1080 (2006)
• Robinson, J., S. Sinton, and Y. Rahmat-Samii,
  Particle swarm, genetic algorithm, and their
  hybrids Optimization of a profiled corrugated
  horn antenna, IEEE International Symposium on
  Antennas Propagat., Vol. 1, 314-317, 2002.
• T. A. A. Victoire and A. E. Jeyakumar, Hybrid
  PSO-SQP for economic dispatch with valve-point
  effect, Electric Power Systems Research, Vol.
  71, No. 1, pp. 51-59, 2004.
• J. Kennedy and R. Eberhart. Particle swarm
  optimization. In Proceedings of the IEEE
  International Conference on Neural Networks,
  pages 1942-1948, IEEE Press, Piscataway, NJ, 1995.