A Study of Evolutionary Multiagent Models Based on Symbiosis

1
A Study of Evolutionary Multiagent ModelsBased
on Symbiosis

• IEEE Transactions on Systems, Man, and
  Cyberrnetics
• Toru Eguchi, Kotaro Hirasawa
• Review By
• Paskorn Champrasert _at_dssg umb

2
Content

• Introduction
• Objectives
• Approach
• Multiagent Systems with Symbiotic Learning and
  Evolution
• Simulation Model
• Simulation Results
• Conclusions

3
Introduction

• Multiagent Systems (MAS)
• are the systems where there are many autonomous
  subjects
• (i.e. agents) interacting each other.
• have been studied and applied to various fields
  such as
• Optimization problem
• Game model
• Analyzing biological phenomena
• Agents have their own objectives.
• Problem
• 1) The strategies of agents become limited
• The variety and flexibility of system are lost.
• 2) The global optimization cannot be obtained
• 1 There are conflicting objectives of agents.
• 2 Nash Equilibrium happens
• Agents find a competitive or compromised
  solution.
• Escaping from Nash Equilibrium is one of the
  important issues to
• improve the performance of MAS.

4
Nash Equilibrium

• In game theory, the Nash equilibrium (named after
  John Nash, who proposed it)
• is a kind of solution concepts of a game
  involving two or more players, where no player
  has anything to gain by changing only his or her
  own strategy.
• If each player has chosen a strategy and no
  player can benefit by changing his or her
  strategy while the other players keep theirs
  unchanged, then the current set of strategy
  choices and the corresponding payoffs constitute
  a Nash equilibrium

5
Pareto Optimality

• Pareto optimality, is an important notion in
  economics with broad applications in game theory,
  engineering and the social sciences.
• Given a set of alternative allocations and a set
  of individuals, a movement from one allocation to
  another that can make at least one individual
  better off, without making any other individual
  worse off, is called a Pareto improvement or
  Pareto optimization.
• An allocation of resources is Pareto efficient or
  Pareto optimal when no further Pareto
  improvements can be made.

6
Objectives

• The authors propose a new concept of MAS named
  Multiagent Systems with Symbiotic Learning and
  Evolution (Masbiole)
• Masbiole
• can escape from Nash Equilibrium
• is expected to apply to many fields
• Economy
• Optimization in engineering problem

7
Design Approach

• In the view of social science, there are two
  important concepts
• Individual Rationality ( Conventional MAS)
• The policy of individual rationality aims to
  optimize only the benefit of individual agent
• It causes Nash Equilibrium
• Group Rationality ( Masbiole)
• The policy of group rationality aims to optimize
  the benefit of the whole group.
• It can solves the Nash Equilibrium problem.

8
Masbiole

• Based on group rationality concept,
• The concept of symbiosis in the ecosystem are
  introduced in the process of learning and
  evolution of MAS.
• There are several relationships among agents
• E.g. mutualism(), harm(--), predation(-),
  altruism(-)
• Agents consider to improve or deteriorate not
  only itself but also an opponent.

9
MAS vs. Masbiole
10
Symbiotic Relations

• Agents can change their strategies according to
  their symbiotic relations.
• Symbiotic relations are defined as the
  combination of improvement/deterioration of
  evaluation of self agent and opponent agent.

11
Symbiotic Relation
S self O opponent
Conventional MAS
For example, if agent s has Mutualism toward
agent o, agent s can change its strategy so that
both the evaluation of agents s and agent o are
improved.
12
Symbiotic Evolution

• Symbiotic evolution is implemented for each
  symbiotic relation sequentially.
• Agent S and agent O are selected
• Only agent S changes its strategy based on its
  symbiotic relation to agent O.
• The strategies of other agents are fixed.
• 3) Do 1) and 2) until a terminal condition meets.

13
Algorithms of Symbiotic Evolution

• This is the algorithm of symbiotic evolution in
  this paper.
• Agents consist of a number of individuals
  (population)
• Each individual has its own strategy and
  evaluation.
• The strategies are changed based on evolutionary
  process using Multiobjective Genetic Algorithm (
  MOGAs)

14
Main Loop of Symbiotic Evoluition

• Initialization
• Strategies and symbiotic relations of agents are
  initialized.
• 2. Selection of Agents
• Agent S and Agent O are selected
• Production of offspring
• Produce offspring of agents S whose strategies
  are changed by genetic operations. (crossover or
  mutation )
• Evaluation
• A pair of individuals selected from agent S and
  agent O is evaluated.
• Symbiotic Ranking
• Rank of a pair of individuals of agent s and
  agent o is calculated using multiobjective
  ranking method.
• Selection of individuals
• The better individuals of agent S with respect
  to the above rank are transferred to the next
  generation
• Terminal condition
• 2-6 are repeated until a terminal condition
  meets.

15
4) Evaluation (1/3)

• At the beginning,
• A pair of individuals is selected from agent S
  and agent O.
• All the individuals of agent S are selected only
  one time.

16
4) Evaluation (2/3)

• The evaluation of agent s and agent o is

Individuals of Agent o and agent others are
selected randomly
17
4) Evaluation (3/3)

• The sets of are called
  Evaluation Point

18
5) Symbiotic Ranking

• The rank of evaluation point is determined to be
  R1 when it is dominated by other R evaluation
  points under the symbiotic relation of agent s
  toward o

19
6) Selection of individual
The better individuals of agent S with respect to
the rank are transferred to the next generation

• No ?Np
• Np parent population
• No offspring population

20
Evaluation Point
21
Symbiotic Pareto Solutions

• The solutions obtained by symbiotic evolution can
  satisfy the Pareto optimality.
• Pareto optimality means a change of an
  individuals strategy that make that can make at
  least one individual get benefit.
• Pareto optimality eliminate Nash Equilibrium
• The authors called Pareto Optimality in Masbiole
  as Symbiotic Pareto Solution (SPS)

22
Definition of SPS

• For the set ?s of the strategy of agent S if and
  only if there is no ?s ? ?s which satisfies the
  following conditions ?s ? ?s is called the
  Symbiotic Pareto Solution (SPS) of agent S toward
  agent o

23
(No Transcript)
24
Simulation Model

• We are playing the tile-world.
• The tile-world
• is a virtual environment
• 2-dimensional grid world
• Consists of autonomous units
• E.g. tiles, obstacles, and holes

25
(No Transcript)
26
How to play

• The units
• can move one cell
• Push the tile
• Drop a tile into a hole
• Score
• Units get score when it can push a tile (Ts) to
  its hole
• However there are disturbance tiles (Td), the
  units can block each other.
• For example, units belonging to agent 1 having
  predation toward agent 2 are expected to drop
  tile Ts into hole 1 (improvement of itself) and
  occupy hole 2 by tile Td.

27
Construction of Agents

• Agents are constructed by genetic network program
  (GNP)
• Genetic Network Program
• Contain directed graph
• Each node is connected by directed branch.
• Is expected to show better performance than GP
• Basic structure of GNP consists of
• Initial Boot Node
• is an origin of node.
• Judgment Node
• has various decision functions
• has several judgment results related to the of
  directed branches.
• Processing Node
• determine a processing to the environment i.e
  action of agent

28
Function Label

• Judgment nodes and Processsing nodes have their
  Function Label
• The library of the function label is prepared in
  advance by designers.

29
Example of GNP
30
Transferring Graph to Bit String

• Phenotype (the previous figure) shows the
  directed graph structure.
• Genotype provides the chromosomes encoded into
  bit-strings.

C connecting node dil delay in branch
NID node ID NT note type
NF Function Label D delay in node
31
(No Transcript)
32
GNP Execution

• GNP boots from the initial boot node
• In the case of exceeding the time delay threshold
  value, GNP stops its node transition.
• The next turn of GNP restarts from the judgment
  node where GNP stopped.

33
GNP Genetic Operators

• Selection Method
• Roulette Selection
• Tournament Selection
• Ranking Selection
• Elite Preservation
• Genetic Operations
• - Mutation
• Crossover

34
Mutation

• Mutation is executed when new offspring is
  generated.
• An individual is selected using selection method
• Some branches are selected randomly for mutation
  with the probability Pm
• The selected branches are changed randomly.
• New offspring is generated

35
(No Transcript)
36
Crossover

• Crossover is executed between two parent
  individuals. In this paper Uniform Crossover is
  implemented.
• Two parents are selected using selection method.
• Some nodes are selected as crossover node with
  the probability Pc.
• Two parents exchange the selected corresponding
  nodes having the same node number and two new
  offspring are generated

37
(No Transcript)
38
GNP and Tile-world game

• Every unit belonging to the same agent is carried
  out by the same GNP of an agent.
• So, the initial boot node has three branches
  corresponding to the units of the agents.
• The turn for the action is
• Unit 1a-gt2a-gt1b-gt2b.. -gt 2c
• The node transition of GNP of a unit stops when
  the unit of GNP of other agents operates. (delay
  in GNP gt threshold). This is called one step.
• Episode is defined as the steps needed to
  complete the task.
• Then, genetic operation happens.

39
(No Transcript)
40
Actions

• Unit can move
• MF move forward
• MB move backward
• ML move left
• MR move right
• SH push tile
• Unit can obtain information on the objects
• SF from front
• SB from back
• SL from left
• SR from right
• The result can be Ts, Td, Unit1,Unit2,Hole1,Hole2
  , Floor, Obstacle

41
Unit action

• Unit can get the directional information of
• Nearest tile ( NTS, NTD)
• Hole (NH1, NH2)
• Unit (NU1, NU2)
• The result can be front,back,left,right,
  several,none)
• several there are many objects
• none there is no object in the view (5 steps).

42
Evaluation Function
43
Simulation configuration
44
(No Transcript)
45
Results
46
Discussion

• The simulation 1 shows the Masbiole.
• Masbiole works well.

47
Simulation 2 Masbiole vs. MAS
Sum Improvement E1 E2 (scalar summation)
48
Result Masbiole vs. MAS
49
Result Masbiole vs. MAS
50
(No Transcript)
51
Conclusions

• Masbiole is modeled to follow the symbiotic
  phenomena in the ecosystem.
• The new kind of evolutionary computation model
  named GNP is proposed.
• Masbiole with mutualism is more useful than
  conventional MAS in terms of overcoming Nash
  Equilibrium.
• Masbiole can be applied to several other examples
  of MAS and Multiobjective Optimization Problems.