Swarm behaviour and traffic simulations

1
Swarm behaviour and traffic simulations

• Using stigmergy to solve algorithmic problems,
  predict and improve vehicle traffic

2
Overview (1)

• Swarms in nature
• Social insects and Stigmergy
• Ant algorithms and application examples
• Foraging in ants
• Using foraging behaviour to solve the TSP
• Labour division among social insects
• Mailmen using adaptive task allocation model

3
Overview (2)

• Traffic simulation by cellular automata
• Adopting the stigmergic process
• Prediction of driver behaviour
• Traffic infrastructure optimization
• Drivers as ant agents
• Signal lights as social insects
• Other real world applications

4
Swarms in nature

• What is a swarm ???

5
Pictures of swarms (1)
6
Pictures of swarms (2)
7
Pictures of swarms (3)
8
Characteristics of swarms

• Aggregation of animals with similar size and
  often similar orientation
• Interaction of animals leads to new intelligent
  forms of behaviour that are not inherited in the
  individuals
• E.g. insects, birds, fish, bacteria

9
The main actor of the presentation
10
Social insects and stigmergy

• Social insect societies are distributed systems
  with highly structered social organization
• They can accomplish complex tasks that far exceed
  the individuals abilities
• Here focus on stigmergy as important means of
  indirect communication paradigm

11
Stigmergy

• Originally defined by Grassé
• Stimulation of workers by the performance they
  have achieved
• Method of indirect communication in a
  self-organizing emergent system where its
  individual parts communicate with each other by
  modifying their local environment
• Here Pheromones ? diffusing chemical substance

12
Stigmergy example

• Example termites buildung nest pillars with soil
  pellets
• Stimulus ? response
• Autocatalytic process

13
Stigmergy behaviour of ants

• Stigmergy behaviour in ants and their transfer to
• computer algorithms
• Foraging and the TSP
• Labour Division and adaptive task allocation

14
Foraging in ants

• Foraging means searching for food
• Ants manage to find the shortest path between
  their nest and a food source
• Achieved through trail-laying and trailfollowing
  behaviour with pheromones
• ? Stigmergy

15
Foraging example

• Two paths with different lengths
• Ants follow way with most pheromones
• Autocatalytic process leads to differential
  length effect

16
The Travelling Salesman Problem

• Consists of a set of given cities
• Goal is to visit all cities in a closed loop of
  shortest length
• Every city must be visited only once!
• E.g. 15 biggest cities of Germany

17
TSP represented by graph theory

• TSP defined more generally by graph theory
• Graphs consist of vertices V and edges E
• Cities are vertices, edges are connections
  between cities
• In the TSP each city is connected to each other!
• Each edge has a certain length
• Example 4 cities A,B,C,D
• represented by vertices
• 6 connnections with lengths
• represented by edges

18
Ants solving the TSP

• Artificial ants exploring the TSP graph
• Artificial pheromones added by ants after
  completion of a complete loop proportional to
  1/length of route
• Probabilistic transition rule for ant k to next
  city j
• City j visited?
• Length of edge gives desirability measure ?ij
  1/length
• Amount of pheromones tij(t) on edge (i, j)
• Evaporation of pheromones over time lets system
  forget bad information

19
Labour division in ants

• Fundamental in social insects Division of
  reproductive castes from worker castes
• Further divisions
• subcastes of workers ? specialists
• subcastes of age and morphology
• again dividing subcastes into behavioural castes
• Plasticity Workers switch tasks in response to
• internal and external pertubations

20
Labour division model

• Based on idea of response threshold
• Stimulus exceeds individuals response threshold
• ? individual engages in task performance
• Stimulus plays role of Stigmergy here
• (can be pheromones, amount of encounters, )

21
Extended labour division model

• More realistic
• Extend previous model by threshold varying in
  time.
• ? if an individual performs a certain task its
  threshold related to this task decreases
• ? the thresholds related to all other tasks, not
  performed meanwhile, are increasing

22
Example Express mail retrieval (1)

• Group of mailmen has to
• pick up letters in a city
• Goal Allocation of mailmen to appearing demands
  should be optimal ? realized with adaptive task
  allocation model
• Each mailmen i reacts with a certain probability
  p to arising demands, depending on
• response threshold ? related to area j with
  demand
• the distance d to the area with demand
• the intensity s of the demand ? stimulus

23
Example Express mail retrieval (2)

• Figure (a) demand of a certain area over time
• At t 2000 the mailman that is specialized on
  this area gets removed
• Figure (b) response threshold of another mailmen
  that is reacting to the loss of the specialist

24
Traffic simulations

• Why would one do that?
• to predict drivers behaviour in order to adjust
  dynamic traffic signs, or propose alternative
  routes in navigation devices or radio
• to improve traffic infrastructure and traffic
  light plans in big, complicated traffic networks
  like cities

25
Cellular automata traffic models (1)

• Two major approaches on traffic simulation
• Fluid-dynamical, with continuous traffic ?
  macroscopic
• Discretized cellular automata model ?
  microscopic
• Focus in presentation discretized cellular
  automata models
• Discretized
• Street is diveded into fixed sites, cars have
  integer velocities
• Each site can be occupied by a car or can be empty

x meters
car 1
car 2
e.g. one lane traffic
site n site n1 site n 2 site
n 3 site n 4
26
Cellular automata traffic models (2)

• Assuming a simple one lane model
• One update of the system consists of the
  following steps performed with each car in
  parallel
• Acceleration or slowing down
• Depending on maximal speed and distance to next
  car
• Randomization
• To contribute human behaviour and external
  influences
• Car motion
• The advancment of sites, according to the speed
• Model shows nontrivial and realistic behaviour

27
Adopting Stigmergy

• Cars adopt the pheromone laying and sniffing
  behaviour of ants
• Leads to very realistic and dynamic system
• Reduces communication between cars to local
  information creation and retrieval ? stigmergy
• Computational costs can be reduced for collision
  checking
• Still, information about traffic signs and other
  environmental signals are non-local
• ? cellular automata ant cars

28
How cars behave like ants

• Each car leaves and sniffs pheromones on the road
• Pheromones fade over time, like with ants
• Faster cars leave longer trails then slower cars
  (a)
• Additional pheromone dropping necessary for
• stopped cars (b)
• quick deceleration
• lane changing (c)
• like using signal and
• brake lights

(a)
(b)
(c)
29
Traffic prediction

• Measurement devices like cameras determine the
  vehicles entering an area
• Implementing foraging behaviour of ants leads to
  realistic system of interacting drivers
• ? drivers follow other drivers and try to escape
  jams
• Various types of driver support
• Adjustment of dynamic traffic signs to avoid
  congestions
• Knowledge of growth of traffic jams allows to
  give reasonable redirections
• Through use of foraging behaviour alternative
  routes can be given more effectively, cars spread
  more

30
Optimizing traffic light plans

• Microscopic traffic model by individual cars with
  individual aims
• 1st Approach
• Cars as agents facilitate change of light plans
  by voting
• Evolutionary process improves overall light
  plans
• 2nd Approach
• Groups of ligths at an intersection behave like
  social insects
• Adaptive task allocation is responsible for
  running plans

31
Cars voting for traffic ligths

• Each car keeps track of two variables
• total driving time dtot
• total waiting time wtot
• ? waiting measure
• Statistics give information about overall fitness
• ? overall waiting measure
• Cars that are stopped at a light vote for it
• Lights with many votes are more probable to be
  mutated
• ? quicker adaption in the evolutionary process

32
Evolutionary process

• Probabilistic mutations of traffic light timings
• Mutation changes length of correlated light
  phases at an intersection (e.g. NS and E-W)
• Simulation of each branch of a new generation
• Survival of the fittest ? with least waiting time
  of cars

simulation 1
mutate
mutate
choose fittest
simulation 2
. . . .
simulation 3
33
Traffic Simulation under SuRJE

• SuRJE Swarms Under RJ using Evolution
• Design environment to build, test and optimize
  traffic scenarios
• Uses swarm based approach and
• evolutionary adaption
• Features
• Enables to build multi-lane road maps
• Car seeding areas define the
• input and output of cars
• Initial lights settings for starting point
• Evolutionary adaption parameters can be set

34
Simulation example in SuRJE

• Example network Looptown (a)
• Figure (b) shows the decrease of overall waiting
  time over generations

35
Traffic lights as social insects

• Traffic lights are implemented as social insects
• ? all lights at an intersection form one insect
• Insect has to perform one traffic light plan out
  of several for its intersection
• Traffic can be modeled by any microscopic traffic
  simulator
• Cars emit pheromones when crossing and waiting at
  an intersection to provide stimulus

36
Use of adaptive task allocation

• Adaptive task allocation model is applied to
  insects
• light-plans are chosen through stimulus
  response strategies
• communication of intersections only by Stigmergy
• Each insect/intersection has individual
  thresholds related to available light-plan
• Stimulus for light-plan j provided by cars
• Reinforcement learning is used to specialize
  intersections
• Threshold j of intersection i
• Learning coefficient

37
Example of stimulus evaluation

• 4 way intersection with two light-plans
• 1st plan gives priority to North-South leading
  lanes
• 2nd plan has priority for West-East leading lanes
• Traffic situation
• Many cars are driving on West East direction
• Few cars on North-South direction
• Initially plan 1 is driven
• Big amount of pheromones for W-E (many, waiting
  cars)
• Small amount for N-S (few, almost not waiting
  cars)
• ? evaluation of stimulus yields higher value for
  2nd plan!

N
W
E
S
38
Traffic simulation - Recapitulation

• Traffic prediction
• Simulation into future by using swarm based
  approach
• Giving the driver useful information about
  choosing the best route
• Traffic light timing improvment by cars as
  agents
• Cars are modeled with swarm based approach
• Improvement of traffic light plans through
  evolution
• Good for simple traffic lights with static timing
  program
• Dynamic traffic light adoption through social
  insects
• Cars are modeled by any microscopic traffic
  simulation
• Intersections choose their light plan through
  adaptive task allocation
• Good for traffic lights with sensors for counting
  cars

39
Examples for real world applications

• Scheduling Problems, e.g. subway, train
• Vehicle Routing, e.g. bus, taxi
• Connection-oriented network routing, e.g.
  internet, TCP/IP
• Connection-less network routing, e.g. bluetooth,
  infrared
• Optical networks routing

40

• Thank you for your attention!