Parallel Simulation' Past, Present and Future

1
Parallel Simulation.Past, Present and Future
C.D. Pham Laboratoire RESAM Universit? Claude
Bernard Lyon 1 cpham_at_resam.univ-lyon1.fr
2
Past

• Introduction
• Discrete Event Simulation (DES)
• Parallel DES and the synchronization problems
• Chandy-Misra-Bryant rules
• Architecture of a conservative LP
• The  Safe is better  approach
• The lookahead ability
• Jefferson?s point of view
• Architecture of an optimistic LP
• Time Warp
• Mixed/adaptive approaches,

3
Present

• Ongoing projects
• SSF,
• TeD/GTW,
• GloMoSim,
• CSAM.

4
Future

• Challenges Perspectives
• Ultra-large scale simulations,
• Wide-area federation-based simulations,
• WEB-based simulations.

5
PASTthe algorithms, only the algorithms!
6
Simulation

• To simulate is to reproduce the behavior of a
  physical system with a model
• Practically, computers are used to numerically
  simulate a logical model
• Simulations are used for performance evaluation
  and prediction of complex systems
• fluids dynamic, chemistry reactions (continous)
• communication network models routing, congestion
  avoidance, mobile? (discrete)
• Simulation is more flexible than analytical
  methods

7
Discrete Event Simulation (DES)

• assumption that a system changes its state at
  discrete points in simulation time

a1
a2
a3
a4
d1
d2
d3
S1
S3
time step
Dt
2Dt
3Dt
4Dt
5Dt
6Dt
0
8
DES concepts

• fundamental concepts
• system state (variables)
• state transitions (events)
• simulation time totally ordered set of values
  representing time in the system being modeled
• the system state can only be modified upon
  reception of an event
• modeling can be
• event-oriented
• process-oriented

9
Life cycle of a DES

• a DES system can be viewed as a collec-tion of
  simulated objects and a sequence of event
  computations
• each event computation contains a time stamp
  indicating when that event occurs in the physical
  system
• each event computation may
• modify state variables
• schedule new events into the simulated future
• events are stored in a local event list
• events are processed in time stamped order
• usually, no more event termination

10
A simple DES model
11
Why it works?

• events are processed in time stamp order
• an event at time t can only generate future
  events with timestamp greater or equal to t (no
  event in the past)
• generated events are put and sorted in the event
  list, according to their timestamp

12
Why change? It ?s so simple!

• models becomes larger and larger
• the simulation time is overwhelming or the
  simulation is just untractable
• example
• parallel programs with millions of lines of
  codes,
• mobile networks with millions of mobile hosts,
• ATM networks with hundreds of complex switches,
• multicast model with thousands of sources,
• ever-growing Internet,
• and much more...

13
Some figures to convince...

• ATM network models
• Simulation at the cell-level,
• 200 switches
• 1000 traffic sources, 50Mbits/s
• 155Mbits/s links,
• 1 simulation event per cell arrival.

More than 26 billions events to simulate 1
second! 30 hours if 1 event is processed in 1us
14
Parallel simulation - principles

• execution of a discrete event simulation on a
  parallel or distributed system with several
  physical processors.
• the simulation model is decomposed into several
  sub-models that can be executed in parallel
• spacial partitioning,
• temporel partitioning,
• radically different from simple simulation
  replications.

15
Parallel simulation - pros cons

• pros
• reduction of the simulation time,
• increase of the model size,
• cons
• causality constraints are difficult to maintain,
• need of special mechanisms to synchronize the
  different processors,
• increase both the model and the simulation kernel
  complexity.
• challenges
• ease of use, transparency.

16
Parallel simulation - example
17
A simple PDES model
local event list
18
Synchronization problems

• fundamental concepts
• each Logical Process (LP) can be at a different
  simulation time
• local causality constraints events in each LP
  must be executed in time stamp order
• synchronization algorithms
• Conservative avoids local causality violations
  by waiting until it ?s safe
• Optimistic allows local causality violations but
  provisions are done to recover from them at
  runtime

19
Chandy-Misra-Bryant rules
Architecture of a conservative LP The  Safe is
better  approach The lookahead ability
20
Architecture of a conservative LP

• LPs communicate by sending non-decreasing
  timestamped messages
• each LP keeps a static FIFO channel for each LP
  with incoming communication
• each FIFO channel (input channel, IC) has a clock
  ci that ticks according to the timestamp of the
  topmost message, if any, otherwise it keeps the
  timestamp of the last message

21
A simple conservative algorithm

• each LP has to process event in time-stamp order
  to avoids local causality violations

The Chandy-Misra-Bryant algorithm
while (simulation is not over) determine the
ICi with the smallest Ci if (ICi empty)
wait for a message else remove topmost
event from ICi process event
22
Safe but has to block
IC1
IC2
IC3
23
Blocks and even deadlocks!
A
merge point
S
M
BLOCKED
B
24
How to solve deadlock null-messages
null-messages for artificial propagation of
simulation time
A
S
M
UNBLOCKED
B
What frequency?
25
How to solve deadlock null-messages
a null-message indicates a Lower Bound Time
Stamp minimum delay between links is 4 LP C
initially at simulation time 0
A
B
C
26
The lookahead ability

• null-messages are sent by an LP to indicate a
  lower bound time stamp on the future messages
  that will be sent
• null-messages rely on the  lookahead  ability
• communication link delays
• server processing time (FIFO)
• lookahead is very application model dependant and
  need to be explicitly identified

27
Lookahead for concurrent processing
TA
TALA
safe event
unsafe event
28
What if lookahead is small?
a null-message indicates a Lower Bound Time
Stamp minimum delay between links is 4 LP C
initially at simulation time 0
1
A
B
C
29
Conservative pros cons

• pros
• simple, easy to implement
• good performance when lookahead is large
  (communication networks, FIFO queue)
• cons
• pessimistic in many cases
• large lookahead is essential for performance
• no transparent exploitation of parallelism
• performances may drop even with small changes in
  the model (adding preemption, adding one small
  lookahead link?)

30
Jefferson?s point of view
Architecture of an optimistic LP Time Warp
31
Architecture of an optimistic LP

• LPs send timestamped messages, not necessarily in
  non-decreasing time stamp order
• no static communication channels between LPs,
  dynamic creation of LPs is easy
• each LP processes events as they are received, no
  need to wait for safe events
• local causality violations are detected and
  corrected at runtime

32
Processing events as they arrive
33
Time Warp. Rollback? How?

• Late messages are handled with a rollback
  mechanism
• undo false/uncorrect local computations,
• state saving save the state variables of an LP
• reverse computation
• undo false/uncorrect remote computations,
• anti-messages anti-messages and (real) messages
  annihilate each other
• process late messages
• re-process previous messages processed events
  are NOT discarded!

34
A pictured-view of a rollback
unprocessed
processed

• The real rollback distance depends on the state
  saving period short period reduces rollback
  overhead but increases state saving overhead

35
Reception of an anti-message

• may initiate a rollback if the corresponding
  positive message has already been processed,

36
Need for a Global Virtual Time

• Motivations
• an indicator that the simulation time advances
• reclaim memory (fossil collection)
• Basically, GVT is the minimum of
• all LPs ? logical simulation time
• timestamp of all messages in transit
• GVT garantees that
• events below GVT are definitive events (I/O)
• no rollback can occur before the GVT
• state points before GVT can be reclaimed
• anti-messages before GVT can be reclaimed

37
A pictured-view of the GVT
WANTED
old GVT
new GVT
conditional event
definitive event
38
Optimistic overheads

• Periodic state savings
• states may be large, very large!
• copies are very costly
• Periodic GVT computations
• difficult in a distributed architecture,
• may block computations,
• Rollback thrashing
• cascaded rollback, no simulation progress!
• Memory!
• memory is THE limitation

39
Optimistic pros cons

• pros
• exploits all the parallelism in the model,
  lookahead is less important,
• transparent to the end-user,
• interactive simulations possible,
• can be general-purpose.
• cons
• very complex, needs lots of memory,
• large overheads (state saving, GVT, rollbacks?)

40
Mixed/adaptive approaches

• General framework that (automatically) switches
  to conservative or optimistic
• Adaptive approaches may determine at runtime the
  amount of conservatism or optimism

messages
41
PRESENThow to survive?
?and how to get money?
42
Parallel simulation today

• Lots of algorithms have been proposed
• variations on conservative and optimistic
• adaptives approaches
• Paradoxically few end-users
• impossible to compete with sequential simulators
  in terms of user interface, generability, ease of
  use...
• Ongoing research mainly focus on
• ultra-large scale simulations of networks,
• tools and execution environments
• composability and interoperability issues

43
Ongoing projects

• DOMAINS/GloMoSim
• SSF
• TeD/GTW
• CSAM

44
DOMAINS/GloMoSim project

• Design of Mobile Adaptive Networks,
  DARPA/DAAB07-97-C-D321
• Provides a library for simulating millions of
  mobile nodes
• Proves the efficiency of parallel simulation for
  scalability issues
• Based on the PARSEC simulation language
• Conservative or optimistic execution

45
Glomo objectives
46
Glomo librairies
47
SSF-Scalable Simulation Framework

• DARPA/ITO (Next Generation Internet Program) and
  NSF/ANIR (Special Projects in Networking)
• SSF proposes discrete event simulations of large
  complex systems, with serial and scalable
  parallel implementations
• SSFNet is a collection of SSF-based models for
  simulating Internet protocols and networks
• Based on YAWNS, a conservative kernel

48
SSFNet, modeling the Internet
49
TeD/GTW

• TeD (Telecommunications Description Language) is
  a language for modeling telecommunicating network
  elements and protocols (PNNI).
• GTW is a general purpose parallel discrete event
  simulation executive using optimistic
  synchronization techniques.
• The TeD compiler translates TeD models into C
  code which uses GTW for parallel simulation

50
Modeling with TeD
51
CSAM

• CSAM Conservative Simulator for ATM network
  Model
• Simulation at the cell-level
• C programming-style, predefined generic model
  of sources, switches, links?

Test-bed for parallel simulations on
high-performance clusters.
52
Test case 78-switch ATM network
Distance-Vector Routing with dynamic link cost
functions Connection setup, admission control
protocols
53
CSAM - Some results...
Routing protocol?s reconfiguration time
54
CSAM - visualization tool
55
FUTUREthe great challenges!
Ultra-Large scale simulations, Wide-area
federation-based simulations, WEB-based
simulations.
56
Ultra-large scale simulations

• Millions of mobile nodes,
• Thousands of multicast connections,
• Full Internet simulation,
• Ultra-large scale simulations require
• lots of memory! lots of CPU!
• new modeling techniques reuse of model
  description, decoupling state from model
  description
• advanced memory management schemes shared
  events, application memory regulation.
•  Out of core  simulations.

57
Federation-based simulations

• Cost of model developping is increasing at a high
  rate.
• Reuse and interoperability is a key issue in the
  development phase of new models.
• Need for a unified framework so that independent
  simulators can run together and achieve a given
  goal.

The DoD has proposed the High-Level
Architecture framework for federation-based
simulations
58
The HLA framework

• Without HLA, simulations are mostly independents
  and interoperability is not easy.

• The High Level Architecture calls for a
  federation of simulations to achieve
  interoperability and reuse of software.

10 Rules for the federation and the federates
behavior
An Interface Specification
An Object Model Template to describe the
simulation objects
59
Wide-area interactive simulations
computer-based sub-marine simulator
display
INTERNET
battle field simulation
human in the loop flight simulator
60
WEB-Based simulation

• Users build models and submit them on the web
  (meta-computing)
• Hides the complexity of parallel simulation
  techniques
• Provides computing resources for the users

ASCII RED 1st rank top 500 list
Cplant
61
JTeD project, the Java-based TeD
62
Summary

• Parallel simulation is a mature field
• Applications, especially communication network
  models, are the centre of interest
• The challenges are for very-large scale
  simulations and re-usability of models
• Real-time interactive simulations are desirable
  on a wide-area interconnection
• As-fast-as possible simulations will likely
  remain  indoor  (cluster, SMP)

63
Requirements put on networking

• In wide-area simulation, data distribution relies
  mainly on multicast and broadcast operations
• Near real-time behaviors are desirable for
  interactive simulation

64
References

• Parallel simulation
• K. M. Chandy and J. Misra, Distributed
  Simulation A Case Study in Design and
  Verification of Distributed Programs, IEEE Trans.
  on Soft. Eng., 1979, pp440-452
• R. Fujimoto, Parallel Discrete Event Simulation,
  Comm. of the ACM, Vol. 33(10), Oct. 90, pp31-53
• HLA
• http//hla.dmso.mil
• Projects
• GlomoSim - http//pcl.cs.ucla.edu/projects/glomosi
  m
• SSF - http//www.ssfnet.org/homePage.html
• TeD/GTW -
• CSAM - http//resam.univ-lyon1.fr/CSAM