A Generic Architecture for LargeScale Distributed Simulations

1
A Generic Architecture for Large-Scale
Distributed Simulations

• Stephen J. TURNER
• School of Computer Engineering,
• Nanyang Technological University,
• Singapore 639798
• ASSJTurner_at_ntu.edu.sg

2
Overview

• Parallel and Distributed Simulation
• Example Application Areas
• Research Issues
• A Generic Architecture
• Federating Parallel Simulators
• HLA vs. Customized Protocol
• Hierarchical Federations Architecture
• Conclusions and Future Research

3
Parallel and Distributed Simulation

• Parallel Discrete Event Simulation
• Aims to reduce the execution time of large
  discrete event simulations.
• The simulation model is partitioned into a number
  of Logical Processes (LPs) that are executed in
  parallel.
• As each LP has its own event list, a
  synchronization protocol is required to preserve
  causality
• A conservative protocol strictly avoids the
  occurrence of any causality error.
• An optimistic protocol detects and recovers from
  causality violations.

4
Parallel and Distributed Simulation

• Distributed Simulation
• Provides a way of linking simulation components
  (federates) of various types at possibly
  different locations to create a common virtual
  environment (federation).

training federate real-time execution
constructive federate time-stepped execution
live component real-time execution with hard
deadlines
constructive federate event driven execution
5
Parallel and Distributed Simulation

• Causality Violation

real world
simulated world
event message
fire
Simulator A (artillery unit)
the observer should see the artillery unit fire
before the target is destroyed.
Simulator B (target)
target destroyed
Simulator C (observer)
Time (wallclock time)
6
Example Application Areas

• Battle Simulation
• Links different types of forces at multiple
  physical locations to create a realistic and
  complex virtual world.
• Multi-player Internet Games
• Requires massive multi-player (10,000) virtual
  world.
• Air Traffic Control
• Simulates airports and airspace sectors to
  provide faster than real-time simulation for
  alternative scenario analysis.
• Supply Chain Management
• Covers the planning and management of material
  and information flow, from the manufacturer
  through the distributors to the customer.

7
Example Application Areas

• Supply Chain Management
• With the globalization of markets, factories and
  distribution centres in a supply chain simulation
  may be dispersed across many different countries.

Shipment to Customers
Wafer Fab 1
ICs
Wafer Fab 2
Assembly Test
Wafer Fab 3
8
Research Issues

• Fast Execution
• The execution time of large-scale simulations may
  be unacceptably large due to the detail and
  complexity.
• Fast simulation is required for what if and
  alternative scenario analysis.
• Reuse and Interoperability
• Large-scale simulations are constructed by
  linking together existing simulation models to
  form a simulation federation.
• These component models may have been implemented
  using different languages or packages and
  developed for different hardware platforms.

9
Research Issues

• Geographical Distribution
• A large-scale simulation may involve linking a
  number of simulation components that are
  geographically distributed.
• Scalability
• As the number of simulation components and the
  size of the network increases, the run-time
  system should be able to handle the
  communications effectively.
• Data Security
• A group of simulation components may need to
  share some sensitive information with each other
  while hiding that data from other simulation
  components in the federation.

10
A Generic Architecture

• Generic Architecture
• A generic architecture for large-scale
  distributed simulation is being developed to
  investigate these research issues.
• As these research issues are common to many
  application areas, the architecture is not
  restricted to any particular application.
• The research issues are addressed by
• Federating Parallel Simulators
• High Level Architecture
• Hierarchical Federations Architecture

11
Federating Parallel Simulators

• Hybrid Distributed/Parallel Simulation
• This is a distributed simulation architecture
  where one or more simulation components
  (federates) is itself partitioned into LPs which
  are executed in parallel.
• Addresses research issue of fast execution.

workstation
Our distributed supply chain simulation can be
speeded up by executing the Assembly and Test
facility as a parallel federate
workstation
workstation
multiprocessor
multiprocessor
12
HLA vs. Customized Protocol

• Option 1 High Level Architecture (HLA) for
  Simulation
• HLA is designed to support reuse and
  interoperability of simulation models through its
  rules, interface specification and object model
  template.

13
HLA vs. Customized Protocol

• Features of High Level Architecture
• Each federate has a simulation object model (SOM)
  that defines the data it is willing to share with
  other federates.
• The federation (set of federates) has a common
  federation object model (FOM).
• With its capability defined by its SOM, a
  federate may be reused in different federations.
• HLA is designed to support distributed
  simulations linking the federates of a federation
  over a LAN or the Internet.
• Time Management can be used to ensure the correct
  ordering of events.
• HLA is an IEEE (1516) and OMG standard.

14
HLA vs. Customized Protocol

• Option 2 Customized Distributed Simulation
  Protocol
• As each federate has its own simulation time, a
  synchronization protocol is required to preserve
  causality.
• A customized distributed simulation protocol can
  be developed based on an existing protocol such
  as the conservative null message approach.
• A standard message passing library such as MPI
  can be used for communication over a LAN or
  Internet.
• Encapsulation of information within the federate
  can be achieved by specifying the interactions
  and data that can be sent as external events.

15
HLA vs. Customized Protocol
16
Simulation over the Internet

• Simulation using HLA-RTI between Singapore and UK
• Three Sun workstations at NTU SGI Sun at
  Oxford.

17
Hierarchical Federations Architecture

• Cluster Based Architecture
• In many distributed simulation applications, the
  individual federates are found to be organized
  into groups.
• Communication traffic within a group is generally
  higher than that between groups, due to closer
  physical and logical proximity.
• Related federates can therefore be grouped into
  clusters, where each cluster is supported by a
  high-speed communication link.
• Each cluster has its own RTI session, with an
  application gateway connecting it to other
  clusters.

18
Hierarchical Federations Architecture
19
Scalability

• Gateway Data Filtering
• The transmission of irrelevant data between
  clusters can be avoided since data filtering
  algorithms can be implemented efficiently at the
  individual gateways.
• Gateway Packet Bundling
• Packet bundling techniques can be implemented at
  the gateways to reduce the bandwidth
  requirements.
• Time Management
• Hierarchical federations can provide more
  efficient time management as the federates are
  more loosely synchronized and lookahead
  restrictions may be relaxed.

20
Reuse and Interoperability

• Heterogeneous Federations
• A cluster based approach can support
  heterogeneous federations, each with its own FOM
• Different federations/clusters may have FOMS at
  different levels of resolution.
• Heterogeneous FOMS allow interoperability with
  legacy simulations, where it is infeasible to
  develop new FOMS.
• Heterogeneous RTIs
• Different RTI implementations may be used for
  different clusters within the hierarchy
• A single RTI might not support all the hardware
  platforms used.
• Some clusters may benefit from specialized RTIs.

21
Data Security

• Information Hiding
• A gateway can provide information hiding, by
  filtering out sensitive data that should not be
  transmitted to other clusters.
• While the HLA allows data to be encapsulated
  within the SOM of a single federate, it does not
  provide a mechanism whereby a subset of federates
  may share information.
• Hierarchical federations allow a group of
  federates within a cluster to share sensitive
  information without making this visible to
  federates outside the group.
• For information hiding, a hybrid gateway/proxy
  architecture has been developed.

22
Conclusions

• Federating Parallel Simulators
• Can provide a solution to the requirement of fast
  execution.
• High Level Architecture
• Facilitates reuse and interoperability of
  component models and supports geographically
  distributed simulations.
• Hierarchical Federations Architecture
• Improves the scalability of distributed
  simulations through bandwidth reduction and
  improved time management.
• Increases the reusability and interoperability
  through heterogeneous federations and RTIs.
• Provides data security through information hiding.

23
Future Research

• Consistency
• Maintaining a consistent view of the virtual
  world.
• Characterization of inconsistency and the
  development of techniques for latency hiding.
• Time Management
• Efficient simulation even with zero or very small
  lookahead.
• Development of alternative mechanisms, e.g.
  causal ordering.
• Reuse and Interoperability
• Tools to support reuse and interoperability at
  the semantic level.
• Automatic generation of gateways in hierarchical
  federations.
• Verification and validation of hierarchical
  federations.

24
The End
Thank You!
HLA-RTI Internet
Questions?
Further Information htttp//www.ntu.edu.sg/home/AS
SJTurner