Adaptive Parallel Computation Over Dynamic and Heterogeneous Networks

1
Adaptive Parallel Computation Over Dynamic and
Heterogeneous Networks
Kaoutar El Maghraoui, Carlos Varela, Boleslaw
Szymanski, James Teresco(1). Worldwide Computing
Lab, Department of Computer ScienceRensselaer
Polytechnic Institutehttp//www.cs.rpi.edu/wwc/
SALSA-MPI Components
Abstract
SALSA/IO architecture

• The Message Passing Interface (MPI) Paradigm
• A set of processes communicating via MPI
  functions
• Restricted to tightly coupled environments such
  as MPPs and clusters.
• Dynamic Grid Paradigm
• Heterogeneous wide area environments
• Highly dynamic environments Resources may join
  and leave at anytime of the computation
• MPI lacks the ability to adapt to the dynamic
  grid world.
• SALSA/IO-MPI Paradigm
• An actor programming language approach to unify
  both parallel and distributed computing
• Support for migration and a middleware for
  dynamic reconfiguration and load balancing over
  widely-distributed networks.

List of nodes
mpirun
MPI Apps
An initial List of peer nodes can be supplied
through mpirun.
An IO Peer Servers
MPI API Calls
Joins the IO network
Create a proxy actor
SALSA-MPI Proxy Actor
SALSA Device
Create, start, and checkpoint MPI processes
Decision Agent
Profiling Actor
According to the decision function, the
profiling agent decides whether to migrate or not
a running MPI process
Keeps monitoring the performance of running MPI
processes
Project Overview
MPI Process
Proposed SALSA-MPI Architecture

• Goal
• provide an infrastructure that allows traditional
  MPI programs to take advantage of dynamic grid
  computational resources.
• The SALSA/IO architecture
• SALSA an actor-oriented programming language
• WWC a distributed runtime environments
• IO A middleware infrastructure for autonomous
  load balancing
• A highly dynamic Infrastructure
• Profiling of active actors resources.
• Migration of actors to optimal execution
  environments if the decision function predicts an
  increase in performance.
• Dynamic addition and removal of nodes in the
  computation.
• SALSA-MPI
• An implementation of the MPI standard that allows
  MPI processes to use the IO middleware for
  dynamic reconfiguration and load balancing.
• The SALSA device
• Intra-machine vendor supplied MPI implementations
  and inter-machine TCP communication.
• Interface to SALSA/IO proxy actors.
• Checkpointing of MPI processes
• Process Serialization and Migration
• Minimal changes to MPI applications

MPI Process
MPI App
MPI App
MPI App
MPI App
MPI Application
MPI Process
Actor wrapped MPI process during migration
MPI API
MPI Implementation
Checkpointing API
The migrating MPI Process Is restarted in
Destination node
Abstract Device Interface (ADI)
References
SALSA Device
IO Middleware services
SALSA Communication Services

• Travis Desell, Kaoutar El Maghraoui, and Carlos
  Varela Load Balancing of Autonomous Actors over
  Dynamic Networks. To appear in proceeding of
  HICSS-37 (2004).
• Varela and Agha, Programming Dynamically
  Reconfigurable Open Systems with SALSA. OOPSLA
  2001 Intriguing Technology Track Proceedings, ACM
  SIGPLAN Notices 36(12)20-34. December, 2001.
• MPI Official web site http//www-unix.mcs.anl.gov
  /mpi/

Migration
Checkpointing
JAVA Process Serialization
Vendor-supplied devices

Ch-p4
Ch-shmem
Globus2
(1) James Teresco is currently at the Department
of Computer Science, Williams College
Transport Layer
Network Layer