Motivation: dynamic apps

1
Motivation dynamic apps

• Rocket center applications
• exhibit irregular structure, dynamic behavior,
  and need adaptive control strategies.
• Geometries are irregular
• Dynamic changes
• burning, pressurization, crack propagation,

2
Motivation modularity

• Rocket center apps are multicomponent
• Codes developed by different teams
• Different discretization schemes may be used,
  etc.
• Multiple alternative strategies may be
  implemented for individual components
• Need
• to develop reusable enabling technologies

3
Need for adaptive strategies

• Computation structure changes over time
• Combustion
• Adaptive techniques in application codes
• Adaptive refinement in structures or even fluid
• Other codes such as crack propagation
• Can affect the load balance dramatically
• One can go from 90 efficiency to less than 25

4
Multi-partition decomposition using objects

• Idea decompose the problem into a number of
  partitions,
• independent of the number of processors
• Partitions gt Processors
• The system maps partitions to processors
• The system should be able to map and re-map
  objects as needed

5
Supporting Multi-partition approach

• A Load balancing framework is needed to support
  such an approach
• Charm
• Migration support
• Automatic instrumentation and object-load
  database
• Re-mapping strategies

6
Charm

• A parallel C library
• Supports data driven objects
• singleton objects, object arrays, groups,
• Many objects per processor, with method execution
  scheduled with availability of data
• System supports automatic instrumentation and
  object migration
• Works with other paradigms MPI, openMP, ..

7
Data driven executionin Charm
Scheduler
Scheduler
Message Q
Message Q
8
Load Balancing Framework

• Aimed at handling ...
• Continuous (slow) load variation
• Abrupt load variation (refinement)
• Workstation clusters in multi-user mode
• Measurement based
• Exploits temporal persistence of computation and
  communication structures
• Very accurate (compared with estimation)
• instrumentation possible via Charm/Converse

9
Object balancing framework
10
Utility of the framework workstation clusters

• Cluster of 8 machines,
• One machine gets another job
• Parallel job slows down on all machines
• Using the framework
• Detection mechanism
• Migrate objects away from overloaded pe
• Restored almost original throughput!

11
Utility of the framework Intrinsic load imbalance

• To test the abilities of the framework
• A simple problem Gauss-Jacobi iterations
• Refine selected sub-domains
• ConSpector web based tool
• Submit parallel jobs
• Monitor performance and application behavior
• Interact with running jobs via GUI interfaces

12
(No Transcript)
13
How to utilize this framework for CSE
applications

• Explicit programming
• Conversion from existing MPI programs
• Higher level frameworks

14
Supporting explicit programming

• Libraries,
• Structured dagger
• Migratable threads

15
Conversion from existing MPI programs

• Is it possible? Yes, with a couple of tricks
• Thread based approach
• No-threads approach

16
Thread based approach

• Each MPI process
• becomes a chunk, with multiple chunks per pe
• Each chunk implemented as a thread
• within a Charm object
• Common MPI calls
• Implemented on top of Charm/threads
• Suspend threads instead of blocking
• Migration of threads tricky but supported

17
Conversion of MPI programs

• Collect all global variables
• in a "chunk" data structure
• (except read-only, for efficiency)
• Replace each occurrence
• of such a global variable x with chunkx
• Provide subroutines
• for packing and unpacking the chunk data
  structure into a buffer

18
No-threads approach

• Use an "irecv with continuation" library
• usual MPI style irecv waitall(f)
• Conversion
• All the steps of the thread approach
• Split subroutines at receives
• Faster context switching than threads, with some
  extra effort

19
Case studies

• ROCFLO using the threads approach
• Crack propagation code
• with a non-thread approach
• Relatively non--invasive, and within reasonable
  effort
• Reviewed/in progress
• ROCSOLID, OPAAL codes

20
Further up In Progress

• Automated conversion from MPI
• Compiler based approach
• FEM framework
• Ghost array framework
• Load balancing strategies
• Component libraries

21
Further up plans

• Geometric Interfaces across modules
• interpolation, data structures
• Support for on-line re-meshing
• Support for Visualization
• Parallel I/O components
• Orchestration
• Controlling simulation at a higher level

22
Higher level framework
Automatic Conversion from MPI
Cross module interpolation
Structured
FEM
MPI-on-Charm
Irecv
Frameworkpath
Load database balancer
Migration path
Charm
Converse