Parallel Simulation of Continuous Systems: A Brief Introduction

1
Parallel Simulation of Continuous SystemsA
Brief Introduction

• Oct. 19, 2005
• CS6236 Lecture

2
Background
Computer simulations

• Sample applications of continuous systems
• Civil engineering building construction
• Aerospace engineering aircraft design
• Mechanical engineering machining
• Systems biology heart simulations
• Computer engineering semiconductor simulations

Discretemodels
Continuous models
3
Outline

• Mathematical models and methods
• Parallel algorithm methodology
• Some active research areas

4
Mathematical Models

• Ordinary/partial differential equations
• Laplace equation
• Heat (diffusion) equation
• Steady-state v.s. time-dependent
• Convert into discrete problem through numerical
  discretization
• Finite difference methods structured grids
• Finite element methods local basis functions
• Spectral methods global basis functions
• Finite volume methods conservation

5
Example 1-D Laplace Equation

• Laplace equation in one dimensionwith boundary
  conditions
• Finite difference approximationwith
• Jacobi iteration

6
Example 2-D Laplace Equation

• Laplace equation in two dimensionwith boundary
  conditions at four sides

7
Parallel Programming Model

• Parallel computation two or more tasks executing
  concurrently
• Task encapsulates sequential program and local
  memory
• Tasks can be mapped to processors in various
  ways, including multiple tasks per processor

8
Performance Considerations

• Load balance work divided evenly
• Concurrency work done simultaneously
• Overhead work not present in serial computation
• Communication
• Synchronization
• Redundant work
• Speculative work

9
Example 1-D Laplace Equation

• Define n tasks, one for each yi
• Program for task i, i1,,n
• Initialize yifor k1, if igt1, send yi to
  task i-1 if iltn, send yi to task i1 if iltn,
  recv yi1 from task i1 if igt1, recv yi-1 from
  task i-1 yi (yi-1yi1)/2 end

10
Design Methodology

• Partition (Decomposition) decompose problem into
  fine-grained tasks to maximize potential
  parallelism
• Communication determine communication pattern
  among tasks
• Agglomeration combine into coarser-grained
  tasks, if necessary, to reduce communication
  requirements or other costs
• Mapping assign tasks to processors, subject to
  tradeoff between communication cost and
  concurrency

11
Design Methodology
12
Types of Partitioning

• Domain decomposition partition data
• Example grid points in 1-, 2-, or 3-D mesh
• Functional decomposition partition computation
• Example components in climate model (atmosphere,
  ocean, land, etc.)

13
Example Domain Decomposition

• 3-D mesh can be partitioned along any combination
  of one, two, or all three of its dimensions

14
Partitioning Checklist

• Identify at least an order of magnitude more
  tasks than processors in target parallel system
• Avoid redundant computation or storage
• Make tasks reasonably uniform in size
• Number of tasks, rather than size of each task,
  should grow as problem size increases

15
Communication Issues

• Latency and bandwidth
• Routing and switching
• Contention, flow control, and aggregate bandwidth
• Collective communication
• One-to-many broadcast, scatter
• Many-to-one gather, reduction, scan
• All-to-all
• Barrier

16
Communication Checklist

• Communication should be reasonably uniform across
  tasks in frequency and volume
• As localized as possible
• Concurrent
• Overlapped with computation, if possible
• Not inhibiting concurrent execution of tasks

17
Agglomeration

• Communication is proportional to surface area of
  subdomain, whereas computation is proportional to
  volume of subdomain
• Higher-dimensional decompositions have more
  favorable communication-to-computation ratio
• Increasing task sizes reduces communication but
  also reduces potential concurrency and flexibility

18
Surface-to-Volume Ratio
19
Example Agglomeration

• Define p tasks, each with n/p of yis
• Program for task j, j1,...p initialize
  yl,...,yh for k1,... if jgt1, send yl to task
  j-1 if jltp, send yh to task j1 if jltp, recv
  yh1 from task j1 if jgt1, recv yl-1 from task
  j-1 for il to h zi (yi-1yi1)/2 end y
  z end

20
Example Overlap Comm/Comp

• Program for task j, j1,...p initialize
  yl,...,yh for k1,... if jgt1, send yl to task
  j-1 if jltp, send yh to task j1 for il1 to
  h-1 zi (yi-1yi1)/2 end if jltp, recv
  yh1 from task j1 zh (yh-1yh1)/2 if
  jgt1, recv yl-1 from task j-1 zl
  (yl-1yl1)/2 y z end

21
Mapping

• Two basic strategies for assigning tasks to
  processors
• Place tasks that can execute concurrently on
  different processors
• Place tasks that communicate frequently on same
  processor
• Problem These two strategies often conflict
• In general, finding optimal solution to this
  tradeoff is NP-complete, so heuristics are used
  to find reasonable compromise
• Dynamic vs static strategies

22
Mapping Issues

• Partitioning
• Granularity
• Mapping
• Scheduling
• Load balancing
• Particularly challenging for irregular problems
• Some software tools Metis, Chaco, Zoltan, etc.

23
Example Atmosphere Model

• Partitioning
• grid points in 3-D
  finite difference model
• Typically yields 105 to 107 tasks
• Communication
• 9-point stencil horizontally and 3-point stencil
  vertically
• Physics computations in vertical columns
• Global operations to compute total mass

24
Example Atmosphere Model
25
Other Equations

• Heat (diffusion) equation
• Laplace equation
• Advection equation
• Wave equation
• Classification of second-order equations
• Parabolic, hyperbolic, and elliptic
• Methods for time-dependent equations
• Explicit v.s. implicit
• Finite-difference, finite-volume, finite-element

26
CFL Condition for Stability

• Necessary condition named after Courant,
  Friedrichs, and Lewy
• Computational domain of dependence must contain
  physical domain of dependence
• Implies time step must satisfy

27
Active Research Areas

• DES of continuous systems

28
Active Research Areas

• Coupling of different physics
• Different mathematical models
• Continuous v.s. discrete techniques
• Load balancing
• Manager-worker model
• Irregular/unstructured problems
• Dynamic load balancing

29
Summary

• Mathematical models for continuous systems
• Ordinary and partial differential equations
• Finite difference, finite volume, and finite
  element
• Parallel algorithm design
• Partitioning
• Communication
• Agglomeration
• Mapping
• Active research areas

30
References

• I. T. Foster, Designing and Building Parallel
  Programs, Addison-Wesley, 1995
• A. Grama, A. Gupta, G. Karypis, and V. Kumar,
  Introduction to Parallel Computing, 2nd. ed.,
  Addison-Wesley, 2003
• M. J. Quinn, Parallel Computing Theory and
  Practice, McGraw-Hill, 1994
• K. M. Chandy and J. Misra, Parallel Program
  Design A Foundation, Addison-Wesley, 1988