Introduction 1

1
Design of Parallel Programs

• Course 2IN10
• College Rudolf Mak r.h.mak_at_tue.nl
• Werkcollege Rudolf Mak
• Peter Veltkamp j.p.veltkamp_at_
  tue.nl
• Practicum Peter Veltkamp
• Rudolf Mak
• Studiewijzer
• http//www.win.tue.nl/wsinmak/Education/2IN10

2
Course organization

• Blok A
• 5 x 2 uur college
• 5 x 2 uur werkcollege
• Blok B
• 4 x 2 uur toelichting practicum opdrachten
• 4 x 2 uur practicum
• Afrekening
• Schriftelijk examen
• Practicum verslag
• Deelresultaten voorgaande jaren blijven geldig

3
Course Overview

• Scope and context of the course
• Concepts and definitions
• Specification and programming language
• Performance metrics
• Design strategies
• Finite linear systems
• Tree-like systems, infinite systems

4
Scope and Context

• Motivation
• Machine model
• Programming model
• Related literature

5
Motivation

• Two important branches of parallel computing are
• High performance computing
• This field is concerned with computationally
  expensive
• (scientific) computations. The main reason to
  consider
• parallelism is computation speed.
• Distributed computing
• This field is concerned with the control of
  complex dis-
• tributed systems. In this case parallelism is
  inherent,
• because the program has to react timely to
  asynchro-
• nous simultaneous stimuli from its environment.

6
Machine Models

• A machine model contains information about the
  nature,
• number, and performance characteristics of the
• Processing units
• Storage units
• Interconnection network
• The machine model influences the choice of
  programming.
• style and the performance analysis of programs.

7
Flynn Taxonomy

• Single Instruction Single Data (SISD)
• Single Instruction Multiple Data (SIMD)
• Multiple Instruction Single Data (MISD)
• Multiple Instruction Multiple Data (MIMD)

8
Multi-computer
A mesh-connected, distributed memory machine
(MIMD)
9
Multiprocessor (UMA)
A shared-memory machine with uniform memory access
10
Programming Model

• Important aspects of a model for parallel
  programming are
• Methodology
• algorithmic parallelism, data parallelism,
  processor farming
• Communication
• message passing or shared variables
• data dependent or data independent behavior
• Grainsize
• fine-grained or coarse-grained or parametric
  grain size

11
Systolic Computation

• A systolic computation is a computation performed
  on a
• multi-computer, also called a systolic array,
  that satisfies
• the following criteria
• Cells are simple and communicate via a fixed
  number of channels
• Communication takes place via uni-directional
  channels
• The network is a regular arrangement of cells
• The communication behavior is independent of the
  values sent and received
• Cells synchronize by message passing only

12
What we do not discuss

• Mapping problem
• Reactive programs (probes, passive processes)
• Dependability
• Safety issues
• Real-time issues

13
Literature

• M.Rem, Lecture Notes in Parallel Programming,
  TU/e dictaat nr 2519, 1995 ( included
  references).
• Michael J. Quinn, Parallel computing theory and
  practice, McGraw-Hill, 1994.
• Ian T. Foster, Designing and Building Parallel
  Programs, Addison-Wesley, 1995.
• Gregory R. Andrews, Foundations of Multithreaded,
  Parallel, and Distributed Programming,
  Addison-Wesley, 1999.
• Ananth Grama et. al., Introduction to Parallel
  Computing, Second Edition, Addison-Wesley, 2003.

14
Literature

• John L. Hennessy and David A. Patterson, Computer
  Architecture A quantitative Approach, 2nd ed.,
  Morgan Kaufmann, 1996.
• Gregory F. Pfister, In Search of Clusters,
  Prentice Hall, 1998.
• Harry F. Jordan and Gita Alaghband, Fundamentals
  of Parallel Processing, Prentice Hall, 2003.
• David Skillikorn and Dominico Talia, Models and
  languages for Parallel Computation, ACM Computing
  Surveys, Vol. 30. Nmb 2, June, 1998.