No.1

1
EE692
Parallel and Distribution Computation Prof.
Song Chong
Ch. 1 Introduction
2
Issues in Parallelization

• Task allocation to processors
• The breakdown of the total workload in small
  tasks assigned to different processors
• Proper sequencing of the tasks when some of them
  are interdependent and cannot be executed
  simultaneously
• Communication of interim computation results
  between processors
• Synchronization of computations of processors
• Synchronous predetermined points for the
  completion of computations or for the arrival of
  data
• Asynchronous no such points
• Performance measures
• Effects of systems architecture on performance

3
Need for Parallel Distributed Computation

• Restrict attention to numerical computation in
  this lecture
• Large-scale fast computation
• Partial differential eq.s (PDEs)
• e.g.) fluid dynamics, weather prediction, image
  processing
• ? Can be decomposed along a spatial dimension
• Each processor manipulates the variables
  associated with a small region in space
• Interaction between variables are local in nature
• Systems of equations, mathematical programming
  (optimization)

4
Need for Parallel Distributed Computation
(Contd)

• Analysis, simulation and optimization of large
  scale interconnected systems
• e.g.) queueing system
• Information acquisition, extraction and control
  in geographically distributed system
• e.g.) sensor networks, communication networks,
  wireless networks
• Synchronization, unreliable communication,
  absence of a central control mechanism

5
Distinction between parallel and distributed
computing systems

• Parallel computing systems
• Processors are located within a small distance of
  each other
• Designed in such a way that processors execute
  jointly a computational task
• Communication between processors is reliable and
  predictable
• Distributed computing systems
• Processors may be far apart (geographically
  distributed)
• Communication delay may be unpredictable
• Communication links may be unreliable
• Topology may undergo changes while operating
• Usually loosely coupled, there is very little
  central coordination and control

6
Parameters to classify parallel and/or
distributed computing systems

• Type and number of processors
• Presence or absence of a global control mechanism
• Synchronous vs. asynchronous operation
• Processor interconnections
• Tow extremes
• Shared memory system
• solve inter-processor computation problem by
  lacing a global memory, but introduce the problem
  of simultaneous accessing of different locations
  of the memory by several processors
• Message-passing system
• local memory with inter-connection networks