159.403/703 Parallel Computing

1
159.403/703 Parallel Computing

• Section 5
• Characteristics of Tasks and Interactions

2
Parallel Algorithm Design

• Decomposition (techniques are for us)
• Identify the concurrency
• Decompose it into tasks
• Mapping
• Take tasks and assign them onto available
  processors
• Properties of tasks and inter-task interactions

3
Characteristics of Tasks

• Following four have a large influence on the
  suitability of a mapping scheme
• Task Generation
• Static task generation all tasks are known
  before the algorithm starts execution,e.g.
• Data decomposition (matrix-multiplication)
• Some recursive decomposition, such as finding the
  minimum
• Dynamic task generation the actual task and the
  task-dependency graph are not explicitly
  available a priori, e.g.recursive decomposition
  in quicksort

4
Task Sizes

• The size of a task is the relative amount of time
  required to complete it
• The complicity of mapping schemes often depends
  on if the tasks are uniform
• Uniform e.g. matrix-mutliplication
• Non-uniform e.g. quicksort

5

• Knowledge of task size
• Known, e.g. matrix-mutliplication
• Unknown e.g. 15-puzzle
• Size of data associated with tasks
• the size and location of these data may determine
  the process that perform the task without
  incurring excessive data-movement overheads

6
Characteristics of Inter-task Interactions (ITI)

• Static versus Dynamic
• based on interaction pattern
• Static if for each task the interactions happen
  at predetermined times, and if the set of tasks
  to interact with these times is known prior to
  the execution of the algorithm
• Static interaction is to programmed easily in the
  message-passing paradigm
• Shard-address space programming can code both
  easily.
• Matrix multiplications vs. 15-puzzle

7
Characteristics of ITI (cont)

• Regular vs. Irregular
• Based on their spatial structure
• An interaction pattern is considered to be
  regular if it has some structure that can be
  exploited for efficient implementation.
• Image dithering vs. sparse matrix-vector
  multiplications

8

• Read-only vs. read-write
• Based on the characterizes of data sharing
• In read-only interactions, tasks require only a
  read-access to the data shared among many
  concurrent tasks

9

• One-way vs. two-way
• The data or work needed by a task or subset of
  tasks is explicitly supplied by another task or
  subset of tasks, such interaction are called
  two-way interaction.
• All read-only can be formulated as one-way
  interaction.
• Read-write can be either one-way or two-way
  interaction.