Introduction to Parallel Processing

1
Introduction to Parallel Processing

• CS147
• Tung Sze Ming

2
Topics Covered

• An Overview of Parallel Processing
• Parallelism in Uniprocessor Systems
• Parallelism in Multiprocessor Systems
• Flynns Classification
• System Topologies

3
An Overview of Parallel Processing

• What is parallel processing?
• Parallel processing is a method to improve
  computer system performance by executing more
  than one instructions at the same time.

4
Misconception

• When speaking of parallel processing, most people
  would think of multi-processor systems

5
Parallelism in Uniprocessor Systems

• It is possible to achieve parallelism with a
  uniprocessor system.
• Some examples are the instruction pipeline,
  arithmetic pipeline, I/O processor.
• Only if the system processes two different
  instructions simultaneously can it be considered
  parallel
• A system that performs different operations on
  the same instruction is not considered parallel.

6
Parallelism in a Uniprocessor System

• A reconfigurable arithmetic pipeline is an
  example of parallelism in a uniprocessor system.
• Each stage of a reconfigurable arithmetic
  pipeline has a multiplexer at its input. The
  multiplexer may pass input data, or the data
  output from other stages, to the stage inputs.
  The control unit of the CPU sets the select
  signals of the multiplexer to control the flow of
  data, thus configuring the pipeline.

7
A Reconfigurable Pipeline With Data Flow for the
Computation Ai ? Bi Ci Di
To memory and registers
0 1 MUX 2 3 S1 S0
0 1 MUX 2 3 S1 S0
0 1 MUX 2 3 S1 S0
0 1 MUX 2 3 S1 S0
LATCH
LATCH
LATCH
Data Inputs
0 0
x x
0 1
1 1
8

• Although arithmetic pipelines can perform many
  iterations of the same operation in parallel,
  they cannot perform different operations
  simultaneously. To perform different arithmetic
  operations in parallel, a CPU must include a
  vectored arithmetic unit.

9
Vector Arithmetic Unit

• A vector arithmetic unit contains multiple
  functional units that perform addition,
  subtraction, and other functions. The control
  unit routes input values to the different
  functional units to allow the CPU to execute
  multiple instructions simultaneously.

10
A Vectored Arithmetic Unit
Data Input Connections
Data Input Connections

-
Data Inputs


A?BC D?E-F
11
Parallelism in Multiprocessor Systems

• Parallel processing systems achieve parallelism
  by having more than one processor performing
  tasks simultaneously. Since multiprocessor
  systems are more complicated than uniprocessor
  systems, there are many different ways to
  organize the processors and memory

12
Flynns Classification

• SISD Single instruction with single data
• SIMD Single instruction with multiple data
• MISD Multiple instruction with single data
• MIMD Multiple instruction with multiple data

13
Topologies

• Topology of a multiprocessor system refers to
  the pattern of connections between its
  processors. Various factors, typically involving
  a cost-performance tradeoff, determine which
  topology a computer designer will select for a
  multiprocessor system.

14
Types of Topology

• Shared Bus Topology
• Ring Topology
• Tree Topology
• Mesh Topology
• Hypercube Topology
• Completely connected Topology

15
Shared Bus Topology

• processors communicate with each other
  exclusively through this bus. However, the bus
  can only handle only one data transmission at a
  time. In most shared busses, processors directly
  communicate with their own local memory.

16
Ring Topology

• The ring topology uses direct connections between
  processors instead of a shared bus. This allows
  all communication links to be active
  simultaneously. Data may have to travel through
  several processors to reach its destination

17
Tree Topology

• Like the ring, it uses direct connections between
  processors each having three connections. There
  is only one unique path between any pair of
  processors

18
Mesh Topology

• In the mesh topology, every processor connects to
  the processors above and below it, and to its
  right and left.

19
Hypercube Topology

• The hypercube is a multidimensional mesh
  topology. Each processor connects to all other
  processors whose binary values differ by one bit.

20
Completely connected Topology

• In the most extreme connection scheme, the
  processors are completely connected. Every
  processor has (n-1) connections, one to each of
  the other processors. This increases the
  complexity of the processors as the system grows,
  but offers maximum communication capabilities.

21

• THE END