Recap

1
Recap
2
Computational Power Improvement
Multiprocessor
Uniprocessor
C.P.I
1 2 . . . .
No. of Processors
3
Summing m Numbers

• Example summing m numbers
• On a sequential computer we have,
• sum a0
• for (i1i
• sum sum ai
• Would expect the running time be be roughly
  proportional to m.

4
Summing m Numbers in Parallel

• What if we have N processors, with each
  calculating the m/N numbers assigned to it?
• We must add these partial sums together to get
  the total sum.

5
Summing Multiprocessors
Suppose we have N processors.

• The algorithm is as follows
• Each processor finds the local sum of its m/N
  numbers
• Each processor passes its local sum to another
  processor in a coordinated way
• The global sum is finally in processor P11.

P11
P13
P12
P21
P23
P22
i
P31
P33
P32
j
6
Shared Memory Multiprocessors
Shared Memory
7
MPP (Massively Parallel Processing)Distributed
Memory Multiprocessors
MB Memory Bus NIC Network Interface Circuitry
MB
MB
P/C
P/C
LM
LM
NIC
NIC
Custom-Designed Network
8
Cluster
LD Local Disk IOB I/O Bus
MB
MB
P/C
P/C
M
M
Bridge
Bridge
LD
LD
IOB
IOB
NIC
NIC
Commodity Network (Ethernet, ATM, Myrinet)
9
Grid
P/C
P/C
P/C
P/C
IOC
IOC
Hub/LAN
Hub/LAN
NIC
NIC
LD
LD
SM
SM
SM
SM
Internet
10
TOP500 architectures