Introduction to Parallel Computing

1
Introduction to Parallel Computing

• Sunggu Lee
• Office 2-415
• Phone [phone]
• Email slee_at_postech.ac.kr

2
Types of Parallel Computers

• Shared Memory
• Multiprocessor
• Symmetric Multiprocessor (SMP)
• Normally Bus Based
• Identical Memory Access Time for all CPUs
• Distributed Memory
• Multicomputer
• Networked Computers
• PC Cluster
• Grid Computer

3
Flynns Classification

• Based on Number of Instruction and Data Streams
• Categories
• SISD
• Single Instruction, Single Data
• SIMD
• MISD
• MIMD

4
Architectural Features

• Static Network (direct, point-to-point)
• Ring, Mesh, Torus
• Tree, Fat Tree
• Hypercube, Cube Connected Cycles, Etc.
• Dynamic Network
• Embedding
• Inter-PE Communication Methods
• Circuit Switching, Packet Switching
• Wormhole Routing, Virtual Circuit Cut-through

5
Networked Computers

• Common Names
• Cluster, NOW, COW, Beowulf
• Uses Commodity Computer Networks
• Bus-based
• Ethernet
• Switch-based
• Switched Ethernet
• Myrinet
• SCI (Scalable Coherent Interface)

6
Performance Evaluation

• Process Granularity
• Speedup
• Efficiency
• Cost
• (execution time) (num. Processors )
• Scalability
• Ahmdahls Law
• Gustafsons Law (Scaled Speedup)