Parallel Computers Today

1
Parallel Computers Today
Two Nvidia 8800 GPUs gt 1 TFLOPS
LANL / IBM Roadrunner gt 1 PFLOPS
Intel 80-core chip gt 1 TFLOPS

• TFLOPS 1012 floating point ops/sec
• PFLOPS 1,000,000,000,000,000 / sec (1015)

2

• Columbia (10240-processor SGI Altix, 50
  Teraflops, NASA Ames Research Center)

3

• Beowulf (18-processor cluster, lab machine)

4

• AMD Opteron quad-core die

5
The nVidia G80 GPU

• 128 streaming floating point processors _at_1.5Ghz
• 1.5 Gb Shared RAM with 86Gb/s bandwidth
• 500 Gflop on one chip (single precision)

6
The Computer Architecture Challenge

• Most high-performance computer designs allocate
  resources to optimize Gaussian elimination on
  large, dense matrices.
• Originally, because linear algebra is the
  middleware of scientific computing.
• Nowadays, mostly for bragging rights.

P

x
7

• Top 500 List
• http//www.top500.org/list/2008/11/100

8
Generic Parallel Machine Architecture
Storage Hierarchy
Proc
Proc
Proc
Cache
Cache
Cache
L2 Cache
L2 Cache
L2 Cache
L3 Cache
L3 Cache
L3 Cache
potential interconnects
Memory
Memory
Memory

• Key architecture question Where is the
  interconnect, and how fast?
• Key algorithm question Where is the data?

9
Multicore SMP Systems
10
More Detail on GPU Architecture
11

• Michael Perrone (IBM) Proper Care and Feeding
  of Multicore Beasts
• http//www.csm.ornl.gov/workshops/HPA/documents/1
  -arch/feeding_the_beast_perrone.pdf

12
Cray XMT (highly multithreaded shared memory)