GPU

1
GPU
2
What it is?

• Helps to create nice pictures?
• A very strong processor?
• A very fast processor?
• A way to make us pay more?

3
Terms to know
? CPU Central Processing Unit ? GPU Graphics
Processing Unit ? GPGPU General Purpose
computation on the GPU ? FLOPS Floating point
Operations Per Second ? GFLOPS Giga-FLOPS
4
Roadmap

• Intro
• Graphics pipeline
• Why is CPU insufficient?
• Why is the solution?
• History evolution of the pipeline
• GPUs
• How they work
• How to use them
• GPU vs. CPU
• GPGPU

5
Graphics pipeline
Transformation / Lighting / Rasterization
6
Transformation

• Model space ? World space
• World space ? Camera space
• Camera space ? Projection space
• Projection space ? Screen space

7
Lighting

• Given
• Vertex properties
• Color
• Normal
• Texture coordinates
• Light sources
• Compute final color

8
Rasterezation

• Given some points describing a primitive in
  screen space, find the set of covered pixels

9
Roadmap

• Intro
• Graphics pipeline
• Why is CPU insufficient?
• Why is the solution?
• History evolution of the pipeline
• GPUs
• How they work
• How to use them
• GPU vs. CPU
• GPGPU

10
Why not CPU?

• Graphics applications are
• Highly computing intensive
• High arithmetic to data ratio
• Massively parallel
• Data stream based
• Straightforward

11
Why not CPU?

• Graphics applications are
• Highly computing intensive
• High arithmetic to data ratio
• Massively parallel
• Data stream based
• Straightforward
• CPUs are
• Serial, one operation at a time
• Control flow based
• Designed to handle huge data volumes
• Flexible
• Obviously not specially adapted to graphics

12
The mess of modern CPU

• Out of order execution (72 in-flight
  instructions)
• Exceptions (addresses, floating points)
• Address translation
• Branch prediction
• Register renaming
• Cache management
• A lot of Silicon!!!

13
Close-up of an AMD
Only a fraction of the chip does computations
14
Roadmap

• Intro
• Graphics pipeline
• Why is CPU insufficient?
• Why is the solution?
• History evolution of the pipeline
• GPUs
• How they work
• How to use them
• GPU vs. CPU
• GPGPU

15
How should graphics work?

• Dont let CPU render the scene
• Pass the data (description of the 3D scene) to
  the graphics card
• Have the GPU render the scene and store it in the
  frame buffer
• Screen is updated from the frame buffer

16
GPU vs. CPU

• GPUs devote hardware to do certain computations
  extremely well (e.g. linear algebra).
• CPUs are versatile and general purpose. They do a
  lot of things with rather well.

17
Roadmap

• Intro
• Graphics pipeline
• Why is CPU insufficient?
• Why is the solution?
• History evolution of the pipeline
• GPUs
• How they work
• How to use them
• GPU vs. CPU
• GPGPU

18
Evolution of the pipeline

• From
• Full software
• Through
• Hardware rasterization
• Hardware TL
• Programmable hardware
• To
• Full programmable hardware
• And beyond

19
Evolution of the pipeline

• From
• Full software
• Through
• Hardware rasterization
• Hardware TL
• Programmable hardware
• To
• Full programmable hardware
• And beyond

Pushed by the entertainment market
20
Software pipeline

• All computations are performed on CPU

Quake, 1996
21
Hardware rasterizer

• 1996 3DFX releases VOODOO
• TL by CPU
• Hardware rasterization

Unreal, 1997
22
Hardware TL

• 1999 NVIDIA releases GeForce256
• TLR by videocard
• CPU performs animations and deformations

Quake III, 2000
23
Programmable pipeline

• 2001 First GPU GeForce3
• Vertex Stage is programmable
• Animations, deformation, etc. move to GPU

Unreal III, 2003
24
Full programmable pipeline

• ATI and nVidia
• Vertex and Pixel Stages are programmable
• Cinematic quality effects are computed by GPUs

Doom III, 2005
25
Whats next

• Real Time Cinema
• Free CPU resources
• Physical simulations

Final Fantasy, the spirit within
26
Whats next

• Real Time Cinema
• Free CPU resources
• Physical simulations

Far Cry
27
Roadmap

• Intro
• Graphics pipeline
• Why is CPU insufficient?
• Why is the solution?
• History evolution of the pipeline
• GPUs
• How they work
• How to use them
• GPU vs. CPU
• GPGPU

28
What is a GPU?

• Hardware support for graphics operations linear
  algebra, lighting, texture, rasterization, and
  others
• High level of parallelism 256 processes
• No cache coherency (up to 16 KB)

29
Parallel Computing

• ? Moore's Law
• Transistor count doubles every two years.
• Increase in transistor count is also a rough
  measure of computer processing speed.
• Has been given to everything that changes
  exponentially.
• ? New Moore's Law
• Microprocessors no longer get faster, just wider.

30
The Ox vs. Chicken Analogy

• Seymour Cray If you were plowing a field, which
  would you rather use a strong ox or 1024
  chickens?

31
The Ox vs. Chicken Analogy

• Seymour Cray If you were plowing a field, which
  would you rather use Two strong oxen or 1024
  chickens?
• Chicken is winning these days
• For many applications, you can run many cores at
  lower freq and come ahead at the speed game.
• For example decrease freq by 20 ? 50 cut in
  power ? can add one more dumb core (chicken) ?
  power budget stays the same but with increased
  performance!

32
GPU in the classic architecture
33
Schematic view
TL
34
Vertex and Fragment Shaders
35
Vertex processor

• Vertex transformation
• Normal transformation and normalization
• Texture Coordinates Generation and transformation
• Lighting (per vertex)
• Color Material (per vertex)

36
Vertex processor
37
Vertex processor
OSG data
OSG modes and attributes
38
Fragment processor

• Scan conversion
• Texture computation
• Blending

39
Geometry shader

• Latest GPU (2008
• Input entire primitive line, triangle
• Output more primitives
• Applications
• Fur simulation
• Geometry processing (for instance widen lines)

40
Programmable processors

• Operate on a single vertex / pixel
• No vertex / pixel information can effect another
  vertex / pixel

41
Programmable GPU pipeline
42
Hardware
43
Hardware
44
Roadmap

• Intro
• Graphics pipeline
• Why is CPU insufficient?
• Why is the solution?
• History evolution of the pipeline
• GPUs
• How they work
• How to use them
• GPU vs. CPU
• GPGPU

45
GPU Programming models
46
GPU Programming models
Linear algebra BLAS, LAPAC, TAUCS, CHOLMOD
OSG
47
OpenGL / GLSL

• An abstraction
• Does not specify what happens
• Specifies what appears to happen
• Not a language, an API or State machine
• The way to communicate with hardware

48
OpenGL / GLSL

• Specifies data XYZ type
• Sets modes, attributes, and parameters
• Allows to rewrite the vertex / fragment /
  geometry shader
• Performs the calculations in software if no GPU
  is present

49
Roadmap

• Intro
• Graphics pipeline
• Why is CPU insufficient?
• Why is the solution?
• History evolution of the pipeline
• GPUs
• How they work
• How to use them
• GPU vs. CPU
• GPGPU

50
GPU vs. CPU
51
GPU vs. CPU
52
Why graphics cards?

• Pros
• Fast
• Cheap
• Low power

• Cons
• Specialized
• Hard to program
• Rapidly changing
• One way

53
Roadmap

• Intro
• Graphics pipeline
• Why is CPU insufficient?
• Why is the solution?
• History evolution of the pipeline
• GPUs
• How they work
• How to use them
• GPU vs. CPU
• GPGPU

54
General Purpose GPU

• Motivation
• Modern GPU are massive computation platforms
• They can reduce the load from CPU by performing
  computation
• Perfect for problems requiring parallel
  processing of data
• Key idea
• Take a general problem and transform it to images
  / graphics

55
GPU examples

• Solving PDE
• Fluid dynamics
• Image / signal processing
• Financial forecasting

56
Problems

• GPUs are designed for independent data
• Temporarily registers are zeroed
• No shared or static data
• No read-modify-write buffers