CSc48206820 Computer Graphics Algorithms Ying Zhu Georgia State University

1
CSc4820/6820 Computer Graphics AlgorithmsYing
ZhuGeorgia State University

• Lecture 18 and 19
• GPU and
• Programmable Shaders

2
Outline

• What is GPU?
• Vertex shaders
• Pixel shaders
• How to develop shaders?
• High level shading languages
• How to control the GPU from the CPU?
• 3D API and shader interfaces

3
Acknowledgement

• Some of the pictures and texts in the following
  slides are taken from
• NVIDIA, ATI, and 3Dlabs presentation slides
• Toms Hardware Guide (http//www.tomshardware.com/
  )

4
What is GPU?

• Graphics Processing Unit (GPU) is the
  microprocessor on a graphics card.
• GPU has access to graphics memory on the graphics
  card
• GPU is connected to the CPU and main memory via
  AGP or PCI-Express bus

NVIDIA GeForce 7800 GTX
ATI Radeon X1800
5
A little history

• Before 1995, SGI was the dominate graphics
  hardware company
• Flagship product Onyx Reality Engine server
• Implements 3D graphics pipeline on hardware
• Very expensive
• Operating system IRIX
• PC graphics cards at that time were rather simple
  (compared with todays standard)
• No hardware 3D pipeline
• Almost all graphics operations are handled by CPU

6
GPU

• The first popular 3D graphics card was the Voodoo
  card by 3DFX (1995)
• Only texture mapping and Z-buffer were
  implemented on graphics card

7
GPU

• Gradually, graphics card companies began to
  implement more and more 3D pipeline functions on
  graphics cards
• By 2000, transformation and lighting were also
  implemented on graphics card (e.g. GeForce2)
• Hardware 3D pipeline on graphics card is complete

8
Programmable GPU

• In 2001, NVDIA introduced the first programmable
  GPU in GeForce3
• Still implements a fixed-function 3D pipeline
• Add support for vertex shader and texture shader
• Can write your own transformation lighting
  programs
• Only supports assembly language shaders

9
Programmable GPU

• In 2003, NVIDIA added pixel shader support in
  GeForceFX
• Pixel shader replaces texture shader
• New high level shading language Cg and HLSL

10
Programmable GPU

• In 2004, NVIDIA releases GeForce 6800 and 6600
• Vertex shaders can now access texture images
• PCI-Express bus

11
Geometry shader

• Geometry shaders are a new feature of DirectX 10
  that allow for dynamic creation of geometry on
  the GPU.
• With the introduction of geometry shaders, the
  GPU can now create and destroy vertices output
  from the vertex shader.
• This capability allows for new features such as
  displacement mapping with tessellation and
  GPU-generated shadow volumes.
• Tutorial
• http//appsrv.cse.cuhk.edu.hk/ymxie/Geometry_Shad
  er/

12
United Shader Model

• Introduced in 2006
• Supported in GeForce 8xxx, Radeon HD 2000 series
  and Intel GMA X3000 series
• Also known as Shader Model 4.0
• Uses a unified Instruction Set Architecture
  across all shader types

13
Why unified shader model?

• NVIDIA designed a single floating point shader
  core with multiple independent processors.
• Each of these independent processors is capable
  of handling any type of shading operation,
  including pixel shading, vertex shading, geometry
  shading, and physics shading.
• GeForce 8800 GPUs can dynamically allocate
  processing power depending on the workload of the
  application, providing unprecedented performance
  and efficiency.

14
Why unified shader model?

• http//www.extremetech.com/article2/0,1697,2053309
  ,00.asp

15
OpenGL 2.0

• In 2004, OpenGL 2.0 is officially released
• Added OpenGL Shading Language to support
  programmable GPU

16
Major GPU companies

• NVIDIA (www.nvidia.com)
• GeForce and Quadro series cards
• High level shading language Cg
• ATI (www.ati.com)
• Radeon and FireGL series cards
• 3DLabs (www.3dlabs.com)
• Wildcat series cards
• High level shading language OpenGL Shading
  Language (GLSL)

17
Why programmable GPU?

• Pre-GPU age fixed function graphics pipeline
• You can change the pipeline parameters in
  OpenGL/DirectX but not the algorithms.
• Programmable GPU gives freedom to developers
• Applications can take control of the processing
  on graphics hardware
• Never-before-seen effects are possible

18
Why programmable GPU?

• Much better graphics performance on GPU than on
  CPU.
• GPUs are designed for extreme speed
• Highly parallel optimized hardware for graphics
  processing
• CPUs are designed for flexibility
• Engineered for wide variety of applications
• For many graphics applications, real-time
  performance now becomes possible.
• Non-graphics applications can also take advantage
  of the powerful GPU.

19
Other Uses of GPU

• Beyond computer graphics application
• Use the powerful GPU as a second CPU
• Speed up non-realtime graphics algorithms
• Real-time ray tracing on GPU
• Real-time photon mapping on GPU
• Real-time radiosity on GPU
• GPU-based volume rendering, etc.
• Non-graphics applications
• Fast Fourier transform
• Interactive collision detection
• Bioinformatic computing
• More information at www.gpgpu.org

20
Major Trends

• Faster GPUs
• Performance doubles every 6 - 9 months
• Increased parallelism with each new generation of
  graphics processors
• Increasing programmability of graphics hardware
• Programmable Processing is the future of graphics
  and visual and cinematic computing
• Enhanced vertex and pixel shader capabilities
• Future shaders may have access to more 3D
  pipeline operations
• E.g. clipping, depth test, stencil test, etc.

21
NVIDIA GeForce 7800 GTX
22
GeForce 8800 GTX

• GeForce 8800 GTX architecture

23
ATI Radeon X1800
24
Whats in a GPU?

• The latest GPUs still implements a fixed-function
  graphics pipeline
• Controlled by OpenGL/Direct3D functions
• But there are two programmable units that can
  take over part of the fixed-function pipeline
• Vertex processor
• Pixel (fragment) processor
• Vertex and pixel processors are controlled by
  shaders

25
Vertex Shader

• A vertex shader is a small program that
  manipulates vertex data and runs on GPUs vertex
  unit
• Provides developer with almost unlimited control
  on how vertices are perturbed.
• Vertex shader can be written in high level
  shading language Cg, HLSL, or GLSL

26
Vertex Shader in Graphics Pipeline
CPU
GPU
27
What is in a vertex shader?

• Vertex Program
• Reads an untransformed, unlit vertex
• Creates a transformed vertex
• Optional operations
• Calculate lighting for vertex
• Creates texture coordinates
• Does not create or delete vertices
• 1 vertex in and 1 vertex out
• No topological information provided
• No edge, face, nor neighboring vertex info

28
Whats in a vertex shader?

• You can implement the following operations in a
  vertex shader
• Multiply model-view and projection matrices with
  vertex coordinates
• Transform normals to eye coordinates
• Normalize normals
• Calculate lighting for each vertex
• Multiply texture matrices with texture
  coordinates
• Access a texture map (shader model 3.0)

29
Whats not in a vertex shader?

• Currently the following operations cannot be
  implemented by a vertex shader
• Perspective division
• Clipping
• Viewport mapping
• Front face determination
• Flat shading or Gouraud shading

30
Why use vertex shader?

• Write your own transformation and lighting
  function
• Visual quality is much better than the fixed
  function Transformation Lighting
• Graphics developers have more control
• Take advantage of the powerful GPU

31
Why use vertex shader?

• Complete control of transform and lighting
  hardware
• Complex vertex operations accelerated in hardware
• Custom vertex lighting
• Custom skinning and blending
• Custom texture coordinate generation
• Custom texture matrix operations
• Custom vertex computations of your choice
• Offloading vertex computations frees up CPU
• More physics and simulation possible!

32
Vertex Shader Example

• Custom transform, lighting, and skinning

33
Vertex Shader Example

• Custom cartoon-style lighting

34
Vertex Shader Example

• Per-vertex set up for per-pixel bump mapping

35
Example

• Character morphing shadow volume projection

36
Example

• Dynamic displacements of surfaces by objects

37
Vertex shader

• You have to load and compile vertex shaders in
  your OpenGL (or DirectX) program
• Can load multiple vertex shaders
• Vertex shaders can be turned on and off like a
  state in OpenGL program
• E.g. turn on shader for some objects
• If turned on, each vertex shader will be executed
  once per frame for each of the vertices on the
  affected objects
• Your vertex shader may run hundreds of times per
  frame

38
What is pixel shader?

• A pixel (fragment) shader is a small program
  which processes pixels and executes on the GPUs
  pixel (fragment) processor
• A pixel shader can be written in high level
  shading language Cg, HLSL, or GLSL

39
Pixel Shader in Graphics Pipeline
CPU
GPU
40
Whats in a pixel shader?

• Pixel shader implements the following
  rasterization operations
• Texture mapping (including filtering, wrapping,
  texture environments, etc.)
• Color sum
• Fog

41
Switch between fixed function and pixel shader
42
Whats not in a pixel shader?

• Currently the following operations are not
  supported by pixel shader
• Blending
• All of the per-fragment operations

43
Why use pixel shader?

• Fully control the texture mapping/blending
  process instead of using the fixed-function
  texture mapping.
• Can generate much better quality images
• E.g. Per-pixel lighting
• Can also use pixel shader for non-graphics
  applications.

44
What can you do with pixel shader?

• Per-pixel lighting
• Looks much better than per-vertex lighting
• True phong shading
• Anisotropic lighting
• Non-Photorealistic-Rendering
• cartoon shading, hatching, gooch lighting, image
  space techniques
• Per-pixel fresnel term
• Volumetric effects
• Advanced bump mapping
• Procedural textures and texture perturbation
• Bidirectional reflectance distribution functions
• And more

45
Pixel Shader Example Natural Light
46
Pixel Shader Example Reflection
47
Pixel Shader Example Bump Mapping
48
Pixel shader

• You have to load and compile pixel shaders in
  your OpenGL (or DirectX) program
• Can load multiple pixel shaders
• Pixel shaders can be turned on or off like a
  state in OpenGL program
• E.g. turn on shader for some objects
• If turned on, each pixel shader will be executed
  once per frame for each of the pixels on the
  affected objects
• Your pixel shader may run hundreds of thousands
  of times per frame

49
How to develop shader programs?

• High level shading languages
• Cg
• Microsoft High Level Shader Language (HLSL)
• OpenGL Shading Language
• Supplementary tools
• ATIs RenderMonkey
• NVIDIA CgFX Composer
• OpenGL Shader Designer
• Assembly language (?)

50
High level shading languages

• Cg
• http//developer.nvidia.com/page/cg_main.html
• Developed by NVIDIA
• Best supported by NVIDIA GeForce cards
• Works on Windows, Linux, and Mac OS
• Looks like C language but with some restrictions
• Can be loaded in both OpenGL and DirectX programs

51
High level shading languages

• MS HLSL
• Part of DirectX 9.0 (http//www.microsoft.com/wind
  ows/directx/)
• Jointly designed by MS and NVIDIA
• Almost identical to Cg
• Supported by GeForce and Radeon cards
• Only runs on Windows
• Only works with DirectX programs

52
High level shading languages

• OpenGL Shading Language (GLSL)
• Developed by 3Dlabs, approved by OpenGL ARB
• http//developer.3dlabs.com/openGL2/index.htm)
• Now part of the OpenGL 2.0 specification
• Supported by latest GeForce and Radeon cards
• Expected to be cross platform
• Only works with OpenGL programs

53
Other high level language for GPU

• Sh
• http//www.libsh.org/
• A metaprogramming language for GPU
• Developed at University of Waterloo
• Brook for GPU
• http//graphics.stanford.edu/projects/brookgpu
• A stream program language for general purpose
  computing on GPU
• Developed at Stanford University

54
Shader development tools

• ATIs RenderMonkey
• http//www.ati.com/developer/rendermonkey/
• http//www2.ati.com/developer/rendermonkey/movies/
  RenderMonkey-Siggraph2004SketchVideo.mov
• A shader development environment for both
  programmers and artists
• Can be used for rapid prototyping shaders
• Supports HLSL and GLSL
• Runs best on ATI Radeon cards
• Free

55
Shader development tools

• NVIDIA FX Composer
• http//developer.nvidia.com/object/fx_composer_hom
  e.html
• http//developer.download.nvidia.com/presentations
  /2007/gdc/NVIDIA_FX_Composer_2.pdf
• Similar to RenderMonkey
• Save files in CgFX or HLSL FX format (A
  metaprogramming file format)
• Runs best on GeForce cards
• Only supports Cg
• Free

56
Interface between shader and 3D API

• Shaders have to be loaded and enabled during
  runtime by an OpenGL or DirectX program
• OpenGL/DirectX program need to interact with
  shaders
• How to create/load/delete shaders?
• How to compile shaders?
• How to enable/disable shaders?
• How to pass data to shaders?

57
Interface between shader and 3D API

• DirectX 9.0 has native functions to interface
  with both HLSL and Cg shaders
• OpenGLCg interface
• NVIDIA Cg runtime library
• NVIDIA OpenGL extensions
• OpenGLGLSL interface
• Newly added OpenGL 2.0 functions
• (older) OpenGL extensions
• OpenGLCgFX interface
• NVIDIA CgFX runtime library

58
Shader developers options

• Write shaders in GLSL, use OpenGL 2.0 functions
  to load/execute shaders
• Not many cards support this yet
• Write shaders in HLSL (e.g. using RenderMonkey),
  use native DirectX functions to load/execute
  shaders
• Windows only solution
• Write shaders in Cg, use NVIDIA OpenGL extensions
  to load/execute shaders
• OpenGL extensions are poorly documented

59
Shader developers options

• Write shaders in Cg, in OpenGL/DirectX program
  use NVIDIA Cg runtime to load/execute shaders
• Perhaps a better choice for now
• Better cross-platform support
• Easier to learn/use
• Develop shaders in NVIDIA FX Composer, save them
  in CgFX files, in OpenGL/DirectX program
  load/execute shaders using CgFX runtime

60
Cg and Cg Runtime a brief introduction

• Cg language
• A C-like language
• Has built-in functions and predefined structures
  to simplify GPU programming
• Cg runtime library
• API that allow OpenGL/DirectX programs to load,
  compile and link, and execute Cg programs at
  runtime

61
How to use Cg shaders?

• Use 3D API calls to specify vertex and pixel
  shaders
• Enable vertex and fragment shaders
• Load/enable texture as usual
• Draw geometry as usual
• Set blend state as usual
• Vertex shader will execute for each vertex
• Pixel shader will execute for each pixel

62
Cg language syntax

• Date types
• float 32 bit floating point
• half 16 bit floating point
• fixed 12 bit fixed point
• bool
• sampler handle to texture map
• struct
• No pointers yet

63
Different kinds of variables

• Uniform variables
• Same for each vertex (in a vertex shader)
• Same for each pixel (in a pixel shader)
• Examples model-view matrix, light color
• Varying variables
• Different for each vertex (in a vertex shader)
• Different for each pixel (in a pixel shader)
• Semantic binding
• Examples position, normal, texture coord.
• Local variables
• Used for intermediate computations within vertex
  or pixel shader

64
Example
Semantic binding
65
Abstracting data flow

• For varying variables, need to use special
  keywords, called semantics, to tell Cg compiler
  witch GPU register to store data in or to read
  data from
• Can use in, out, and inout modifiers to describe
  varying variables
• in varying variables that are inputs to a shader
• out varying variables that are outputs by a
  shader
• inout varying variables that are inputs but also
  written by the shader

66
Example
67
Array / vector / matrix
68
Function overloading
69
Support for vectors and matrices
70
New operators
71
A simple Cg vertex shader

• struct appdata
• float4 position POSITION
• float3 normal NORMAL
• float3 color DIFFUSE
• float3 TestColor SPECULAR
• struct vfconn
• float4 HPOS POSITION
• float4 COL0 COLOR0

72
A simple Cg vertex shader

• vfconn main(appdata IN,
• uniform float4 Kd,
• uniform float4x4 ModelViewProj)
• vfconn OUT
• OUT.HPOS mul(ModelViewProj, IN.position)
• OUT.COL0.xyz Kd.xyz IN.TestColor.xyz
• OUT.COL0.w 1.0
• return OUT
• // main

73
Two ways to compile Cg

• Offline compilation
• Compile Cg code into assembly code
• Assembly code is linked to OpenGL program at
  runtime
• Runtime compilation
• Cg program is compiled and linked at runtime.
• Good future compatibility
• No dependency limitations
• Flexible input parameter management

74
What is Cg runtime?

• An API that allow application to compile and link
  Cg programs at runtime
• Three parts
• Core Cg runtime
• OpenGL Cg runtime
• Direct3D Cg runtime
• Well focus on Core Cg and OpenGL Cg runtime

75
Where are the files?

• Cg runtime comes with Cg Toolkit
  (http//developer.nvidia.com/object/cg_toolkit.htm
  l)
• Normally under C\Program Files\NVIDIA
  Corporation\Cg
• Cg.dll, cgGL.dll, cgD3D9.dll, etc.
• Cg.lib, cgGL.lib, cgD3D9.lib, etc.
• Cg.h, cgGL.h, cgD3D9.h, etc.
• Download and install Cg Toolkit
• Make sure Cg runtime header files and lib files
  are on VCs search path
• Make sure Cg runtime DLL files are on system
  search path

76
Basic Cg runtime concepts

• Context
• A container for multiple Cg programs
• An application may have multiple contexts
• Program
• Compile a Cg program by adding it to a context
• Parameters
• A program has several parameters

77
Cg Language Profiles

• Different GPUs support different set of graphics
  capabilities
• A Cg profile defines a subset of full Cg language
  that is supported on a particular hardware
  platform or API
• CG_PROFILE_VP30, CG_PROFILE_FP30, etc.
• A full list can found on Cg Users Manual
• A Cg program has to be compiled for a particular
  profile

78
Basic Steps of using Cg Runtime

• In your OpenGL program, do the following
• Create Cg context(s)
• Initialize profiles
• Compile Cg program(s)
• Load Cg program(s)
• (a) Modify Cg program parameters
• (b) Enable profile(s)

79
Basic Steps of using Cg Runtime

• (c) Bind a Cg program
• (d) Draw 3D object(s)
• (e) Disable the profile
• Repeat (a) (e) if necessary
• Destroy Cg program(s)
• Destroy Cg context(s)

80
Create a Context

• You have to create at least one context if you
  want to use Cg
• Its the first to be created and the last to be
  destroyed
• CGcontext context cgCreateContext()

81
Get Cg profiles

• CGprofile vertexProfile cgGLGetLatestProfile(CG
  _GL_VERTEX)
• CGprofile fragmentProfile cgGLGetLatestProfile(
  CG_GL_FRAGMENT)

82
Compiling a program

• CGprogram vertexProgram cgCreateProgram(context,
  CG_SOURCE, myVertexProgramString, vertexProfile,
  main, args)

83
Loading a program

• Pass the compiled object code to OpenGL
• cgGLLoadProgram(vertexProgram)

84
Modify Cg Program Parameters

• You need to set Cg program parameters before its
  execution
• Get Cg program parameters
• CGparameter baseTexture cgGetNamedParameter(frag
  mentProgram, baseTexture)
• Set uniform parameters that dont change during
  program execution
• cgGLSetTextureParameter(baseTexture, texture)
• cgGLSetParameter4fv(someColor, constantColor)

85
Modify Cg Program Parameters

• Set the varying parameters
• cgGLEnableClientState(texCoord)
• cgGLSetParameterPointer(texCoord, 2, GL_FLOAT, 0,
  vertexTexCoords)
• Set the uniform parameters that may change every
  frame
• cgGLSetStateMatrixParameter(modelViewMatrix,
  CG_GL_MODELVIEW_PROJECT_MATRIX,
  CG_GL_MATRIX_IDENTITY)

86
Enable the profiles

• cgGLEnableProfile(vertexProfile)
• cgGLEnableProfile(fragmentProfile)

87
Bind the programs

• You can bind only one vertex program and one
  fragment program at a time.
• cgGLBindProgram(vertexProgram)
• cgGLBindProgram(fragmentProgram)

88
Disable the profiles

• This automatically unbind the corresponding
  vertex and/or fragment program
• cgGLDisableProfile(vertexProfile)
• cgGLDsiableProfile(fragmentProfile)

89
Destroy Cg program

• Release the resources allocated to the Cg program
• cgDestroyProgram(vertexProgram)

90
Destroy Context

• Release the resources allocated to the Cg context
• cgDestroyContext(context)

91
Cg Runtime Example

• Example from Cg Users Manual, page 57 60
• http//developer.nvidia.com/object/cg_toolkit.htm
  l

Vertex shader
92
Cg Runtime Example (cont.)
Pixel shader
93
Cg Runtime Example (cont.)
OpenGL program
94
Cg Runtime Example (cont.)
95
Cg Runtime Example (cont.)
96
Cg Runtime Example (cont.)
97
Examples

• A complete Cg/Cg runtime example in Hello, Cg!
• http//developer.nvidia.com/object/hello_cg_tutori
  al.html
• More examples comes with Cg Toolkit
• Installed under C\Program Files\NVIDIA
  Corporation\Cg\examples\runtime_ogl
• cgGL_vertex_example.c
• cgGL_vertex_example.cg

98
Summary

• Cg runtime is a bridge between Cg program and the
  host OpenGL/D3D application
• Quite straightforward to learn/use
• Perhaps the more difficult part is to learn how
  to get/set various Cg program parameters
• A good understanding of various Cg profiles are
  also important

99
Summary

• Programmable GPU is the biggest revolution in
  graphics in 10 years and the foundation for the
  next 10
• Use shading languages to write vertex or pixel
  shaders that run on GPU
• OpenGL Shading Language, Cg, High Level Shading
  Language, etc.

100
Readings

• Introduction to the hardware graphics pipeline
• http//developer.nvidia.com/object/eg_2004_present
  ations.html
• Hello, Cg! http//developer.nvidia.com/object/hell
  o_cg_tutorial.html
• Cg in two pages (http//developer.nvidia.com/att
  ach/7076 )
• (Optional) Cg Users Manual
• http//developer.nvidia.com/object/cg_toolkit.html
  

101
GPU references

• NVIDIA technical presentations
• http//developer.nvidia.com/page/documentation.htm
  l
• ATI technical presentations
• http//www.ati.com/developer/techpapers.html
• 3Dlabs technical presentations
• http//developer.3dlabs.com/openGL2/presentations/
  index.htm

102
Cg and Cg runtime References

• Cg toolkit
• http//developer.nvidia.com/object/cg_toolkit.html
  
• Cg Users Manual 1.5
• Comes with Cg Toolkit
• A thorough discussion of Cg and Cg runtime.
• Appendix A Cg language specification
• www.shadertech.com
• The Cg Tutorial, by R. Fernando and M. J.
  Kilgard, Addison-Wesley, 2003

103
GLSL references

• OpenGL 2.0 specification
• http//www.opengl.org/documentation/specs/version2
  .0/glspec20.pdf
• GLSL Introductions GLSL master class
• http//developer.3dlabs.com/openGL2/presentations/
  index.htm
• OpenGL Shading Language, by R. J. Rost, 2nd
  edition, Addison-Wesley, 2006
• http//www.3dshaders.com/

104
Next lecture

• Overview of modeling