Shading Languages

1
Shading Languages Vertex Shader Pixel
ShaderLecture 12
Dr Abdennour El Rhalibi a.elrhalibi_at_ljmu.ac.uk
2
Shaders

• Procedural way to implement some process of
  rendering
• Transformation
• Lighting
• Texturing
• Rasterization
• Pixel fill-in
• Programmable Shading Language
• Vertex shader
• Pixel shader

3
Powered by Shader

• Per-pixel lighting
• Motion blur
• Volume / Height fog
• Volume lines
• Depth of field
• Fur shading
• Reflection / Refraction
• NPR
• Shadow
• Linear algebra operators
• Perlin noise
• Quaternion
• Sparse matrix solvers
• Skin bone deformation
• Normal map
• Displacement map
• Particle shader

• Procedural Morphing
• Water Simulation

4
Game Models

• Geometry
• Position / vertex normals / vertex colors /
  texture coordinates
• Topology
• Primitive
• Lines / triangles / surfaces /
• Property
• Materials
• Textures
• Motion
• Hierarchy
• Level-of-detail
• Etc

5
Geometry Data

• Vertex position
• (x, y, z, w)
• In model space or screen space
• Vertex normal
• (nx, ny, nz)
• Vertex color
• (r, g, b) or (diffuse, specular)
• Texture coordinates on vertex
• (u1, v1), (u2, v2),
• Skin weights
• (bone1, w1, bone2, w2, )

6
Indexed Triangles

• Geometric data
• Vertex data
• v0, v1, v2, v3,
• (x, y, z, nx, ny, nz, tu, tv)
• or (x, y, z, cr, cg, cb, tu, tv, )
• Topology
• Face v0 v3 v6 v7
• Edge table

polygon normal
vertex normal
v0
v7
v3
v6
Right-hand rule for index
7
Triangle Strips
v0
v6
v2
v4
T0
T4
T2
T5
T3
T1
v5
v1
v7
v3
v0 , v1 , v2 , v3 , v4 , v5 , v6 , v7
Get great performance to use triangle strips for
rendering on current hardware
8
Materials

• Material
• Ambient
• Environment
• Non-lighted area
• Diffuse
• Dynamic lighting
• Emissive
• Self-lighting
• Specular with shineness
• Hi-light
• View-dependent
• Not good for hardware rendering
• Local illumination
• For fixed function rendering pipeline

9
Textures

• Textures
• Single texture
• Texture coordinate animation
• Texture animation
• Multiple textures
• Alphamap

Lightmap
Base color texture
Material or vertex colors
10
The Graphics Pipeline

• The process to generate two-dimensional images
  from given virtual cameras and 3D objects
• The pipeline stages implement various core
  graphics rendering algorithms
• Why should you know the pipeline?
• Understand the various graphics algorithms
• Program low level graphics systems
• Necessary for programming GPUs
• Help analyze the performance bottleneck

11
The Graphics Pipeline

• The basic construction three conceptual stages
• Each stage can be a pipeline or parallelized
• Graphics performance is determined by the slowest
  stage
• Modern graphics systems
• software
• hardware

Application
Geometry
Rasterizer
Image
12
The Graphics Pipeline
13
The Geometry Stage
(local space polygons)
eye space
Modeling and Viewing
Vertex Lighting
Projection
clip space
screen space
Perspective
Viewport Mapping
Clipping
to rasterizer stage
(screen space lit polygon vertices)
14
The Geometry Stage

• Transform coordinates and normal
• Model-gtworld
• World-gteye
• Normalize the normal vectors
• Compute vertex lighting
• Generate (if necessary) and transform texture
  coordinates
• Transform to clip space (by projection)
• Assemble vertices into primitives
• Clip against viewing frustum
• Perspective
• Viewport transformation
• Back face culling

Introduce vertex dependences
15
The Rasterizer Stage

• Per-pixel operation assign colors to the pixels
  in the frame buffer (a.k.a scan conversion)
• Main steps
• Setup
• Sampling (convert a primitive to fragments)
• Interpolation (lighting, texturing, z values,
  etc)
• Color combinations (illumination and texture
  colors)
• Other pixel tests (alpha, stencil tests etc)
• Visibility (depth test)
• Blending/compositing

(frame buffer)
16
The Rasterization Stage

• Convert each primitive into fragments (not
  pixels)
• Fragment transient data structures
• position (x,y) depth color texture
  coordinates etc
• Fragments from the rasterized polygons are then
  selected (z buffer comparison for instance) to
  form the frame buffer pixels

17
Programmable GPUs

• So far we only discuss fixed graphics pipeline
• Fixed TL algorithms
• Fixed Fragment processing steps
• New GPU trends programmable vertex and fragment
  (Pixel) processing
• nVidia GeForce 3 and up
• Latest ATI Radeon cards
• Opengl and DirectX 8 / 9 (Cg, HLSL, GL2)

18
Programmable Shading Hardware

• Graphic hardware with programmable stages of
  the rendering pipeline is called GPU.
• Vertex shader and pixel shader can be
  paralleized to achievehigher polygon-per-secondr
  ates and pixel fill rates.

19
Shader History I

• 1995/1996 3dfx Voodoo 1 first mass market GPU,
  hw accelerated rasterization GLIDE API, 16bit
  buffers, texturing shading Quake using OpenGL!
• 1998 3dfx Voodoo 2 / Banshee AGP port, but no
  AGP texturing two texture units for single-pass
  multitexturing
• 1999 Nvidia Geforce 256 / Matrox G400 fixed
  function graphics pipeline (TL) first hardware
  OpenGL support memory introduction bump mapping

20
Shader History II

• 2000 3dfx Voodoo 3 / Nvidia Geforce
  2 multi-texturing unit, per pixel shading,
  hardware transform clipping shading, full
  screen antialiasing
• 2001 Nvidia Geforce 3 programmable TL
• 2002 Nvidia Geforce 4Ti / DirectX 9.0 first
  full hardware/software shader support
• 2003 Nvidia Geforce FX / ATI Radeon
  9800XT 256-bit memory port, displacement
  mapping, 128-bit color precision, Ultra
  Shadow, AGP 8x

21
Current Graphic Cards

• Geforce FX 5950 Ultra
• Graphics Core/Interface 256-bit
• Fill Rate 3.8 billion texels/sec.
• Vertices/sec. 356 million
• Textures per Pixel 16
• Pixel Shaders (2.0)/Vertex Shaders
• Supports OpenGL 1.5 / DX9.0

22
Current Graphic Cards II

• ATI Radeon 9800XT
• Graphics Core/Interface 256-bit
• Fill Rate 3.3 gpixels/sec.
• Vertices/sec. 412 million
• full screen antialiasing
• Pixel Shaders (2.0)/Vertex Shaders
• Supports OpenGL 1.5 / DX9.0
• SmartShader, Truform 2.0
• Videoshader

23
Shading Language

• Examples of Shading Languages
• Cooks Shade Tree
• Expression tree of shading operations.
• Quake III Arena scripting language.
• The RenderMan specification of a shading
  language.
• Stanford real-time shading language (RTSL)
• DX9s High Level Shading Language(HLSL)
• nVIDIAs Cg.
• ATIs RenderMonkey.

24
2001 Programmable Vertex Shader
25
Vertex Program
Vertex Program
Clipping
Programmable vertex processing
Fixed vertex processing
26
Vertex Program

• Used to be only assembly language interface to
  TL unit (2002)
• GPU instruction set to perform all vertex math
• Reads an untransformed, unlit vertex
• Creates a transformed vertex
• Optionally creates
• Lights a vertex
• Creates texture coordinates
• Creates fog coordinates
• Creates point sizes
• High level programming language APIs are now
  available (GLSL, Cg, HLSL, etc)

27
Programming Model
V0 V15
Vertex Source
Program Constants
c0 c96
16x4 registers
OHPOS OCOL0 OCOL1 OFOGP OPSIZ OTEX0
OTEX7
Vertex Program
96x4 registers
R0 R11
Temporary Registers
128 instructions
12x4 registers
Vertex Output
15x4 registers
28
2002-2003 Programmable Pixel Shader
29
Fragment Programs
Fog Alpha, s, z tests Blending
Fragment program
Programmable fragment processing
Fixed fragment pipeline
30
2004 Shader Model 3.0 and 64-Bit Colors
31
Overview

• Why were shaders developed?
• Vertex Shader was to replace hardware transform
  and lighting(TL) unit.
• Developers gain more control over the lighting
  model of vertices.
• Optimization can be made for models with special
  conditions.
• Special effects and object deformations.
• Pixel Shader was to replace Texturing Stages.
• Developers can manipulate each pixels directly,
  making handcrafted phong shading and other
  per-pixel effects possible.
• Can modify the z-depth value.
• Make texture accessing and manipulation even
  easier
• No limit on when and how texture fetches are
  performed.
• Access to up to 16 textures.
• Manipulation of texture coordinates, like scaling
  and offsetting.

32
Overview

• How to write them?
• Assembly-like languages.
• Shading Languages High-Level Shading Languages
  (HLSL), like the nVIDIAs Cg and ATIs
  RenderMonkey.
• Difference between V.S and P.S.
• V.S can only modify values associated with
  vertices, but P.S provides per-pixel shading.
• V.S have flow control such as branch and jump,
  but P.S. currently dont have.
• P.S. is not necessarily language-based, for
  example, for example, nVIDIA enhance each texture
  stage with a register combiner (offering
  arithmetic instructions).

33
Vertex Shader

• Hardware TL versus Vertex Shader
• Hardware TL
• The task of TL of vertices was offloaded to
  graphic hardware because we want to free CPU for
  other tasks.
• The drawback was that it forced the use of basic
  hard-wired Gouraud/ Phong lighting model.
• Vertex Shader
• Hardware-assisted and programmable version of
  Hardware TL stage.
• V.S can emulate all of the functionality of the
  Hardware TL, and with optimizations it can even
  be faster than the hard-wired path.

34
Vertex Shader

• Object deformation.
• Deform or even define the objects vertices by
  V.S.
• Page curls, heat haze, water ripples.
• By using the entire frame buffers contents as a
  texture on a mesh.
• The Present and Future of V.S.
• DX9
• Instruction count up to 256 steps.
• Constant memory up to 256 vectors.
• Special register for displacement mapping.
• New data formats.
• Flow control instructions.
• DX10
• Predicate conditionals.
• Subroutine stack.
• More integration with pixel shaders.

35
Pixel Shader

• General specification
• As in V.S., data are stored as vector.
• No flow control instructions.
• Three sets of inputs
• The interpolated/ lamped diffuse and specular
  colors and alphas.
• 8 constants registers.
• 4 or more texture coordinates.
• These texture coordinates actually can
  represent anything. Ex light vector (x,y,z),
  half angle (x,y,z) or planar coordinates (x,y).
• Texture addressing instructions.
• ALU. (perform arithmetic instructions)
• ALU must be executed after texture addressing
  instructions.

36
DX9 API Object List

• IDirect3DSwapChain9
• IDirect3DTexture9
• IDirect3DVolume9
• IDirect3DVertexBuffer9
• IDirect3DIndexBuffer9
• IDirect3DSurface9
• RenderTargets, DepthSurfaces, TransferSurfaces
• IDirect3DStateBlock9
• IDirect3DVDecl9
• IDirect3DVertexShader9
• IDirect3DPixelShader9

37
DX9 API Object Model
D3DDevice9
D3DVDecl9
DMA Tess
D3DVertexShader9
Vertex ALU
D3DPixelShader9
Pixel ALU
38
Pipeline Overview

• Create VB Object
• Set up Vertex Stream VBs
• Define Vertex Decl Object
• Vertex Shader Object
• Pixel Shader Object
• FB blender

39
Vertex Shader 2.0

• Now supports
• 256 instructions (double DX8)
• Constant-based flow control logic
• based on boolean and integer registers
• At least 256 constants
• A 4-D address register for indexing them

40
DirectX Graphics Architecture
Vector Data
VB
Vertex Shader
Geometry Ops
Vertex Components
Primitive Ops
Pixel Shader
Pixel Ops
Image Surface
Samplers
Output pixels
41
Vertex Shader Architecture
Vec4
Vec15

A0
Const0
R0
Const1
R1
Vertex ALU
Const2
R2
R3
Const3


R11
Const95
Hpos
Color0
Color1
42
Pixel Shader 2.0 Architecture
v0
v1
t4
t7

c0
r0
c1
r1
Pixel ALU
c2
r2
r3
c3


r11
c31
43
HLSL Features from C

• HLSL provides structures and arrays incl.
  Multidimensional arrays
• Comments (//, / /)
• All Cs arithmetic operators (, , /, etc.)
• bool type with bolean and relational operators
  (, , !, etc.)
• Increment/decrement (, --)
• Conditional expression (? )
• Assigment expressions (, etc.)
• C comma operator
• User defined functions (not recursive functions)
• A subset of Cs flow constructs (do, while, for,
  if, break, continue) Not goto, switch
• define, ifdef etc.

44
HL Shading Language

• The benefits of using higher level shading
  language
• Ease of use.
• Easy to write.
• Easy to maintain.
• Portability.
• Optimization.
• The shading language compiler can perform
  optimization on the generated codes.
• In fact, these benefits can apply to any high
  level programming languages, like C over assembly
  codes.

45
(No Transcript)
46
(No Transcript)
47
Summary

• 3D hardware previously used fixed-function
  pipelines in which, for example, one was stuck
  with the lighting model chosen by the hardware
  vendor.
• Graphics hardware was able to do transformation
  and lighting (TL for short) on the card, but it
  was not flexible.
• A vertex shader sidesteps the TL stage in the
  pipeline and lets the user add on to it.
• Pixel shaders operate after the geometry pipeline
  and before final rasterization.
• They operate in parallel with multitexturing to
  produce a final pixel color and z-value for the
  final, rasterization step in the graphics
  pipeline (where alpha blending and depth testing
  occur).

48
References
http//www.shaderx.com/download.htm http//mirror
.ati.com/developer/indexsc.htmldirectx9 http//m
sdn.microsoft.com/library/default.asp?url/library
/en-us/dnhlsl/html/shaderx2_introductionto.asp
49
Revision

• Examples of Questions for the Advanced Games
  Technology Modules -
• Open questions and essay style questions from the
  following topics
• 3D Graphics Pipeline Concepts and Principles
• D3D Concepts, Data Structure Representation,
  Rendering Issues and Solutions
• Frustrum Culling Concept, Issues, Solutions
• Level of Detail Concepts, Discrete/Continuous,
  Applications
• Terrain Generations Algorithms
• Spatial Data Structures
• Game AI and RBS Concepts, Issues,
  Representation, Forward / Backward Chaining
• Game Networking Concepts, Issues, Solutions
• Shaders Languages Concepts, Principles
• The exam will be 3 questions to choose from 6
  questions.
• Review the notes, the book recommended and the
  papers referred to.