SHADERS

1
SHADERS

• Akshay Loke

2
Overview

• GPU Pipeline
• What are Shaders???
• Different types of shaders
• Vertex shader
• Pixel shader
• Geometry shader
• Hair shaders
• Fur shaders
• Film and game demos
• The Future lies ahead!!
• Questions and comments??

3

• Basic question
• What is a shader???
• Technically speaking, Shaders are microprograms
  controlling vertex and pixel engines in graphics
  hardware.
• ???... To be explained

4
GPU Pipeline

• So where are these shaders that everyone keeps
  talking about??
• First an overview of the GPU pipelines
• Fixed
• Programmable
• Unified core

5
Fixed GPU pipeline
GPU
CPU
Graphics State
Xformed, Lit Vertices (2D)
Screenspace triangles (2D)
Fragments (pre-pixels)
Final Pixels (Color, Depth)
Application
Transform Light
Rasterize
Shade
AssemblePrimitives
Vertices (3D)
VideoMemory(Textures)
Render-to-texture

• This was before the advent of shaders and
  programmable graphics hardware.

6
Programmable GPU pipeline
GPU
CPU
Graphics State
FragmentProcessor
VertexProcessor
Xformed, Lit Vertices (2D)
Screenspace triangles (2D)
Fragments (pre-pixels)
Final Pixels (Color, Depth)
Transform
Application
Rasterize
Shade
AssemblePrimitives
Vertices (3D)
VideoMemory(Textures)
Render-to-texture

• Programmers wanted more control on how they deal
  with vertices and pixels.
• But this cannot change the geometry on out
  application we wanted that too so that gave
  rise to the

7
Unified core architecture
GPU
CPU
Graphics State
GeometryProcessor
Xformed, Lit Vertices (2D)
Screenspace triangles (2D)
Fragments (pre-pixels)
Final Pixels (Color, Depth)
Application
VertexProcessor
Rasterize
FragmentProcessor
AssemblePrimitives
Vertices (3D)
VideoMemory(Textures)
Render-to-texture

• Here we have the programmable primitive assembly
  aka the geometry shader/processor, where we can
  add and delete geometry elements!!!

8
What are shaders???

• A shader is a program that is run for every
  shading sample on the surface.
• So, a vertex shader is called for every vertex
  and pixel shader is called for every fragment
  (pre-pixel) on the surface.
• A shader is given a bunch of parameters by the
  renderer (position, normal, light position, color
  etc.) and by the user (surface color, shininess,
  type of surface, texture names etc.) and
  calculates what the final surface color and
  opacity should be.
• E.g. all the materials that we see in Maya are in
  fact shaders.
• Basically a shader is what you want it to be!!

9
Types of shaders

• Vertex shader/processor
• Pixel/Fragment shader/processor
• Geometry shader/processor

10
Vertex shader

• Vertex shaders perform mathematical operations on
  vertex data.
• Each vertex is associated with a variety of data
  variables.
• Eg.
• 3 vertices each having
• Position (x,y,z,w)
• Color (r,g,b,a)
• Texture(1 to n) (s,t,r,q)
• Lighting values like specular color etc.
• Vertex shader does not create or delete vertices.
• Vertex shaders are a natural extension to the
  hardwired TL engines of earlier GPUs.
• Only one vertex shader can be active at a time
  hence should be written on a per task basis.

11
Vertex shader contd.

• Complex character animation
• Environmental effects like fog or smoke
• Procedural deformation
• Motion blur
• Lens effects

12
Pixel Shader

• A graphics function that calculates effects on a
  per pixel basis.
• Provide unprecedented control for determining
  light, shade and color for each pixel.
• E.g. characters with facial hair and blemishes,
  golf balls with dimples, red leather chair, wood
  with a grainy texture etc

13
Pixel Shader

• Realism
• Bump mapping
• Effects
• NPR
• Etc

14
Geometry Shader

• New feature of the DirectX10 architecture
• Create and destroy vertices output from the
  vertex shader
• Input point lists, line lists or strips,
  triangle lists or strips, with or without
  adjacency info
• Output point lists, line strips, triangle strips

15
Geometry Shader contd.

• Complex physics n hair simulations
• Water simulations
• Single pass env maps
• Motion blur
• Displacement mapping
• Multiple shadow maps
• Most of these can be done on CPU, but that is way
  too slow!!

16
Hair n Fur shaders

• What is the difference between hair and fur??
• Scientific American says None whatsoever!!
• Fur is just hair that stops growing at a specific
  length.
• Even in CG, hair and fur rendering is considered
  synonymous and most of the algorithms and
  methodologies used for one can be extended for
  the other.
• Hair and fur shading isnt just about rendering
  it also depends very much on the way it is
  represented and animated.

17
Hair shaders

• Effective modeling and rendering of hair crucial
  for realistic effect in CG
• Issues
• Lots n lots of hair strands per head
• Each hair extremely small yet distinct
• Can be modeled as volume or individual strands
• But no industry standard as yet
• 3 basic Hair modeling stages
• Hair styling
• Hair animation
• Hair rendering

18
Hair styling

• Consists of modeling the shape of the hair,
  specifying hair geometry and density,
  distribution and orientation of hair strands
• 3 steps
• Attaching hair to scalp
• Giving an overall shape
• Managing finer hair properties

19
Hair styling

• Attaching hair to the scalp
• 2D
• T.-Y. Kim and U. Neumann, Interactive
  multiresolution hair modeling and editing
• 3D

20
Hair styling

• Giving the hair an overall shape
• Geometry based
• Parametric surfaces
• P. Noble and W. Tang, Modelling and animating
  cartoon hair with nurbs surfaces
• Wisps and generalized cylinders
• Multi resolution editing

21
Hair styling

• Physically based
• Cantilever beam
• K. Anjyo, Y. Usami, and T. Kurihara, A simple
  method for extracting the natural beauty of
  hair, 1992
• Fluid flow
• S. Hadap and N. Magnenat-Thalmann, Interactive
  hair styler based on fluid flow, 2000
• Generation of hair style from images
• Managing finer hair properties
• For appearance such as curls
• For effects like wetness
• Generally taken care of during rendering stages

22
Hair animation

• Dynamics of individual hair strands
• Mass spring systems
• Single hair strand modeled as set of particles
  with stiff springs and hinges
• 1D projective equations/Cantilever beam systems
• Rigid multi body serial chain
• FK used for animation through the chain
• Hair Dance Demo
• Dynamic super-helices
• F. Bertails, B. Audoly, M.-P. Cani, B. Querleux,
  F. Leroy, and J.-L.Leveque, Super-helices for
  predicting the dynamics of natural hair,, 2006
• Shape of each hair strand is a piecewise helix,
  with a finite number of degrees of freedom
• Accounts for typical nonlinear behavior of hair
  and also bending and twisting.

23
Hair animation

• Dynamics of a full hairstyle
• Collision detection and response to be handled
  while animating a full hairstyle
• Hair as a continuous medium
• Simulating each strand computationally expensive
• Anisotropic continuous medium
• 2 possible approaches
• Fluid dynamics
• Captures complex interactions of hair strands
• expensive
• FFD
• Lattice is deformed as a particle system and hair
  strands follow the deformation using interpolation

24
Hair animation

• Dynamics of a full hairstyle
• Hair as disjoint groups
• Grouping nearby hair strands and simulating these
  disjoint groups as independent, interacting
  entities.
• Reaches interactive frame rates and captures
  realistic features of hair, esp for fast motion.
• 2 possible approaches
• Strips of hair
• Faster due to fewer control points
• Realtime ATIs Ruby more on this later
• Wisps
• Hair represented as fixed set of deformable
  volumetric wisps
• Skeleton curve for large scale motion and
  deformation
• Deformable envelope coats the skeleton and
  accounts for deformation
• Fixed number of hair strands inside the envelope
  used only during rendering

25
Hair Rendering

• Light Scattering in hair
• Plays the same role to that played by local
  illumination model in surface rendering
• Kajiya Kay model primarily for fur, but widely
  used for hair as well
• Specular and diffuse component of light
• Marschners model
• S. Marschner, H.W. Jensen, M. Cammarano,
  S.Worley, and P. Hanrahan,Light scattering from
  human hair fibers, 2003
• Improves upon the K-K model by breaking down
  specular highlight into 3 - R, TRT, TT

26
Hair Rendering contd.

• Light Scattering in wet hair
• Increased specular component, decreased diffuse
  component
• Total internal reflection at water-air interface
  and increased absorption leads to increased
  darkening
• Gupta and N.M. Thalmann Interactive Rendering of
  Optical Effects in Wet Hair

27
Hair Rendering contd.

• Hair self-shadowing
• Ray casting through volumetric representation
• Directly ray-trace volume density or
• 2 pass shadowing scheme
• Shadow maps
• Depth image of hair rendered from lights point
  of view
• Deep shadow maps
• Store transmittance function per pixel instead of
  single depth value
• Opacity shadow maps
• Store opacity value instead of transmittance
  function

28
Hair Rendering contd.

• Rendering acceleration techniques
• Approximating hair geometry
• Surfaces and strips
• LOD representations
• Interactive volume rendering
• Light-oriented voxel grid to store hair density
  values
• Graphics hardware
• M. Koster, J. Haber, and H.-P. Seidel, Real-time
  rendering of human hair using programmable
  graphics hardware,, 2004
• Storing all opacity maps in GPU texture
• Pixel shader program for calculating anisotropic
  reflection and self shadowing
• Coming up!! Graphics card wars for hair
  rendering!!...

29
NVidias Nalu v/s ATIs Ruby
30
Nalu

• Long, blonde hair
• Lots of hair, so fast shading required
• 3 visible highlights
• Shadows much more visible
• Process broken down into
• Hair geometry and dynamics
• Rendering

31
Nalu

• Hair geometry and dynamics
• 2D patch with control hairs
• 762 control hairs, wach with 7 vertices
• Control hair -gt 6 bezier curves
• Tesselate
• Interpolate 3 at a time
• Result Lots of vertices!!

32
Nalu

• Rendering
• Marschners model for light scattering
• 3 highlights
• R white primary highlight
• TRT colored secondary highlight
• TT forward scattering component
• Shadowing using opacity maps

33
Ruby

• Hair geometry and dynamics
• Layered 2D polygon patches
• Lowers the load on vertex processor
• Simplifies back-to-front sorting while rendering
• Limited dynamics

34
Ruby

• Rendering
• Based on Kajiya-Kay and Marschners model
• Diffuse
• We use scaled and biased term (0.75 N.L 0.25)
  instead of N.L
• Brightens up the side of hair facing away from
  light and creates a softer look
• Specular
• Only R and TRT terms considered
• Shaders used
• Vertex shader
• Passes down tangent, normal, view vector, light
  vector, ambient occlusion term
• Pixel shader
• Calculate diffuse term
• Calculate specular terms and shift them in
  opposite directions along hair length to
  pronounce highlight
• 4 pass rendering algorithm

35
Fur rendering

• 2 basic models of fur rendering
• Kajiya Kay model
• Hypertextures
• Real time fur rendering
• Lengyels algorithm
• Improvement on Lengyels algo

36
Kajiya Kay model

• J. Kajiya and T. Kay, Rendering fur with three
  dimensional textures, 1989
• Represent fur as light-scattering volumes called
  texels
• Raytrace through the model
• For lighting, consider hair as a smooth cylinder.
• Diffuse Kdsin(theta)
• Specular Ks cosn(phi theta pi)
• Where (phi theta pi) represents the angle
  between cone and eye vector

37
Hypertextures

• Shape texture hypertexture
• Model objects as distributions of density
• Soft objects
• Density function D(x)
• D 1 -gt inside the object
• D 0 -gt outside the object
• 0 lt D lt 1 -gt region of nonzero thickness
  inbetween
• Fur exists in objects soft region, growing
  inside out
• Use volume rendering techniques, but slow
• But suitable for parallel implementations, since
  function is evaluated at each sample point

38
Real time fur rendering

• Lengyel, Praun, Finkelstein, Hoppe, Real-Time
  Fur over Arbitrary Surfaces, 2001
• Input
• Triangle mesh
• Parametric model for hair
• Preprocessing
• Geometric
• Compute lapped texture patches over the surface
• Covers an arbitrary surface by pasting small
  patches
• Texture
• Grow a geometric model of patch of hair and
  sample it into shell and fin textures

39
Real time fur rendering contd.

• Realtime rendering
• Surface rendering
• For each frame, render an opaque version of the
  whole mesh
• Fin rendering
• Render texture fins
• Shell rendering
• Render the offset shells from innermost to
  outermost

40
User customizable realtime fur

• Lengyels approach extended by Isidoro, Mitchell
  User Customizable Real-Time Fur, 2002
• Shell and fin extrusion using vertex shaders
• Each shell vertex moved in the direction of
  normal
• Each fin extruded out along normal direction
• E.g. game characters having shrinking or growing
  hair
• Per pixel anisotropic lighting on fins and shells
• Allows each hair to capture diffuse and specular
  lighting
• Breaks up homogeneous look of fur
• Shells and fins could appear animated by
  perturbing the surface normal
• Nvidia Wolfman Demo

41
Hair n fur in industry

• Almost all CG films nowadays
• Ratatouille
• Shrek
• Narnia
• Monsters Inc
• Final Fantasy http//youtube.com/watch?v4qUtzNuj
  qig

42
The Future

• lies ahead!!
• Unified shader architecture giving more
  flexibility to applications
• Geometry shaders
• Even more realistic realtime fur and hair
• Physically based realism for cosmetic prototyping
• Visually based realism for movies
• Computationally accelerated for games and virtual
  environments

43
Questions???
44
Bibliography

• General-Purpose Computation on Graphics Hardware,
  David Luebke, University of Virginia
• NVIDIA GeForce 8800 GPU Architecture Overview
• F. Bertails, T.-Y. Kim, M.-P. Cani, and U.
  Neumann, Adaptive wisp tree
• - a multiresolution control structure for
  simulating dynamic clustering in
• hair motion, in ACM SIGGRAPH, July 2003
• S. Hadap and N. Magnenat-Thalmann, Modeling
  dynamic hair as a
• continuum, Computer Graphics Forum, 2001
• F. Bertails, B. Audoly, M.-P. Cani, B. Querleux,
  F. Leroy, and J.-L.
• Leveque, Super-helices for predicting the
  dynamics of natural hair,
• in ACM SIGGRAPH, August 2006
• A. Daldegan, N. M. Thalmann, T. Kurihara, and D.
  Thalmann, An integrated
• system for modeling, animating and rendering
  hair, Computer
• Graphics Forum, 1993
• A. M. LeBlanc, R. Turner, and D. Thalmann,
  Rendering hair using
• pixel blending and shadow buffers, The Journal
  of Visualization and
• Computer Animation, 1991
• J. Kajiya and T. Kay, Rendering fur with three
  dimensional textures,
• ACM SIGGRAPH 89,
• S. Marschner, H. W. Jensen, M. Cammarano, S.
  Worley, and P. Hanrahan,