Using the Stencil Buffer

1
Using the Stencil Buffer

• Advanced D3D Programming
• Sim Dietrich
• SDietrich_at_nvidia.com

2
Overview

• What is a Stencil Buffer?
• Direct3D Stencil Pipeline
• Stencil Comparison Function
• Stencil Operations
• Example - Masking a Car Interior
• Example - Depth Complexity
• Example - Shadow / Light Volumes

3
What is a Stencil Buffer?

• Stencil bits are stored with the Z Buffer
• Typically consists of 1 or 8 bits
• Stencil allows apps to tag various regions of
  the frame buffer
• Can be used for shadow/light volumes, masking,
  reflections and measuring depth complexity

4
Stencil Pipeline
STENCIL_REF
STENCIL_MASK
Stencil Buffer


Stencil Test
Stencil Write Mask
STENCIL_FAIL
Fail
Pixel Rejected
Pass



Depth Test
STENCIL_ZFAIL
STENCIL_PASS
Fail
Pass
Pixel Rejected
Pixel Accepted
5
Stencil Comparison Function

• D3DCMPFUNC ( Just like Depth Test )
• D3DCMP_NEVER
• D3DCMP_LESS
• D3DCMP_EQUAL
• D3DCMP_LESSEQUAL
• D3DCMP_GREATER
• D3DCMP_NOTEQUAL
• D3DCMP_GREATEREQUAL
• D3DCMP_ALWAYS
• D3DSTENCIL_WRITEMASK
• Limits updates to bits of the stencil buffer

6
Stencil Operations

• Set for each of three cases
• STENCIL_FAIL ( Failed Stencil Test, pixel is
  dropped )
• STENCIL_ZFAIL ( Passed Stencil, Failed Z )
• STENCIL_PASS ( Passed Both Stencil and Z )
• D3DSTENCILOP_KEEP
• Leave current stencil value alone

7
Stencil Operations (cont.)

• D3DSTENCILOP_ZERO
• Set Stencil to Zero
• D3DSTENCILOP_REPLACE
• Set to Stencil Reference Value
• D3DSTENCILOP_INCRSAT
• Add one to Stencil, clamp to maximum value
• D3DSTENCILOP_DECRSAT
• Minus one from Stencil, clamp to Zero

8
Stencil Operations (cont.)

• D3DSTENCILOP_INVERT
• If bit is 1, make it 0
• If bit is 0, make it 1
• D3DSTENCILOP_INCR
• Add 1 to stencil, wrap to 0
• D3DSTENCILOP_DECR
• Minus 1 from stencil, wrap to maximum value
• Maximum Value is 2( of Stencil Bits) - 1

9
Example - Masking a Car Interior

• Imagine a in-car view of a driving game
• Why re-draw the cars interior each frame?
• Stencil can be used to Mask it
• Clear Stencil Buffer to 0 when Z buffer is
  cleared
• Set Stencil Compare Function D3DCMP_ALWAYS
• Set Stencil Operation for STENCIL_PASS to
  D3DSTENCILOP_INCRSAT
• Set Stencil Write Mask to 0xFF
• Draw car interior, thereby setting stencil gt 1
  for each pixel it covers

10
Masking a Car Interior (cont.)

• Now draw the rest of the scene
• Set Stencil Reference Value to Zero
• Set Stencil Compare FunctionD3DCMP_EQUAL
• Set Stencil Operation for STENCIL_PASS ( for
  passing both Z and Stencil Tests ) to
  D3DSTENCILOP_KEEP
• This tells the card to only draw where it sees a
  Zero in the Stencil Buffer
• The car interior was drawn with stencil of 1 or
  higher, so it will not be overwritten.

11
Measuring Depth Complexity

• Depth Complexity is the average of times a
  pixel is redrawn in one scene
• Determines the maximum performance limit at
  various resolutions
• Maximum Frame Rate lt
  ( Fill Rate ) / ( Depth
  Complexity Resolution )
• Stencil can be used to count each time a pixel is
  drawn, thereby measuring Depth Complexity of any
  scene

12
Measuring Depth Complexity

• Clear Stencil Buffer to 0 when Z buffer is
  cleared
• Set Stencil Compare Function to D3DCMP_ALWAYS
• Set Stencil Operation for STENCIL_PASS and
  STENCIL_ZFAIL to D3DSTENCILOP_INCRSAT
• Set Stencil Write Mask to 0xFF
• Draw scene, thereby setting stencil gt 1 for each
  pixel drawn

13
Measuring Depth Complexity

• Stencil Buffer now holds a count of how many
  times each pixel was drawn
• Lock and read back the stencil buffer
• Sum all stencil buffer values and divide by
  resolution of the viewport
• That number is the scenes depth complexity
• Set the Z Fail operation to INCRSAT to count Z
  rejected pixels

14
Shadow and Light Volumes

• Many shadow algorithms project on to the floor or
  ground plane
• They dont handle objects that are inside another
  objects shadow
• To handle this case, apps have to test vertices
  against the shadowed area
• Stencil and depth buffer can perform per-pixel
  intersection calculations instead

15
Shadow And Light Volumes

• Shadow volumes are best used when the
  shadow-casting object is simple, and the shadowed
  geometry is complex
• Think of the volume of light that is blocked by
  an object from the light direction
• Find the silhouette of our shadowing object
• Use the lowest detail level model if possible
• Create a volume from the silhouettes projection
  from the light direction

16
Shadow Volume Example
17
Shadow and Light Volumes

• We want the stencil bit for the shadow volume to
  be 1 for each pixel that should be darkened,
  and Zero for all others
• We can use the depth buffer to perform an
  intersection operation, and record the result in
  the stencil buffer
• We draw the back and front faces of the shadow
  volume into the stencil buffer only

18
Shadow and Light Volumes

• We want to mark all pixels that are inside the
  shadow volume
• In front of the back faces of the shadow volume
• In back of the front faces of the shadow volume
• So, a pixel inside the shadow volume should cause
  the the first depth test to fail, and the second
  one to pass. Other cases will pass or fail or
  twice.

19
Setting Up the Stencil Pipeline

• Here is the process
• Clear the Stencil Buffer to 0 with Depth clear
• Render entire lighted scene as usual
• Set the Stencil Mask for the lights bit (ie
  0x01)
• Set the Stencil Write Mask for the lights bit
  ( ie 0x01 )
• We can have a separate bit for each light that
  casts shadows, up to the number of stencil bits,
  unless we wish perform multiple stencil passes
  for more lights

20
Setting Up the Stencil Pipeline

• Set the Stencil Comparison Function
  to D3DCMP_ALWAYS
• Set the Stencil Z Fail to D3DSTENCILOP_INVERT
• Set the Stencil Pass Function to
  D3DSTENCILOP_KEEP
• The Stencil Fail function is a dont care,
  because we set the stencil test to always pass
• Set the Stencil Reference value to 0xFF

21
Shadow and Light Volumes

• Set cull mode to D3DCULL_NONE
• We need the both back and front faces of the
  shadow volume to be drawn into the stencil
  buffer
• We dont want them in the frame buffer, so set
  the alpha blend mode to
• D3DRENDERSTATE SRCBLEND D3DBLEND_ZERO
• D3DRENDERSTATE DESTBLEND D3DBLEND_ONE

22
Shadow And Light Volumes

• Turn off Z writes
• D3DRENDERSTATE_ZWRITEENABLE FALSE
• Draw the Shadow volume
• The stencil buffer will now contain a 1 at the
  appropriate bit for each pixel in the scene that
  lies within the shadow volume

23
Scene before Shadow Volume
Depth .6 Stencil 0
Z Axis
Depth .5 Stencil 0
Light Vector
Depth .4 Stencil 0
Occluding Object
X Axis
24
After Front Face of Volume
Depth .6 Stencil 0
Z Axis
Depth .5 Stencil 0
Light Vector
Depth .4 Stencil 1
Occluding Object
X Axis
Front Face of Shadow Volume
25
After Back Face of Volume
Depth .6 Stencil 0
Z Axis
Depth .5 Stencil 1
Light Vector
Depth .4 Stencil 0
Occluding Object
X Axis
Back Face of Shadow Volume
26
Shadow And Light Volumes

• Now, we need to apply an effect to these
  pixels...
• Set up the stencil tests to only affect pixels
  that correspond to the appropriate stencil bit
  for our light
• Set the Stencil Comparison Function
  to D3DCMP_EQUAL
• Set the Stencil Mask value to match the lights
  bit ( ie 0x01 )

27
Shadow and Light Volumes

• Set the Stencil Write Mask to match the lights
  bit ( ie 0x01 )
• Set the Alpha Blend function for our effect
• Make the rectangle out of the light color and use
• ONE, ONE for a Volumetric Additive effect
• ZERO, SRCALPHA for a Darkening Shadow
• ONE, ZERO for a color replacement effect
• Draw the rectangle and the effect is composited
  into the scene

28
Handling Multiple Lights

• We can use as many lights as the stencil buffer
  has bits, ie 1 or 8.
• We could handle more with multiple stencil
  passes, by changing which bits refer to which
  lights
• We need to do one blend operation per light
• We can do the all volumes for one light by using
  a big enough rectangle

29
Handling Complex Volumes

• If your geometry produces a self-intersecting
  volume, INVERT wont work
• Instead, you can render the volumes back faces
  with ZFAIL set to INCRSAT, and the volumes front
  faces with DECRSAT
• Pixels in the volume will have non-zero stencil
  bits
• But, multiple lights require more passes

30
Caveats

• When clipping your volume, make sure it remains
  closed - You need an even number of intersections
  with the shadow volume to avoid being darkened
• If the viewer is inside a shadow volume, there
  will be only one intersection with that volume,
  thus reversing what is in or out of shadow,
  unless you cap your volume with a polygon at the
  near clipping plane

31
Caveats

• We can draw the front or back faces of the volume
  instead of the screen-aligned rectangle, but it
  is likely to be more expensive, and really should
  be sorted for some blending effects
• The shadow volume is capped at the view frustum,
  which means that shadows can be cast onto more
  than one wall if not clipped to the first wall.

32
Other Approaches

• One fundamental problem with the darkening
  approach presented here is that it is a purely
  pixel-space process. You have lost all
  information about materials, fog, etc. The
  advantage is that you render the scene only once.
• An alternative is to re-render the stencil-marked
  pixels within the shadow volume with the light
  turned off for more accuracy

33
Other Approaches

• A more accurate way to simulate objects in shadow
  is to avoid lighting them with the blocked lights
  to begin with, rather than just darkening them
  after the fact
• Render flat-shaded scene into depth buffer only
• Careful with alpha-tested transparent texures
• Render shadow volume into stencil
• Render textured scene with the light ON to
  everywhere but shadowed area
• Render scene with light OFF to shadowed area

34
Other Approaches

• For a smoother shadow edge on the floor or walls,
  connect a scaled-up version of the shadow volume
  to the original shadow volume. Set the Alpha on
  the original vertices to 0xff and the Alpha on
  the scaled up version to 0 and connect them.
• Note that this only works for surfaces that you
  cap the volume to ( like walls or a floor ), not
  for intermediate objects in the shadow volume

35
Questions
?
?
?
?
?
?
?
Sim Dietrich SDietrich_at_nvidia.com www.nvidia.com