Ray Tracing with Existing Graphics Systems

1
Ray Tracing with Existing Graphics Systems

• Jeremy Sugerman,
• FLASHG 31 January 2006

2
Why Consider Tracing Rays?

• Some techniques are hard to mimic with
  rasterization.
• Shadows (arbitrary view visibility)
• Reflection, Refraction effects
• Global, Indirect Illumination

3
Why Now?

• Raytracing is getting fast enough to use
  interactively.
• New architectures seem potentially suited or not
  too far from being suited.

4
Interactive, Really?

• Intel MLTRA (SIGGRAPH 2005)
• 20-36 fps (20-36 MRays/sec) on one 3.2 GHz
• Without MLTRA, still 7-12 fps / MRays/sec
• My toy SSE raytracer
• About 2 MRays/sec (primary rays)
• Quick Hack Cell system at SIGGRAPH
• Claims 20-25 fps / MRays/sec (unreviewed)

5
What about Rasterization?

• 60 fps over 1024x768 on 99 GPUs
• Ray tracing would need 50 MRays/sec casting
  primary rays alone.
• GPUs are getting faster at a phenomenal rate.
• Ray tracing is DOOMED!

6
Its Not About Rasterization

• The vast majority of render time is actually
  shading in most 3D apps.
• True for offline rendering too.
• Whether fragments are rasterized or ray traced,
  shading is the same
• To the extent the fragments are the same

7
And Who Traces Primary Rays?

• We want ray tracing for shadows,
  reflection/refraction, and indirect illumination.
• Those are all applications of secondary rays.
• The rasterizer already produces (x, y, z),
  normals, etc. from primary hits.
• Might as well rasterize them if you have a fast
  existing mechanism. It doesnt matter.

8
Synthesis (Thesis Antithesis)

• Bolt a ray tracer onto a conventional
  rasterization based system.
• Add a bit of ray tracer friendliness to a GPU.
• Or, wire together a Cell and a GPU (dont tell
  Sony).
• Window systems (2D), text are rasterization tasks
  fundamentally (plus optional compositing)

9
Ray Tracer Friendliness?

• Ray tracing strongly favours threaded
  architectures over SIMD.
• Packet tracing leverages bandwidth at the cost of
  very simple horizontal communication.
• Being able to use queues / write buffers seems
  critical.

10
How Would It Work?

• Operate on hits
• Analogous to fragments, plus weights
• Each hit feeds any of three (independent) shading
  choices
• Gather rays
• Shadowing and local lighting contribution
• Secondary rays
• Each produces a colour and a weight which are
  accumulated into a final pixel colour

11
Shadowing And Local Lighting

• Effectively runs
• Interpolate shading data (BRDF, Normal, etc.)
• Generate as many shadow rays as are desired
• Foreach shadow ray
• If (shadow ray hits light)
• Compute local light contribution from the light
• Fits in the same per-fragment storage
• Shadow computations are indepedent

12
Gather and Secondary Rays

• Gather rays just perform a weighted lookup in a
  data structure (e.g. photon map)
• Secondary rays are generated based on the surface
  shading information (BRDF, Normal, )
• Hits are fed back into the same pipeline
• Once generated, independent of parent

13
Kinda Like a GPU (If you squint)

• GPU shading just does the weighted local lighting
  calculation.
• But a fragment program can generate and trace
  shadow rays to mask local lighting
• And a fragment program can generate rays for
  final gathers.
• All this formulation does is expose parallelism
  and offer natural places to optimize hardware

14
I Saw You Palm That Card!

• Secondary rays are a bit harder to cram into a
  fragment program.
• No variable output buffers
• No recursion
• Once generated, secondary rays are completely
  independent.
• Only need an unordered write buffer
• No state, so recursion becomes iteration
• So start with only shadow and gather rays
• Exact same system supports them, though

15
What About Coherence?

• Rasterization lie Rasterization systems exploit
  coherence vastly better than ray tracing.
• Really means shading coherence
• Coherence between fragments in texture lookups
• Can bind a single material shader and rasterize
  only relevant geometry
• Can perform (expensive!) shading math at a
  different resolution than visibility math.

16
Ray Tracing Is Coherent Too

• Packet tracing is useful because of visibility
  coherence.
• The same coherence is also somewhat relevant for
  local shading and lighting.
• Even, render only objects with the same material
  in each pass and z-cull.
• Just as coherent as rasterization

17
Ray Tracing Is Coherent Too

• Packet tracing literature demonstrates ray
  coherence helps visibility most, but also shadow
  / secondary rays and shading.
• Rasterization systems save overdraw by rendering
  a depth-only pass before material shading with
  z-culling.
• The same technique works with rays!

18
Global Effects Are Incoherent

• Nearly by definition, indirect and inter-object
  shading effects are incoherent.
• In rasterization or ray-traced systems!
• Packet tracing literature indicates secondary
  rays retain some coherence.
• Geometry level of detail can regain coherence.

19
Coherence Bottom Line

• Without secondary rays, ray tracing shares the
  coherence characteristics of rasterization
  (modulo implementation).
• With secondary rays, ray tracing offers natural
  mechanisms for effects that are only awkwardly
  kludged via rasterization.
• Let the developer / director pick the tradeoff.

20
What Needs To Be Done?

• Persuade Pat to get a PS3 dev kit?
• Get a good ray tracer running on Cell and/or a
  GPU.
• Simulate various extensions
• Cram code in fragment programs
• Readback to the CPU for now
• Readback to a Cell
• Talk to the Smart Memories folks?
• Talk to GPU vendors (unlikely, theyre busy folks)

21
Whats Behind the Curtain?

• K-d tree building and updates
• But, Rasterization systems do this today
• K-d tree storage and bandwidth needs
• Hardware changes for traversing k-d trees or
  intersecting triangles
• Unclear if its even necessary initially
• Level of detail with secondary rays
• Decoupling visibility and shading resolutions

22
Contributions and Inspirations

• Tim, Pat
• Gordon Stoll
• Bill Mark
• Phil Slusallek and the Saarland crew
• Tim Purcell, John Nickolls (NVIDIA)

23
Rasterization Lies

• Rasterization has coherence Ray tracing lacks.
• The rasterization argument goes rasterize all
  the geometry of a given material and theres
  excellent coherence of shading samples.
• Without secondary rays, ray tracing is the same.
  With secondary rays, you get better pictures.
  The programmer gets to choose.