RealTime Relighting

1
Real-Time Relighting

• Digital Image Synthesis
• Yung-Yu Chuang
• 1/10/2008

with slides by Ravi Ramamoorthi, Robin Green and
Milos Hasan
2
Realistic rendering

• We have talked about photorealistic rendering for
  complex materials, complex geometry and complex
  lighting. They are realistic but slow.

3
Real-time rendering

• Its goal is to achieve interactive rendering with
  reasonable quality. Its important in many
  applications such as games, visualization,
  computer-aided design,

4
Real-Time relighting

• Lighting is the process of adjusting lights. It
  is an important but time-consuming step in
  animation production pipeline.
• Relighting algorithms for two kinds of lights
• Distant environment lights
• Near-field lights for production

5
Relighting algorithms for distant environment
lights
6
Natural illumination

• People perceive materials more easily under
  natural illumination than simplified illumination.

Images courtesy Ron Dror and Ted Adelson
7
Natural illumination

• Rendering with natural illumination is more
  expensive compared to using simplified
  illumination

directional source
natural illumination
8
Reflection maps
Blinn and Newell, 1976
9
Environment maps
Miller and Hoffman, 1984
10
HDR lighting
11
Examples of complex environment light
12
Examples of complex environment light
13
Direct lighting with complex illumination
q
p
14
Function approximation

• G(x) the function to approximate
• B1(x), B2(x), Bn(x) basis functions
• We want
• Storing a finite number of coefficients ci gives
  an approximation of G(x)

15
Function approximation

• How to find coefficients ci?
• Minimize an error measure
• What error measure?
• L2 error
• Coefficients

16
Function approximation

• Basis Functions are pieces of signal that can be
  used to produce approximations to a function

17
Function approximation

• We can then use these coefficients to reconstruct
  an approximation to the original signal

18
Function approximation

• We can then use these coefficients to reconstruct
  an approximation to the original signal

19
Orthogonal basis functions

• Orthogonal Basis Functions
• These are families of functions with special
  properties
• Intuitively, its like functions dont overlap
  each others footprint
• A bit like the way a Fourier transform breaks a
  functions into component sine waves

20
Integral of product
21
Basis functions

• Transform data to a space in which we can capture
  the essence of the data better
• Spherical harmonics, similar to Fourier transform
  in spherical domain, is used in PRT.

22
Real spherical harmonics

• A system of signed, orthogonal functions over the
  sphere
• Represented in spherical coordinates by the
  function
• where l is the band and m is the index within the
  band

23
Real spherical harmonics
24
Reading SH diagrams
Thisdirection


Not thisdirection
25
Reading SH diagrams
Thisdirection


Not thisdirection
26
The SH functions
27
The SH functions
28
Spherical harmonics
29
Spherical harmonics
m
0
l
1
2
-1
-2
0
1
2
30
SH projection

• First we define a strict order for SH functions
• Project a spherical function into a vector ofSH
  coefficients

31
SH reconstruction

• To reconstruct the approximation to a function
• We truncate the infinite series of SH functions
  to give a low frequency approximation

32
Examples of reconstruction
33
An example

• Take a function comprised of two area light
  sources
• SH project them into 4 bands 16 coefficients

34
Low frequency light source

• We reconstruct the signal
• Using only these coefficients to find a low
  frequency approximation to the original light
  source

35
Harr wavelets

• Scaling functions (Vj)
• Wavelet functions (Wj)
• The set of scaling functions and wavelet
  functions forms an orthogonal basis

36
Harr wavelets
37
Example for wavelet transform

• Delta functions, f(9,7,3,5) in V2

38
Wavelet transform

• V1, W1

39
Example for wavelet transform

• V0, W0 , W1

40
Example for wavelet transform
41
Quadratic Bspline scaling and wavelets
42
2D Harr wavelets
43
Example for 2D Harr wavelets
44
Applications
19 5 L2
1 15 L2
3 10 L2
45
Relighting algorithms for animation production
46
Relighting for production

• Lighting is a time-consuming process.
• Artists adjust lighting parameters and wait for a
  couple of hours or days to get feedback.
• Local shading with complex scene and many lights
• Interactive relighting
• Interative visual eedback
• Fixed scene and camera
• Lower quality
• Scalable with sene complexity and number of lights

47
Deep framebuffer

• Gershbein and Hanrahan, SIGGRAPH 2000

48
Deep framebuffer
49
Deep framebuffer
50
LPICS

• Pixar, SIGGRPH 2005. A practical realization for
  the deep framebuffer approach on GPUs

LPICS 0.1s
Final renderer 2,000s
video
51
Lightspeed

• ILM, SIGGRAPH 2007
• An even more practical system with automatic
  shader conversion. (2.7s v.s. 57m)

52
Direct-to-indirect transfer

• Hasan et. al. SIGGRAPH 2006
• Deep framebuffer approaches only support local
  shading, but not indirect lighting

direct lighting
With indirect lighting
53
Concept

• Distribute gather samples on scene surfaces

54
Concept

• Direct illumination on both gather samples and
  view samples

55
Concept

• Inter-reflections between gather samples

56
Concept

• Final gather on view samples

57
Inter-reflections between gather samples
gather sample
gather sample
58
Inter-reflections between gather samples

• Assume all gather samples are diffuse

59
Inter-reflections between gather samples
60
Inter-reflections between gather samples
61
Final gathering
62
Concept
Direct on view
Transfer matrix
Final
Direct on gather
Indirect on view
63
Scene Still Life
Precomputation 1.6 hours
Polygon 107k
11.4 18.7 fps
64
Scene Temple
Precomputation 2.5 hours
8.5 25.8 fps
Polygon 2M
65
Scene Hair Ball
Precomputation 2.9 hours
9.7 24.7 fps
Polygon 320k
66
Scene Sponza Atrium
Precomputation 1.5 hours
13.7 24.9 fps
Polygon 66k
67
Comparison
DTI 8-25 fps (2.5 hr precomputation)
Monte Carlo path tracer 32 hours