Signal Processing Framework for Reflection

1

• Signal Processing Framework for Reflection
• Lectures 7 and 8

Thanks to Ravi Ramamoorthi, Pat Hanrahan, Ronen
Basri, David Jacobs, Ron Dror, Ted Adelson. Ravi
Ramamoorthis homepage is an excellent source for
papers, videos, PPTs on this topic. Many of the
slides in these classes are obtained from his
website. http//www.cs.columbia.edu/ravir/
2
Illumination Illusion

• People perceive materials more easily under
  natural illumination than simplified
  illumination.

Images courtesy Ron Dror and Ted Adelson
3
Illumination Illusion

• People perceive materials more easily under
  natural illumination than simplified
  illumination.

Images courtesy Ron Dror and Ted Adelson
4
Material Recognition
Photographs of 4 spheres in 3 different lighting
conditions courtesy Dror and Adelson
5
Dror, Adelson, Wilsky
6
Surface Appearance - RECAP
sensor
source
normal
surface element
Image intensities f ( normal, surface
reflectance, illumination ) Surface Reflection
depends on both the viewing and illumination
direction.
7
BRDF Bidirectional Reflectance Distribution
Function
source
z
incident direction
viewing direction
normal
y
surface element
x
Irradiance at Surface in direction
Radiance of Surface in direction
BRDF
8
Derivation of the Scene Radiance Equation
From the definition of BRDF
9
Derivation of the Scene Radiance Equation
Important!
From the definition of BRDF
Write Surface Irradiance in terms of Source
Radiance
Integrate over entire hemisphere of possible
source directions
Convert from solid angle to theta-phi
representation
10
Assumptions

• Known geometry

11
Assumptions

• Known geometry
• Convex curved surfaces no shadows,
  interreflection

Complex geometry use surface normal
12
Assumptions

• Known geometry
• Convex curved surfaces no shadows,
  interreflection
• Distant illumination

Illumination Grace Cathedral courtesy Paul
Debevec
Photograph of mirror sphere
13
Assumptions

• Known geometry
• Convex curved surfaces no shadows,
  interreflection
• Distant illumination
• Homogeneous isotropic materials

Anisotropic
Isotropic
14
Assumptions

• Known geometry
• Convex curved surfaces no shadows,
  interreflection
• Distant illumination
• Homogeneous isotropic materials
• Later, practical algorithms relax some
  assumptions

15
Reflection
16
Reflection as Convolution (2D)
L
B
17
Reflection as Convolution (2D)
18
Reflection as Convolution (2D)
19
Convolution
u
Signal f(x)
Output h(u)
Filter g(x)
20
Convolution
u1
u
x
Signal f(x)
Output h(u)
Filter g(x)
21
Convolution
u2
u
x
Signal f(x)
Output h(u)
Filter g(x)
22
Convolution
u3
u
x
Signal f(x)
Output h(u)
Filter g(x)
23
Convolution
u
x
Signal f(x)
Output h(u)
Filter g(x)
24
Reflection as Convolution (2D)
Fourier analysis
R. Ramamoorthi and P. Hanrahan Analysis of
Planar Light Fields from Homogeneous Convex
Curved Surfaces under Distant Illumination SPIE
Photonics West 2001 Human Vision and Electronic
Imaging VI pp 195-208
25
(No Transcript)
26
(No Transcript)
27
Spherical Harmonics (3D)

• Polynomials of polar and azimuth angles.
• Represent all rotations on the sphere.
• Solutions to the angular part of Laplacian
  Equation in 3D
• - do not depend on radius of sphere.
• - very important in physics problems.
• They are Orthonormal basis on the sphere.
• Any function on the sphere can be expanded using
  a sum of
• spherical harmonics of different orders (like
  Fourier series in 2D)

28
Spherical Harmonics
0
1
2 . . .
-1
-2
0
1
2
29
Spherical Harmonic Analysis
2D
3D
30
Environment Maps
Miller and Hoffman, 1984
31
Irradiance Environment Maps
Incident Radiance (Illumination Environment Map)
Irradiance Environment Map
32
Diffuse Reflection
Reflectance (albedo/texture)
Radiosity (image intensity)
Irradiance (incoming light)


quake light map
33
Computing Irradiance

• Classically, hemispherical integral for each
  pixel
• Lambertian surface is like low pass filter
• Frequency-space analysis

Incident Radiance
Irradiance
34
Assumptions

• Diffuse surfaces
• Distant illumination
• No shadowing, interreflection
• Hence, Irradiance is a function of surface normal

35
Spherical Harmonic Expansion

• Expand lighting (L), irradiance (E) in basis
  functions

.67
.36

36
Computing Light Coefficients

• Compute 9 lighting coefficients Llm
• 9 numbers instead of integrals for every pixel
• Lighting coefficients are moments of lighting
• Weighted sum of pixels in the environment map

37
Analytic Irradiance Formula

• Lambertian surface acts like low-pass filter

38
Computing Irradiance

• Classically, hemispherical integral for each
  pixel
• Lambertian surface is like low pass filter
• Frequency-space analysis

Incident Radiance
Irradiance
39
9 Parameter Approximation
Order 0 1 term (constant)
Exact image
RMS error 25
40
9 Parameter Approximation
Order 1 4 terms (linear)
Exact image
RMS Error 8
41
9 Parameter Approximation
Order 2 9 terms (quadratic)
Exact image
RMS Error 1
For any illumination, average error lt 2 Basri
Jacobs 01
42
Comparison
Irradiance map Texture 256x256 Hemispherical Inte
gration 2Hrs
Irradiance map Texture 256x256 Spherical
Harmonic Coefficients 1sec
Incident illumination 300x300
43
Dual Representation

• Diffuse BRDF Filter width small in frequency
  domain
• Specular Filter width small in spatial (angular)
  domain
• Practical Representation Dual angular,
  frequency-space

44
Complex Geometry

• Assume no shadowing Simply use surface normal

45
Lighting Design

• Final image sum of 3D basis functions scaled by
  Llm
• Alter appearance by changing weights of basis
  functions

46
Insights Signal Processing

• Signal processing framework for reflection
• Light is the signal
• BRDF is the filter
• Reflection on a curved surface is convolution

47
Insights Signal Processing

• Signal processing framework for reflection
• Light is the signal
• BRDF is the filter
• Reflection on a curved surface is convolution

Filter is Delta function Output Signal
48
(No Transcript)
49
Insights Signal Processing

• Signal processing framework for reflection
• Light is the signal
• BRDF is the filter
• Reflection on a curved surface is convolution

Signal is Delta function Output Filter
50
(No Transcript)
51
Phong, Microfacet Models
Mirror
Illumination estimation ill-posed for rough
surfaces
Analytic formulae in R. Ramamoorthi and P.
Hanrahan A Signal-Processing Framework for
Inverse Rendering SIGGRAPH 2001 pp 117-128
52
Lambertian
Incident radiance (mirror sphere)
Irradiance (Lambertian)
53
(No Transcript)
54
Estimating BRDF and Lighting
Photographs
Geometric model
55
Estimating BRDF and Lighting
Forward RenderingAlgorithm
Photographs
BRDF
Rendering
Lighting
Geometric model
56
Estimating BRDF and Lighting
Forward RenderingAlgorithm
Photographs
BRDF
Novel lighting
Rendering
Geometric model
57
Inverse Problems Difficulties
Ill-posed (ambiguous)
58
Motivation

• Understand nature of reflection and illumination
• Applications in computer graphics
• Real-time forward rendering
• Inverse rendering

59
Inverse Lighting
Given B,? find L

• Well-posed unless denominator vanishes
• BRDF should contain high frequencies Sharp
  highlights
• Diffuse reflectors low pass filters Inverse
  lighting ill-posed

60
Inverse BRDF
Given B,L find ?

• Well-posed unless Llm vanishes
• Lighting should have sharp features (point
  sources, edges)
• BRDF estimation ill-conditioned for soft lighting

Area source Same BRDF
Directional Source
61
Factoring the Light Field

• Light Field can be factored
• Up to global scale factor
• Assumes reciprocity of BRDF
• Can be ill-conditioned
• Analytic formula derived

Given B find L and ?
More knowns (4D) than unknowns (2D/3D)
62
Factoring the Light Field
Lighting coefficients are independent of viewing
directions (indices L and M are independent of P
and Q).
BRDF Reciprocity
63
Factoring the Light Field
Bootstrapping Method for Factorization (Start by
assuming DC component of Lighting)
64
Algorithm Validation
Photograph
True values
Kd 0.91
Ks 0.09
µ 1.85
s 0.13
65
Algorithm Validation
Photograph
Renderings
Image RMS error 5
Known lighting
Unknown lighting
True values
Kd 0.91 0.89 0.87
Ks 0.09 0.11 0.13
µ 1.85 1.78 1.48
s 0.13 0.12 0.14
66
Inverse BRDF Spheres
Photographs
Renderings (Recovered BRDF)
67
Complications

• Challenge Complex geometry with concavities
  Self shadowing
• Solution
• Use associativity of convolution
• Blur lighting, treat specular BRDF term as mirror
• Single ray for shadowing, easy in ray tracer

68
Complex Geometry

• 3 photographs of a sculpture
• Complex unknown illumination
• Geometry known
• Estimate microfacet BRDF and distant lighting

69
Comparison
70
New View, Lighting
Photograph
Rendering
71
Textured Objects
Rendering
Photograph