Lambertian Reflectance and Linear Subspaces

1
Lambertian Reflectance and Linear Subspaces
Ronen Basri David Jacobs
Weizmann NEC
2
Lighting affects appearance
3
How Complicated is Lighting?

• Lighting gt infinite DOFs.
• Set of possible images infinite dimensional
  (Belhumeur and Kriegman)

• But, in many cases, lighting gt 9 DOFs.

4
Prior Empirical Study
(Epstein, Hallinan and Yuille see also
Hallinan Belhumeur and Kriegman)
5
Our Results

• Convex, Lambertian objects 9D linear space
  captures gt98 of reflectance.
• Explains previous empirical results.
  (Epstein, Hallinan and Yuille Hallinan
  Belhumeur and Kriegman)
• For lighting, justifies low-dim methods.
• Simple, analytic form.
  gt New recognition algorithms.

6
Domain
Domain

• Lambertian
• No cast shadows
• Lights far and isotropic

n
l
q
llmax (cosq, 0)
7
(No Transcript)
8
Lighting to Reflectance Intuition
9
(See DZmura, 91 Ramamoorthi and Hanrahan 00)
10
Spherical Harmonics

• Orthonormal basis for functions on the sphere
• Funk-Hecke convolution theorem
• Rotation Phase Shift
• nth order harmonic has 2n1 components.

11
Amplitudes of Kernel
n
12
Reflectance functions near
low-dimensional linear subspace
Yields 9D linear subspace.
13
How accurate is approximation?

• Accuracy depends on lighting.
• For point source 9D space captures 99.2 of
  energy
• For any lighting 9D space captures gt98 of
  energy.

14
Forming Harmonic images
l
lZ
lY
lX
lXY
lXZ
lYZ
15
Accuracy of Approximation of Images

• Normals present to varying amounts.
• Albedo makes some pixels more important.
• Worst case approximation arbitrarily bad.
• Average case approximation should be good.

16
Query
17
Comparison Methods

• Linear
• Non-negative light
• (See Georghides, Belhumeur and Kriegman)
• Non-negative light, first order approximation
  

18
Previous Linear Methods

• Shashua. With no shadows, illn
• with B lX,lY,lZ.
• First harmonic, no DC
• Koenderink van Doorn heuristically suggest
  using l too.

19

• PCA on many images

Amano, Hiura, Yamaguti, and Inokuchi Atick and
Redlich Bakry, Abo-Elsoud, and Kamel
Belhumeur, Hespanha, and Kriegman Bhatnagar,
Shaw, and Williams Black and Jepson Brennan
and Principe Campbell and Flynn Casasent, Sipe
and Talukder Chan, Nasrabadi and Torrieri
Chung, Kee and Kim Cootes, Taylor, Cooper and
Graham Covell Cui and Weng Daily and
Cottrell Demir, Akarun, and Alpaydin Duta,
Jain and Dubuisson-Jolly Hallinan Han and
Tewfik Jebara and Pentland Kagesawa, Ueno,
Kasushi, and Kashiwagi King and Xu Kalocsai,
Zhao, and Elagin Lee, Jung, Kwon and Hong Liu
and Wechsler Menser and Muller Moghaddam
Moon and Philips Murase and Nayar Nishino,
Sato, and Ikeuchi Novak, and Owirka Nishino,
Sato, and Ikeuchi Ohta, Kohtaro and Ikeuchi
Ong and Gong Penev and Atick Penev and
Sirivitch Lorente and Torres Pentland,
Moghaddam, and Starner Ramanathan, Sum, and
Soon Reiter and Matas Romdhani, Gong and
Psarrou Shan, Gao, Chen, and Ma Shen, Fu, Xu,
Hsu, Chang, and Meng Sirivitch and Kirby
Song, Chang, and Shaowei Torres, Reutter, and
Lorente Turk and Pentland Watta, Gandhi, and
Lakshmanan Weng and Chen Yuela, Dai, and
Feng Yuille, Snow, Epstein, and Belhumeur
Zhao, Chellappa, and Krishnaswamy Zhao and Yang.
20
Comparison to PCA

• Space built analytically
• Size and accuracy known
• More efficient
• time,
• When pose unknown, rendering and PCA
• done at run time.

21
Experiments

• 3-D Models of 42 faces acquired with scanner.
• 30 query images for each of 10 faces (300
  images).
• Pose automatically computed using manually
  selected features (Blicher and Roy).
• Best lighting found for each model best fitting
  model wins.

22
(No Transcript)
23
Results

• 9D Linear Method 90 correct.
• 9D Non-negative light 88 correct.
• Ongoing work Most errors seem due to pose
  problems. With better poses, results seem near
  100.

24
(No Transcript)
25
(No Transcript)
26
Summary

• We characterize images object produces.
• Useful for recognition with 3D model.
• Also tells us how to generalize from images.