Computer Vision

1
Computer Vision

• Spring 2006 15-385,-685
• Instructor S. Narasimhan
• Wean 5403
• T-R 300pm 420pm
• Lecture 12

2
Midterm March 9
Syllabus until and including Lightness and
Retinex Closed book, closed notes exam in
class. Time 300pm 420pm Midterm review
class next Tuesday (March 7) (Email me by March 6
specific questions) If you have read the notes
and readings, attended all classes, done
assignments well, it should be a walk in the
park?
3
Mechanisms of Reflection
source
incident direction
surface reflection
body reflection
surface

• Surface Reflection
• Specular Reflection
• Glossy Appearance
• Highlights
• Dominant for Metals

• Body Reflection
• Diffuse Reflection
• Matte Appearance
• Non-Homogeneous Medium
• Clay, paper, etc

Image Intensity Body Reflection Surface
Reflection
4
Example Surfaces
Surface Reflection Specular Reflection Glossy
Appearance Highlights Dominant for Metals
Body Reflection Diffuse Reflection Matte
Appearance Non-Homogeneous Medium Clay, paper,
etc
Many materials exhibit both Reflections
5
Diffuse Reflection and Lambertian BRDF
source intensity I
incident direction
normal
viewing direction
surface element

• Surface appears equally bright from ALL
  directions! (independent of )

albedo

• Lambertian BRDF is simply a constant

• Surface Radiance

source intensity

• Commonly used in Vision and Graphics!

6
Diffuse Reflection and Lambertian BRDF
7
White-out Snow and Overcast Skies
CANT perceive the shape of the snow covered
terrain!
CAN perceive shape in regions lit by the
street lamp!! WHY?
8
Diffuse Reflection from Uniform Sky

• Assume Lambertian Surface with Albedo 1 (no
  absorption)
• Assume Sky radiance is constant
• Substituting in above Equation

Radiance of any patch is the same as Sky radiance
!! (white-out condition)
9
Specular Reflection and Mirror BRDF
source intensity I
specular/mirror direction
incident direction
normal
viewing direction
surface element

• Valid for very smooth surfaces.
• All incident light energy reflected in a SINGLE
  direction (only when ).

• Mirror BRDF is simply a double-delta function

specular albedo

• Surface Radiance

10
Combing Specular and Diffuse Dichromatic
Reflection
Observed Image Color a x Body Color b x
Specular Reflection Color
Klinker-Shafer-Kanade 1988
R
Color of Source (Specular reflection)
Does not specify any specific model
for Diffuse/specular reflection
G
Color of Surface (Diffuse/Body Reflection)
B
11
Diffuse and Specular Reflection
diffuse
specular
diffusespecular
12

• Photometric Stereo
• Lecture 12

13
Image Intensity and 3D Geometry

• Shading as a cue for shape reconstruction
• What is the relation between intensity and shape?
• Reflectance Map

14
Surface Normal
surface normal
15
Surface Normal
16
Gradient Space
17
Reflectance Map

• Relates image irradiance I(x,y) to surface
  orientation (p,q) for given source direction and
  surface reflectance
• Lambertian case

18
Reflectance Map

• Lambertian case

19
Reflectance Map

• Lambertian case

iso-brightness contour
Note is maximum when
20
Reflectance Map

• Glossy surfaces (Torrance-Sparrow reflectance
  model)

diffuse term
specular term
21
Shape from a Single Image?

• Given a single image of an object with known
  surface reflectance taken under a known light
  source, can we recover the shape of the object?
• Given R(p,q) ( (pS,qS) and surface reflectance)
  can we determine (p,q) uniquely for each image
  point?

NO
22
Solution

• Take more images
• Photometric stereo
• Add more constraints
• Shape-from-shading (next class)

23
Photometric Stereo
24
Photometric Stereo
Lambertian case
Image irradiance

• We can write this in matrix form

25
Solving the Equations
26
More than Three Light Sources

• Get better results by using more lights

27
Color Images

• The case of RGB images
• get three sets of equations, one per color
  channel
• Simple solution first solve for using one
  channel
• Then substitute known into above equations to
  get
• Or combine three channels and solve for

28
Computing light source directions

• Trick place a chrome sphere in the scene
• the location of the highlight tells you the
  source direction

29
Specular Reflection - Recap

• For a perfect mirror, light is reflected about N

• We see a highlight when
• Then is given as follows

30
Computing the Light Source Direction
Chrome sphere that has a highlight at position h
in the image
N
h
H
rN
c
C
sphere in 3D
image plane

• Can compute N by studying this figure
• Hints
• use this equation
• can measure c, h, and r in the image

31
Depth from Normals

• Get a similar equation for V2
• Each normal gives us two linear constraints on z
• compute z values by solving a matrix equation

32
Limitations

• Big problems
• Doesnt work for shiny things, semi-translucent
  things
• Shadows, inter-reflections
• Smaller problems
• Camera and lights have to be distant
• Calibration requirements
• measure light source directions, intensities
• camera response function

33
Trick for Handling Shadows

• Weight each equation by the pixel brightness

• Gives weighted least-squares matrix equation

• Solve for as before

34
Original Images
35
Results - Shape
Shallow reconstruction (effect of
interreflections)
Accurate reconstruction (after removing
interreflections)
36
Results - Albedo
No Shading Information
37
Original Images
38
Results - Shape
39
Results - Albedo
40
Results

1. Estimate light source directions
2. Compute surface normals
3. Compute albedo values
4. Estimate depth from surface normals
5. Relight the object (with original texture and
   uniform albedo)

41
Next Class

• Shape from Shading
• Reading Horn, Chapter 11.