Physics based Methods in Vision (a.k.a. Appearance Modeling)

1
Physics based Methods in Vision(a.k.a.
Appearance Modeling)

• Instructor Narasimhan
• Monday - Wednesday
• 130pm - 250pm
• Wean Hall W4165A

2
A Picture is Worth 1000 Words
3
A Picture is Worth 100,000 Words
4
A Picture is Worth a Million Words
5
A Picture is Worth a ...?
Neckers Cube Reversal
6
A Picture is Worth a ...?
Checker Shadow Illusion E. H. Adelson
7
A Picture is Worth a ...?
Checker Shadow Illusion E. H. Adelson
8
Human Vision

• Can do amazing things like
• Recognize people and objects
• Navigate through obstacles
• Understand mood in the scene
• Imagine stories
• But still is not perfect
• Suffers from Illusions
• Ignores many details
• Ambiguous description of the world
• Doesnt care about accuracy of world

9
Computer Vision
What we see
What a computer sees
10
Components of a Computer Vision System
Camera
Lighting
Computer
Scene
11
What is Computer Vision?

• Inverse Optics
• Intelligent interpretation of Imagery
• Building a Visual Cortex
• No matter what your definition is
• Vision is hard.
• But is fun...

12
What is Physics-based Vision?

• How did the Pixel get its value?
• - Jitendra Malik, UC Berkeley
• We must understand scene appearance

13
What is Appearance?
14
Lets see some pictures!
15
Light and Shadows
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
Reflections
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
Refractions
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
Interreflections
32
(No Transcript)
33
Scattering
34
(No Transcript)
35
(No Transcript)
36
De-hazed
Haze
37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
More Complex Appearances
46
(No Transcript)
47
Hair
Marschner et al.
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
Concepts in Optics you Learn
Reflection Refraction Polarization Diffraction
Caustics Interreflections Scattering Interferen
ce
52
Why Understand Appearance?
53
Appearance in Vision
54
Good Vision in Bad Weather
Mist
Haze
Rain
Fog
Images Courtesy Steve and Carol Sheldon
55
Driving in Bad Weather
56
Applications in Graphics
Final Fantasy
Shrek
57
Computer Vision
Appearance
Underwater Imaging
Computer Graphics
Satellite Imaging
Medical Imaging
58
Exploring the Visual Appearance of Nature
Optics Mathematics
Cameras
Capture
Modeling
Synthesis
Interpreting
Vision
Graphics
59
What to model?How much to model?
60
Scale is Everything Geometry vs. Reflectance vs.
Statistics vs. Sensor
61
Course Format

• One Midterm assignment -
  20
• Semester-long research project -
  30
• One Final Exam - 20
• Nature Photography Competition - 15
  
• One Paper Presentation - 15
• IMPORTANT
• Do you have access to a good camera?
• Do you have access to a good machine with Matlab?

62
Lectures Format
Each Week First Lecture Optical
Phenomenon Second Lecture Applications in
Vision and Graphics
63
Research Project

• Teams of 2 or 3 for a project.
• Meet me this or next week for initial
  discussion.
• Set small goals throughout the semester.
• Meet me once in two weeks to show progress
• and discuss next steps.
• Give final presentation/demo in December.
• If you get great results, we can even try to
  write a paper.

64
Sample Research Projects
Dimensionality of Outdoor Lighting and Reduced
Dimension models Lighting insensitive features,
shadow detection and elimination Reconstructing
Illumination from Images Image based rendering
from Webcam data Rendering appearance effect
X,Y,Z,A,B,C More details next class
65
Webcams everywhere!
Image-based Rendering Lighting Insensitive
Recognition Reduced dimensional models for
appearance
66
Photography Competition
Take 10 Great Photographs of the
appearance effects discussed in class. Give a
five minute slide presentation with only those
pictures (point to the effects). Vote for the
best photographer. A few faculty will also
vote. Win a Prize! Nonot a round trip to
Hawaii!