OUTLINE

1
OUTLINE

• What is Photography?
• What is The Photographic Signal?
• Perfecting Film-Like Photography Old Problems,
  New Approaches
• Photography Beyond Film New Goals, Methods,
  Expressions

2
(No Transcript)
3
A2 Core Concepts(15 minutes)
Jack Tumblin Northwestern University
4
Film-Like Photography

• Film Camera design assumptions
• Instantaneous light measurement
• Of focal plane image behind a lens.
• Reproduce those amounts of light.

Implied What we see is ?
focal-plane intensities. well, nowe see
much more! (seeing is deeply cognitive)
5
Our Definitions

• Film-like Photography
• Static instantaneous record of the 2D image
  formed by a lens
• Display image ? sensor image
• Computational Photography
• displayed image ? sensor image
• A more expressive, controllable displayed result,
• from transformed, merged, decoded sensor data

6
What is Photography?

• A bucket word a neat container for messy
  notions(e.g. aviation, music, comprehension)
• A record of what we see,or would like to see,in
  tangible form.
• Does film photography always capture it? no.
• So, what do we see?

Harold Doc Edgerton 1936
7
What is Photography?
PHYSICAL
PERCEIVED
Exposure Control,tone map
3D Scene light sources, BRDFs, shapes, positions,
movements, Eyepoint position,
movement, projection,
Scene light sources, BRDFs, shapes, positions, mov
ements, Eyepoint position, movement, projection
,
Light Optics
Display RGB(x,y,tn)
Image I(x,y,?,t)
Vision
Photo A Tangible Record Editable, storable
asFilm or Pixels
8
Ultimate Photographic Goals
PERCEIVED or UNDERSTOOD
PHYSICAL
3D Scene light sources, BRDFs, shapes, positions,
movements, Eyepoint position,
movement, projection,
Light Optics
3D Scene? light sources, BRDFs, shapes, positions
, movements, Eyepoint? position,
movement, projection, Meaning
Visual Stimulus
Sensor(s)
Vision
Computing
Photo A Tangible Record Scene estimates we
can capture, edit, store, display
9
Missing Viewpoint Freedom

• Multiple-Center-of-Projection Images
  Rademacher, P, Bishop, G., SIGGRAPH '98

10
Missing Reliable Visual Boundaries

• 5 ray sets ? explicit geometric occlusion
  boundaries

Ramesh Raskar, MERL, 2004
11
Missing Expressive Time Manipulations

• What other waysbetter reveal appearance to human
  viewers?
• (Without direct shape measurement? )

Can you understand this shape better?
Time for space wiggle. Gasparini, 1998.
12
Photographic Signal Pixels Rays

• Core ideas are ancient, simple, seem obvious
• Lighting ray sources
• Optics ray bending/folding devices
• Sensor measure light
• Processing assess it
• Display reproduce it
• Ancient Greekseye rays wipe the worldto feel
  its contents

http//www.mlahanas.de/Greeks/Optics.htm
13
The Photographic Signal Path

• Computing can improve every component

Light Sources
Sensors
Data Types,Processing
Optics
Optics
Rays
Display
Scene
Rays
14
Review How many Rays in a 3-D Scene?

• A 4-D set of infinitesimal members.
• Imagine
• Convex Enclosure of a 3D scene
• Inward-facing ray camera at every surface point
• Pick the rays you need for ANY camera outside.
• 2D surface of cameras,2D ray set for each
  camera,
• ? 4D set of rays.

(Levoy et al. SIGG96)
(Gortler et al. 96)

15
4-D Light Field / Lumigraph

• Measure all the outgoing light rays.

16
4-D Illumination Field

• Same Idea Measure all the incoming light rays

17
4D x 4D 8-D Reflectance Field

• Ratio Rij (outgoing rayi) / (incoming rayj)

18
Future Photography
Novel Illuminators
Lights
Novel Cameras
Modulators
General Optics 4D Ray Benders
Generalized Sensor
General Optics 4D Ray Benders
Ray Reconstructor
GeneralizedProcessing
4D Incident Lighting
4D Ray Sampler
Viewed 4D Light Field
Novel Displays
Generalized Display
Recreated 4D Light field
Scene 8D Ray Modulator
19
Expand Optics Into Software

• Programmable Optical Devices enable new
  forms of
• Nayar
• Omni-Directional Lens Systems
• (Hi-Def 360o video)
• Assorted Pixels Sensors (Robust HDR,
  multispectral)
• Lensless Adaptive-Aperture Cameras
  (tracking without panning)

20
Expand Scientific / Medical Imaging

• 4D light sources 4D cameras enable new
  forms of
• Levoy
• Synthetic Aperture Imaging (see through
  trees)
• Tomography (3-D volumetric imaging)
• Confocal Scanning (look inside muddy
  water)

21
The Ideal Photographic Signal

• I CLAIM IT IS
• All Rays? Some Rays? Changes in Some Rays
• Photographic ray space is vast and redundantgt8
  dimensions 4D view, 4D light, time, ?,
• ? Gather only visually significant ray
  changes ?
• ? What rays should we measure ?
• ? How should we combine them ?
• ? How should we display them ?

22
Beyond Film-Like Photography

• Call it Computational Photography
• To make meaningful ray changes tangible,
• Optics can do more
• Sensors can do more
• Light Sources can do more
• Processing can do more
• by applying low-cost storage,
  computation, and control.

23
A.2 Concepts Tumblin