Light Field = Array of (virtual) Cameras

1
Light Field Array of (virtual) Cameras
Sub-aperture
Virtual Camera Sub-aperture View
2
Sensor
Microlens array
Plenoptic Camera
Heterodyne Camera

• Samples individual rays
• Predefined spectrum for lenses
• Chromatic abberration
• High alignment precision
• Peripheral pixels wasted pixels
• Negligible Light Loss

• Samples coded combination of rays
• Supports any wavelength
• Reconfigurable f/, Easier alignment
• No wastage
• High resolution image for parts of scene in
  focus
• 50 Light Loss due to mask

3
x1 x1 ?iz
?i
?j
x1
?j
x2
Shear of Light Field
?i
?j
x1
x2
x1
x'1
4
Light Propagation (Defocus Blur)
Captured Photo
FFT of Captured Photo
5
Space of LF representationsTime-frequency
representations Phase space representations Quasi
light field
6
Quasi light fieldsthe utility of light fields,
the versatility of Maxwell

• We form coherent images by
• formulating,
• capturing,
• and integrating
• quasi light fields.

7
(i) Observable Light Field

• move aperture across plane
• look at directional spread
• continuous form of plenoptic camera

scene
8
(ii) Augmented Light Field with LF Transformer
WDF
Augmented LF
Light Field
Interaction at the optical elements
9
Virtual light projector with real valued
(possibly negative radiance) along a ray
first null (OPD ?/2)
real projector
virtual light projector
real projector
10
(ii) ALF with LF Transformer
11
(iii) Rihaczek Distribution FunctionTradeoff
between cross-interference terms and localization
u
y
(i) Spectrogram non-negative localization
(ii) Wigner localization cross terms
(iii) Rihaczek localization complex
3 m
u
0 m
y
y
y
0 m
3 m
0 m
3 m
0 m
3 m
12
Property of the Representation
Constant along rays Non-negativity Coherence Wavelength Interference Cross term
Traditional LF always constant always positive only incoherent zero no
Observable LF nearly constant always positive any coherence state any yes
Augmented LF only in the paraxial region positive and negative any any yes
WDF only in the paraxial region positive and negative any any yes
Rihaczek DF no linear drift complex any any reduced
13
Benefits Limitations of the Representation
Ability to propagate Modeling wave optics Simplicity of computation Adaptability to current pipe line Near Field Far Field
Traditional LF x-shear no very simple high no yes
Observable LF not x-shear yes modest low yes yes
Augmented LF x-shear yes modest high no yes
WDF x-shear yes modest low yes yes
Rihaczek DF x-shear yes better than WDF, not as simple as LF low no yes
14
Motivation

• What is the difference between a hologram and a
  lenticular screen?
• How they capture phase of a wavefront for
  telescope applications?
• What is wavefront coding lens for extended
  depth of field imaging?

15
Application - Wavefront Coding
Dowski and Cathey 1995
same aberrant blur regardless of depth of focus
16
Can they be part of Computer Vision?Moving away
from 2D images or 4D lightfields?
Rendering New perspective projection methods
Holography Reference targets
Wavefront coding WLC mobile phone cameras
Rotating PSF Depth from defocus
Gaussian beam lasersModern active illumination
17
Computational Photography
http//computationalphotography.org

• Epsilon Photography
• Low-level Vision Pixels
• Multiphotos by bracketing (HDR, panorama)
• Ultimate camera
• Coded Photography
• Mid-Level Cues
• Regions, Edges, Motion, Direct/global
• Single/few snapshot
• Reversible encoding of data, Lightfield
• Additional sensors/optics/illum
• Smart Camera
• Essence Photography
• Not mimic human eye
• Beyond single view/illum
• New artform

18
Resources

• Website
• http//scripts.mit.edu/raskar/lightfields/
• Or follow http//cvpr2009.org tutorial pages
• Key new papers
• Wigner Distributions and How They Relate to the
  Light Field Zhengyun Zhang and Marc Levoy, ICCP
  2009 (best paper)
• Augmenting Light Field to Model Wave Optics
  Effects , Se Baek Oh, Barbastathis, Raskar (in
  Preparation)
• Quasi light fields extending the light field to
  coherent radiation , Anthony Accardi, Wornell
  (in Preparation)

19
Acknowledgements

• Darthmuth
• Marcus Testorf,
• MIT
• Ankit Mohan, Ahmed Kirmani, Jaewon Kim
• George Barbastathis
• Stanford
• Marc Levoy, Ren Ng, Andrew Adams
• Adobe
• Todor Georgiev,
• MERL
• Ashok Veeraraghavan, Amit Agrawal

20
Light Fields___
Camera Culture
Ramesh Raskar
MIT Media Lab http// CameraCulture . info/
21
Light Fields in Ray and Wave Optics

• Introduction to Light Fields Ramesh Raskar
• Wigner Distribution Function to explain Light
  Fields Zhengyun Zhang
• Augmenting LF to explain Wigner Distribution
  Function Se Baek Oh
• QA
• Break
• Light Fields with Coherent Light Anthony
  Accardi
• New Opportunities and Applications Raskar
  and Oh
• QA All