A Proposed Strategy

1
Camera Culture
Ramesh Raskar
Camera Culture
Associate Professor, MIT Media Lab
2

• We focus on creating tools to better capture and
  share visual information
• The goal is to create an entirely new class of
  imaging platforms that have an understanding of
  the world that far exceeds human ability and
  produce meaningful abstractions that are well
  within human comprehensibility

Ramesh Raskar
3
(No Transcript)
4
Where are the cameras?
5
Where are the cameras?
6
Questions

• What will a camera look like in 10,20 years?

7
Tools for Visual Computing
Shadow
Refractive
Reflective
Fernald, Science Sept 2006
8
Questions
9
Approach

• Not just USE but CHANGE camera
• Optics, illumination, sensor, movement
• Exploit wavelength, speed, depth, polarization
  etc
• Probes, actuators
• We have exhausted bits in pixels
• Scene understanding is challenging
• Build feature-revealing cameras
• Process photons
• Technology, Applications, Society
• We study Impact of Imaging on all fronts

10

• Capture
• Cameras Everywhere
• Deep pervasive sensing
• Analysis
• Computer Vision
• Mo-cap
• Personalized services, tracking in real world
• Animation
• Simulation of bio/chemical/physical processes at
  all scales
• Synthesis
• Virtual Human, Digital Actors, Tele-avatars
• Exa and Zeta-scale computing
• simulate every neural activity, predict weather
  for weeks, simulate impact of global warming
• Display
• Real world AR
• Realistic Displays 6D or 8D

11
Mask based Light Field Camera
Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan,
Siggraph 2007
12
Optical Heterodyning
Incident Modulated Signal
Photographic Signal(Light Field)
Carrier
ReferenceCarrier
13
Cosine Mask Used
Mask Tile
1/f0
14
Captured 2D Photo
15
Captured Blurred Photo
16
Refocused on Person
17
Computational Camera and Photography
18
Projects

• Lightweight Medical Imaging
• High speed Tomography
• Muscle, blood flow activity with wearable devices
  for patients
• Femto-second Analysis of Light Transport
• Building and modeling future ultra-high speed
  cameras
• Avoid car-crashes, analyze complex scenes
• Programmable Wavelength in Thermal Range
• Facial expressions, Healthcare
• Human-emotion aware computing, Fast diagnosis
• Second skin
• Wearable fabric for bio-I/O via high speed
  optical motion capture
• Record and mimic any human motion, Care for
  elderly, Teach a robot

19

• We focus on creating tools to better capture and
  share visual information
• The goal is to create an entirely new class of
  imaging platforms that have an understanding of
  the world that far exceeds human ability and
  produce meaningful abstractions that are well
  within human comprehensibility

Ramesh Raskar
20
Computational Illumination
Curved
Planar
Non-planar
Pocket-Proj
Objects
1997
1998
2002
2002
SingleProjector
1998
2002
1999
2003
1998
MultipleProjectors
Computational Camera and Photography
21
Flutter Shutter Camera
Raskar, Agrawal, Tumblin Siggraph2006
LCD opacity switched in coded sequence
22
(No Transcript)
23
Fourier Transform
Sharp Photo
Blurred Photo
PSF Broadband Function
Flutter Shutter Shutter is OPEN and CLOSED
24
Coded Aperture Camera

• The aperture of a 100 mm lens is modified

Insert a coded mask with chosen binary pattern
Rest of the camera is unmodified
25
Out of Focus Photo Coded Aperture
26
Less is More
Blocking Light More Information
Coding in Time
Coding in Space
27
Digital Refocusing
Ng et al 2005
Can we achieve this with a Mask alone?
28
Coded Computational Photography

• Coded Exposure
• Motion Deblurring 2006
• Coded Aperture
• Focus Deblurring 2007
• Glare reduction 2008
• Optical Heterodyning
• Light Field Capture 2007
• Coded Illumination
• Motion Capture 2007
• Multi-flash Shape Contours 2004
• Coded Spectrum
• Agile Wavelength Profile 2008

http//raskar.info
29
Dependence on incident angle
30
Dependence on incident angle
31
(No Transcript)
32
Towards a 6D DisplayPassive Reflectance Field
Display
2D
2D
2D
1
2
1
1

• Martin Fuchs, Ramesh Raskar,
• Hans-Peter Seidel, Hendrik P. A. Lensch
• Siggraph 2008

1 MPI Informatik, Germany 2 MIT
33
6D building block light sensitive 4D display
34
Single shot visual hull
35
Single shot Visual Hulls Simultaneous
Projections using Masks
36
Long Distance Bar-codes

• Barcode size 3mm x 3mm
• Distance from camera upto 5m

Woo, Mohan, Raskar, 2008
37
(No Transcript)
38
Coded Imaging
39
Future Computational Photography

• Coded Exposure
• Motion Deblurring 2006
• Coded Aperture
• Focus Deblurring 2007
• Glare reduction 2008
• Optical Heterodyning
• Light Field Capture 2007
• Coded Illumination
• Motion Capture 2007
• Multi-flash Shape Contours 2004
• Coded Spectrum
• Agile Wavelength Profile 2008

http//raskar.info