The Plenoptic Function

1
The Plenoptic Function

• Lázaro Hermoso Beltrán

2
Previous Concepts

• The body of the air is full of an infinite
  number of radiant pyramids caused by the objects
  located in it Leonardo Da Vinci
• Pencil of rays The set of light rays passing
  through any point in space
• Plenoptic function
• Plenus complete or full
• Optic

3
Definition (I)

• Radiance received along any direction V arriving
  at any point E in space, at any time t and over
  any range of wavelength
• 7-Dimensional function

4
Geometric Components of the pencil of ray lights

• ANGULAR COORDINATES
• E (Ex, Ey, Ez)
• Viewpoint
• .
• Direction of the ray light passin trough the
  Viewpoint
• CARTESIAN COORDINATES
• Comonly used in machine vision

5
How would obtain plenoptic function

• Placing an imaginary eye at every position
• Record the intensity of light at every angle
• For every wavelength
• At every time t
• It does not need specify the
• direction of gaze!!

6
The Holographic Movie Example

• Black and white photo
• Color photograph
• Color movie
• Color holographic movie

7
Plenoptic Measurements in Human Vision

• PF ?potential iformation available
• Observer takes samples from PF
• Not all the information is usefull. Enviroment
  has determined what is usefull (movements ?
  predators). Enviroment constraints

8
P.M. in Human Vision

• Measurements has limited
• resolution and number of
• Samples
• Space and time the more importants

9
P.M. in Human Vision

• An example of what we can not measure
• http//web.mit.edu/persci/demos/Lightness/gaz-teac
  hing/flash/koffka-movie.swf

10
P.M. in Human Vision
11
Plenoptic Measurements Red Bar Example
(Vx, Vy, Vz)
12
P.M. in Machine Vision

• Develop a taxonomy of derivative types
• Periodic Table
• of visual elements
• Constraints ?
• Available information (PF)

13
Aplications Image Based Rendering

• IBR Sampling and Rendering
• Problem, PF is 7D function.We need reduce it!
• RGB representation
• Plenoptic modelling
• Light Field/Lumigraph P(s, t, u, v)
• 2D-Image Mosaicing

14
RGB Representation

• Three 5-D functions

15
Plenoptic Modelling

• Without time and wavelength
• Taking samples with cameras in enough close
  positions
• We can reconstruct PF interpolating the samples

16
Light Field

• Assumption that that rays does not change
  intensity along direction
• Parametrization rays with 2 parallel planes

17
2D-Image Mosacing

• To paste serial 2D images to obtain a wider
  image. The images share the same proyection point
• Panoramic mosaic or Panorama
• We reconstruct a 2D Plenoptic function

18
Interpolate Views
19
References

• E.H. Adelson and J. Bergen, The plenoptic
  function andthe elements of early vision In
  Computational Models ofVisual Processing, pages
  3-20. MIT Press, 1991
• The Plenoptic Illumination Function
  Tien-TsinWong1 Chi-Wing Fu2 Pheng-Ann Heng1
  Chi-Sing Leung3
• McMillan, Bishop. Plenotic Modeling An
  Image-Based Rendering System
• Shum, Kang. A Review of Image-based Rendering
  Techniques