Plenoptic Modeling

1
Plenoptic Modeling

• An Image-Based Rendering System

2
Plenoptic Function

• p P(?, f, ?, Vx, Vy, Vz, t) where
• ? Horizontal angle
• f Vertical angle
• ? Band of Wavelengths (ex. visible light,
  x-rays)
• Vx X position of eye
• Vy Y position of eye
• Vz Z position of eye
• t time (ex. video)

3
P(?, f, ?, Vx, Vy, Vz, t)
4
Plenoptic Function

• The Plenoptic function can be thought of as a
  representation of a scene and given the inputs of
  location and direction of viewing (and band of
  wavelengths and time) it gives an image.

5
Image-Based Rendering

• An attempt to reconstruct the plenoptic function
  from a sample set.
• Complete Sample A full spherical map from a
  given viewpoint and time.
• Incomplete Sample A solid angle subset of a
  spherical map from a given viewpoint and time.

6
Sample Representation

• Cylindrical projections because
• Easy to unroll onto a planar map.
• Easier to store than a spherical projection.
• Acquisition is easier than for a sphere.
• Just set down the tripod and pan.

7
Acquisition of Cylinders

• Video Camera on a tripod /w continuous panning.
• Two planar perspective projections of a scene
  which share the same viewpoint are related by the
  following transform

8
The Transform

• The transform is composed of two parts
• The Rotational component from panning
• The Intrinsic component (such as tilt, roll
  angle, etc.) which is assumed to remain the same
  from image to image. Comes from the cameras
  properties and maybe how it sits on the tripod as
  well.

9
The set of Transforms
the vector of position values (u, v, w)
the set of homogenous transforms
the vector of position values (x, y, 1)
the rotational matrix from panning
the intrinsic matrix from camera properties
10
Acquiring the Rotational Matrices

• Get the translation in x from each image to the
  next image and use these values to approximate
  the rotation angles.

N is the number of images ti is the translation
in pixels in the x direction f is the focal
length in pixels
Using the Newton method f can be found and with f
the rotation angles can be approximated.
11
Getting the Intrinsic Matrix
Tilt Angle Matrix
Intrinsic Matrix
Projection Matrix
Roll Angle Matrix
Skew deviation from rectilinear grid
Sampling grids aspect ratio
Focal length in pixels
12
Getting the Intrinsic Matrix (contd)

• Minimize the following error function
• Starting with the initial values
• Using Powells multivariable minimization method

13
Relating Cylindrical Projections

• Get a set of corresponding points that are
  visible in both views (Can be user defined)
• Treat points as rays by starting at the center of
  projection and going out to the corresponding
  points
• Find the centers of projection and rotational
  offsets using the point that is closest to each
  pair of corresponding rays

14
Create Rays

• Formula of Ray Formula of Direction

Series of points making up the ray
Center of projection of the cylinder
Direction of ray
Horizontal angle of point
Scanline of point
Scale factor which determines vertical field of
view
Scanline where the center of projection would
project onto the scene
Rotational offset aligns the angular orientation
of cylinders to a common frame
15
Closest point to corresponding rays
is the closest point calculated by the following
formulas
16
Finding the Centers of Projection and Rotational
Offsets

• Using the formulas for the closest point to
  corresponding rays, minimize the distance between
  rays using Powells method to solve for the Ca,
  Cb, fa, and fb

17
Automate Finding Corresponding Points

• Use cylindrical epipolar geometry to find points
  that correspond on the same curve.
• Curves across the cylinders that correspond to
  each other (projection of rays across the surface
  of one cylinder and specified by position on
  another cylinder).

Formula for epipolar curve
18
Epipolar Curves on Cylinders
First half of Cylinder projection
Second half of Cylinder projection
19
Generating Image Warps
( Z )
20
Relate a to ß

• Using these formulas

21
Or Map directly to a Plane

• Using these formulas

22
What is u, v, and o from previous slide?
23
Occlusions

• Use toroidal sheets and order them back to front
  so the front point is drawn last.

24
Source Cylinders
The top and bottom are the same but with epipolar
curves as well. The top two are the only ones
used for the results shown on the next slides.
25
Results