ImageBased Rendering

1
Image-Based Rendering

• Part I
• Introduction
• Panoramas
• Light Field, Lumigraph

2
What is Image-Based Rendering?

• Not just using images on geometry
• Built on desire to bypass the manual modeling
  phase
• Can you use images (of some kind) either directly
  or at least for modeling?

3
Types of IBR

• Images, MovieMaps, Lippmann, S80
• Panoramas/Image Mosaics
• QuicktimeVR, Chen S95
• Others, like Szeliski S97, products
• Light Fields, Lumigraph
• Depth Images
• Geometry from Images

4
Plenoptic Function

• From Adelson and Bergen, The Plenoptic Function
  and Elements of Early Vision.
• Captures the light received
• at any point in space,
• from any direction,
• at any time.
• Function of location, direction of gaze,
  wavelength, and time.

5
5D Plenoptic Function

• Eliminate time
• Use 3 component color
• Left with 5D
• Position
• Orientation

6
Panoramas

• 2D Plenoptic Function
• Single center of projection
• Only rotations
• Stitched (QTVR, PhotoVista, etc.)
• Panorama Tools at
• http//www.fh-furtwangen.de/dersch/
• IBM tools
• Dual Fisheye lenses

7
Characteristics of Panorama

• Relatively easy to make
• 3D Navigation bad
• Hot spots to click.
• Cant really tell where to go next.
• No sense of 3D

8
Columbias Omnicam

• Web Site

9
Examples of video panoramas

• BeHere
• http//www.behere.com
• iMove
• http//www.imoveinc.com/

10
Light Field and Lumigraph

• Take advantage of empty space to reduce Plenoptic
  Function to 4D
• Database
• How and Why?

11
Empty Space ? 4D

• Color is OK anywhere
• along ray
• as long as in empty space

12
Problem How do we store and find the rays?

• In other words
• How do we parameterize?
• Issues
• Efficient Calculation
• Uniformity in Sampling

13
Two Plane Parameterization
14
Image to Lumigraph Mapping
15
Object Based

• Can be made a cube around object

16
One Plane

• Can have one plane at infinity

17
Images from uv Plane
18
Line Space
19
Sampling
20
Sampling
21
Resolution

• In Lumigraph paper they suggested that far plane
  needs to have resolution of desired image.
• Near can have lower
• Far 128 to 512
• Near 16 to 64
• See SIGGRAPH 2000 paper, Plenoptic Sampling

22
Filtering of Light Field
23
Making Light Field/Lumigraph

• Rendered from synthetic model
• Made from real world
• With gantry
• Handheld

24
From Synthetic Model

• Sheared perspective projection

25
Gantry

• Lazy Susan
• Manually rotated
• XY Positioner
• Lights turn with lazy susan
• Correctness by construction.

26
Handheld Camera

• Blue screen on stage
• Walls moveable, can turn

27
Camera Pose

• Intrinsic parameters
• Lens calibration
• Pinhole projection
• Radial distortion
• Used Tsais algorithm (and code)
• Extrinsic
• Where is camera?
• Use targets

28
Approximate Geometry

• Blue screen to find object in each image
• Use octree to represent volume of object
• Project, hierarchically, the octree cells
• Subdivide if not fully in/out.
• External polygons collected
• Smoothed
• Used 4 levels of octree

29
Reconstruction of Lumigraph From Sparse Set of
Samples

• Pictures not taken uniformly
• Used Splat, Pull, Push algorithm
• Splat onto full res
• Pull to lower res
• Push back to fill gaps

30
Compression

• Treated more in Light Field paper
• Vector Quantization
• Lempel-Ziv
• Compression 1201.
• 402 MB to 3.4MB

31
Reconstruction
32
Depth Correction of Lumigraph

• For ray (s,u), is (si1, up) the best sample?

33
Changed Basis
34
Quadralinear Interpolation

• Interpolate over nearest locations (16 pts
  because they used a linear/bilinear approx.)

35
Depth Correction

• Note they say for well behaved radiance
  function.
• They mean non-specular.
• Lumigraph changes basis (filtering support) based
  on local geometry (next slide).

36
Comparison
37
Comparison
38
Run Example
39
Results

• Light Field

40
Light Field Pages

• Videos
• Main page
• http//www.graphics.stanford.edu/projects/lightfie
  ld/
• Viewers for PC and SGI available.
• Drag left mouse button to rotate the object.
• Drag both buttons to zoom in and out.
• Spacebar back to the starting position.

41
Concentric Mosaics

• Different parameterization (3D)
• Make vertical slit (line scan) images at
  concentric circles

42
Store Mosaic

• As a panorama

43
Indexing into Mosaic

• Find tangents at proper locations.

44
Off-Plane Rays

• Not correct. They manually select a depth.

45
Making Mosaic

• Use standard camera. Tangential or radial.

46
(No Transcript)
47
(No Transcript)
48
Show MPEG
49
Other Parameterization

• Aliaga, Plenoptic Stitching, SIGGRAPH 2001