Image Based Modeling and Rendering

1
Image Based Modeling and Rendering

• Synopsis of Papers Presented

2
QuickTime VR

• Goal
• Create a virtual environment using a cylindrical
  panorama
• Rotate a camera on a tripod, then stitch the
  images together
• Weaknesses
• No parallax
• No smooth transition between cylinders

3
View Interpolation for Image Synthesis

• Idea
• Given a set of images of a scene, determine pixel
  correspondences between images, then provide a
  morph that will allow the user to view the scene
  from arbitrary locations, by morphing between
  images
• Drawbacks
• Camera position and orientation, as well as range
  data, needed

4
View Interpolation for Image Synthesis (2)

• Main Points
• Computing pixel correspondences
• Compositing images
• Overlaps
• Holes
• Optimizations
• Block compression
• View independent visible priority

5
Plenoptic Modeling

• Provides framework for IBMR
• Plenoptic function
• Describes all light at a particular location in
  space viewed a particular direction

6
Plenoptic Modeling (2)

• Take cylindrical images
• Use epipolar curves to find pixel correspondences
• Ordering algorithm developed to speed up rendering

7
Plenoptic Stitching

• 4D paramaterization of the plenoptic function
• Provides interactive walkthroughs of environments
• Capture the environment using an omnidirectional
  camera
• Construct image loops through the environment
• Image reconstruction based on the loop the viewer
  is currently in

8
Modeling and Rendering Architecture from
Photographs

• Goal
• Render architectural scenes without complete
  modeling
• Three contributions
• Photogrammetric modeling
• View-dependent texture mapping
• Model-based stereo

9
Modeling and Rendering Architecture from
Photographs

• Photogrammetric Modeling
• Mark edges on images, correspond them to blocks
  in the model
• Images are mapped to the blocks
• View-Dependent Texture Mapping
• Map original images onto the model based on the
  current viewpoint
• Model-based steropsis
• Compute depth map for key image
• Use that map for the image reprojection

10
View Morphing

• Goal
• create transitions between two images that are
  smooth and require no 3D information
• Method
• Prewarp the two images
• Morph
• Postwarp the image

11
View Morphing (2)

• Want shape preserving image interpolation
• Need
• Two images
• Their projection matrices
• Correspondence between pixels in the images
• Prewarp both images to common plane
• Morph interpolate positions an colors between
  prewarped images to new position
• Postwarp to put new image to correct position

12
Rendering with Concentric Mosaics

• 3D approximation of the plenoptic function
• Mount a camera on a beam
• Rotate the beam, taking images as you go
• Construct novel views based on slit images from
  the images you have taken
• Gives horizontal parallax

13
Multiple Center-of-Projection Images

• Rather than using a number of images, use a
  single image that has been captured from multiple
  locations
• Take slit images, moving the camera to new
  locations for each new slit image
• Combine the slit images into a single MCOP image
• Knowing the camera position for each slit image,
  the image can be projected for novel views

14
Video Mosaics for Virtual Environments

• Goal
• Mosaic images together automatically
• How
• Align different pieces of a scene into a larger
  picture, then seamlessly blend them together
• Must compute transformations that relate the
  pieces together
• Iterate using an error minimization routine

15
Video Mosaics for Virtual Environments (2)

• To deal with local minima
• Hierarchical matching
• Phase correlation
• Sidebar in the paper that discusses projective
  transformations

16
The Lumigraph

• 4D approximation to the Plenoptic Function
• Assume no participating media
• Images parameterized by two parallel planes (s,
  t, u, v)
• Capture images using hand-held camera and
  calibration markers
• Construct novel views from the given images

17
Light Field Rendering

• Similar to Lumigraph
• They use a computer controlled gantry to gather
  images
• Compression
• Vector quanitization
• Entropy coding

18
Layered Depth Images

• Main Ideas
• Use images to generate novel views
• Need more than traditional texture mapping
• Main contributions
• Sprites with depth
• Layered depth images

19
Layered Depth Images (2)

• Sprites with Depth
• Keep per pixel depths for sprites
• Gives internal parallax
• Layered depth images
• Depth pixel
• Store RGBA, Z, and splat index
• Layered Depth Pixel
• store of layers, array of depth pixels
• Layered Depth Image
• Store camera info, 2D array of layered depth
  pixels

20
Layered Depth Images (3)

• Viewing an LDI
• Warp LDI to new view
• Splat pixels to output image
• Perform warp using McMillans ordering algorithm
  based on the epipolar point

21
Tour Into The Picture

• Idea
• Make an animation from a 2D image
• Steps
• Separate foreground objects from background
• Determine vanishing point, set up perspective
• Model background with 5 polygons
• Create billboards for foreground objects
• Allow the user to fly through the scene