Image-Based Rendering

1
Image-Based Rendering
2
3D Scene Shape Shading
Source Leonard mcMillan, UNC-CH
3
Modeling is Hard!

• Good 3D geometric models need a lot of work.
• Getting correct material properties is even
  harder
• R, G, B, Ka, Kd, Ks, specularity for Phong model.
• BRDF
• Subsurface reflectance, diffractionetc.

4
How to Create 3D Contents

• AutoCAD used for architectures (buildings)
• 3D Studio Max, Blenderetc.
• Maya is a major production tool used in movie
  studios.
• Problems? It takes an artist, and its still
  hard to make it look real!

5
QuickTime VR
6
3D Photography

• Can building 3D models be as easy as taking 2D
  photos?
• How do we digitize the massive assets in various
  museums?
• QuickTime VR object movies
• 3D Scans Cyberware scanner, Digital Michelangeo

Source www.cyberware.com
7
Image-Based Rendering

• Can we build 3D contents from photographs
  directly?
• Difference from computer vision?
• Can we make the objects look more real?
• Difference from texture mapping?

8
Top Level Survey
3D Graphics
Geometry or Surface Based Rendering Modeling
Sample-Based Graphics
Image-Based Rendering Modeling
Volume Rendering
9
Traditional Computer Graphics

• Input Geometry, Material Properties (Color,
  Reflectance,etc.), Lighting.
• Transformation and Rasterization.

3D Graphics Pipeline
10
Role of Images

• Used as textures.
• Or, as input to computer vision methods in order
  to recover the 3D models.

Vision
11
Image-Based Rendering

• To bypass the 3D models altogether.

12
Image-Based Rendering

• Input Regular Images or Depth Images."
• No 3D model is constructed.
• Example 3D Warping.

13
3D Warping Another Example

• Reading room of UNC CS department
• Source images contain depths in each pixel.
• The depths are obtained from a laser range finder.

14
Why IBR?
Geometry IBR
Modeling Difficult Easy
Complexity triangles pixels
Fidelity Synthetic Acquired

• Problems of triangle-based graphics
• Always starts from scratch.
• Millions of sub-pixel triangles.

15
Why is It Possible?

• 5D Plenoptic Function.
• Color f(x, y, z, ?, ?)
• (x, y, z) defines the viewpoint.
• (?, ?) defines the view direction.
• 4D Light Field/Lumigraph
• Color f(u, v, s, t)
• (u, v) defines the viewpoint.
• (s, t) defines the pixel coord.

Picture source Leonard McMillan
16
3D Image Warping

• Each pixel in the source images has coordinates
  (u1, v1), depth info d1, and color.
• Warping Equation is applied to each pixel
• (u2, v2) f(u1, v1, d1)
• ( a?u1b?v1cd?d1 , e?u1f?v1gh?d1 )
• i?u1j?v1kl?d1 i?u1j?v1kl?d1
  
• where variables a to l are fixed for the same
  view.
• Rendering Time O(pixels)

17
(No Transcript)
18
v1
v2
u1
u2
19
Artifacts of 3D Image Warping

• Surfaces that were occluded in source images.
• Non-uniform sampling (an example in the next
  slide).

20
Reconstruction
21
Using Multiple Source Images
22
IBR Survey
Image-Based Rendering Modeling
Light Field Rendering (or Lumigraph) Stanford/Micr
osoft
3D Warping (or Plenoptic Modeling) UNC-Chapel Hill
23
Light Field Lumigraph
24
Images as 4D Samples

• Consider each image pixel a sample of 4D Light
  Field.

25
Does it Matter Where We Place the Planes?

• Yes!
• Depth correction in Lumigraphs

26
Concentric Mosaic

• Hold a camera on a stick, then sweep a circle.
• Viewpoint is constrained on a 2D plane.
• Reducing the 4D light field to a 3D subspace.

27
Surface Light Field

• May be considered a compression scheme for light
  field data.
• 3D geometry required.

28
LYTRO Light Field Camera

• See https//www.lytro.com/science_inside

29
Further Reading on Light Field

• See Marc Levoys IEEE Computer 2006 article at
  http//graphics.stanford.edu/papers/lfphoto/levoy-
  lfphoto-ieee06.pdf

30
Camera Array
Source http//sabia.tic.udc.es/gc/Contenidos20ad
icionales/trabajos/Peliculas/FX/ej3.html
31
Image-Based Lighting -- Light Stage

• Take photos under single point light at various
  positions.
• Trivial questions how to produce new images at
• Two point lights?
• Area light?
• Environment light (captured by light probe)?

32
(No Transcript)