CSCE 641 Computer Graphics: Image-based Rendering (cont.)

1
CSCE 641 Computer Graphics Image-based
Rendering (cont.)

• Jinxiang Chai

2
Outline

• Light field rendering
• Plenoptic sampling (light field sampling)
• Layered depth image/Post-Rendering 3D Warping
• View-dependent texture mapping
• Unstructured lumigraph

3
Layered depth image Shade et al, SIGGRAPH98
Layered depth image - image with depths
4
Layered depth image Shade et al, SIGGRAPH98
Layered depth image - rays with colors and
depths
5
Layered depth image Shade et al, SIGGRAPH98
Layered depth image (r,g,b,depth) - image with
depths - rays with colors and depths
6
Layered depth image Shade et al, SIGGRAPH98
Rendering from layered depth image
7
Layered depth image Shade et al, SIGGRAPH98
Rendering from layered depth image

• - Incremental in X and Y
• - Forward warping one pixel with depth

8
Layered depth image Shade et al, SIGGRAPH98
Rendering from layered depth image

• - Incremental in X and Y
• - Forward warping one pixel with depth

9
Layered depth image Shade et al, SIGGRAPH98
Rendering from layered depth image
How to deal with occlusion/visibility problem?

• - Incremental in X and Y
• - Forward warping one pixel with depth

10
How to form LDIs

• Synthetic world with known geometry and texture
• - from multiple depth images
• - modified ray tracer
• Real images
• - reconstruct geometry from multiple images
  (e.g.,
• voxel coloring, stereo reconstruction)
• - form LDIs using multiple images and
• reconstructed geometry
• Kinect sensors
• - record both image data and depth data

11
Image-based Rendering Using Kinect Sensors

• Capture both video/depth data using kinect
  sensors
• Using 3D warping to render a video from a novel
  view point e.g., Post-Rendering 3D Warping
• Demo click here

12
Outline

• Light field rendering
• Plenoptic sampling (light field sampling)
• Layered depth image/Post-Rendering 3D Warping
• View-dependent texture mapping
• Unstructured lumigraph

13
View-dependent surface representation
From multiple input image - reconstruct the
geometry - view-dependent texture
14
View-dependent surface representation
From multiple input image - reconstruct the
geometry - view-dependent texture
15
View-dependent surface representation
From multiple input image - reconstruct the
geometry - view-dependent texture
16
View-dependent surface representation
From multiple input image - reconstruct the
geometry - view-dependent texture
17
View-dependent texture mapping
Debevec et al 98
18
View-dependent texture mapping

• - Virtual camera at point D
• Textures from camera Ci mapped onto triangle
  faces
• Blending weights in vertex V
• Angle ?i is used to compute the weight values
• wi exp(-?i2/2s2)

Subject's 3D proxy points
V
q
q
0
3
q
q
1
2
C
C
C
C
0
2
3
1
D
19
Videos view-dependent texture mapping
20
Outline

• Light field rendering
• Plenoptic sampling (light field sampling)
• Layered depth image/Post-Rendering 3D Warping
• View-dependent texture mapping
• Unstructured lumigraph

21
The Image-Based Rendering Problem

• Synthesize novel views from reference images
• Static scenes, fixed lighting
• Flexible geometry and camera configurations

22
The ULR Algorithm Siggraph01

• Designed to work over a range of image and
  geometry configurations

of Images
Geometric Fidelity
23
The ULR Algorithm Siggraph01

• Designed to work over a range of image and
  geometry configurations

of Images
ULR
Geometric Fidelity
24
The ULR Algorithm Siggraph01

• Designed to work over a range of image and
  geometry configurations
• Designed to satisfy desirable properties

of Images
ULR
Geometric Fidelity
25
Light Field Rendering, SIGGRAPH 96
Desired color interpolated from nearest cameras
26
Light Field Rendering, SIGGRAPH 96
27
The Lumigraph, SIGGRAPH 96
28
The Lumigraph, SIGGRAPH 96
29
The Lumigraph, SIGGRAPH 96
30
The Lumigraph, SIGGRAPH 96
Rebinning
Note all images are resampled.
31
The Lumigraph, SIGGRAPH 96
32
View-Dependent Texture Mapping, SIGGRAPH 96,
EGRW 98
33
View-Dependent Texture Mapping, SIGGRAPH 96,
EGRW 98
34
View-Dependent Texture Mapping, SIGGRAPH 96,
EGRW 98
?1
?3
?2
35
View-Dependent Texture Mapping, SIGGRAPH 96,
EGRW 98
?1
?3
?2
36
The Scene
37
(No Transcript)
38
Demo