Multiview Radial Catadioptric Imaging for Scene Capture

1
Multiview Radial Catadioptric Imaging for Scene
Capture
Sujit Kuthirummal Shree K. Nayar
Columbia University
SIGGRAPH Conference July 2006, Boston, USA
Supported by NSF
2
(No Transcript)
3
(No Transcript)
4
(No Transcript)
5
Multiview Radial Catadioptric Imaging
Multiview Radial Catadioptric Imaging
Cameras Viewpoint
6
Multiview Radial Catadioptric Imaging
Virtual Viewpoint 1
Cameras Viewpoint
Virtual Viewpoint 2
7
Multiview Radial Catadioptric Imaging
Virtual Viewpoint 1
Cameras Viewpoint
Virtual Viewpoint 2
8
Multiview Radial Catadioptric Imaging
Circular Viewpoint Locus
9
(No Transcript)
10
Captured Image
11
Captured Image
Captured Image
12
Captured Image
Captured Image
Epipolar Lines are Radial
13
(No Transcript)
14
Catadioptric Systems in Graphics

• Several catadioptric systems have been proposed
• Dana 2001, Han et al 2003, Unger et al
  2003, Levoy et al 2004, Hawkins et al 2005, etc
• Most designed for a specific application
• We propose a family of catadioptric imaging
  systems
• Specific members are suited for different
  applications

15
Characterizing a Radial Imaging System
16
Family of Radial Imaging Systems
17
Characterizing a Radial Imaging System
18
(No Transcript)
19
Properties of Radial Imaging Systems

• Virtual Viewpoint Locus
• Fields of View
• Resolution Characteristics

Locus Radius
Locus Distance
Real Viewpoint
Virtual Viewpoint
Radial
Tangential
20
Properties of Radial Imaging Systems

• Virtual Viewpoint Locus
• Fields of View
• Resolution Characteristics

Locus Radius
Locus Distance
Real Viewpoint
Virtual Viewpoint
Radial
Tangential
21
Properties of Radial Imaging Systems

• Virtual Viewpoint Locus
• Fields of View
• Resolution Characteristics

Locus Radius
Locus Distance
Real Viewpoint
Virtual Viewpoint
Radial
Tangential
22
Properties of Radial Imaging Systems

• Virtual Viewpoint Locus
• Fields of View
• Resolution Characteristics

23
(No Transcript)
24
Applications

• Reconstructing 3D Objects
• Reconstructing 3D Textures
• BRDF Sampling
• Complete Texture Maps
• Complete Object Geometry

25
Applications

• Reconstructing 3D Objects

• BRDF Estimation

• Complete Texture and Object Geometry

26
Reconstructing 3D Objects
27
Reconstructing 3D Objects
Viewpoint Locus
28
Reconstructing 3D Objects
Binocular
Trinocular
Binocular
Field of View
Vergence
29
Reconstructing 3D Objects
Binocular
Trinocular
Binocular
Vergence
30
Reconstructing 3D Objects

• Related Work
• Nayar 1988, Southwell et al 1996, Nene et
  al 1998, Gluckman et al 1999,
• Radial Epipolar Geometry
• More Robust Structure Recovery

31
Reconstructing Faces
Camera
Mirror
Subject
32
Reconstructing Faces
33
Stereo Views
34
Reconstructing Faces
35
(No Transcript)
36
Reconstructing 3D Textures
37
Reconstructing 3D Textures
Viewpoint Locus
38
Reconstructing 3D Textures
Binocular
Trinocular
Binocular
Field of View
39
Reconstructing 3D Textures
Mirror
Object
Camera
40
Reconstructing 3D Textures
Bark
Bread
41
Reconstructing 3D Textures
Bark
Bread
42
Synthesizing Textures
(R,G,B) Texture
(R,G,B) Texture
2D Texture Synthesis Efros et al 1999, Efros et
al 2001, Kwatra et al 2003,
43
Synthesizing 3D Textures
(R,G,B,Z) Texture
(R,G,B,Z) Texture
Modified Image Quilting Efros et al 2001
44
Synthesizing 3D Textures
45
(No Transcript)
46
(No Transcript)
47
One Reflection
48
Two Reflections
49
Virtual Viewpoints 1 reflection
Virtual Viewpoints 2 reflections
n Reflections n Circular Loci of
Virtual Viewpoints
50
Sampling and Estimating BRDFs
Ward 1992, Dana 2001, Han and Perlin 2003,
Hawkins et al 2005
51
Sampling and Estimating BRDFs
Mirror
Sample
Camera
(4 Reflections)
52
Sampling and Estimating BRDFs
Metallic Paint
Red Satin Paint
53
Sampling and Estimating BRDFs
Light Direction
Normal
Specular Direction
54
Sampling and Estimating BRDFs
Light Direction
Normal
Specular Direction
55
Sampling and Estimating BRDFs
Light Direction
Normal
Viewing Directions
Specular Direction
56
Sampling and Estimating BRDFs
Light Direction
Normal
Viewing Directions
Specular Direction
57
Sampling and Estimating BRDFs
Analytic BRDF Model Oren-Nayar (Diffuse)
Torrance-Sparrow (Specular)
Metallic Paint
Red Satin Paint
58
Sampling and Estimating BRDFs
Analytic BRDF Model Oren-Nayar (Diffuse)
Torrance-Sparrow (Specular)
Metallic Paint
Red Satin Paint
59
Sampling and Estimating BRDFs
Red Satin Paint
Metallic Paint
60
(No Transcript)
61
Complete Texture Maps of Convex Objects
62
Complete Texture Maps of Convex Objects
Viewpoint Locus
63
Complete Texture Maps of Convex Objects
Field of View
64
Complete Texture Maps of Convex Objects
Field of View
65
Complete Texture Maps of Convex Objects
66
Complete Texture Maps of Convex Objects
Davidhazy 1987, Seitz and Kim 2002
67
Complete Texture Maps of Convex Objects
Davidhazy 1987, Seitz and Kim 2002
68
Complete Texture Maps of Convex Objects
Camera
Sample
Mirror
69
Complete Texture Maps of Convex Objects
Conical Object
Cylindrical Object
70
Complete Texture Maps of Convex Objects
Cylindrical Object
Conical Object
71
Complete Geometry of Convex Objects
72
Complete Geometry of Convex Objects
Farther Image
Nearer Image
Toy Head
73
Complete Geometry of Convex Objects
74
(No Transcript)
75
Resolution

• Benefits come at the price of resolution
• Multiple views projected on single image detector
• Camera resolution is increasing rapidly

76
(No Transcript)
77
Multiview Radial Catadioptric Imaging

• Family of Imaging Systems
• Analyzed Properties

78
(No Transcript)
79
Reconstructing Faces
Camera
Mirror
Subject
80
Reconstruction Accuracy
Plane at 40 cm Rms Error of Best Fit Plane
0.083 cm
81
Reconstructing 3D Textures
Mirror
Object
Camera
82
Reconstruction Accuracy
Radius of circular cross-section 3.739 cm Best
fit circles radius 3.557
cm Rms Error of Best Fit Circle 0.026 cm
83
(No Transcript)
84
Resolution Tangential
Captured Image
85
Resolution Tangential
Captured Image
Tangential Resolution is DepthDependent

86
Resolution Tangential
Captured Image