A High Resolution Geometry Capture System for Facial Performance

1
(No Transcript)
2
A High Resolution Geometry Capture System for
Facial Performance

• Wan-Chun Ma
• Andrew Jones
• Tim Hawkins
• Jen-Yuan Chiang
• Paul Debevec
• USC Institute for Creative Technologies

3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
Gradient illumination
Real-time acquisition
Motion capture
7
Other Acquisition Systems
Gradient Illumination
Wrinkle measurement
Structured light
Ma et al. 2007
Li Zhang et al. 2004
Bickel et al. 2007
Song Zhang et al. 2006
8
LCD Projector
Canon 1ds Mark III Stereo Cameras
9
Light Stage 5 162 White LEDS
10
Spherical Gradient Normals
X
y
z
Ma et al. Rapid Acquisition of Diffuse and
Specular Normal Maps from Polarized Spherical
Gradient Illumination, EGSR 2007
11
Structured light scan Diffuse normals
12
Spherical Gradient Normals
X
y
z
Ma et al. Rapid Acquisition of Diffuse and
Specular Normal Maps from Polarized Spherical
Gradient Illumination, EGSR 2007
13
Structured light scan Diffuse normals
14
Structured light scan Specular normals
15
Light Stage 5
LCD projector
V10
Canon 1ds Mark III
16
Light Stage 5
MULE high-speed projector
V10
Vision Research Phantom V10 cameras
17
(No Transcript)
18
High-speed video
19
Gradient illumination
Projector patterns
Repeat patterns 24-30 times per second
20
Mechanical polarizer flipper
21
Ferro-electric shutter
Diffuse
Specular
22
Polarized beam splitter
Ferroelectric shutter
Diffuse
Specular
23
Color space seperation
Ferroelectric shutter
Diffuse
Specular
Beam splitter
24
(No Transcript)
25
Rendering using color-space normal maps
26
(No Transcript)
27
(No Transcript)
28
Face replacement

• Rendered face (closeup)

29
Face replacement

• Composite

30
?
31
Tension Maps / Wrinkle maps
Christopher Oat, SIGGRAPH 2007 Courses Blend
between artistically drawn bump maps
32
Can we learn where wrinkles form?
33
Capture geometry motion capture
34
(No Transcript)
35
Register texture space
Texture space
Image space
36
Register texture space
Texture space
Image space
37
Wrinkle Changes vs. Deformation
38
Wrinkle Changes vs. Deformation
Angry
39
Wrinkle Changes vs. Deformation
Surprise
40
Polynomial Displacement Maps
(u,v)
2D strain space
Du,v(du,dv) a0du2 a1dv2 a2dudv a3du
a4dv a5
Displacement maps
41
Polynomial Texture Maps
(u,v)
2D lighting space
Lu,v(lu,lv) a0lu2 a1lv2 a2lulv a3lu
a4lv a5
Displacement maps
Malzbender et al. 2001
42
(No Transcript)
43
Neutral expression
Neutral expression without detail
44
Thin shell deformation using mocap points Botsch
2004
45
Add medium resolution 3D displacement
46
Result
Ground truth
Add high resolution 1D displacement
47
Result
Ground truth
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
(No Transcript)
52
(No Transcript)
53
(No Transcript)
54
Conclusions
Dynamic detail detail is important
High-resolution synthesis
High-resolution capture
55
Future Work

• Ear-to-ear geometry
• Eyes, mouth, hair
• Improved reflectance properties
• Optimal training dataset

56
SIGGRAPH 2008collaboration with
Tech Talk Getting Real with Emily Hall G, Room
2 August 13th, 1-230pm
57

• Website http//gl.ict.usc.edu

Wan-Chun Ma, A Framework for Capture and
Synthesis of High Resolution Facial Geometry and
Performance, PhD Thesis, National Taiwan
University, July 2008 Thanks Jay Busch,
Abhijeet Ghosh, Magnus Lang, Matthias
Hullin, Michael Weigand, Monica Nichelson, Tom
Pereira
58
(No Transcript)