Spacetime Faces High-Resolution Capture for Modeling and Animation

1
Spacetime FacesHigh-Resolution Capture for
Modeling and Animation

• Li Zhang, Noah Snavely, Brian Curless, Steve
  Seitz
• University of Washington

2
Motivation
Anatomy-based Methods
Data-driven Methods
3
Dynamic Shape Capture
4
Overview
5
Overview
6
Camera rig
7
Capture session
8
Input videos (640x480, 60fps)
9
Overview
10
Overview
11
(No Transcript)
12

Face surface
13

Face surface
14

Face surface
15

16
(No Transcript)
17
time
time
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
Shape sequence
Window locations
Window sizes and shears
27
(No Transcript)
28
Spacetime Stereo Results
29
Overview
30
Overview
31
A sequence of meshes
Template mesh
32
Warped template
33
Template mesh
Depth map
Warped template
Fitted template
34
Template mesh
Depth map
Warped template
Fitted template
35
Template mesh
time2
Depth map
Warped template
Fitted template
36
Template mesh
time2
Depth map
time2
Fitted template
time1
37
Fitted template
38
Fitted template
39
Fitted template
40
Overview
41
Overview
42
(No Transcript)
43
(No Transcript)
44
Weight


45
Weight
46






47
Overview
48
Overview
49
Data-Driven Animation
Input Sequence
50
d( , )
51
i
i 1
j
52
(No Transcript)
53
Upper face weight map
Random walk state
Composite face
Upper face graph

Lower face graph
Random walk state
Lower face weight map
54
(No Transcript)
55
Keyframe 2
Keyframe 1
56
Keyframe 2
Keyframe 1
57
(No Transcript)
58

59

60
Two-region linear interpolation
Data-driven interpolation
61
(No Transcript)
62
Contributions

• High spatiotemporal resolution shape capture
• High resolution motion capture
• Modeling captured facial dynamics for
• editing
• animation

63
Limitations and future work
? Adaptive window
? Infra red projection
? Better template fitting
? Better extrapolation
64
Limitations and future work
? Infra red projection
Spacetime stereo reconstruction
Modified DLP projector
65
More future work

• DOF of an individual face
• Decomposition of expression and speech
• Style description for many faces

66
Acknowledgments

• Kiera Henning, Terri Moore, Ethel Evans
• Brett Allen, Jiwon Kim, Wilmot Li,
• Yung-Yu Chuang, Nan Li
• National Science Foundation
• UW Animation Research Labs
• Office of Naval Research
• Microsoft Cooperation

67
(No Transcript)