Editing and approximating deforming surfaces

1
Editing and approximating deforming surfaces

• Preliminary Examination
• Scott Kircher

2
Deforming surfaces
3
Why edit deforming surfaces?

• Avoid endless hours of tweaking simulations
• Get close enough, then edit

4
Why edit deforming surfaces?

• Modify motion-captured data

5
Why edit deforming surfaces?

• Reuse existing deforming surface data

6
Overview

• Approximating
• Published SCA 2005
• Spatial editing (GSP modeling)
• To appear in SIGGRAPH 2006
• Temporal editing (TSP keyframing)
• Ongoing proposed work

7
Approximation
Progressive Multiresolution Meshes for Deforming
Surfaces Symposium on Computer Animation, 2005
8
Visual Abstract
17,489v
9
Visual Abstract
17,489v
3,200v
10
Visual Abstract
11
Visual Abstract
12
Motivation

• Too much detail.
• Too much motion.
• Per-frame simplification wasteful and incoherent.

13
Related Work
14
Static/Static Simplification

• Progressive Meshes Hoppe 1996
• Quadric Error Metric Garland Heckbert 1997

Garland Heckbert 1997
Hoppe1996
15
Static/Dynamic Simplification

• Deformation sensitive decimation Mohr Gleicher
  2003
• Pose independent simplification DeCoro
  Rusinkiewicz 2005

16
Dynamic/Static Simplification

• Selective Refinement and Vertex Hierarchies
• View dependent simplification
• Xia Varshney 1996
• Hoppe 1997
• Luebke Erikson 1997

Hoppe 1997
17
Dynamic/Dynamic Simplification

• Time-dependent Directed Acyclic Graph
  (T-DAG) Shamir et al. 2000, 2001

18
Vertex Hierarchies as Hierarchical Clustering
(our view)
19
Vertex Hierarchies as Hierarchical Clustering
(our view)
20
Vertex Hierarchies as Hierarchical Clustering
(our view)
21
Prior Reclustering Work

• Kernighan-Lin algorithm 1970
• Multilevel scheme for graph partitioning Karypis
  Kumar 1998
• Probabilistic mesh reclustering using bloom
  filters Carr Hart 2004

Carr Hart 2004
22
The Main Idea
23
Approximating Deforming Surfaces

• Build initial approximation hierarchy
• Transfer hierarchy to next frame
• Improve hierarchy incrementally

24
How Does it Work?
25
Reclustering the Multilevel Mesh

• Edge-contraction based simplification
• Contractions specify clusters as well as
  approximation
• Reclustering
• Clusters can be used to rebuild approximation
• Changing clusters produces new approximation

26
Reclustering via swapping

• The fundamental swap operation (v,a,b)

27
Reclustering via swapping

• The fundamental swap operation (v,a,b)

28
Swap Validity and Priority

• Validity When can we swap?
• Priority When should we swap?

?
29
Swap Validity

• Validity When can we swap?
• We can guarantee
• Approximation could have been obtained by normal
  edge-contraction.
• Clusters remain connected
• For any arbitrary original mesh.

30
Swap Validity

• Validity When can we swap?
• We can guarantee
• Approximation could have been obtained by normal
  edge-contraction.
• Clusters remain connected
• For any arbitrary original mesh.
• Approximation is homeomorphic to original surface
• For manifolds
• Optional

31
Swap Priority

• Priority When should we swap?
• Perform swaps of highest benefit first
• Benefit is how much swap reduces error
• Quadric Error Metric Garland Heckbert 1997

32
Hierarchical Reclustering

• To improve entire hierarchy
• Perform reclustering at each level, from coarsest
  to finest
• Error metric is weighted sum of quadric errors
  from each level

33
Deforming Surfaces Revisted
34
Approximating Deforming Surfaces

• Sequence of swaps can be recorded
• Can be later played back in real-time
• Progressive format
• Initial hierarchy
• Swaps for each frame

35
Results
36
Horse To Man Morph
37
Collapsing Horse
38
Selective Refinement
39
Wind Whipped Cloth

• Homeomorphism preservation Texture mapping

40
Example Cloth LOD
41
Comparison
Our Result
Original
Our Mesh
42
Comparison
Our Result
Original
Our Mesh
Uniform Mesh
43
Performance

• Analysis phase comparable to edge-contraction
  based simplification (seconds per frame)
• Full hierarchy updating (for VDR)
• Input and output dependent
• A few milliseconds for medium size meshes
• Connectivity updating (non-VDR)
• Output dependent only
• See flying cows video
• 20 of frame time at 20fps.
• Vertex count lt 0.5 of original for 550 capes.

44
Approximation Conclusion

• Multiresolution meshes for deforming surfaces
• Use reclustering to update hierarchy as surface
  deforms
• Can guarantee various validity properties
• Cluster connectedness
• Homeomorphism to original surface
• Progressive representation
• Store sequence of updates at each frame
• Real-time playback of updates
• Compatible with existing VDR methods

45
Spatial Editing
Editing Arbitrarily Deforming Surface
Animations To appear in SIGGRAPH 2006
46
Visual Abstract
47
Motivation

• Review
• Ease simulated surface development

48
Motivation

• Review
• Ease simulated surface development
• Modify motion captured cloth

49
Motivation

• Review
• Ease simulated surface development
• Modify motion captured cloth
• Reuse deforming surface data

50
Related Work
51
Related Work

• Irregular mesh multiresolution editing
• Interactive multi-resolution modeling on
  arbitrary surfaces Kobbelt1998
• Multiresolution Signal Processing for Meshes
  Guskov1999

52
Related Work

• Deforming surface editing
• Skinning Mesh Animations Twigg2005

53
The Main Idea
54
Main Idea

• Multiresolution editing
• Preserve surface detail
• Better edit transport behavior
• Geometric signal processing

55
Main Idea

• Irregular mesh multiresolution hierarchy depends
  on geometry
• Temporal coherence important
• Use swap-based hierarchy adaptation method
• Basis smoothing

56
Why change the hierarchy?

• Hierarchy encodes frequency content of surface

Vital for signal processing
57
Why change the hierarchy?
Static wavelet basis
58
Why change the hierarchy?
Static wavelet basis
59
Why change the hierarchy?
Static wavelet basis
Adaptive relaxation operators (Static hierarchy)
60
Why change the hierarchy?
Static wavelet basis
Adaptive relaxation operators (Static hierarchy)
61
Why change the hierarchy?
Static wavelet basis
Fully adaptive basis (Our result)
Static hierarchy
62
How Does it Work?
63
Mesh Pyramid Transform
Multilevel Mesh Hierarchy
64
Mesh Pyramid Transform
Subdivide and Relax
Relaxation Stencil
Multilevel Mesh Hierarchy
65
Mesh Pyramid Transform
Subdivide and Relax
Subtract subdivided M2 geometry from original M1
geometry to get detail vectors. Store in local
coordinate frames.
66
Time-Varying Transform (TVT)

• Adapt hierarchy using swap-based method
• And recompute relaxation coefficients
• Still not fully coherent

67
Time-Varying Transform (TVT)

• Basis Smoothing (Basis Voting)
• Average editing results over nearby bases
• Diffuses effects of combinatorial changes

Important Smooth over space of bases, not over
time
68
Time-Varying Transform (TVT)

• Signal processing result with basis smoothing

69
Blockification

• Acceleration technique for basis smoothing
• Dont need a new basis every frame
• Break sequence into blocks

70
Editing

• Direct manipulation
• Multiresolution Embossing
• Constraint Editing
• Signal Processing

71
Transporting Edits

• Edit replicators
• Associate all edits with finest level mesh

72
Transporting Edits

• Edit replicators
• Associate all edits with finest level mesh

73
Transporting Edits

• Edit replicators
• Associate all edits with finest level mesh

74
Transporting Edits

• Edit replicators
• Associate all edits with finest level mesh

75
Transporting Edits

• Edit replicators
• Associate all edits with finest level mesh

76
Transporting Edits

• Edit replicators
• Associate all edits with finest level mesh

77
Multiresolution Embossing
78
Multiresolution Embossing
79
Multiresolution Embossing
80
Multiresolution Embossing
81
Constraints

• Ignore detail vector, move directly to
  constrained location when reconstructing

82
Results
83
Wheres my skeleton!?
84
Embossing Comparison
85
More Embossing Results
86
Stretch and Crumple
87
A napkin? No, its a sail!
88
Barnyard Superheroes
89
Spatial Editing Conclusion

• Time-Varying Transform
• Supports multiscale spatial operations
• Maintains temporal coherence
• Editing Arbitrarily Deforming Surfaces
• Direct Manipulation
• Multiresolution Embossing
• Constraint Editing
• Signal Processing

90
Temporal Signal Processing andKeyframing
Ongoing and Proposed Work
91
Goals

• Signal processing of motion itself
• Orthogonal to spatial signal processing

92
Goals

• Signal processing of motion itself
• Orthogonal to spatial signal processing
• Inserting new keyframes into existing motion

93
Goals

• Signal processing of motion itself
• Orthogonal to spatial signal processing
• Inserting new keyframes into existing motion
• Intuitive authoring of new keyframes
• Possibly by analyzing existing animation

94
Related Work
95
Deformation Gradients

• Deformation Transfer for Triangle Meshes
  Sumner2004
• Mesh-based Inverse Kinematics Sumner2005
• Deformation gradient decomposition
• Pose editing

96
Articulated Motion Editing

• Motion Signal Processing Bruderlin1995
• Motion Warping Witkin1995

Witkin1995
97
Current Ideas Preliminary Results
98
Motion Representation

• Canonical Deformation Gradient
• Affine transformation of canonical triangle to
  current triangle
• Decomposed into rotational and stretch/skew parts

(q1,S1)
99
Motion Representation

• Triangle Path
• Sequence of canonical deformation gradients over
  time for single triangle
• Can modify each path separately
• Recombine using least-squares solve

100
Temporal Signal Processing

• Apply standard 1D multi-resolution pyramid
  transform to triangle paths
• Linear part (stretch/skew) easy
• Angular part (rotations) requires quaternion
  operations

101
Frequency Decomposition

• Original Horse Animation

102
Frequency Decomposition

• DC component

103
Frequency Decomposition

• Low angular frequencies

104
Frequency Decomposition

• Medium angular frequencies

105
Frequency Decomposition

• High angular frequencies

106
Frequency Decomposition

• Low linear frequencies (amplitude ?5)

107
Frequency Decomposition

• Medium linear frequencies (amplitude ?5)

108
Motion Enhancement

• Original Horse Animation (Reminder)

109
Motion Enhancement

• Medium-high angular band exaggerated

110
Cloth Signal Processing

• Medium-high frequencies quintupled
• Highest frequency band damped

111
Keyframing

• Compute residual between keyframe and exiting
  frame
• Blend residual into other frames

112
Keyframing
113
Summary
114
Proposed Thesis Components

• Dynamic Surface
• Approximation
• Adaptive Multiresolution Hierarchy
• Spatial Multiresolution Editing
• Geometric Signal Processing
• Direct Manipulation
• Embossing
• Constraint Editing
• Temporal (Motion) Editing (Work in progress)
• Temporal Signal Processing
• Keyframe Insertion
• Intuitive Keyframe Authoring (Planned work)

115
Questions?