Estimating Cloth Simulation Parameters from Video

1
Estimating Cloth Simulation Parameters from Video

• Kiran S. Bhat, Christopher D. Twigg, Jessica K.
  Hodgins, Pradeep K. Khosla, Zoran Popovic and
  Steven M. Seitz,
• School of Computer Science, Carnegie Mellon
  University
• Department of Computer Science and Engineering,
  University of Washington
• Presented by
• Ratko Jagodic

2
Cloth

• Need realism for mixed CG/real scenes
• Realism now possible

3
Simulating Cloth

• Cloth characteristics determined by resistance
  to
• Bending, stretching, shearing, external forces,
  aerodynamic effects, friction and collisions
• Great results are achievable with the right set
  of parameters
• BUT, difficult and time consuming to choose
  parameters
• Different approach estimate parameters from
  real-life video of cloth movement

4
Related Work

• Various cloth models
• Using stiff springs (Baraff and Witkin)
• Bending energy model (Choi and Ko)
• Modeling cloth collisions and friction
• Untangling cloth
• Preemptively avoiding collisions
• Estimating parameters
• Mechanical tests measuring force needed for
  deforming (Breen)
• Searching for parameters that match a piece of
  real cloth draped over a sphere (Jojic and Huang)
• Neither considered dynamic parameters

5
Proposed Approach

• Estimate parameters from the video of moving
  cloth pieces
• Choose simulated cloth parameters based on a
  cloth model
• Apply an optimization function until the
  simulated cloth behaves like the real one from
  the video

6
Cloth Model

• Cloth model requirements
• Realism and practicality
• Scalable to varying resolutions of cloth
• Input parameters independent of meshing
• Baraff and Witkin cloth model was chosen with
  minor adjustments
• Collisions based on Bridsons model (again,
  slightly modified)

7
A Metric for Matching Simulation to Video

• Two sequences compared frame-by-frame
• Average error is computed across the whole
  sequence
• Perceptually motivated metric
• Human perceptual system is sensitive to moving
  edges in video
• Moving edges folds in cloth
• The metric returns the difference in folds
  between the real and simulated cloth

8
Fold Detection and Representation

• Project striped pattern on pieces of cloth to
  eliminate lighting and material reflectance
• Compute dominant orientations of edge pixels ?
  angle map
• Threshold the gradient of the angle map ?
  gradient mask

9
Fold and Silhouette Comparison

• Angle map from video Angle map from simulation
• Angle map difference Angle map diff x
  gradient mask
• Silhouette Comparison the difference between
  the shapes

10
Optimization

• Before optimization
• 4 values are chosen for each parameter across its
  entire range
• Then simulate the fabric for each of those 4
  points and compare with the real fabric.
• Use the best of the 4 points to start the
  optimization with
• Optimization
• Uses simulated annealing for n-variable
  optimization

11
Experiments

• Perform simpler experiments that would yield
  parameters for the cloth model calibration
• Fabric
• Linen, fleece, satin and knit
• Static Test
• How the fabric naturally hangs under gravity
• Waving Test
• One corner is fixed and the other moved back and
  forth
• Swatches of cloth were used
• give reasonable estimates without the need to
  optimize on complex fabric geometries

12
Experimental Setup

• Setup
• Vicon motion capture system for locating two top
  corners of fabric
• Motion capture markers for calibrating the camera
  and the projector
• 2.8 GHz Xeon processor
• Trials
• 2 trials per fabric
• 1 trial 50 frames of video
• each takes 50 hours to optimize
• Parameters
• Bend, stretch, shear, bend damping, stretch
  damping, shear damping, linear drag, quadratic
  drag, drag degradation

13
Results Static Test

• Good estimates for the parameters that have low
  variability (stretch and bend)
• Bad estimates for high variability parameters
  since they do not contribute much to the total
  error (dynamic parameters like damping)

14
Results Waving Test

• Added air drag parameters
• Results good but probably needed longer or more
  complex sequences since a few parameters still
  had high variability

15
More Results

• Evaluated obtained parameters on longer sequences
  (150 frames) good match
• Evaluated on a more complex piece of cloth (a
  skirt) approximately correct

16
Conclusion

• The estimation produces parameters that yield
  appealing results.
• Realism in the end is still dependent on the
  cloth model so good ones are needed
• Future work
• Different metrics could be explored (e.g. time
  varying as opposed to frame-by-frame)
• More complex garments may need additional test
  that mimic behaviors better suited to a
  particular application of fabric (such as form
  fitting pants)
• Different parameters are needed for more complex
  effects (e.g. stretching in one direction causing
  shrinking in the other)

17
Video

• Movie

18
Questions?
19
Simulated Annealing