Definition and realisation of modeling methods and motion computation algorithms for virtual humans

1
Definition and realisation of modeling methods
and motion computation algorithms for virtual
humans

• Nicolas Pronost

Advisor Bruno Arnaldi
Co-advisor Georges Dumont
Team IRISA - SIAMES
Thursday 7 December 2006
2
Where are they found ?

• Animation

• Video games
• Movies

• Motion sciences

• Biomechanics
• Medicine, Health
• Sports

• Robotics

• Bipedal robot

• Anthropology

• Modern human
• Fossilised hominids

• Simulation

• Motion analysis
• Motion synthesis

3
Fondamental aspects

• Modeling the human
• Symbolical and controlable representation
• Kinematical and physical properties
• Modeling the motion
• Manipulable mathematical representation
• Dependant of the editing methods
• Editing methods
• Based on motions laws
• Looking for generiness

4
Outline

• Related works
• Overview and motivations
• Analysis / synthesis loop
• Kinematical adaptation
• Evaluation of the dynamics
• Forward dynamics synthesis
• Conclusion and future work

5
Outline

• Related works
• Overview and motivations
• Analysis / synthesis loop
• Kinematical adaptation
• Evaluation of the dynamics
• Forward dynamics synthesis
• Conclusion and future works

6
Modeling a virtual human

• Simplification through a hierarchical
  representation of rigid bodies H-Anim 06
• Mechanical joints are perfect and the number of
  limbs is limited

• Consensus between the anatomical reality and
  motion control
• Manipulation of the rotational degrees of freedom

7
Biomechanical modeling

• The physical properties describe the movement
  capacity of the limbs
• At least the masses and inertias
• These data are avalaible from anthropometrical
  tables and computable from regression laws
  Vaughan et al. 99
• Cadaverical data Dempster 55, Winter 90
• Gamma radiography Zatsiorsky 90
• Various definitions of limbs Chandler et al. 75,
  De Leva 96
• Additionnal data
• Articular limits, muscular activations Liu et
  al. 05, articular elasticities

8
Modeling the motion

• Motion is a sequence of postures
• Mareys work Marey 1894 on motion decomposition
• Kinematical model
• Positions, velocities and acceleration
• Description of the possibility of motions
• Static and quasi-static (kinetical) models
• Usefull for slow motions
• Balance of internal and external forces
• Dynamical model
• Use of motor forces at joints

9
Motion editing methods

• Methods of kinematics and inverse kinematics
• Representation by splines Zeltzer 82, Bruderlin
  and Calvert 93
• Local linearisation and secondary tasks Boulic
  and Thalmann 92 Tolani and Badler 96

?1
?2
X
10
Motion editing methods

• Kinematics and control of the center of mass
• Important on quasi-static positions Phillips 91
• Projection of the center of mass on the
  sustentation polygon Boulic et al. 94
• Resolution using inverse kinematics with a
  priority formulation Boulic et al. 97
• Conservation of the Zero Moment Point
• Point of null result of linear momentum of the
  limbs Tak et al. 00
• Dynamics filtering of motions Yamane and
  Nakamura 03

11
Motion editing methods

• Methods of dynamics and inverse dynamics
• Animation engine using a system of secondary
  order differential equations Hodgins 98
• Newtons laws of motion and fondamental physical
  laws
• Virtual works and Lagrangian formalism Rémion 00

12
Manipulation of real movements

• Real movements intrinsically have all of the
  information of the motion
• Correct perception of the realism from few
  positions of caracteristical joints Johansson
  73
• Good realism of animations
• Low generiness
• Usable database for generating new motions

13
Manipulation of real movements

• Adaptation to new characters retargeting

14
Manipulation of real movements

• Adaptation to new characters retargeting
• With different morphologies Gleicher 98 thanks
  to spacetime constraints
• Use of intermediate skeletons Monzani et al. 00,
  Ménardais 03
• Decomposition of articular trajectories into
  hierarchical splines Lee and Shin 99
• Take account of muscular forces Komura et al. 00

15
Manipulation of real movements

• Modification of the motion
• Frequential description of the articular
  trajectories and deformation function Bruderlin
  and Williams 95
• displacement map, conservation of the high
  frequencies
• Description by key postures and interpolation on
  the parameters Witkin and Popovic 95
• Interpolation of the deformation function (scale
  and translation)

16
Manipulation of real movements

• Combining motions
• The motion graphs Kovar et al. 02
• node key posture
• arc possible transition
• Motion blending
• Synchronisation by dynamic time warping
  Bruderlin and Williams 95
• Blending by linear combinaison or weighted sum
  Guo and Robergé 96, Park et al. 02, Rose et al.
  98, Ménardais et al. 04

17
Manipulation of real movements

• Motion database
• Efficient interpolation on simple motions Wiley
  et Hahn 97
• Behavioral or frequential decomposition
• Fourier space Unuma et al. 95
• Radial basis function Rose et al. 96
• Hidden Markov chains Brand and Hertzmann 00
• Static models using PCA Bowden 00
• Physical simulations Zordan et al. 05, Arikan et
  al. 05, Tang et al. 06

18
Bipedal locomotion

• The locomotion is a cyclic movement
• Decomposable into phases Marrey 1894, Plat and
  Veil 83
• A step is a half cycle
• walk sequence of single and double support
  phases
• run sequence of single support and flying
  phases
• Specific modeling of locomotion
• With cyclic state machines Multon 98
• With deformed hypertorus from PCA Martineau 06
• Large number of biomechanical data
• Articular trajectories Alexander 84, Patla 91
• Support phases Girard 87

19
Summary

• The analysis and the synthesis of virtual human
  motions Gibet 02 answer to very differents
  constraints according to the application
• Compromise interactivity / realism / generiness

Kinematical corrections Interactivity of natural looking motions
Dynamical constraints Physical realism on specific motions
Spacetime resolution Offline and kinematical high control
Physics-based simulation Generic offline production of motions
Motions editing Balanced applications
20
Outline

• Related works
• Overview and motivations
• Analysis / synthesis loop
• Kinematical adaptation
• Evaluation of the dynamics
• Forward dynamics synthesis
• Conclusion and future works

21
Overview
Database of motions
Adaptation algorithm
adapted motion
synthetised motion
Synthesis by forward dynamics
Analysis of the dynamics
resulting forces

• Motivation to study and to realise a process of
  modeling tools and motion computation
• Application to locomotion

22
Outline

• Related works
• Overview and motivations
• Analysis / synthesis loop
• Kinematical adaptation
• Evaluation of the dynamics
• Forward dynamics synthesis
• Conclusion and future works

23
Kinematical adaptation
24
Modeling the human

• Definition of a kinematical chain with 11 dof
• Spherical joints at pelvis and hips
• Pin joints at knees

world
3 rotations
pelvis reference frame
3 rotations
3 rotations
right femur reference frame
left femur reference frame
1 rotation
1 rotation
right tibia reference frame
left tibia reference frame
25
Why use this model ?

• Application field in paleoanthropology
• Study of the bipedalism of fossilised hominids
• Australopithecus afarensis Lucy (A.L. 288-1)

pictures courtesy of G. Berillon
26
Modeling the locomotion

• Treatments on the motion
• Homogeneous reconstruction Ménardais 03
• Orientation of the locomotion
• Identification of the cycles
• Definition of the movement of the articular
  centers
• Computed from real landmarks
• Accurate positions of articular
  centers and virtual points

27
Modeling the locomotion

• A parametrical representation of the locomotion
  the poulaine

• Definition the Cartesian displacement of the
  ankle in the root reference frame

• Modeled by a cubic curve using 4 characteristic
  points of the cycle

28
Method of computation

• The principle of dimensional interpolation in the
  database
• Definition of the step size on x
• Definition of the step shift on y
• Definition of the rest posture on z

29
Post-treatments

• Adding the temporal dimension
• Use of an average profile of speed, normalised by
  the time cycle and distance on the ground
• Representation of the profile by a polynomial
  function
• Integrating the function, computing the
  curvilinear x-coordinate and the parameters of
  the cubic curves

• Synchronisation of the left and right poulaines

• By minimisation of vertical differences
• By minimisation of ground sliding

30
Post-treatments

• Computation of the postures by an IK solver
• Proposition of secondary tasks Nicolas et al.
  04
• (C1) Maximal distance from joint limits
• (C2) Minimisation of the kinematical energy of
  rotation
• (C3) Search of the closest posture to the rest
  posture
• Evaluation of these tasks by 3 criteria
• The total Jerk, third derivate of the angles
• The difference between the final and the initial
  posture
• The internal work

31
Post-treatments

• Preparation of the animation
• Construction of the foot and the ankle angle
• Feet lenghts from anthropometrical tables De
  Leva 96
• Trajectories of ankles computed by corrections of
  the ground penetrations
• To go to the global motion
• Global displacement minimising the sliding
• Upper body movement
• Adapted to the morphology and synchronised with
  the real motion

32
Results of the adaptation

• Validation of the interpolation

x axis x axis z axis Average
RMS (cm) 2.74 1.22 1.24 1.73
S.D. (cm) 1.18 1.12 0.76 0.82
33
Results of the adaptation

• Validation of the adaptation
• Comparison between real angular trajectories and
  adapted trajectories of 7 subjects in the database

Pelv. incl. Pelv. obliq. Pelv. rot. int/ext Hip flex/ext
Average (rad) 0.03 0.01 0.01 0.05
S.D. (rad) 0.03 0.01 0.01 0.03
Hip abd/add Hip rot. int/ext Knee flex/ext
Average (rad) 0.04 0.03 0.02
S.D. (rad) 0.04 0.01 0.01
34
Results of the adaptation

• Simulations

35
Partial summary

• The method computes a plausible locomotion from
  biomechanical knowledge and rules Pronost et al.
  05
• Controled by limbs sizes, bones configuration,
  physical parameters of the limbs, joints types,
  footprints, articular limits and the style of
  motion
• Applied to paleoanthropology Pronost et al. 06
• Future work
• Combining the method with a real time adaptation
  to the environment
• To drive the extrapolation by physical properties
• A global resolution of the IK to reduce
  discontinuity of pelvis angles
• Increase the size and the diversity of the
  database

36
Outline

• Related works
• Overview and motivations
• Analysis / synthesis loop
• Kinematical adaptation
• Evaluation of the dynamics
• Forward dynamics synthesis
• Conclusion and future works

37
Evaluation of the dynamics
38
Modeling the human

• Creation of a biomechanical model
• Description of Denavit-Hartenberg Hartenberg and
  Denavit 55
• Parameters of rotation
  user
• Parameters of translation
  auto
• Gender and nature of limbs
• Gender user
• Nature auto
• Using anthropometrical tables
  deLeva 96 and regression laws
  Vaughan et al. 99

39
Modeling the motion

• The mapping issue
• An iterative method on kinematical
  chains from the root joint to
  the effectors
• Using a sequencing of the articular
  systems
• Treatment according to the number
  of degrees of freedom
• 1dof gt pin joint, minimisation of the error
• 3 dof gt spherical joint, an infinity of
  solutions
• with constraints gt minimisation of the future
  error
• without constraints gt minimal rotation

40
Method of computation

• In order to solve the inverse dynamics issue, we
  have to know the external forces applied to the
  system, for locomotion
• the gravity
• constant value for any motion
• the aerodynamical forces
• supposed negligible
• the ground reaction forces
• When are they applied ?
• Support phase recognition

41
Support phase recognition

• Evaluation of 4 methods of ground contacts
  recognition
• hand-labeled, method of reference, accurate at
  the frequency of the motion capture system
• speed, evaluation of a speed threshold for the
  effectors
• height, evaluation of a height threshold for the
  effectors
• configuration, particular configurations of the
  effectors
• By four criteria
• the number of failures
• the average error
• its S.D.
• the normalised S.D. of the thresholds

42
Support phase recognition

• Results with 12 x 2 (left/right) x 2 (flex/ext)
  48 contacts
• Results of the thresholds estimation with heels
  and toes effectors
• Our algorithm chooses the best method according
  to the configuration of the effectors and the
  evaluation of the criteria

Criteria/rank 1 2 3
Number of failure (0) speed (0) config (16) Height
Average error (2.3) speed (5.6) height (9.7) config
S.D. (1.6) speed (2.1) config (2.4) height
S.D. norm (-) config (0.41) height (0.5) speed
Heel strike Toe off
Height () 13.0 13.37
Speed (m/s) 0.73 0.39
43
Application of Newtons law

• Application of the FBD principle
• Free Body Diagram on each segment
• Study of forces and torques applied to the limbs
• Application of Newtons second law of motion
• on limbs s

44
Resolution of the equations

• The translation form
• Single support, from the free foot to
  the support foot
• Double support, global resolution
• No support, independent resolution
• The rotation form
• Iterative resolution of the equation

45
Validation of the resolution

• Forces (in N) at left toe and torques (in N.m) at
  left knee of 6 adapted locomotions

Forces (N)
Torques (N.m)
cycle
cycle
cycle
46
Validation of the resolution

• Comparison between 3 real ground reaction forces
  (black plots) and analysed forces (blue plot)
  from characters with similar biomechanical
  properties

GRF (N)
47
Validation of the resolution

• Ground reaction forces of different styles of
  real locomotions
• run
• jump
• walk

48
Influence of the retargeting

• The global scale
• Most used parameter
• Large influence on the dynamics of the motion
• Linear relation (c.c. 0.87) between the scale
  and the relative values of the GRF
• Experimental validation between 0.7 , 1.2 scales

GRF (N)
relative GRF
cycle
global scale
49
Influence of the retargeting

• The femur/tibia ratio
• To evaluate errors on articular centers (relative
  length of limbs)
• Experimental validation between 0.8 , 1.2
  ratios
• The GRF are not compensated by relative lenghts
  of the limbs

RMS error
GRF (N)
cycle
femur/tibia ratio
50
Influence of the retargeting

• The structure of the skeleton
• Models with 33 and 21 dof (pin joints at knees,
  ankles and elbows)
• Kinematical influence 1.4 cm per limbs
• Dynamical influence mostly on fore-aft
  acceleration
• Here, corresponds to 2.5 kg reduction (4.5 of
  the mass)

GRF (N)
RMS error
cycle
cycle
51
Influence of the kinematical interpolation

• The step size
• To study the kinematical correction of support
  phases
• Results are in agreement with the interpolation
  stage of valid corrections (database of motions)
• To improve this database with more motions, with
  more step sizes

GRF (N)
RMS error
cycle
step size factor
52
Influence of the kinematical interpolation

• The motion style
• Defined by the rest posture, i.e. the erect
  percentile
• More bent style nonsignificant errors
• More erect style increase of the error
• Large steps can not be done with a hight erect
  posture

GRF (N)
RMS error
cycle
erect percentile factor
53
Influence of the kinematical interpolation

• The character velocity
• Used for generating new motions
• Velocity , error but physics OK
• Velocity , double hump amplitude
• Need to change the style of locomotion (ex run)

RMS error
GRF (N)
cycle
velocity factor
54
Partial summary

• Method of evaluation of the dynamics influences
  of a locomotion editing method Pronost and
  Dumont 06 a
• The analysis is automatic, generic and
  independent Pronost and Dumont 06 b
• Applied on a retargeting approach and a
  kinematical interpolation in a database
• The editing method is valid on the experimental
  range of validation
• Future work
• To validate the analysis using more experimental
  data
• To overcome standard limitation of inverse
  dynamics issue, such as complex articular systems
  and muscular activation
• To use the dynamics-based analysis to correct
  motions

55
Outline

• Related works
• Overview and motivations
• Analysis / synthesis loop
• Kinematical adaptation
• Evaluation of the dynamics
• Forward dynamics synthesis
• Conclusion and future work

56
Forward dynamics synthesis
57
Modeling

• Of the human
• Representation of the rigid bodies
• masses, inertias and rotation matrix
• Representation of the mechanical joints
• type free (6 dof), spherical (3 dof), pin (1
  dof)
• mechanical parameters
• spring, damper, and limits
• Of the motion
• Values of the degrees of freedom of the
  mechanical system

58
Method of computation

• Normalisation of the forces and torques
• Principle
• Preliminary study on 5 motions

Subject 1 Subject 2
Normal N1,1,N1,2 N2
Bent F1 F2
N1,1 N1,2 F1 N2 F2
RMS(1) 0 0.234 0.621 0.512 1.001
M.E. 0 0.026 0.068 0.056 0.109
M.STDP (x10-3) 0 8.7 13.9 3.5 23.0
C.C. 1 0.973 0.681 0.980 0.399
C.ACP 1 0.995 0.322 0.332 0.328
Deriv (x10-4) 0 12.0 23.0 6.4 42.0
N1,1 N1,2 F1 N2 F2
RMS(1) 0 0.150 0.426 0.141 0.946
M.E. 0 0.025 0.071 0.026 0.148
M.STDP (x10-2) 0 22.6 55.6 22.9 129.3
C.C. 1 0.938 0.826 0.954 0.702
C.ACP 1 0.945 0.848 0.962 0.802
Deriv (x10-3) 0 36.5 74.1 37.3 162.2
Differences and similarities of root positions
Differences and similarities of normalised GRF
59
Preliminary results

• The forward dynamics resolution
• using the NMECAM library Arnaldi 89, Dumont 90
• library defining a symbolic solver of motion
  equations
• Results on a virtual human without external
  forces and with independent limbs

60
Partial summary

• The whole of the principles are not yet
  implemented
• Apprehension of the phenomena of the forward
  dynamics issue
• Usefull approach of normalisation in many fields
  such as kinematics and kinetics synthesis
• To make choices according to physical properties
  of the synthetised motion
• Future work
• Simulation of a complete chain of a human with
  external forces
• To organise a database including physical
  properties such as normalised profiles of forces
  and torques

61
Outline

• Related works
• Overview and motivations
• Analysis / synthesis loop
• Kinematical adaptation
• Evaluation of the dynamics
• Forward dynamics synthesis
• Conclusion and future work

62
Conclusion

• To understand and to simulate human motions
• Analysis and synthesis approaches
• Many applicatives fields
• animation, biomechanics, mechanics, anthropology
• Kinematical adaptation of captured motions
  preserving the physical credibility of the
  synthetised motions
• Proposition of methods and algorithms combining
  kinematical and dynamical approaches
• Adapted to the morphology and the locomotion
• Normalisation of the data, of the morphological
  structures and the forces and torques
• Use of principles and data from biomechanics

63
Limitations and future work

• On the methods
• Study of other motions with external forces
• To increase the size and the diversity of the
  database and the experimental data
• for the interpolation and the physical validation
• To use the analysis to drive the kinematical
  synthesis thanks to physical simulations
• To do a unique method with the proposed
  approaches
• On the applications
• Motion editing, biomechanics, anthropology
• But also on the modeling and the use of data on
  virtual humans

64
Thank you for your attention
Definition and realisation of modeling methods
and motion computation algorithms for virtual
humans
Nicolas Pronost
Advisor Bruno Arnaldi
Co-advisor Georges Dumont
Team IRISA - SIAMES