Human Motion Analysis 2

1
Human Motion Analysis 2

• Shunsuke Kudoh
• Katsushi Ikeuchi

2
Last week
? Motion graph
Concatenating short motion clips
Dynamic response ?
Creating dynamic responses by simulation for a
short time period between two motion clips.
3
Synthesizing Physically Realistic Human Motion in
Low-Dimensional, Behavior-Specific Spaces

• A. Safonova, J. K. Hodgins, N. S. Pollard
• (SIGGRAPH 2004)

4
Overview

• Creating motion by optimization calculation
• Motion-captured data
• Reducing dimension of a posture by PCA
• Strong correlation among body parts
• Constraints that users can control
• Posture, contact to environment, timing
• Physical (dynamic) constraints
• Using the previous method presented in
  SIGGRAPH2003

5
System overview

• keypose
• contact
• timing

6
Example
(top) captured data (2nd row) initial
result (3rd row) final result (bottom) captured
similar data
7
Observation 1

• Original motion vs. low-dimensional motion
• (running, walking, jumping, climbing,
  stretching, boxing, drinking,
• playing football, lifting objects, sitting
  down, and getting up)

8
Original vs. low-dimensional
9
Observation 2

• Error of 20 forward jump motions projected on 6
  low-dimensional spaces
• (original jumps, similar 3 jumps, 20 jumps,1
  mid-range jump, mix of 150 motions, 20 running
  motions)

Jump 1
Jump 2
Jump 20
PCA based on several motions
Average of square errors
PCA
PCA
PCA
Low-dim. Jump 1
Low-dim. Jump 2
Low-dim. Jump 20
10
Error of jump motions
11
Knowledge from observation

• Human behaviors can be represented with more than
  five to ten dimensions.
• Low-dimensional space constructed from the
  motions of the same behavior can represent the
  motion quite well with seven dimensions.

12
Low-dimensional motion expression
? A(t) is represented by B-spline
13
Selecting dimensionality

• Smallest such that

14
Contact constraints

• The low-dimensional space cannot include contact
  constraints for feet or hands.

15
Contact constraints by IK
16
Constraints 1

• User-specified constraints
• Posture constraints
• Initial, final, and key postures
• Specified in full-dimensional space
• Contact constraints
• Contact between feet and the ground
• Time constraints
• Duration of various phase of a motion
• High-level constraints
• Height of a jump, speed of walking

17
Constraints 2

• Constraints on physical validity of motions
• (automatically added by the system)
• Joint angle limits
• Simple inequations
• Joint torque limits
• Calculating joint torque by inverse dynamics
• Force constraints
• Using a method by Fang et al. Fang2003
• Considering conservation of momentum during
  flight and ground contact force

18
Objective function
19
Result 1
(top) captured data (2nd row) initial
result (3rd row) final result (bottom) captured
similar data
20
Result 2
Creating basis from 8 running and 1 running
across stepping stones (top) running with
different step length (bottom) running across
stepping stones with varying placement and speed
21
Result 3
Creating bases from 7 walking and 1 walking with
the step over an obstacle (top) normal
walking (middle) exaggerated walking with very
large step length (bottom) motion-captured
exaggerated walking (for comparison)
22
Result 4
Creating bases from 1 acrobatic back flip
motion (top) back flip of different height and
distance (bottom) motion-captured data (for
comparison)
23
summary

• Creating human motions by optimization
  calculation in low-dimensional space
• More natural result
• Strong correlation among body parts
• Low computation cost

24
Efficient Content-Based Retrieval of Motion
Capture Data

• Meinard Müller, Tido Röder, Michael Clausen
• (SIGGRAPH2005)

25
Overview
Retrieving motions from database using geometric
features
26
Motion representation
Human body is modeled as a kinematic chain
Name of joints
? End effectors (e.g. toes and fingers) are
regarded as joints for simplicity
27
Geometric features

• Each feature is represented as a boolean function
• Feature function is a set of boolean features
• F(P) is referred as a feature vector

28
Example of geometric features
Whether does rtoe lie in front of or behind the
plane spanned by lankle, lhip, and root
Plane spanned by the 3 points
Which side does the point exist?
29
Generalization
Function that takes 4 points on 3D space as
arguments
Geometric feature defined by B()
30
Walking
right foot
left foot
right left foot
31
Adding offset
Test whether a hand is in front of the body ?
Introduction of offset makes the test robust
32
Other geometric features
A plane specified by two joints (a normal vector
and a point)
Whether is a joint bent or stretched? (empirically
, 120 degrees is a good threshold)
Whether are two joints in certain distance or not?
33
Geometric features in this system
There are 31 features, which are divided into
three groups.
34
Segmentation
F-equivalent
F-segment
F-feature sequence
35
Example of segmentation
36
Properties of segmentation
Fine features induce segmentations with many
short segments.
37
Properties of segmentation
Normal wolk
Walk with short steps
For the both case, feature sequences are same ?
Essentially same motions can be extracted. (time
invariance)
38
Searching motion
Set of indexes in the database
We want to find it!
39
Algorithm of search
Calculating inverted lists for all feature
vectors beforehand
A set of indexes to be searched can be calculated
as an intersection of the inverted lists
40
Example
?
41
Example
Inverted list
The set of all F-hits
42
Fuzzy search
Using a sequence of sets of feature vectors as a
query, instead of a sequence of feature vectors
Same calculation as exact search except queries
Calculating an intersection by inverted lists
43
Adaptive fuzzy search

• Problems of fuzzy search
• Fuzzy in spatial domain
• Not fuzzy in temporal domain

44
Adaptive fuzzy search
45
Example (fuzzy search)
Inverted list
The set of fuzzy hits
? ???!
46
Example (adaptive fuzzy search)
The set of adaptive fuzzy hits
? There are two hits
47
Indexing

• The number of inverted lists that we calculate
  beforehand exponentially increases
• For 31 geometric features
• Dividing the set of features into small groups
• In this case, 31 features are divided into 3
  groups

48
Result

• The number of segments is smaller than that of
  frames (312).
• Index sizes are proportional to the number of
  segments.
• The total indexing time is linear in the number
  of frames.
• Half of the total indexing time is spent on
  reading the data.

49
Result
50
Geometric scene description
right foot kick followed by a left hand punch
Query Q
51
Time alignment
matching two motions using information of
geometric features
It is very effective when we use classical DTW
techniques after a computing coarse segment-wise
alignment based on geometric features.
52
Summary

• Geometric features
• Evaluating postures by boolean feature functions
• Segmentation
• Sequence of a certain feature vector
• Preliminary
• Calculation of inverted list
• Extracting motion sequences
• Normal search, fuzzy search, adaptive fuzzy search

53
Summary

• Geometric features
• ?????????????????????
• ?????????
• ????????????????
• ???
• Inverted list ???
• ??????
• Normal search, fuzzy search, adaptive fuzzy search

54
Synthesizing Animations of Human Manipulation
Tasks

• Katsu Yamane, James Kuffner, Jessica Hodgins
• (SIGGRAPH2004)

55
Overview
56
Overview

• Input by users
• Start and goal location of an object
• Hand grasp constraints
• External constraints (e.g. feet and floor)
• Character model
• (skeleton kinematics, joint limits, geometry
  data, dynamic properties, etc.)
• Output
• Human behavior of manipulation

57
Overview

• Planning
• Path of an object (external constraints etc.)
• Postures of characters (balance maintenance etc.)
• Post-processing
• Creating smooth trajectory
• Applying a velocity profile

58
Object path (C-space)

• Pos rot of an object (3D position and
  quaternion)
• Distance between two configurations

59
Object path (growing a graph)

• Repeatedly selecting a sample node and growing
  the tree

60
Object path (algorithm)
61
Object path (algorithm)
62
Characters pose

• Solving IK by constrained optimization
• Position of the feet and position of the object
• ? hard constraints
• Poses from the captured data with similar
  constraints
• ? soft constraints
• Projection of the center of the gravity on the
  ground
• ? soft constraints (for balance maintenance)

63
Creating posture database

• 41 markers for representing a posture
• 4 box manipulation performances
• high?high, high?low, low?high, low?low
• Performed by one subject
• Constructed so that distance between every two
  poses is greater than a threshold
• Before calculating distance, the marker positions
  are translated and rotated so that the squared
  sum of the marker distances is minimized

64
Computing pose

• Marker position
• Hard constraints

65
Computing pose
66
Smoothing
Selecting two points Interpolating (slerp for
rotation) Checking collisions ?accept or reject
Repeating it until no progress can be made
Collision among the character, environment, and
object
67
From posture sequence to motion

• So far, posture sequences are obtained
• Converting them to motions
• ?velocity profile
• 25 pick-and-place manipulation trajectories
• Selecting a path closest to the smoothed path
  from the database and using its velocity profile

68
Velocity profile

• Normalizing motion of markers
• RBF(radial basis function)

Velocity profile is extremely stereotypical (psy
chophysical knowledge)
69
Normalized motion

• Normalizing traveled distance and marker position

70
Approximation by RBF

• Normalized marker position and its velocity
  represented as functions of the normalized
  traveled distance

71
Generating motion
is represented as a function of normalized
time. The following equation is repeated until
.
duration and length of created and database
trajectory
time at the last step in the above iteration.
72
Gaze synthesis

• Eye motion is important for natural human motion
• Most motion capture systems do not include gaze
  tracking

• Approximate gaze model from biomechanical
  observations
• The visual target is acquired in the center of
  the retina
• While reaching for an object, the target is the
  location of the object
• After acquiring the object, the target is the
  destination location
• The gaze motion is initiated at the same time or
  slightly before the arm motion

73
Result

• Changing a character

74
Result

• Changing an object

75
Result

• Changing motion

76
Result

• Complex motion (combination of 7 planners)

77
Summary

• Generating motion of manipulation tasks
• Generating object trajectories by path planning
• Using motion-captured data ? natural results
• Considering balance (only COG static balance)
• Converting a pose sequence to motion by velocity
  profile