Gestures: Computational Analysis

1
Gestures Computational Analysis

• Kanav Kahol
• Research Associate
• Center for Cognitive Ubiquitous Computing
• Department of Computer Science and Engineering
• www.public.asu.edu/kkahol

2
Gesture Segmentation and Tagging
Database Of Gestures And Patterns
Segmenting Of Gestures
Tag the Gestures
3
State-space models
Gestures are modeled as probabilistic sequences
of static poses
Start position
End position
4
My Approach
Segmentation Followed by Tagging
5
Computational Approach to gesture segmentation

• Separate from tagging algorithm
• Accounts for dynamic characteristic of the human
  body hierarchy.
• Development of an effective measure of movement
  in the human body.

6
Dynamic Human Body Hierarchy

• The Human Body can be divided into segments which
  create the movement
• This hierarchy is however dynamic in nature

7
Distinctive Characteristics
Every local minima in the total body force is a
potential gesture boundary
8
Movement Creation Original Score
9
Motion Capture
10
Detailed Score
11
Dynamic Characteristics-II
12
Gesture Segmentation
13
Segmentation Results
14
Recognition based on human anatomy

• The human body hierarchy and its basic structure
  is the same for all subjects and for all
  movements
• It is thereby natural to try and model gestures
  based on events in the segments.
• This approach will lead to a generic state space
  model with a fixed number of states. It could do
  what phoneme based HMMs did for speech
  recognition.

15
Modeling of Gestures

• A gesture in this model can be coded as a series
  of events in segments and joints
• An event in the segment is a local minimum in the
  segmental force
• An event in the joint is stabilization of angle.
• An example Walk-gtfoot,calf, ankle, thigh, knee,
  hip.

16
Walk
17
Human Anatomy Based Coupled HMM (CHMM)
coupling
2
3
4
5
14
1

18
Distance Based coupling

• Coupling of HMM is NP-Complete.
• This algorithm is adapted from coupling HMM for
  Mobile Stations where coupling strength differs
  based on distance of mobile station
• In our algorithm the coupling differs based on
  body-distance of a segment from a joint in our
  hierarchical model

19
Testing the model

• 185 widely accepted gestures, each with 6
  instances were used.
• These 6 instances were provided by 2 subjects.
• 3 used for training 3 used for testing.
• The model gave 91.2 average recognition
  accuracy.
• In a variation, 3 instances used for training
  were from one subject and 3 used for testing were
  from the other. The model still gave 90.8
  accuracy

20
Future Work

• Segmentation and Recognition algorithms have
  given good results and further testing will
  determine their bounds
• Move onto semantics and syntax of gestures
• Applications towards visual memory prosthetics