Gesture Recognition

1

• Gesture Recognition
• Assaf Feldman
• MASJ622
• The Media Laboratory
• Massachusetts Institute of Technology

2
Introduction

• ReachMedia is an on the move interactive system
  for delivering Just-in-Time information on every
  day objects
• A wristband with an RFID reader detects objects
  that the user touches or holds (implicit input).

3
Challenges

• Supply information in an unobtrusive way.
• Requirments.
• Hands free
• Socially acceptable

4
Current interfaces

• Too high tech
• Prevent normal activities such flip book pages,
  make gestures or shake hands.

Cyber Glove
Twiddler
5
Approach

• Rely on existing hardware
• Use the wristband as an explicit input method via
  gesture recognition.

6
Sensor

• Accelerometrs
• Cheap
• Small
• Low power

7
Data

• 3 axis acceleration (labeled by hand)
• 100 samples per class

Classes
Noise
8
Gesture Recognition Problem

• unknown class consists of everything else.
• High accuracy (small FP rate) require.
• Preferably user independent (pre trained models)
  - Classifier must generalize over all exemplars
  of one class.

9
Gesture Recognition Problem

• unknown class consists of everything else.
• High accuracy (small FP rate) required.
• Preferably user independent (pre trained models)
  - Classifier must generalize over all exemplars
  of one class.

10
Classification Algorithm Choice

• Nature of Data
• Training is possible
• Real Time classification needed can use raw
  data, no ffts, no convolutions

time series recognition techniques

• Temporal data

11
HMM performance w/wo noise

• No noise - High accuracy between 3 gestures
•  
• With noise Confusion (either FP or low
  accuracy)

12
Choosing HMM model

• Bad result for uniform number of states
• 3 gestures are quite similar, big difference
  between 3 gestures and unknown class.
• Need more states for unknown class

How Many?
13
Choosing unknown class model

• Fix number of states per gesture
• Find evidence for unknown class with 10 fold
  cross validation
• Repeat

14
Results unknown class
3 states HMM for gestures 5 states for unknown
tested on 30 samples per class
15
Cons for unknown modeling

• Easy to create a training interface for gestures
  - use a game or a small taks
• How do you get a user to train unknown class

16
Alternative Use HMM outputs

• Maybe the log probability outputs of the 3 models
  will be sufficient for creating a statistical
  rejection model
• Method
• Find HMM models for the 3 class problem
• Use test data to collect log probability outputs
  for hits
• Trains a GMM on the with the HMM outputs
• Use cross validation to decide on a threshold for
  the distance

17
Partial Results

• Distinctive clusters
• Not separable (some noise in the middle of class
  3
• Need to test classifier (NB GMM)

18
Future Work

• Finish testing binary claasification of HMM,
  maybe with some more time domain features added
  to the vector