A Practical Approach to Recognizing Physical Activities

1
A Practical Approach to Recognizing Physical
Activities

• Jonathan Lester
• Tanzeem Choudhury
• Gaetano Borriello

2
The Idea
What am I doing?
3
Requirements, and intriguing questions

• Single point of location on the body and location
  independence Does it matter?
• Work for any person Personalization only
  increases accuracy How much variation across
  users?
• Cost Effective How many sensors are needed?

4
Sensors Locations - Single versus Multiple

• Majority of research with single sensor modality,
  at multiple locations
• Sensor placement and data collection becomes
  cumbersome
• Use of single sensor (single location) reduced
  accuracy by 35 1
• Compensate for accuracy lost using multiple
  sensor modalities2
• More comfortable for user to wear at single
  location
• Can integrate into existing mobile platforms
• 1 Bao, L., Intille, S. Activity Recognition
  from User-Annotated Acceleration Data.
• In Proc. Proc. Pervasive (2004) 1-17
• 2 Choudhury, T., Lester, J., Kern, N.,
  Borriello, G., Hannaford, B.. A Hybrid
  Discriminative
• Discriminative/Generative Approach for Modeling
  Human Activities.

5
Components

• Three main components
• Sensing Module
• Gathers low level information about activities
• microphone, accelerometer, light sensors, etc
• Feature Processing and Selection Module
• Process raw data from sensors into features
• Features discriminate activities
• Classification Module
• Tags the activity being performed from predefined
  types

6
The Experimental System

• A Multi-Mode Sensor Board (MSB)
• Bluetooth Intel Mote (iMote)
• USB rechargeable battery board

7
Methodology

• Data collected across 12 individuals performing 8
  activities
• 8 in mid twenties, 4 in their thirties
• 2/3 of data collected in a computer science
  building, 1/3 collected in office building
• Data collected from 3 MSBs located at shoulder
  strap, side of waist, right wrist
• Given a sequence of activities - sitting followed
  by climbing stairs followed by brushing teeth
• Data collected on laptop, annotated by an
  observer using iPaq

8
Feature Extraction

• About 18,000 samples per second
• How to bring out the important details compute
  features
• Features linear, log scale FFT frequency
  coefficients, spectral entropy, band pass filter
  coefficients, correlations, integrals, means, etc
• Not every feature for every sensor
• Total of 651 features computed
• Need to pick few important features to avoid
  confusing classification algorithm

9
Training the Classifiers

• Have all the data with the features extracted
• Need to separate the data into training and
  testing sections
• Data separated into 4 folds
• 3 folds used for training the classifier
• 1 fold used for testing
• Done with all combinations of 3 of 4 folds
• Temperature and humidity sensor data not used

10
Classifier Structure

• Modified version of AdaBoost1 used to select
  best features and rank them based on
  classification performance
• Most discriminative sub-set of features selected
  to learn static classification of activities
• Usage of Hidden Markov model (HMM) as a second
  layer using the class probabilities
• Discriminative classifier tuned to make
  activities more distinguishable, while HMM
  ensures temporal smoothness
• 1 Viola, P., Jones, M. Rapid Object Detection
  using a Boosted Cascade of Simple
• Features. In Proc. Computer Vision and Pattern
  Recognition (2001)

11
Confusion Matrix The Results
Table Confusion matrix for the static and HMM
classifier trained using a single stream of
sensor data from all three locations on the body
12
Location Sensitivity

• Sensors at different locations on body give
  better classification
• Want to make the sensors invisible to the user
• Ideally classification should work accurately
  with data from different locations on the body
• Allows user to carry at any convenient location
• How to determine which location is best to place
  the sensor board?
• Train four sets of classifiers (1) data from
  all 3 locations
• (2) data from shoulder (3) data from waist
  (4) data from wrist

13
Location Sensitivity Results
Table Overall precision/recall for the static
and HMM classifiers trained/tested on
all locations (top row) and a single location
(bottom rows).

• Precision True Positive / (True Positive
  False Positive)
• Recall True Positive / (True Positive False
  Negative)
• Overall accuracy (True Positive True
  Negative) / Total number of example

14
Variation across users

• Is there any need for training period for
  particular end user?
• User would want it to work immediately upon
  purchase
• Good if it gives better results with more usage
  but reasonable performance to begin with
• Train the classifier with lot of data collected
  from diverse group of people
• Use the classifier for new individual No need
  to collect any training data, no need to retrain
  the classifier

15
User Variation Test 1

• Data collected from 12 users. Select N (N 1 to
  12) users data for training. Test on all 12
  users data
• Idea To show that accuracy improves with
  increase in training data

16
User Variation Test 2

• Data collected from 12 users. Select N (N 1 to
  12) users data for training. Test on remaining
  12 N users data
• Idea To show improvement in accuracy due to
  general classifier and not increased training
  data size

17
Sensors Necessary

• Do not necessarily need all sensors on MSB to
  accurately classify
• Results shown here do not use temperature and
  humidity sensors
• Reducing number of sensors makes system less
  susceptible to environmental changes
• Increasing number of sensors increases
  complexity, power and computational requirements,
  thus higher costs
• Fewer sensors means smaller sizes, more practical
  to use
• 3 important sensors accelerometer, audio and
  barometric pressure sensor

18
Single Versus Multiple Sensors
Table Classifiers trained using a single sensor.
Overall accuracies approximately 65
Table Classifiers trained using three sensors.
Overall accuracies approximately 90
19
Conclusion

• Accurate recognition of range of activities can
  be achieved
• Have answered the three questions
• Single-board activity recognition system
  generalizes well. No need to learn location
  specific activity models
• For the data set, no need for customization to
  specific individuals
• Although 7 different modalities on board, but
  three main audio, barometric pressure and
  accelerometer
• Participants were young healthy individuals
• How to handle activities that do not fall into
  predefined classes?
• How to handle ambiguities associated with
  compound activities?