Rowing Motion Capture System

1
Rowing Motion Capture System

• Simon Fothergill
• Ph.D. student, Digital Technology Group, Computer
  Laboratory

Jesus College graduate conference May 2009
2
Overview

• The Bigger Picture
• Previous work
• Problem
• Process
• Data Capture System
• Results
• Future work

3
The Bigger Picture

• Sentient Computing!
• Computer Vision
• Pattern Recognition Machine learning
• A long way to go!

4
The Bigger Picture Watching Humans

• Physical Performances
• Heath care
• What are they doing?
• How well are they doing it?
• How should be improved?
• How should they be told?

5
Previous Work - Activity / Gesture recognition

• Motion capture methods have included
• Blob tracking
• Point trajectories
• Recognition techniques have included
• Single frame
• Multiple frame
• Parametric

6
Learn the quality of a performance from body part
trajectories

• Minimise markers using redundancy
• Complex trajectories, continuous score
• Flexible rubrics require learning
• Different types of expert labelling
• Explanations
• Non-specific / specific
• Different granularities of quality
• Which sections of the trajectory are how
  relevant?
• One section of a can depend on many aspects

7
Process

• Learning

Extract and select features
Trajectories
Features
Capture motion
Performance
Inference model
Video
Capture video
Expert coach labels with their judgement
Learn
Judging
Performance
Features
Judgement
Extract and select features
Capture motion
Evaluate
Trajectories
Inference model
8
Data Capture System
ECS
Erg
Power
Control
Motion sensitive LED markers
Wii controllers
9
Data Capture System - Architecture
Fire wire
TCP/IP
Video camera (30Hz)
PC
PC
Java / C client
C server
Wii library
Bluetooth library
Bluetooth
Bluetooth
Nintendo Wii controller  
Nintendo Wii controller  
IR 1024x768 camera (100Hz)
IR 1024x768 camera (100Hz)
C server
Buffer
Wii controller
Wii controller
10
Data Capture System Calibration and operation
Server
Client
Storage
Batch
4 x 2D coordinates
Stereo calibration
Calibrate WMCS
Calibration
Erg calibration
Calibrate labeller
Label markers
Triangulation
Update ECS if necessary
ECS
Transform to ECS
Live operation
4 x 3D coordinates
Detect strokes
Calculate stats
Save picture
Display on GUI
Control camera
Log data
Log files
Encodes video
11
Data Capture System

• Example video

12
Preliminary Results

• Preliminary results have been obtained using a
  dataset of 6 rowers and the complete trajectory
  of the erg handle only. Binary classification
  over stroke quality was done using tempo-spatial
  features of the trajectory and a neural network.
  Two training methods were compared.

Classification accuracy across given number of
performers, for quality of individual aspects of
technique.
13
Summary and Further Work

• Data capture system and how it fits into the
  bigger picture
• More information is available on the feature
  extraction selection and inference algorithms.
  
• A larger data set would allow conclusive results
  to be obtained
• Feature extraction and selection methods that
  address using the relevant segments of the
  relevant trajectories
• More sophisticated modelling based on particle
  filters
• Supports multiple body parts and labelling
  methods
• Uses a distribution of motion vectors to
  probabilistically track the quality so far as
  the stroke evolves.

14
In Conclusion

• Advertisement!
• Acknowledgements
• Professor Andy Hopper, Dr Sean Holden, Dr Robert
  Harle
• Members of the DTG and Rainbow groups, Computer
  Laboratory
• Jesus College, JCBC and the Graduate society
• References
• Optical tracking using commodity hardware, Hay,
  S. Newman, J. Harle, R. ISMAR 2008.
  Page(s)159 - 160
• Thank you!
• Questions?

Please come down to the boathouse and use the
data capture system!