Learning Riemannian metrics for motion classification

1
Learning Riemannian metrics for motion
classification

• Fabio Cuzzolin
• INRIA Rhone-Alpes
• Computational Imaging Group,
• Pompeu Fabra University, Barcellona
• 25/1/2007

2
Myself

• Masters thesis on gesture recognition
• at the University of Padova
• Visiting student, ESSRL, Washington University in
  St. Louis
• Ph.D. thesis on the theory of belief functions
• Young researcher in Milan with the Image and
  Sound Processing group
• Post-doc at UCLA in the Vision Lab
• Marie Curie fellowship, INRIA Rhone-Alpes

3
My research

• object and body tracking
• data association
• gesture and action recognition
• identity recognition

research
4
Todays talk

• Motion classification is one of most popular
  vision problems
• Applications surveillance, biometric,
  human-computer interaction

5
Riemannian metrics for classification
Distances between dynamical models Learning a
metric from a training set Pullback
metrics Spaces of linear systems and Fisher
metric Experiments on scalar models
6
Distances between dynamical models

• Problem motion classification
• Approach representing each movement as a
  linear dynamical model
• for instance, each image sequence can be mapped
  to an ARMA, or AR linear model
• Classification is then reduced to find a suitable
  distance function in the space of dynamical
  models
• We can then use this distance in any
  distance-based classification scheme k-NN, SVM,
  etc.

7
A review of the literature

• Some distances have been proposed
• a family of probability distributions depending
  on a n-dimensional parameter can be regarded in
  fact as an n-dimensional manifold, with Fisher
  information matrix Amari
• Kullback-Leibler divergence
• Gap metric Zames,El-Sakkary compares graphs
  associated with linear systems thought of as
  input-output maps
• Cepstrum norm Martin
• Subspace angles between column spaces of the
  observability matrices

8
Riemannian metrics for classification
Distances between dynamical models Learning a
metric from a training set Pullback
metrics Spaces of linear systems and Fisher
metric Experiments on scalar models
9
Learning metrics from a training set

• All those metrics are task-specific
• Besides, it makes no sense to choose a single
  distance for all possible classification problems
  as
• Labels can be assigned arbitrarily to dynamical
  systems, no matter what the underlying structure
  is
• When some a-priori info is available (training
  set)..
• .. we can learn in a supervised fashion the
  best metric for the classification problem!
• A feasible approach volume minimization of
• pullback metrics

10
Learning distances

• Of course many unsupervised algorithms take an
  input dataset and embed it in some other space,
  implicitly learning a metric (LLE, Laplacian
  Eigenmaps, etc.)
• they fail to learn a full metric for the whole
  input space, but only images of a set of samples
• Xing, Jordan maximizes classification
  performance for linear maps yA1/2 x ?gt optimal
  Mahalanobis distance
• reduces to convex optimization
• Shental et al relevant component analysis
  changes the feature space by a global linear
  transformation which assigns large weights to
  relevant dimensions" and low weights to
  irrelevant dimensions

11
Riemannian metrics for classification
Distances between dynamical models Learning a
metric from a training set Pullback
metrics Spaces of linear systems and Fisher
metric Experiments on scalar models
12
Learning pullback metrics

• Some notions of differential geometry give us a
  tool to build a parameterized family of metrics

• Consider than a family of diffeomorphisms Fl
  between the original space M and a metric space N

• The geodesics of the pullback metric are the
  liftings of the geodesics associated with the
  original metric

13
Pullback metrics - detail

• Diffeomorphism on M

14
Inverse volume maximization

• The natural criterion would be to optimize the
  classification performance
• In a nonlinear setup this is hard to formulate
  and solve
• Reasonable to choose a different but related
  objective function

• Effect finding the manifold which better
  interpolates the data (i.e. forcing the geodesics
  to pass through crowded regions)

15
Riemannian metrics for classification
Distances between dynamical models Learning a
metric from a training set Pullback
metrics Spaces of linear systems and Fisher
metric Experiments on scalar models
16
Space of AR(2) models

• Given an input sequence, we can identify the
  parameters of the linear model which better
  describes it
• We chose the class of autoregressive models of
  order 2 AR(2)

17
Space of M(1,1,1) models

• Consider instead the class of stable
  discrete-time linear systems of order 1
• After choosing a canonical setting c 1 the
  transfer function becomes h(z) b/(z ? a)

18
Families of diffeomorphisms

• We chose two different families of diffeomorphisms

19
Riemannian metrics for classification
Distances between dynamical models Learning a
metric from a training set Pullback
metrics Spaces of linear systems and Fisher
metric Experiments on scalar models
20
MOBO database

• Mobo database 25 people performing 4 different
  walking actions, from 6 cameras
• Each sequence has three labels action, id, view

21
Classification of scalar models

• recognition of actions and identities from image
  sequences
• scalar feature, AR(2) and M(1,1,1) models

22
Results - action

• Action recognition performance, all views
  considered second best distance function

23
Results action 2

• Recognition performance of the second-best
  distance (blue) and the optimal pull-back metric
  (red), increasing size of training set

24
Effect of the training set

• The size of the training set obviously affects
  the recognition rate
• Systems of the class M(1,1,1)
• Increasing size of the training set on the
  abscissae

All views considered
25
Conclusions

• Movements can be represented as dynamical systems

• having a training set of such models we can learn
  the best metric for a given classification
  problem

• and use it to classify new sequences

• Design of a family of diffeomorphisms