Gait Recognition and Inverse Biometrics

1
Gait Recognition and Inverse Biometrics

• Sudeep Sarkar
• (Zongyi Liu, Pranab Mohanty)
• Computer Science and Engineering
• University of South Florida, Tampa

2
Biometrics from far
3
Paul Taylor on Walking

• And walking is the most revealing. A walk is
  like a fingerprint. No two people walk the same.
  Paul Taylor

4
Gait Research
Gait Recognition publications/year (Google)

• Human Perception
• Point light displays
• Johansson (1973), Cutting Kozlowski, Mather
  Mudock., Neri, Pavlova
• Gender discrimination
• Stevenage and Nixon (1999)

Gait Challenge Problem
5
The HumanID Gait Challenge Problem

• Data set of gait video
• Number of subjects (122)
• Exercise 5 covariates
• view, shoe, surface, carrying condition, time
• 1.2 TB of data
• Challenge experiments of increasing difficulty
• Baseline recognition algorithm to measure
  progress
• Joint effort of USF, NIST, and ND.

6
Samples within bounding boxes
Shoe
Briefcase
View
Grass
Surface
Time
7
Gallery and Probes
Shoe A
Concrete
8
Challenge Experiments
9
Baseline Algorithm Silhouette Detection

• Estimate background
• RGB mean, covariance
• Compute Mahanalobis distance
• Smooth the distances
• Expectation maximization
• Foreground vs. background
• Feature Mahanalobis distance
• Assume Gaussian distribution for each class
• Initialize labels by random threshold
• Iterate to maximize likelihood

10
Post Processing of Silhouettes

• Select the largest connected component
• Scale the silhouettes
• Final size 128 by 88.
• The height scaled to 128 pixels
• Horizontally centered so that the column with
  most number of foreground pixel is at column 44
• Some amount of scale invariance

11
Baseline Algorithm Similarity Measure
Gallery (N frames)
Nprobe
Correlate
Correlate
Correlate
Median (or Mean, Maximum)
SIMILARITY MEASURE
12
Impact of the Challenge Problem
13
Impact on full dataset
14
Summary of Identification Rates
15
Significant findings based on this dataset

• Walking surface and Time are the most significant
  covariates
• Shape over dynamics (Maryland, CMU, USF)
• Quality of silhouettes do not seem to be the
  bottleneck (USF)
• 3D approaches are starting to emerge (Maryland)

16
Dynamics Normalized Gait Recognition

• Emphasize gait shape over gait dynamics
• Shape gt silhouette shape for each stance
• Dynamics gt transition between stances
• Normalize the dynamics of any given gait sequence
  into a generic one
• Emphasize differences in gait shape between
  subjects

17
Generic Gait Model Population HMM

• Learnt from the manually specified silhouettes of
  71 subjects. (Baum-Welch)
• Exponential Observation Model

18
Dynamics Normalized Gait

• Map given frames to generic stances
• Viterbi (dynamic programming)
• For each stance, average all frames mapped to it
• Final Dynamics normalized gait cycle over 20
  stances

19
Similarity Computation
Gallery
Project
LDA Subspaces
S1
S2
S20
Project
Probe
20
Performance Gait Challenge (122 subjects)
21
Performance on UMD Data (55 subjects)
Without retraining
22
Recognition Performance SpeedCMU Mobo Database
(25 subjects)
Without retraining
23
Face, Gait Publications/Year
FERET
GAIT CHALLENGE
24
Inverse Biometrics From Scores to Templates
Gallery
Recognition Algorithm
Break in set
Score
Can you recreate this template?
25
Two step process
Modeling
Face Recognition System
Affine transformation that approximates the
recognition algorithm
Embedding Reconstruction
Face Recognition System
Match Scores
Embedding
Reconstruction
Reconstructed Target
Unknown Target
26
Modeling Recognition Algorithm
Computed distances between faces in the break-in
set using the face recognition method to be
modeled
Define an Transformation (A) that operates on a
face (xi) to embed it into a lower dimensional
space not necessarily orthogonal
27
Two part transformation
A Anr Ar

• a rigid transformation
• independent of the recognition algorithm
• derived from the orthonormal subspace analysis
  e.g. PCA of images in break-in set

• non-rigid transformation
• depends on the specific recognition algorithm
• approximate the recognition algorithm through
  sheer and stretching of the image space
• derived using classical MDS

28
Embedding Reconstruction
Unknown target
Inverse Transformation
?
?
Original image space
Modeled Space

• Observe the distances of selected templates from
  break-in set to unknown target
• Calculate the co-ordinates of the unknown target
  in the transformed space
• Use inverse transformation to reconstruct the
  unknown target template in the original affine
  space

29
Dataset Recognition Algorithms

• Databases FERET and FRGC
• Modeling results
• Break-In Results
• Algorithms
• FRGC baseline algorithm (PCA)
• Independent Component Analysis (ICA)
• Bayesian intrapersonal/extrapersonal classifier
  (template based)
• Elastic Bunch Graph Matching (EBGM)
• Commercial Face Recognition System (feature based)

30
Modeling results

• Quality of modeling is evaluated using ROC curves
• Training set 600 images from150 subjects from
  FERET
• Subjects different from the test set
• FERET data set with fa-fb and dup I probe set
• Bayesian, Commercial, PCAMahacosine (ICA, EBGM)
• On FRGC dataset
• Only for the commercial and baseline

31
Affine modeling quality (Bayesian)
Different Days
Different expressions

• On FERET with 1196 gallery subjects

32
Affine modeling quality (Com.)
Different Days
Different expressions

• On FERET with 1196 gallery subjects

33
Affine modeling quality (Com.)
Exp II (indoor, time)
Exp III (indoor vs. outdoor)

• On FRGC data

34
Face Template Reconstruction

• On FERET Gallery of 1196 subjects using a FRGC
  break-in set
• On FRGC Gallery of 466 subjects using a FERET
  break-in set
• Break-in in presence of score quantization
• Break-in performance Probability of breaking a
  randomly chosen face template
• Algorithms are set to operate at 1 False
  Acceptance Rate
• Commercial and Baseline (PCA)

35
Template reconstruction (FERET)
Original
Baseline
Reconstructed
Commercial
Bayesian
36
Template reconstruction (FRGC)
37
Template reconstruction (contd.)
38
Break-in probability (FERET)

• On FERET Gallery (1196 subjects)
• Probability of breaking a randomly chosen face
  template

39
Break-in probability (FRGC)

• On FRGC Gallery (466 subjects)
• Probability of breaking a randomly chosen face
  template

40
Break-in probability (Quantization)
(10 levels)
(100 levels)
(1000 levels)
(10000 levels)
Number of quantized levels

• FERET data
• Score quantization is a countermeasure against
  the hill climbing attack
• Probability of breaking a randomly chosen face
  template

41
Questions
Thank You