Silhouette Analysis-Based Gait Recognition

1
Silhouette Analysis-Based Gait Recognition for
Human Identification
Liang Wang, Tieniu Tan, Huazhong Ning, and
Weiming Hu December 2003 IEEE TRANSACTIONS ON
PATTERN ANALYSIS AND MACHINE INTELLIGENCE Present
er Xi Chen
2
Background and Research Problem

• Biometrics is a technology that makes use of the
  physiological or behavioral characteristics to
  authenticate the identities of people.
• The interest of vision-based human
  identification at a distance is driven by the
  need for automated person identification systems
  for visual surveillance and monitoring
  applications in security-sensitive environments
  such as banks, parking lots, and airports
• So far, gait is probably the only perceivable
  biometric feature from a great distance. In
  comparison with other first-generation biometric
  features such as fingerprint and iris, gait has
  the advantage of being unobtrusive.
• Gait recognition, also called gait-based human
  identifications, is a
• relatively new research direction in
  biometrics, which aims to
• discriminate individuals by the way they walk.
  

3
Related Work

• Current gait recognition approaches may be
  classified into two main classes, model-based and
  motion-based methods.
• Model-based approaches aim to model human body
  or motion and perform model matching in each
  frame of a walking sequence.
• Motion-based approaches can be divided into two
  main classes. The first class, state-space method
  considers gait motion to be composed of a
  sequence of static body poses. The second class,
  spatiotemporal methods, characterizes the
  spatiotemporal distribution generated by gait
  motion in its continuum.
• A number of approaches have already shown that
  it is possible to recognize people by gait.
  

4
Overview of Approach
5
Purpose and Contribution

• The proposed method has several desirable
  properties
• It is easy to comprehend and implement.
• It is insensitive to the color and texture of
  cloth as a silhouette-based approach.
• Some additional features related to pace,
  stride, and build are used to improve recognition
  accuracy.
• Experimental results demonstrate that it has a
  relatively low computational cost.

6
Human Detection and Tracking

• Background Modeling
• The main assumption made here is that the camera
  is static,
• and the only moving object in video sequences
  is the
• walker.
• Here, the LMedS (Least Median of Squares) method
  is used to
• construct the background. Let I represent a
  sequence including
• N images. The resulting background can be
  computed by

Where p is the background brightness value to be
determined for the pixel location (x,y), med
represents the median value, and t represents the
frame index ranging within 1-N.
7
Human Detection and Tracking
Differencing between the Background and the
Current Image In this paper, the following
extraction function is used to indirectly
perform differencing
Where a(x,y) and b(x,y) are the brightness of
current image and the background at the pixel
position (x,y). This function can detect the
change sensitivity of the difference value
according to the brightness level of each pixel
in the background image.
8
Human Detection and Tracking
9
Human Detection and Tracking

• Postprocessing and Tracking
• Differencing process is independently performed
  for each
• component R, G, and B. For a given pixel, if
  one of the three
• components determines it as the changing point,
  then the pixel
• will be set to the foreground.
• No change detection algorithm is perfect. So, it
  is imperative to
• remove as much noise and distortion as possible
  from the segmented
• foreground. Morphological operators such as
  erosion and dilation
• are used to further filter spurious pixels, and
  small holes inside the
• extracted silhouettes are then filled.

10
Feature Extraction

• Silhouette Representation
• For the sake of computational efficiency, we
  convert these 2D silhouette
• changes into an associated 1D signals to
  approximate temporal pattern
• gait.
• After the shape centroid is obtained, by
  choosing it as the reference origin, we can
  unwrap the outer contour counterclockwise to turn
  it into
• a distance signal.

11
Feature Extraction
Silhouette Representation
12
Feature Extraction

• Principal Component Analysis (PCA) Training
• The purpose of PCA training is to obtain several
  principal components
• to represent the original gait features from a
  high-dimensional
• measurement space to a low-dimensional
  eigenspace.
• Given s classes for training, and each one
  represents a sequence of
• distance signals of one subjects gait.
• Let Dij be the jth distance signal in class i
  and Ni the number of such
• distance signals in the ith class. The total
  number of training samples is

13
Feature Extraction

• PCA Training
• The mean and the global covariance matrix of
  such a data set can be
• calculated as

14
Feature Extraction

• PCA Training
• We can compute the N nonzero eigenvalues
  and the associated
• eigenvectors based on SVD
  (Singular Value Decomposition).
• Considering the memory efficiency in practical
  applications, only those
• bigger eigenvalues and corresponding
  eigenvectors are kept using a
• threshold Ts0.95.

Where Wk is the accumulated variance of the first
k largest eigenvalues with respect to all
eigenvalues.
15
Feature Extraction

• Projection to the Eigenspace
• Taking only kltN largest eigenvalues and their
  associated eigenvectores,
• the transform matrix can be
  constructed to project an original
• distance signal into a point in the
  k-dimensional eigenspace.

• k is usually much smaller than then original
  data dimension N. That is
• to say, eigenspace analysis can drastically
  reduce the dimensionality
• of input samples. For each training sequence,
  the projection centroid
• is given by

16
Recognition

• Similarity Measures
• Spatiotemporal Correlation for two input
  sequences, they are projected into and
  in the eigenspace. The similarity between
  two such vector sequences can be computed by

Where is a dynamic time warping
vector from with respect to time
stretching and shifting for an approximation of
the temporal alignment between the two sequences.
The selection of the parameters a and b depends
on the relative stride frequency and phase
difference within a gait period respectively.
17
Recognition
Gait Period Analysis serves to determine the
frequency and phase of each observed sequence so
as to align sequences before matching.
18
Recognition

• Similarity Measures
• The selection of the parameters a and b depends
  on the relative stride frequency and phase
  difference within a gait period respectively.
• Let and denote the frequencies of the
  two gait sequences, then we can get
  .
• By cropping a subsequence of length from the
  second sequence vector and stretching it with a,
  we may obtain its correlation with .
• The minimum of all prominent valleys of the
  correlation results determines their similarity.
  

19
Recognition
Similarity Measures (2) The Normalized
Euclidean Distance (NED) the computational cost
will increase quickly if the similarity is
performed in the spatiotemporal domain,
especially when time stretching and shifting is
taken into account. The NED is used to measure
the similarity of two gait sequences only with
the projection centroids. Each projection
centroid implicitly represents a principal
structural shape of certain subject in the
eigenspace. The normalized Euclidean distance
between the two sequential projection centroids
can be defined by
20
Recognition

• Classifier
• The classification process is carried out via
  two simple different classification methods, the
  nearest neighbor classifier (NN) and the nearest
  neighbor classifier with respect to class
  exemplars (ENN).
• represents a test sequence and
  represents the ith reference sequence. This test
  sequence is classified into class that can
  minimize the similarity distance between the test
  sequence and all reference patterns by

21
Experiments

• Data Acquisition
• A new gait database, called NLPR database, is
  established for the experiment.
• All subjects walk along a straight-line path in
  three different views with respect to the image
  plane, laterally, obliquely, and frontally.
• The resulting NLPR database includes 20 subjects
  and 4 sequences for each viewing angle per
  subject.

22
Experiments

• Preprocessing and Training
• For each image sequence, motion segmentation
  and tracking are performed to extract the
  silhouette of the walking subjects.
• The extracted silhouettes are converted into an
  associated sequence of 1D distance signals before
  training and projection.

23
Experiments

• Preprocessing and Training
• The first 15 eigenvalues and their associated
  eigenvectors are kept to form the eigenspace
  transformation matrix.
• Then each silhouette image can be mapped to one
  point in a 15-dimensional eigenspace.

24
Results and Analysis

• Identification Mode
• A useful classification performance measure,
  defined as the cumulative probability that the
  real class of a test measurement is among its top
  k matches, is used. To show the performance
  graphically, the rank k is plotted in the
  horizontal axis, and the vertical axis is the
  match score.
• The leave-one-out cross-validation rule is used
  with the NLPR to estimate the performance of the
  proposed method.
• After computing the similarity between the test
  sample and the training data, the NN and ENN are
  then applied for classification.

25
Results and Analysis
Identification Mode
26
Results and Analysis

• Identification Mode
• The identification performance using NED is, in
  general, better than that using STC. In theory,
  STC should better capture spatiotemporal
  characteristics of gait motion than NED. This
  result probably due to the fact that segmentation
  errors maybe accumulated into a quick-enlarged
  match error.
• The NED based on the exemplar projection
  centroid performs better than NED using only a
  single projection centroid.
• The recognition performance under frontal
  walking is the best. This result is probably due
  to the averaging associated with the silhouette
  shape analysis because there are less severe
  variations of silhouette appearances in such gait
  patterns compared with other views.

27
Results and Analysis

• Verification Mode
• In this paper, the FAR (False Acceptance Rate)
  and FRR (False Reject Rate) are also estimated
  via the leave-one-out rule in verification mode.

28
Results and Analysis

• Validation Based on Physical Features
• In experiments, recognition errors often
  occurred when the two smallest values of
  similarity function are very close.
• When the difference between the last two minima
  is lower than a predefined threshold, some
  additional features available from the training
  sequences are introduced to validate the final
  decision.
• The features can be body height, build, and
  stride length, which can be obtained in the
  training process.

29
Results and Analysis
Validation Based on Physical Features
30
Results and Analysis

• Validation Based on Physical Features
• The experiment results shows that the additional
  physical features improve the performance of the
  classifier without validation.

31
Comparisons

• Compared with other algorithms, the algorithm
  proposed has better
• performance and the advantage of lowest
  computational cost.

32
Discussion and Future Work

• To provide a general approach to automatic
  person identification in
• unconstrained environments, much remains to be
  done.
• Further evaluation on a much larger and
  most-varied database is still
• needed. We are planning to set up such a database
  with more subjects,
• more sequences and more variations in conditions.
• It is more efficient for recognition to extract
  dynamic information such
• as the oscillatory trajectories of joints. Future
  work may try to combine
• both static and dynamic features of gait for
  recognition.
• Also, seeking better similarity measures,
  designing more sophisticated
• classifiers, gait segmentation, and the
  evaluation of different scenarios
• deserve more attention in future work.