Brief Survey of Static Face Recognition Methods

1
Brief Survey of (Static) Face Recognition Methods

• Surveillance Methods Seminar
• Dan Koppel
• 10/6/04

2
Survey paper

• Face Recognition A Literature Survey
• W. Zhao (Sarnoff)
• R. Chellappa (Univ. of Maryland)
• P.J. Phillips (National Institute of Standards
  and Technology)
• A. Rosenfeld (Univ. of Maryland)

3
Overview

• Applications

4

• Commercial Products

5

• General Architecture of Face Recognition Systems

6
Subtasks in Face Recognition

• Detection and normalization of faces within an
  image
• Feature extraction
• Identification/Verification

7
Segmentation/Detection

• Until mid-90s work focused on single-face
  segmentation
• Methods
• Whole face template
• Deformable feature-based template
• Skin color
• Neural network

8

• Appearance/image-based methods
• Based on training using large image databases
• Better results than feature-based or template
  matching methods
• Detection that handles rotation
• Training occurs with samples captured with
  several pose angles
• Better performance than methods using invariant
  features when angle is large (35 degrees)

9
Feature Extraction

• The use of specific features enhances the
  performance of holistic methods (that take the
  face image as a whole without breaking down into
  features eg. eigenface method)
• Position of features are used by holistic
  methods to normalize the image before processing
• Believed by psychologists that human visual
  system operates in this combined mode.

10
Types of feature extraction methods

• Generic methods based on edges, lines and curves
• Feature-template based method
• Uses predefined parameterized template
• Energy function is minimized wrt parameters
• Structural matching methods which use geometric
  constraints on the features (regularization)
• Combines statistical shape models with
  statistical texture models
• Training set of 400 face images with 68 landmark
  points gt landmark vectors PCA generates mean
  shape and orthogonal mapping matrix
  (eigen-features)

11
Contd

• After warping image to correspond to mean shape,
  a shape-free texture is obtained
• PCA is applied to this data (after getting mean
  texture) and a shape-free texture model is
  obtained
• Correlation between shape and texture can be
  established by concatenating the 2
  (reduced-dimension) vectors into 1 vector and
  again performing PCA
• To establish model parameters from a given image,
  a minimization between actual and synthetic
  (given a set of shape and texture parameters)
  images is performed.
• Additional DOFs are translation and intensity
  scaling

12
Example
13
Comparing methods

• Early methods used templates for single features
• Had robustness problems when configuration
  changed (eg. eye closed or glasses or mouth
  opened/closed)
• Structural matching methods (eg. Active Shape
  Model)
• More robust relative to intensity changes and
  variations in feature shapes

14
Recognition Techniques

• 3 basic categories
• Holistic methods
• Use of entire face image
• Eg eigenpictures Sirovich and Kirby 1987
  Kirby and Sirovich 1990 and eigenfaces Turk
  and Pentland 1991
• Feature-based matching
• Features are extracted
• Geometric/appearance data or statistics are fed
  into a classifier
• Hybrid
• Human visual system believed to also be a hybrid

15
Categories
16
PCA Principal Component Analysis

• Given a high-dimensional space, a point
  corresponds to an instance (eg. a specific
  image)
• Points may approximately reside in a lower
  dimensional subspace
• Given a new point (eg. an image), we can test to
  see if the point is in this subspace.
• If not, then it does not belong to this category
  (ie. it is not a face)

17
Contd

• If it does, then project this point onto the
  subspace to eliminate any possible error (the
  point should be very close to the mathematical
  subspace)
• Look for the nearest training point on this
  subspace to identify a match
• Actual method for doing this involves building a
  matrix of all training points (vectors) and using
  SVD to build a basis of orthogonal vectors
  spanning this space.
• Only vectors corresponding to large singular
  values are kept and these span the lower
  dimensional space that we are looking for.

18
Different Classifiers used in conjunction with PCA

• Nearest neighbor (eigenfaces)
• Feature-line-based methods (distance from test
  point to line connecting 2 training points)
• Linear/Fisher discriminant analysis
  (Fisherfaces)
• Bayesian methods using a probabilistic distance
  metric
• Support Vector Machines (SVM)

19
Holistic approach

• Low-dimensional representation based on existence
  of statistical redundancy in face images
• By normalizing wrt translation, orientation and
  scale, even larger redundancy exists.
• PCA decorrelates the outputs
• Robustness to noise is achieved
• Blurring
• Partial occlusion
• Differing backgrounds (assuming segmentation is
  not complete)

20
Robustness of method
21
Robustness of identification
22
Holistic eigenpicture,eigenface

• Must solve the following eigenvalue problem
• (where C is covariance matrix of image-vectors)

23
Bayesian extension to eigenfaces

• Instead of using a simple Euclidean dissimilarity
  measure Turk and Pentland 1991, a probabilistic
  measure was introduced Moghaddam and Pentland
  1997
• Drawback need to estimate probability
  distributions in a high-dimensional space with
  only a limited number of training samples per
  class

24
Contd

• A 2-class problem is created based on classifying
  images into 2 mutually exclusive categories
• One based on intra-personal variations of several
  images for the same individual
• One based on extra-personal variations among
  different individuals
• Gaussian distributions are assumed for both cases

25
Contd

• Using these distributions and Maximum A
  Posteriori criterion, two face images are deemed
  to belong to the same person if
• where
• Large performance improvement over standard
  nearest-neighbor eigenspace matching (using large
  face datasets like FERET)

26
LDA/FLD methods

• Training based on scatter-matrix analysis
• 2 scattering matrices are defined for within
  classes and between classes

27
Contd

• Discriminatory power can be quantified by the
  ratio of the determinants of both matrices
• The basis T is selected to maximize J

28
Comparing different linear projection algorithms

• 4 different methods were compared in Belhumeur
  1997
• Correlation-based method
• A linear subspace method by Shashua 1994
• Eigenface method Turk and Pentland 1991
• Fisherface method using subspace projection LDA
• Using 500 images, the Fisherface method performed
  best

29
Additional Methods

• Evolution Pursuit-based Adaptive
  Representation uses genetic algorithm to
  optimally find projection basis (subspace)
• Balances minimizing the empirical risk (for the
  offline training images) with minimizing the
  guaranteed risk (for the online test images)
• ICA Independent Component Analysis
• Generalization of PCA that decorrelates
  higher-order moments (not just 2nd order moments)
• Proponents argue that most face information is
  contained in these higher moments

30
Contd
31
Feature-Based Methods

• Early methods based on geometrical relationships
  of feature locations Kanade 1973
• Additional processing of geometric information
• Use of DLAs (Dynamic Link Architecture) extends
  Artificial Neural Networks by making the synaptic
  weights variables whose values are controlled by
  the signal correlations between the two neurons.
• Wavelets (Gabor)
• Graph matching
• Above 3 concepts used in the Elastic Bunch Graph
  Matching (EBGM) algorithm Okada 1998 Wiskott
  1997 face detection/recognition, pose
  estimation, gender classification,
  sketch-image-based recognition

32
Contd
33
Gabor Wavelets

• Wavelet coefficients used to describe local
  features are robust to changes in
• Illumination
• Translation
• Distortion
• Rotation
• Scaling

34
Hybrid Approaches

• Extension of eigenface method Pentland 1994
  called view-based eigenspaces
• Faces captured from different angles
• Separate eigenspaces formed for each viewpoint
  (instead of lumping all views together)
• Has been shown to perform better

35
Contd

• Another extension eigenspace concept used on
  individual extracted features eigenfeatures
• When number of eigenvectors is kept small, this
  method outperforms eigenfaces.
• Keeping low-order eigenfaces (as regularizer) and
  using eigenfeatures with higher orders is
  suggested by author
• Combination of holistic methods with
  feature-based methods mimics human recognition
  system (according to psychologists) and should be
  used algorithmically.

36
Closing Comments

• Implementation details have large impact on
  performance. Examples
• 7 different distance metrics for finding
  nearest-neighbors in PCA method were compared
  Moon and Phillips 2001 and significant
  performance variation was found.
• Different amounts normalization produce differing
  performance
• Translation, rotation, scale
• Masking and affine warping (to cancel differing
  head poses)

37
Contd

• Machine face recognition/detection is still a
  developing field and has not reached performance
  of human beings
• Specific issues for future research
• Achieving accuracy/robustness of
  feature-locations with variations in
• Illumination
• Pose
  
  
  
  
  
  
  
  
  
  
  

38

• Thank you for your attention!