Face recognition: Opportunities and Challenges

1
Face recognition Opportunities and Challenges
Microsoft Research March 17, 2006

• Yu Hen Hu
• University of Wisconsin Madison
• Dept. Electrical and Computer Enginerring
• Madison, WI 53706
• yhhu_at_wisc.edu
• Collaborators Nigel Boston, Charles Dyer,
• Weiyang Lin, Ryan Wong, Goudong Guo

2
Agenda

• Human Face Recognition Problem
• Recent Progress and Findings
• Research at UW-Madison
• Invariant of transformation group
• Existing Invariants
• Integral invariant
• Summation invariant
• Experiment Environment
• Face Recognition Grand Challenge
• BEE Biometric Experimentation Environment
• Preliminary FRGC Results

3
Face Recognition Problem
probe image
Probe

• gallery

4
Face Recognition by Human

• A. Adler and J. Maclean,
• Performance comparison of human and automatic
• face recognition
• Biometrics Consortium Conference 2004

5
Understanding Human Face Recognition

• Many physiological and psychological studies have
  been conducted. Eg. Pawan Sinha _at_MIT. For
  example, we learned
• facial configuration plays an important role in
  human judgments of identity Sinha05

Celebrity faces created using identikits
Sinha05
C\users\yuhen\1Research\face\sinha\
6
Human Face Recognition Limitations
Harmon and Julesz, 1973
Degraded images
Human can handle low resolution celebrity images
quite well.
Sinha05
7
Face Recognition by Machine
Pre-processing
Feature Extraction
Pattern Classification
8
Face Recognition Process

• A pattern classification problem.
• Two steps
• Feature extraction
• Classification
• Feature extraction properties
• Invariant
• Discriminant
• Classification
• Often use distance based method

Query image
Feature Extraction
Query image feature vector
Gallery face images feature vectors
Pattern Classification
Recognition results
Wei-Yang Lin
9
Challenges

• Sinha et al 2005 use this example to illustrate
  the difficulty of finding a suitable similarity
  measure to gauge similarity between a pair of
  faces.
• In this example, the outer two faces actually
  belong to the same person while the middle one
  does not. But conventional pixel-based measures
  who say otherwise.
• Common variations in pose (this case), lighting,
  expression, distance, aging remain challenges to
  face recognition.

10
Face Recognition Challenges
Table 1 Face Recognition Technology Evaluation
Size
Note that the MCINT portion of FRVT 2002 is the
only test in this chart that included video
signatures.Signatures in all other tests were a
single still image.
http//www.frvt.org/FRVT2002/default.htm
11
Face Recognition Grand Challenge

• Goal to advance performance of face recognition
  by 10-fold (20 ? 2 verification rate _at_0.1
  false alarm rate)
• Focus on five different scenarios.
• Status on-going to be concluded by the end of
  2005

12
Object Recognition

• Three Approaches for object recognition (Powen
  Sinha)
• Transformationist approach
• Requires normalization
• Computationally expensive.
• View-based approach
• Store all possible views of the same object
• Expensive on storage.
• Invariant-based approach
• Different views ? same invariant features
• Desired properties of features
• Invariant to variations of the same object
• Discriminate to separate similar objects

13
Geometric Transformation Groups
14
Moment Invariants

• Introduced by M. K. Hu in 1962
• Advantages
• Do NOT require parameterization.
• Not sensitive to noise.
• Limitations
• Low discriminating power.
• Local characteristics can NOT
• be extracted.

15
Differential Invariants

• Two examples in 2D
• Limitation
• sensitive to noise

16
Integral Invariants

• Hann and Hickman 2002 extend transformation to
  integrals
• Advantages
• do NOT require derivatives
• local characteristics can be extracted
• Invariants can be systematically generated
• Limitation
• Require analytical expression of shape

17
Summation Invariants

• Introduced by Lin et al. 2005
• Advantages
• systematical approach
• robustness
• high discriminating power
• Limitation
• require parameterization

18
Method of Moving Frame

• The method of moving frame, introduce by Elie
  Cartan, is a tool for finding invariants under
  group actions.
• Definition A moving frame is a smooth,
  G-equivariant map

19
Example Differential Invariants of E(2)
20
Example Integral Invariants of E(2)
21
Example Summation Invariants of E(2)
22
Euclidean Summation Invariants of Curves

• Given a curve under Euclidean transformation
• We can find a moving frame by solving

23
Euclidean Summation Invariants of Curves

• From the moving frame, a family of invariant
  functions can be derived

24
Euclidean Summation Invariants of Curves

• The first summation invariants are explicitly
  shown below
• where

25
Euclidean Summation Invariants of Surfaces

• Similarly, the family of surface invariants
• where

26
Face Recognition Grand Challenge (FRGC)

• Organized by NIST to facilitate the development
  of FR technology
• Provide challenging problems and facial images
• FRGC v2.0 dataset contains 50,000 recordings,
  including
• high resolution still images
• 3D images

27
Four FRGC Experiments

• Controlled indoor
• Multiple images
• 3D images
• Controlled vs. uncontrolled

28
Baseline _at_ FAR 0.1
29
Biometric Experimentation Environment (BEE)
Image Preprocessing
BioBox
Sub-similarity Generation
Similarity Normalization
Analysis
30
Sub-similarity Generation
31
FRGC 3D Experiment
32
FRGC 3D Baseline Algorithm
33
Proposed Algorithm
shape
curve SI
PCA
similarity score
similarity score

similarity score
PCA
surface SI
shape
34
Experiment Setup

• Use only 3D shape. Note 2D texture is not
  utilized in our experiment.
• Specified the region of interest
• At each pixel, compute summation invariants from
  a specified window region
• Perform PCA to reduce dimensionality

35
3D Facial Surface and its Summation Invariants
36
3D Facial Surface and its Summation Invariants
37
ROC Performance of Curve Invariants
38
ROC Performance of Surface Invariants
39
Fusion of Summation Invariants
40
Comparison with Baseline Algorithm
41
Conclusion

• Summation invariants
• novel geometric feature
• provide useful shape information
• fusion further improve recognition performance
• In principle, one can apply summation invariants
  to unnormalized images