Face Recognition in Hyperspectral Images

1
Face Recognition in Hyperspectral Images

• Z. Pan, G. Healey, M. Prasad and B. Tromberg
• University of California
• Published at IEEE Trans. on PAMI
• Vol 25, No. 12, December 2003.

2
Introduction
What is a hyperspectral Image?
visible electromagnetic spectrum
0.4
0.7 µm
Red,
RGB
Green,
Blue Channels
3
Introduction
What is a hyperspectral Image?
UV Ultra Violet Vis VisibleNIR Near
infraredSWIR Short wavelength infraredMWIR
Medium wavelength infraredLWIR Long wavelength
infrared
4
Introduction

• Hyperspectral cameras provide useful
  discriminants for human face that cannot be
  obtained by other imaging methods.

5
Introduction

• The utility of using near-infrared (NIR)
  hyperspectral images for face recognition is
  studied
• Spectral measurements over the NIR allow sensing
  subsurface tissue structures
• Subsurface tissue
• Significantly different from person to person,
• Relatively stable over time,
• Nearly invariant to face orientations and
  expressions.

6
Introduction
Significantly different from person to person
7
Introduction
Nearly invariant to face orientations
8
Data Collection

• 200 subjects
• 31 spectral bands (0.7-1.0µm)
• Tunable filter
• 468x498 spatial resolution
• Uniform illumination
• 10 seconds each image.

9
Data Collection
10
Data Collection
7 images for each subject and at most 5 regions
(17x17) sampled
20 subjects took part of different imaging
sessions
11
Experiments

• Setup
• Cumulative Match Characteristic (CMC) curves.
• Minimum Mahalanobis Distance from query to
  gallery

where ?x is 1 or 0, if region x was sampled or
not Dx(i, j) is computed from the average
intensities of the sampled region x of i and j.
12
First Experiment
- Verification of utility of various tissues
types for hyperspectral face recognition - Only
frontal images were used (Gallery fg Query fa,
fb).

13
First Experiment
Better performance is achieved when different
tissues are combined
14
First Experiment
Changes in expression do not impact significantly
the hyperspectral discriminants
15
First Experiment
Forehead is the least affected by change of
expressions
16
Second Experiment
- Examination of the impact of changes in face
orientation for hyperspectral face
recognition - Only frontal images were used
(Gallery fg Query all others).
17
Second Experiment
45 - 75 for n 1 and 94 for n 5 90 - 80
for n 10. The distance function assumes that
tissue spectral reflectance does not depend on
photometric angles.
18
Second Experiment
Performance degrades as the size of the subset
considered increases.
19
Analyses of First and Second Experiment
20
Analyses of First and Second Experiment
21
Third Experiment

• Examination of variance of hyperspectral
  discriminants over time
• 20 subjects imaged between 3 days and 5 weeks
  after the first session
• The same 200 subject gallery is used.

22
Third Experiment
- Similar results for images from different
times - Significant reduction of performance
over single day images
23
Third Experiment
The difference in performance can be attributed
to changes in subject condition - blood
flow - water concentration - blood
oxygenation - melanin concentration Also -
sensor characteristics.
24
Questions?
25
Face Recognition on Fitting a 3D Morphable Model

• V. Blanz and T. Vetter
• Published at IEEE Trans. on PAMI
• Vol 25, No. 9, September 2003.

26
Introduction

• Color values in a face image do not depend only
  on the person identity (pose and illumination)
• Goal separate the characteristics of a face
  (shape and texture) from conditions of image
  acquisition
• The conditions may be described consistently
  across the entire image by a small set of
  extrinsic parameters

27
Introduction

• The algorithm developed combines deformable 3D
  models with CG simulations of illumination and
  projection
• It makes face shape and texture fully independent
  of extrinsic parameters
• Given a single image of a person, the algorithm
  automatically estimates face 3D shape, texture,
  and all relevant 3D scene parameters.

28
Model-Based Recognition
29
Morphable Model

• Vector space constructed such that any convex
  combination of shape and texture vectors Si and
  Ti describes a human face
• Continuous changes in model parameters generate a
  smooth transition that moves the initial surface
  toward a final one

30
Database of 3D Laser Scans

• Laser scans of 200 faces were used to create the
  morphable model

31
Correspondence

• Establish dense point-to-point correspondence
  between each face and a reference face
• Generalization of Optical Flow to 3D surfaces
  is used to determine the vector field

Vi
32
Generalized Optical Flow
To find the face vector field, the following
expression must be minimized for a neighborhood
R (5x5)
33
Face Vectors

• One scanned face is chosen as reference I0
• Reference shape and texture vectors are defined
  from conversion of each cylindrical coordinate to
  Cartesian coordinates

34
Face Vectors

• For a novel scan I, the flow field from I0 to I
  is computed and converted to cartesian
  coordinates (S and T).

35
Principal Component Analysis

• PCA is performed on Si and Ti
• Shape and texture eigenvectors (si and ti) and
  variances (sS and sT) are computed

36
Model Fitting

• Given a novel face image, the parameters and
  are found to provide the reconstruction of
  the 3D shape
• Pose, camera focal length, light intensity, color
  and direction are automatically found

37
Model Fitting
38
Model Fitting

• Optimization of shape coefficients and
  texture coefficients , along with pose
  angles, translation and focal length parameters,
  Lambertian light intensity and direction,
  contrast, and gains and offsets of color
  channels (?)
• Cost Function
• Optimization method Stochastic Newton
  Algorithm.
• Similar to stochastic gradient descent
  algorithm
• Makes use of first derivative of E

39
Experiments

• Model fitting and identification were tested on
  PIE (4488 images) and FERET (1940 images)
  databases
• None of the faces are in the model database
• Feature points manually defined
• Gallery and Query recognition approach.

40
Results of Model Fitting
41
Results of Model Fitting
42
Results of Recognition

• Metrics used for comparison
• Sum of Mahalanobis Distances dM c1-c22
• Cosine of the angle between two vectors
  dAltc1,c2gt/c1.c2
• Maximum-Likelihood and LDA
• c is a face, represented by shape and texture
  coefficients

dW is superior because it takes into account
fitting inaccuracy (different coefficients for
the same subject)
43
Results of Recognition
44
Results of Recognition
45
Results of Recognition
46
Comment

• Fitting process depends on user interaction and
  takes 4.5 minutes on a Pentium 3 2GHz.

47
Questions?