Multiscale Integral Invariants For Facial Landmark Detection in 2'5D Data

1
Multi-scale Integral Invariants For Facial
Landmark Detection in 2.5D Data
MMSP07, Chania, Crete Greece Oct. 2, 2007

• Adam Slater, Yu Hen Hu, Nigel Boston
• University of Wisconsin Madison
• Dept. Electrical and Computer Engineering
• 1415 Engineering Drive
• Madison, WI 53706
• hu_at_engr.wisc.edu

2
Overview

• we propose a 3D integral invariant feature for 3D
  surface
• inspired by the work of integral invariant
  signatures by Manay et al. (Soatos group) But
  ours is for 3D contour, and theirs is 2D curve
• an efficient, incremental feature extraction
  method of the proposed 3D integral invariant
• apply it to detect the nose tip landmark for a
  given range image of human face.
• A hierarchical multi-modal 3D surface landmark
  detection method for locating nose tip using both
  3D range image and corresponding 2D color image

3
Outline

• Integral invariant signature
• Feature extraction
• Applications to nose tip detection
• Experimental Results

4
3D Face Detection and Recognition

• State of art face recognition has been mostly
  dealing with 2D facial images.
• 3D surfaces registration is still
  computation-intensive.
• E.g. ICP (Iterative Closest Point) algorithm for
  registration.
• Existing 3D surface registration methods
• Extracting feature points based on local invarant
  features, eg. Curvature,

5
Point Signature

• Given a point p on a 3D surface, contour C is
  the intersection of a ball of radius r, centered
  at p with the surface.
• Fitting C into a planer circle C yields a normal
  vector n1 of C.
• Distance from C to C measured at different
  angles on C form a signature for point p.
  Chua00
• Computation intensive and sensitive to noise

6
3D Integral Invariants General

• ? a surface, p a point on ?, volume
  enclosed by ?
• dµ(x)
• infinitesimal geodesic distance on the surface.
• h(p, x) a kernel function satisfying
• where G is a group and gp is the point p under
  group action of g in G.

7
We Choose

• Br(p) a sphere of radius r centered at the point
  p
• ?() an indicator function
• h(p, x) is invariant with respect to the special
  Euclidean group, translations and rotations
• Resulting integral invariant
• represents the volume of intersection of a sphere
  of radius r, centered at point p, and volume
  enclosed by the surface ?.
• Question How to choose r?

8
Revised Kernel

• Some value of r would be more relevant to
  characterize the surface surrounding p.
• To exploit such feature, we consider a
  differential representation of the integral
  invaraint.
• An integral invariant equivalent to the volume of
  intersection of the surface ? and a spherical
  shell with interior radius rk-1 and exterior
  radius rk.

9
Integral Invariant Feature

• Approximated integral invariant at each rk
  yields a number.
• For rk range from 0 to a maximum value, these
  integral invariants form a feature vector.
• The feature dimension is dependent on particular
  type of 3D surface under study and hence would
  best be determined empirically.
• We use 3D (2.5 D range image) data from FRGC v2.0
  dataset.
• 100 set of range scan of facial images
• Loosely (manually) registered
• Goal set to detect the nose tip position
• Training and testing data set
• select nose tip and other points on range face
  images
• Compute feature vectors with r varying from 4mm
  to 100 mm in 2mm increment.

10
Feature Dimension Selection

• Feature vectors belonging to nose tip form a
  K-dim pdf. The non-nose tip vectors form another
  K-dim pdf.
• Approach select K such that the two
  distributions are most separated
• Separation is measured using Mahalanobis distance
  
• m1, m2 mean vectors
• S Covariance matrix of the non-nose
  distribution
• We choose K 60mm.

11
Multi-modal Classifiers

• Facial region
• Featur 2D color space
• Yield a probability map of whether a point belong
  to facial region
• Medial canthus (inside eye corner) Detector
• Detecting medial canthus using the 3D integral
  invariant feature
• Used as a land mark
• Distance from each canthus to nose tip are
  classified
• Nose tip classifier
• Determine nose tip based on 3D integral invariant

12
Mahalanobis Distance

• Average Mahalanobis distance of facial points
  from the mean, evaluated on 100
  loosely-registered face scans with feature radii
  from 4mm to 100mm in increments of 2mm. Note the
  bright spots around medial canthus

13
Two-Class Distance Based Classifier

• Each feature vector is labeled with class 0 or 1
  (nose vs non-nose, face vs non-face, etc)
• Distance between each pair of feature vectors are
  calculated.
• Distance between vectors of the same label should
  be small
• Distance between vectors of different labels
  should be large
• Yield a (normalized) distance map where distance
  between the same vectors are discarded.
• Corresponds to a leave-one-out cross validation
  method
• A simple fusion of three normalized distance maps
  corresponding to three classifiers is conducted
  to yield a combined distance map.

14
Experiment Set UP

• FRGC 2.0 database of 4007 2.5D range scans and
  associated 2D color images of human faces.
• many include hair and clothing as well as pose
  and expression variation.
• Preprocessing a median filter and removal of
  very large outlier points, as well as resampling
  on a 1mm square rectangular grid.
• 101 were used as a training set, and the
  algorithm was evaluated on the remaining 3906.
• 10 radii ranging from 4mm to 60mm.
• Baseline comparison
• For the sake of comparison, we also implemented
  an earlier algorithm proposed by Xu et al.12,
  chosen for its thorough algorithm description.
• The parameters used for this algorithm were the
  same given in the paper.

15
Cumulative Distribution of Nose Tip Detection
Cumulative Probability
Distance from nose tip (mm)
16
Results and Discussion

• Examine the percentage of noses which were
  detected within 1cm of the manually selected
  position.
• Using only the integral invariant, 98.08 of
  these were detected to this tolerance.
• With skin color segmentation, this improved to
  98.52.
• Using the canthus distance further improved
  classification to 99.08.
• Our baseline method determined 45.3 of the noses
  to this degree of accuracy. This was due in part
  to a high number of false positives due to hair
  and clothing variations and early culling of
  important feature points, possibly due to the
  density of our range scans or a small amount of
  surface noise.