Face Analysis

1
Face Analysis

• Jim Rehg
• CS 7636 Computational Perception
• Lecture 3
• Wed Jan 15, 2003

2
Face Modeling

• Beginning with eigenfaces work of Turk and
  Pentland, face recognition has spurred the
  revival of learning and data-driven approaches in
  vision.
• Face applications
• Detection
• Recognition
• Identity
• Age, Gender, Ethnicity, Beauty
• Tracking
• Head pose
• Eye gaze
• Analysis and synthesis (facial animation)
• Facial expressions (i.e. emotion)
• Speechreading

3
Sources of Variability in Faces

• Head pose
• Illumination
• Facial hair, glasses, etc.
• Facial expressions
• Identity
• Age, gender, ethnicity, etc.

4
Issues in Face Modeling

• Parts-based vs. holistic

Parts-based
Holistic
Each face image is unwrapped into a big vector
Each face image is composed of regions
5
Issues in Face Modeling

• Parts-based vs. holistic
• Explicit 3-D vs. 2-D view-based

Subspace model
6
Issues in Face Modeling

• Parts-based vs. holistic
• 3-D vs. 2-D
• Generative vs. discriminative

Faces
7
Subspace of face images
Same lighting
Pose and lighting have a dramatic effect on
appearance(Moses Ullman 01)
Same person

• Pixel coordinates are not semantically useful
• Do faces form a compact subspace (manifold) in
  the space of all possible images?
• Then we can recover the intrinsic manifold
  dimensions (identity, lighting, etc.) with
  semantic meaning.

8
Uses of subspace models
Projection
Subspace defined by a set of basis vectors
Inside face manifold?

• Reconstruction (universality)
• Detection and recognition
• Synthesis

9
Game Plan

• We will adopt the holistic approach and treat
  each face image as a single vector.
• Use dimensionality reduction to identify
  subspace.
• PCA and Probabilistic PCA
• Relate back to Lambertian model properties
• Shape Texture decomposition to improve
  robustness.
• Morphable Models (aka Active Appearance Models)
• Model effects of pose and illumination as
  manifolds.

10
Dimensionality Reduction
Subspace model
11
Principle Component Analysis

• PCA is an orthonormal projection of a random
  vector X onto a lower-dimensional subspace Y that
  minimizes mean square error.
• Equivalently, PCA yields a distribution for Y
  with
• Uncorrelated components
• Maximum variance

12
PCA Derivation
13
PCA Properties

• Advantages
• Optimal linear method
• Direct computation from data
• Usage involves simple matrix operations
• Global in space, local in frequency
• Disadvantages
• Does not define a proper probability model
• Basis computation can be expensive.
• How to handle missing data?
• Global in space, local in frequency