Principal Component Analysis and Eigenfaces: A Derivation

1
Principal Component Analysis and Eigenfaces A
Derivation

• Chris Beaumont

2
Mathematical Setup Principal Component Analysis

• Let F1, F2, , FM be a set of database images
  (each of which has n pixels)
• T1/M (F1 F2 FM ) the average face
• Gi Fi T the deviation of face i from the
  average face
• We seek a set of basis vectors v1, v2, , vk
  (k
• 1 if ij, 0 otherwise (orthonormal)
• 2 2 2 is a
  maximum
• The average projection of the basis vector along
  each face image is maximized

3
Geometrical Illustration in two dimensional space
1st Principal Component 1st Basis Vector
2nd Principal Component 2nd Basis Vector
4
PCA Contd

• A(G 1 G 2 G M ) is an n x M matrix of all
  faces
• Consider the matrix CAAt
• The autocorrelation matrix
• Symmetric and real (Self Adjoint)
• Because it is self adjoint, we may choose an ON
  set of vectors v1,v2,,vn such that Cvi?ivi
  where the ?s are in decreasing order
• If A consists of M images, C has M nonzero
  distinct eigenvalues
• Claim the eigenvectors of C are the proper basis
  vectors
• Consider an x in Spanvi

So /x2 is maximized when xv1
5
Our Goal

• Find an ON set of vectors v1,,vk such that
• 2 2 is maximized.
• ( ) x ( Gm )t
• (vit x A) (vit A)t
• (vit x A) (At x vi)
• vit x A x At x vi
• /vi2
• So the vs are the ordered eigenvectors of the
  autocorrelation matrix

6
Principal Components

• v1, , vk Form an ON basis for some subspace of
  the general image space this subspace is the
  optimal k-dim subspace for representing the M
  images of the training set. These are eigenfaces.
• For a face image x, x Saivi
• ai
• The coordinates ai in face space form a new set
  of vector distances for face comparison