Face Detection C library with Skin and Motion analysis

1
Face Detection C library with Skin and Motion
analysis Yuriy V. Chesnokov (http//www.codeprojec
t.com/audio/face_detection.asp)
2
The first step in intelligent image\video
processing for face recognition in uncontrolled
scenery with complex background (outdoor
environments, airports, train\bus stations) is
face detection. Upon precision of the later
heavily depends your face recognition results.
The project demonstrates face detection SSE
optimized C library for color and gray scale
data with skin detection, motion estimation for
faster processing, small sized support vector
machine (SVM) and artificial neural network (ANN)
rough face prefiltering, PCA\LDA\ICA\any
dimensionality reduction\projection and final ANN
classification. It detects multiple faces
on conventional desktop PC with 2.0Ghz processor
at least at 15fps on 640x480 RGB video stream.
With motion and skin estimation detection rate
could be as high as 50fps.
3
The algorithm

• Downscale the image with Haar transform to 80x60
  to reduce search space, remove high freq noise,
  convert it to gray scale.
• Estimate motion vector to reduce search space.
• Detect skin regions (optional) and combine search
  region with motion vector.
• Build image pyramid from 80x60 image and smooth
  the data with Gaussian filter.
• Browse through histogram equalized 19x19
  rectangles on the image pyramid in the search
  space only.
• Reject non-face rectangles with small sized
  classifiers 2 support vectors linear SVM, 3
  layered ANN with 2-4 hidden neurons.
• Project positive face rectangles to PCA/LDA/ICA
  or any other linear projection basis to reduce
  dimensionality of 19x19 361 dimensional image
  vector.
• Classify the projected 19x19 rectangles with ANN
  classifier.
• Estimate the ANN output on image pyramid and
  detects multiple faces.

4
Image downscaling
Haar based wavelet transform http//www.codeproj
ect.com/useritems/Dyadic_down_sampling.asp
5
Skin detection
Collect skin samples and non-skin samples. Train
ANN to discriminate between them. Backpropagation
Artificial Neural Network in C (http//www.codep
roject.com/useritems/neural_network.asp) Skin
detection results on independent test set Se
97.77 Sp 99.22 Pp 98.30 Np 98.97 Ac
98.76 Sensitivity (Se), specificity (Sp),
positive predictivity (Pp), negative
predictivity, (Np) accuracy (Ac).
6
Rough non-face rejection
Arrange small classifiers processing time
efficient to reject majority of non-face
rectangles. Linear SVM with 2 support vectors
representing means from face set and non-face
set. Results on 1800 faces and 18000 non-faces
samples. Se 93.85 Sp 76.72 Pp 28.76 Np
99.20 Ac 78.28 High negative predictivity rate
99. Very precise in identifying non-faces.
7
Rough non-face rejection
3 layered ANN with 2-4 neurons in hidden
layer Results on 1800 faces and 18000 non-faces
samples. Se 95.08 Sp 99.16 Pp 91.89 Np
99.51 Ac 98.79 Even higher specificity and
negative predictivity rate 99 compared to SVM.
Very precise in identifying non faces.
8
Some of the faces used in training set
9
Some of the non-faces used in training set
10
PCA projection
Obtain PCA projection basis from the face,
non-face training set. 40 eigenvectors with 95
of covariance retained.
11
Project the data to the PCA basis
Error bar plot shows projection of the training
set.
12
LDA projected data
LDA projection separates only 1st component of
the face, non-face data, worse performance
13
Train ANN classifier of PCA projected data
4 layered ANN with 40-20-10-1 configuration. Use
cross validation to avoid overfitting. Results
on 1800 faces and 35000 non-faces samples divided
to 50 for train set and 50 to validation and
test set. Train set 893 faces, 17154
non-faces Se 99.55 Sp 100.00 Pp 100.00 Np
99.98 Ac 99.98 Validation set 447 faces, 8578
non-faces Se 96.42 Sp 99.90 Pp 97.95 Np 99.81
Ac 99.72 Test set 447 faces, 8578
non-faces Se 97.32 Sp 99.90 Pp 97.97 Np 99.86
Ac 99.77
14
Some Results on Real Time Face Detectionin the
next few slides
15
12 of motion, 9 of skin. 27fps on 2.0 Ghz
single core
16
24 of motion, 14 of skin. 19fps on 2.0 Ghz
single core
17
12 of motion, 2 of skin. 55fps on 2.0 Ghz
single core
18
Contacts
Yuriy V. Chesnokov Krasnodar, Russia.
chesnokov_yuriy_at_mail.ru chesnokov.yuriy_at_gmail.ru
Articles available online http//www.codeproject
.com/script/Articles/list_articles.asp?userid4613
314