Face Photo Recognition Using Sketch

1
Face Photo Recognition Using Sketch

• Xiaoou Tang, Xiaogang Wang
• Department of Information Engineering
• The Chinese University of Hong Kong
• Shatin, Hong Kong
• Xtang_at_ie.cuhk.edu.hk

2
Abstract

• We develop a novel face photo retrieval
  algorithm using face sketches. By transforming a
  photo image into a sketch, we reduce the
  difference between photo and sketch
  significantly, thus allowing effective matching
  between the two. Experiments over a data set
  containing 188 people clearly demonstrate the
  efficacy of the algorithm.

3
Task Description

• Application. Searching a face image database by
  sketch drawing is useful in law enforcement, when
  the photo of the suspect is not available. Since
  the database is quite large, it is valuable to
  build a system capable of automatically recognize
  faces from sketches.
• Procedure

Sketch
Identify
Photo database
4
Database for Research

• There are 188 photo sketch pairs in the database.
• Sample

Photo
Sketch
5
Recognition Diagram

• Basic idea for an input photo, generate a
  pseudo-sketch from the training set, to reduce
  the difference between photo and real sketch.

6
Eigensketch Transformation

• Fit the input photo as a linear combination of
  the training photos, and render the pseudo-sketch
  by replacing the training photos with the
  corresponding sketches. PCA is an optimal signal
  reconstruction in the sense of mean square error.
  Reconstructing Photo by PCA can achieves an
  optimal linear combination from the training set.
  So we develop the Eigensketch transformation
  based on PCA to generate the pseudo-sketch from
  the training set.

7
Eigensketch Transformation


Pseudo-Sketch
The weight vector is got from the
face photo Eigenspace
8
Eigensketch Transformation

• Some pseudo-sketch generated by computer

Photo
Reconstructed Photo
Sketch
Pseudo-sketch

• The pseudo-sketches are matched with the real
  sketches in the sketch Eigenspace for recognition

9
Experiment

• Database
• 88 photos and corresponding 88 sketches for
  training, 100 photos for reference and the 100
  corresponding sketches for test.

• Method
• Geometry Method (26 measures of geometry
  distance)
• Eigenface Method
• New method

10
Experiment
Table 1. Cumulative match score for the three
methods
11
Experiment
Figure 1. Cumulative match score for the three
methods
12
Experiment

• Discussion
• The recognition rate is low using geometry
  feature and Eigenface. Only 30 accuracy is
  obtained for first math 70 for the tenth rank.
  This shows that photo and sketch have great
  difference on texture and shape
• The Eigentransformation method greatly improved
  the recognition accuracy to 96 for the top 10
  accuracy.
• Although the small size of database, the relative
  performance of the three methods has demonstrated
  the advantage of the new method.

13
Conclusion

• Sketch and photo have great difference in texture
  and shape. This is the main difficulty for
  recognition.

• Our method transforms a photo to sketch, thus
  significantly eliminating their difference.

• The great improvement on experiment to the
  traditional algorithm, clearly demonstrated the
  efficacy of this method.