My Project Title

1
My Project Title

• -Sridhar Godavarthy

2
Contents

• A Little Background Blink
• A Lot More Background Strain as a Soft
  Forensic Evidence
• Facial Recognition
• Culprits
• Human anatomy as a feature
• Strain Measurement
• Micro expression Detection using Strain Patterns
• Challenges
• Sample Strain patterns
• References

3
Contents

• A Little Background Blink
• A Lot More Background Strain as a Soft
  Forensic Evidence
• Facial Recognition
• Culprits
• Human anatomy as a feature
• Strain Measurement
• Micro expression Detection using Strain Patterns
• Challenges
• Sample Strain patterns
• References

4
BLINK!!!
A Little Background
5
Introduction Blink

• Why are some people brilliant decision makers?
• How do some people act upon instincts?
• Why are we unable to explain some decisions?

6
Blink Contd

• Great decision makers are not ones that process
  the most information
• Malcolm Gladwells The statue that didnt look
  right
• They are those who have perfected the art of
  Thin Slicing
• Filtering out the very few factors that matter.

7
Navarasas the Nine Emotions
8
Contents

• A Little Background Blink
• A Lot More Background Strain as a Soft
  Forensic Evidence
• Facial Recognition
• Culprits
• Human anatomy as a feature
• Strain Measurement
• Micro expression Detection using Strain Patterns
• Challenges
• Sample Strain patterns
• References

9
Facial Strain Pattern as a Soft Forensic Evidence
A Lot More Background

• V.Manohar, D.B.Goldgof, S.Sarkar,Y.Zhang

Some slides have been adapted from the Authors
presentation
10
Facial Recognition

• Face recognition has made huge advances
• Picasas Web Albums
• Sonys say cheese( or is it CHEERS) detection
• Almost perfect
• Picasa still confuses between closely related
  faces
• Canon almost always never detects my face
• Some say - might be because of my hair -)
• Has anyone used the Lenovo Face ID?
• Because they use static images
• Could be supplemented for better performance.

11
Culprits (ICHE)

• Illumination
• Camouflage(Makeup/glasses)
• Facial Hair
• Expressions
• The Solution Use methods based on Human Anatomy

12
Methods based on Human Anatomy

• Iris scan
• Retina scan
• Skull X-ray
• Disadvantage
• Require Specialized equipment
• Intrusive
• Proposed Alternative
• Skin and tissues of the face

13
Elasticity
Authentic Author Slide

• Different materials have different elasticity
• Elasticity can be modeled

Known
Calculate
14
Facial Strain

• What is Facial Strain?
• Strain on soft tissue when expressions are made.
• Anatomical method
• Uses a pair of frames to measure deformation

15
Facial Strain

• Why Facial Strain?
• As it is a difference, it is independent of all
  the earlier mentioned culprits(ICHE)

16
Facial Strain

• Visual Pattern is unique to every face.
• Easily quantifiable by elasticity
• Hard to measure non-linear, inverse equations
• Can be represented by strain pattern under
  specific boundary conditions
• Is unique to a person.

17
Measurement of Facial Strain

• Contact strain measurement equipment is already
  available.
• Cannot be used if we are looking to identify
  people at a Casino/Airport
• Did I mention the actual applications of this
  paper
• Soft forensics based on surveillance videos

18
Measurement of Facial Strain Contd

• Two major steps
• Obtain motion field between two frames
• Compute strain image from above Motion field.

19
First Step Obtaining Motion Field

• Feature Based
• Need to identify features Difficult!
• Features may be ill defined( when camouflaged)
• Usually requires manual intervention
• Produces a sparse motion field
• Produce Good correspondence in large motion
• Optical Flow based
• Fully automated
• Dense Motion field.
• Requires constant illumination

20
First Step Optical Flow
Adapted Author Slide

• Observed motion over sequential image frames

21
Second Step Strain Computation Type

• 3D Strain
• Ideal
• No high speed equipment available to capture
  range images
• 2D Strain
• Well not much of a choice
• Authors could use existing data.

22
Second Step Optical Strain
Authentic Author Slide

• Variation of displacement values obtained from
  optical flow
• Calculated by taking the derivative of each pixel

• Sobel operator (central difference)

23
Strain Computation - methods

• Finite Element Method
• Forward modeling when Dirichlet condition is
  satisfied
• Good at handling irregular shapes
• Computationally expensive
• This method is an approximation to the solution
• Finite Difference Method
• Strain, a tensor, can be expressed derivatives of
  the displacement vector
• This can be approximated by a Finite Difference
  Method.
• Very efficient when carried out on a regular
  grid.
• This method is an approximation to the
  differential equation

24
Finite Difference Method

• Finite Strain tensor
• Cauchy tensor

25
Integrating Strain Patterns

• Motion is mostly vertical
• Strain pattern is dominated by its normal
  components
• The strain magnitudes are scaled to gray levels
• White highest strain
• Black lowest strain
• It is now a pattern matching problem.

26
Review of Choices

• Motion field Based on Optical flow
• Strain Type 2-D
• Computation Finite Difference Method

27
Examples
28
Identification and matching

• Strain Magnitude is now 1-D
• Use PCA to perform matching

29
Experiments

• Experiments performed on
• Normal light
• Low light
• Shadow light
• Regular face
• Camouflaged face
• Frontal view
• Profile view
• Neutral expression
• Open mouth

30
Experiments Contd

• Subject may not perform the expression to the
  same extent every time
• Experiments repeated on shorter, subsampled
  videos

31
Results

• Strain measurement seems to be logically correct
• We do not discuss the PCA and hence the
  recognition results as they are outside the scope
  of this discussion.( But they were good)
• Acts as a supplement to existing recognition
  methods.

32
Contents

• A Little Background Blink
• A Lot More Background Strain as a Soft
  Forensic Evidence
• Facial Recognition
• Culprits
• Human anatomy as a feature
• Strain Measurement
• Micro expression Detection using Strain Patterns
• Challenges
• Sample Strain patterns
• References

33
Micro expression Detection using Strain Patterns
34
Macro Vs Micro expressions

• Macro Expressions
• Large movement
• Smile
• Talking
• Shaking head
• Micro expressions
• Raising eyebrow
• Fast blinking

35
Can you classify?
36
Where will it be used?

• Supplement lie detection
• Very little noise
• As part of a general discussion
• Bond might not have lost even the first time!

37
Ideal Frame Sequence
c. 1-4
b. 3-4
a. 1-2
1
2
3
4
5
n
6
d. 1-3
e. 4-6
f. 1-6

a 1-2 100
b 3-4 200
c 1-4 300
d 1-3 200
e 4-6 200
f 1-6 400
38
Strain Measurement for a Practical Frame Sequence
39
Challenges

• Small movements are inevitable
• Macro expressions also possible
• Eyes always blink. Need to detect changes in
  speed of blinking
• Need to identify the frames to be used

Solution Normalize
40
References

• V.manohar, D.B. Goldgof, S.Sarkar, Y. Zhang,
  "Facial Strain Pattern as a Soft Forensic
  Evidence", IEEE Workshop on Applications of
  Computer Vision (WACV'07),pp 42-42
• Vasant Manohar, Matthew Shreve, Dmitry Goldgof
  and Sudeep Sarkar, "Finite Element Modeling of
  Facial Deformation in Videos for Computing Strain
  Pattern", International Conference on Pattern
  Recognition, Dec. 2008
• Matthew A. Shreve, Shaun J. Canavan, Yong Zhang,
  John R. Sullins, and Rupali Patil, "Imaging And
  Characterization Of Facial Strain In Long Video
  Sequences",xxxx
• Malcolm Gladwell, Blink The Power of Thinking
  Without Thinking, Back Bay Books (April 3, 2007)

41
Thank You!

• Sridhar Godavarthy
• Dept. Of Computer Science and Engineering
• University of South Florida
• sgodavar_at_cse.usf.edu