Speaker Independent Lipreading

1
Speaker Independent Lipreading

• By Steve Davis

2
Introduction

• Visual only system
• Supplement to audio system
• Speaker independent
• Small vocabulary
• Whole word vocabulary
• 2 stage visual transform
• 1 stage statistical model

3
Video stream

• 8 bit grayscale
• 180 x 120 pixel
• Mouth centered in frame
• 30 frames per second

4
Canny Edge Detector

• 5 pixel wide sobel operator
• High threshold 200
• Low threshold 50

5
Corner Identification

• Search for corners with high eigenvalues
• Perform non-maxima suppression over 3x3
  neighborhoods
• Quality level .4
• Minimum separation of 4 pixels
• Data reduction of at least 64 times

6
Statistical manipulation

• Hidden markov model
• Left right model, unconstrained
• Discrete valued
• Training forward-backward algorithm, baum welch
  reestimation (linear probabilities)
• Recognition viterbi algorithm
• 10 words
• 45 frame sequence length

7
(No Transcript)
8
Conclusions

• 2 stage visual transform reduces data size
• High frame rate
• High resolution
• Speaker independent