Myoelectric Signals in Speech Recognition

1
Myoelectric SignalsinSpeech Recognition

• Adrian Chan
• PhD Candidate
• Institute of Biomedical Engineering
• University of New Brunswick

2
Outline

• Conventional Speech Recognition
• Myoelectric Speech Recognition
• Experimental Procedures
• Results
• Conclusions

3
Speech Recognition on a Jet
4
Conventional Speech Recognition
Seven
7
5
The Problem with Noise
Seven
11
6
Myoelectric Speech Recognition
Seven
7
7
Advantages of Using Myoelectric Signals

• not corrupted by audio noise
• there are similar sounding words with unique
  mouth positions implying unique myoelectric
  signals
• Example sign and fine

8
Feasibility Study
Is there speech information within the
myoelectric signal???
9
Experimental Setup
Data Collection
Acoustic Data
Myoelectric Data
10
Experimental Setup
Data Collection
Acoustic Data
Myoelectric Data
11
Experiment 1

• Vocabulary Digits 0 to 9
• Isolated words
• Repeated words in nonrandom fashion
• 5 Surface myoelectric signals
• (simultaneously recorded with acoustic data)
• 5 subjects (4 male, 1 female)
• Canadian English

0 0 0 0 0 0 0 0 0
1 1 1 1 1 1 1 1 1
2 2 2 2 2 2 2 2 2
9 9 9 9 9 9 9 9 9
12
ElectrodePlacement

• Levator
• Anguli Oris

• Zygomaticus
• Major

• Platysma

• Depressor
• Anguli Oris

• Anterior
• Belly of the
• Digastric

13
Electrodes in Mask
14
Experimental Setup
Data Collection
Acoustic Data
Myoelectric Data
15
Data Segmentation
acoustic
LAI
ZYG
PLT
DAO
ABD
Time (s)
16
Data Segmentation
Trigger
acoustic
LAI
ZYG
PLT
DAO
ABD
Time (s)
17
Data Segmentation
1024 ms
Trigger
acoustic
LAI
ZYG
PLT
DAO
ABD
Time (s)
18
Data Segmentation
1024 ms
Trigger
acoustic
LAI
ZYG
PLT
DAO
ABD
Time (s)
19
Data Segmentation
acoustic
LAI
ZYG
PLT
DAO
ABD
Time (s)
20
Data Segmentation
Pretrigger
acoustic
LAI
ZYG
PLT
DAO
ABD
Time (s)
21
Data Segmentation
Pretrigger
acoustic
LAI
ZYG
PLT
DAO
ABD
Time (s)
22
Experimental Setup
Data Collection
Acoustic Data
Myoelectric Data
23
Data Classification
Myoelectric Data
Feature Extraction
PCA
LDA
Word Spoken
24
Effect of Pretrigger
15
S1
S2
S3
10
S4
Classification Error ()
S5
5
0
0
100
200
300
400
500
600
700
Pretrigger (ms)
25
Effect of Pretrigger
15
S1
S2
S3
10
S4
Classification Error ()
S5
5
0
0
100
200
300
400
500
600
700
Pretrigger (ms)
26
Effect of Pretrigger
15
S1
S2
S3
10
S4
Classification Error ()
S5
5
0
0
100
200
300
400
500
600
700
Pretrigger (ms)
27
Effect of Pretrigger
15
S1
S2
S3
10
S4
Classification Error ()
S5
5
0
0
100
200
300
400
500
600
700
Pretrigger (ms)
28
Experiment 2

• Vocabulary Digits 0 to 9
• Isolated words
• Repeated words in random fashion
• 5 Surface myoelectric signals
• (simultaneously recorded with acoustic data)
• 2 subjects (2 male)
• Canadian English

6 2 9 1 3 7 9 1 3
29
Randomization of Words
30
Exp1
25
20
S1
S2
Classification Error ()
15
10
5
0
0
100
200
300
400
500
600
700
Pretrigger (ms)
30
Conclusions

• The myoelectric signal does contain speech
  information
• Myoelectric signal precedes the acoustic
  information
• Optimal pretrigger 400 500 ms

31
Future Work
Data Collection
Myoelectric Data
Acoustic Data
Data Segmentation
Data Classification
32
Future Work
Data Collection
Myoelectric Data
Acoustic Data
Data Segmentation
Data Classification
33
Acknowledgements

• Supervisors
• Dr. Kevin Englehart
• Dr. Bernard Hudgins
• Dr. Dennis Lovely
• Defense Evaluation Research Agency
• National Sciences and Engineering Research
  Council of Canada

34
Questions
35
Intersubject Differences
36
Positive Pressure Breathing