Hidden Markov Model Classification of Myoelectric Signals in Speech

1
Hidden Markov Model Classification of Myoelectric
Signals in Speech

• Adrian D.C. Chan
• PhD Candidate
• Institute of Biomedical Engineering
• University of New Brunswick
• Canada

2
Alternative Control

• Complex instrumentation panel

3
Speech Recognition

• Simplifies user interface
• Improves safety by encouraging head-up flying

4
Speech Recognition

• Simplifies user interface
• Improves safety by encouraging head-up flying

5
Speech Recognition Problems
80
60
Classification Error ()
40
20
0
20
22
24
26
28
INF
Signal to Noise Ratio (dB)
6
Research Objective

• Develop a robust multi-expert speech recognition
  system

7
Speech Recognition Problems
80
60
Classification Error ()
40
20
0
20
22
24
26
28
INF
Signal to Noise Ratio (dB)
8
Conventional Acoustic System
FINE
Acoustic Speech Expert
9
Multi-Expert System
FINE
Acoustic Speech Expert
Myoelectric Speech Expert
10
Advantages of Myoelectric Signals

• Immune to acoustic noise
• Acoustically similar words may be distinct in the
  myoelectric signal
• e.g. fine and sign

11
Advantages of Myoelectric Signals

• Immune to acoustic noise
• Acoustically similar words may be distinct in the
  myoelectric signal
• e.g. fine and sign

12
Myoelectric Speech Expert

• Hidden Markov Models are used extensively in
  acoustic speech recognition system

13
Myoelectric Speech Expert

• Hidden Markov Models classify on sequences of
  features
• Hidden Markov Models resilient to time-scale
  variance

14
Myoelectric Speech Expert

• Hidden Markov Models classify on sequences of
  features
• Hidden Markov Models resilient to time-scale
  variance

15
Experiment Objective

• Evaluate Hidden Markov Model myoelectric speech
  experts resilience to temporal variance
• Compare against another method (Linear
  Discriminant Analysis)

16
Experiment Methods

• 2 male subjects
• 5 facial muscles
• Vocabulary
• zero to nine
• 45 repetition per word
• randomized order

17
Experiment Methods

• 2 male subjects
• 5 facial muscles
• Vocabulary
• zero to nine
• 45 repetition per word
• randomized order

18
Experiment Methods

• 2 male subjects
• 5 facial muscles
• Vocabulary
• zero to nine
• 45 repetition per word
• randomized order

19
Data Segmentation
acoustic
Time (s)
20
Data Segmentation
acoustic
myo1
Trigger
myo2
myo3
myo4
myo5
Time (s)
21
Data Segmentation
Window 1
Window 2
Train Set
Test Set
acoustic
myo1
myo2
myo3
myo4
myo5
Time (s)
22
Data Segmentation
Temporal Variance
Train Set
Test Set
acoustic
myo1
myo2
myo3
myo4
myo5
Time (s)
23
Temporal Variance
Train Set
Test Set
acoustic
myo1
myo2
myo3
myo4
myo5
Time (s)
24
Temporal Variance
Train Set
Test Set
acoustic
myo1
myo2
myo3
myo4
myo5
Time (s)
25
Temporal Variance
Train Set
Test Set
acoustic
myo1
myo2
myo3
myo4
myo5
Time (s)
26
Hidden Markov Model

• 6 state left-right Hidden Markov Model
• 64 ms observations windows
• 2 autoregressive coefficients
• integrated absolute value

27
Hidden Markov Model

• 6 state left-right Hidden Markov Model
• 64 ms observations windows
• 2 autoregressive coefficients
• integrated absolute value

28
Hidden Markov Model

• 6 state left-right Hidden Markov Model
• 64 ms observations windows
• 2 autoregressive coefficients
• integrated absolute value

29
Hidden Markov Model

• 6 state left-right Hidden Markov Model
• 64 ms observations windows
• 2 autoregressive coefficients
• integrated absolute value

30
Linear Discriminant Analysis
Englehart et al, 1999
31
Results
60
Linear Discriminant Analysis
AC LDA
50
JR LDA
40
30
Classification Error ()
20
10
0
-100
-50
0
50
100
Misalignment of Test Set to Train Set (ms)
32
Results
60
Linear Discriminant Analysis
AC LDA
50
JR LDA
40
30
Classification Error ()
20
10
0
-100
-50
0
50
100
Misalignment of Test Set to Train Set (ms)
33
Results
60
AC LDA
50
JR LDA
40
30
Classification Error ()
20
10
0
-100
-50
0
50
100
Misalignment of Test Set to Train Set (ms)
34
Multi-Expert System
Acoustic Speech Expert
Myoelectric Speech Expert
35
Multi-Expert Results
Borda Count
80
Multiplying
Adding
60
Classification Error ()
40
20
0
20
22
24
26
28
INF
Signal to Noise Ratio (dB)
36
Conclusions

• A Hidden Markov Model based classifier is
  resilient to temporal variance
• A myoelectric speech expert can augment an
  acoustic speech expert to create a robust
  multi-expert speech recognition system

37
Conclusions

• A Hidden Markov Model based classifier is
  resilient to temporal variance
• A myoelectric speech expert can augment an
  acoustic speech expert to create a robust
  multi-expert speech recognition system

38
Hidden Markov Model Classification of Myoelectric
Signals in Speech

• Adrian D.C. Chan
• PhD Candidate
• Institute of Biomedical Engineering
• University of New Brunswick
• Canada

39
Positive Pressure Breathing
40
Other Applications
41
Inter-individual Differences
42
Problem of Facial Hair