Title: Investigating the Role of Glottal Features in Classifying Clinical Depression
1Investigating the Role of Glottal Features
inClassifying Clinical Depression
Elliot Moore II1, Mark Clements1, John Peifer2
and Lydia Weisser3
- 1School of Electrical and Computer Engineering,
Georgia Tech, Atlanta, GA - 2Interactive Media Technology Center, Georgia
Tech, Atlanta, GA - 3Department of Psychiatry and Behavioral Health,
Medical College of Georgia, Augusta, GA
2Overview
- Research Synopsis
- Glottal Wave and Speech Production
- Feature Extraction
- Analysis Method
- Classification Procedure
- Results / Conclusions
3Research Synopsis
- Part of overall research effort in stress/emotion
in speech - Goals
- Investigate extracted features of glottal
dynamics - Preliminary separation of control/depressed group
- Potential Areas of Application
- Clinical evaluation/monitoring tools
- Monitor improvement/digression
- Evaluate potential warnings of depression
- Telemedicine treatment
- Additional data for remote clinical sessions
- General Stress analysis
- 911, Forensic analysis, psychological evaluation,
customer call centers
4Glottal Wave?
- Three broad categories of speech analysis
- Prosodics Relative pitch, intensity, rhythm, and
rate of speech - Vocal Tract Related to shape and resonances of
the mouth and other vocal articulators - Glottal Waveform
- Glottal Waveform
- Related to volume velocity air profile through
vocal folds - Shape affected by vocal effort, vocal cord
tension, etc. - High correlation with speaker states
- Speaker characterization
- Dialect Information
- Speaker stress styles
5Speech Production Face Model
6Speech Production
- Source Filter theory
- S(z) R(z)G(z)V(z)
- R(z) Lip radiation
- G(z) Glottal volume velocity transformation
- V(z) Upper Vocal tract (Formant frequencies)
- Inverse filtering
- G(z) S(z) / ( R(z)V(z) )
- R(z) modeled as single pole (I.e., R(z) 1
z0z-1) (z0 ? 1) - Estimate V(z) from acoustic waveform (all-pole
filter) - P Prediction Order
- ai ith Linear Prediction Coefficient
7Ideal Glottal Flow
8Glottal Flow Example
Acoustic Signal
Glottal Flow
9Problem Areas in Emotion/stress Research
- Data acquisition
- Difficult to generate authentic response
- Difficult to assess level of stress/emotion
induced in subject - Stereotypical responses and exaggeration in actor
responses - Lack of definite labels and definitions (i.e.,
anger as hot anger or cold anger - No model
- Wide variety of human responses make a working
model extremely difficult - Lack of consistent deviations in traditional
prosodic measures across speakers
10Research Overview
- Depression in speech
- One of the more common emotional disorder
- Can be a precursor to suicide
- Primary Research Efforts in Prosodics
- Objectives
- Investigation of speech features related to
depression and vocal affect - Determine a robust method of vocal affect
analysis - Determine potential significance of glottal
features
11Database Collection
- Experimental Group - diagnosed with some form of
depressive illness - 9 females, 6 males
- From outpatient clinic (Psychiatry and Behavioral
Health Department, MCG) - Control Group no prior diagnosis of any
depressive illness - 9 females, 9 males
- From staff, students, etc. at MCG
- 3-5 minutes of total speech per subject
- Short story broken into 65 sentences
12Glottal Feature Extraction
- Timing
- Timing Ratios
- Amplitude Shimmer
- Spectrum
13Feature Extraction - Timing
- Timing
- Open Phase (O)
- Opening Phase (OP) onset of airflow through
vocal folds - Closing Phase (CP) termination of airflow
through vocal folds - Closed Phase
- Minimal or no airflow through vocal folds
14Feature Extraction Timing Ratios
- Timing Ratios
- rCPOP - Ratio of closing phase to opening phase
- rOTC - Ratio of open phase to total cycle (TC)
- rCTC - Ratio of closed phase to total cycle (TC)
- rOPO - Ratio of opening phase to open phase
- rCPO - Ratio of closing phase to open phase
15Feature Extraction - Amplitude
- Amplitude Shimmer
- Peak amplitude variation
- Measured at peak glottal opening
16Feature Extraction - Spectrum
- Spectral tilt and bias estimated by fitting a
line to the glottal frequency response over a set
interval. - Spectral Tilt Slope of the spectral roll-off of
the glottal frequency response - Spectral Bias Intercept term
- Two frequency intervals used for estimation
- Between the peak frequency response and 1000 Hz
- (gSt1000-spectral tilt gSb1000-spectral bias)
- Between the peak frequency response and 3700 Hz
- (gSt3700-spectral tilt gSb3700-spectral bias).
17Feature Extraction Spectrum Example
18Analysis Method
- Assumption Individual sentences carry contextual
affective content - Statistical analysis for affect is broken into
two categories - Intra-sentence statistics (within a sentence)
- Related to momentary affective expression as
fitting to the context of the story (i.e.,
character expression, narrator expression, etc.) - Inter-sentence statistics (between sentences)
- Attempts to capture the overall affect of the
speaker by monitoring the variation of affective
expression across the passage - Quantifies intra-sentence statistics
19Sentence Analysis Setup
S1 , SK , SK1, S2K, S65-K1 S65
20Analysis Setup
- Within Each Sentence (Intra-)
- 15 extracted glottal features (timing, timing
ratios, etc.) - 8 intra-sentence statistics
- 15 8 120 feature statistics (vector)
- Create observations
- Group 1 (G1) 13 observations of 5 sentences
- Group 2 (G2) 5 observations of 13 sentences
- Between Grouped Sentences (Inter-)
- 120 intra-sentence statistics per sentence
- 8 inter-sentence statistics
- 120 8 960 statistical measures per speaker
- One-Way Analysis of Variance (ANOVA)
21Classification Setup
- Separation accuracy
- Evaluate individual inter-sentence feature
statistics - Subject-based accuracy (I.e., use of entire set
of observations) - Gaussian Mixtures
- 2 Models ?c for controls ?p for patients
- N observations (based on grouping, G1, G2)
- Choose class with maximum log-sum probability
- Leave-One Out testing and evaluation
22Classification Accuracy (Males, G1)
23Classification Accuracy (Males, G2)
24Classification Accuracy (Females, G1)
25Classification Accuracy (Females, G2)
26Results (Males)
- Males
- Single Feature Accuracy (87)
- 2 misclassified subjects (1 patient, 1 control)
- Best feature pair Accuracy (grouping G2, 93)
- Closing Phase - Intra DRNG Inter Max
- Closing Phase - Intra IQR Inter DRng
- 1 misclassified control
- Best overall single feature statistics
- Glottal Spectrum
- Glottal Ratios
- G1 vs. G2
- Observational length affected feature selection
slightly - Overall accuracy similar for both
27Results (Females)
- Females
- Single Feature Accuracy
- 100 separation for (gSt3700 Intra MAX Inter
Std) in G1 - Feature Pair
- No improvement in overall accuracy for G2
- Best overall single feature statistics
- Glottal Timing
- Glottal Spectrum
- G1 vs. G2
- Best overall separation in G1 (I.e., shorter
observational length, more observations)
28Discussion
- Conclusions
- Relatively high separation accuracy males/females
- Exclusive use of glottal features
- Potentially more robust than prosodic features
- Further Work
- Subject accuracy vs. Observational Accuracy
- Subject Accuracy Classifying subject utilizing
all observations (3-5 minutes of speech) for
majority vote decision - Observational Accuracy Accuracy classifying
single observations at random - High Observational Accuracy ultimate goal
- Finding optimal set of class separators
- Understanding perceptual link