Intelligent and Noise-Robust Interfaces for MEMS Acoustic Sensors: Smart Microphone

1
Representation of Timbre in
the Auditory System
Shihab A. Shamma

Center for Auditory and Acoustic Research
Institute for Systems Research
Electrical and Computer Engineering
University of Maryland, College Park
2
(No Transcript)
3
A
t
t
r
i
b
u
t
e
s

o
f

C
o
m
p
l
e
x

S
o
u
n
d
s
A
n
a
t
o
m
y

o
f

t
h
e

A
u
d
i
t
o
r
y
Location
Timbre
Pitch
S
y
s
t
e
m
C
e
n
t
r
a
l

A
u
d
i
t
o
r
y
S
t
a
g
e
s
Spatial maps
Computing pitch
MGB
IC

C
o
l
l
i
c
u
l
a
r
S
t
a
g
e
s
N
L
L
Harmonic templates
ILD, ITD Spectral cues
L
L
M
i
d
b
r
a
i
n
N
u
c
l
e
i
T
B
The auditory spectrum



D
C
N
P
V
C
N
E
a
r
l
y

A
u
d
i
t
o
r
y
A
V
C
N
S
t
a
g
e
s
s
o
u
n
d
4
(No Transcript)
5
(No Transcript)
6
Auditory-Nerve Response
Patterns to Two-Tone Stimulus
average response
4000
2000
1000
500
250
60
Time(
ms
)
7
(No Transcript)
8
Estimated stimulus spectrum
Lateral Inhibition
Cochlear Analysis
A
Sound
B
Characteristic Frequency Axis (CF)
Auditory-nerve fibers
Time (msec)
60
Basilar membrane vibrations
C
Hair cells along the tonotopic axis
500
Time (msec)
9
Down-Shift
Normal
Dilate
Compress
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
/come/ /home/ /right/ /away/
Three envelopes of modulation Slow (lt 30
Hz) Intemediate (lt 500 Hz) Fast (lt 4 kHz)
14
Decomposing a Spectrogram into Dynamic Ripples
S
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
Multiscale Cortical Representation of a
Spectrogram
Rate (Hz)
Frequency
20
Scale-Rate Decomposition
Reconstruction
21
MUSICAL TIMBRE
22
(No Transcript)
23
Patterns of Musical Timbre
24
(No Transcript)
25
Timbre Metric for Musical Instruments
Guitar Harp Violin Pizz. Violin Bowed Bass
Synth A Synth B Oboe Clarinet Flute
Horn Trumpet
Guitar Harp Violin Pizz. Violin Bowed Bass
Synth A Synth B Oboe Clarinet Flute
Horn Trumpet
Guitar Harp Violin Pizz. Violin Bowed Bass
Synth A Synth B Oboe Clarinet Flute
Horn Trumpet
Guitar Harp Violin Pizz. Violin Bowed Bass
Synth A Synth B Oboe Clarinet Flute
Horn Trumpet
Subjects (1-24)
Spectral cues
Temporal cues
Spectro-temporal cues
26
Mapping musical instruments
Guitar
Trumpet
A Melody with the Trumpar
ACE Chord
Trumpar
27
Speech AnalysisAssessment of Inteligibility
28
/come/ /home/ /right/ /away/
Three envelopes of modulation Slow (lt 30
Hz) Intemediate (lt 500 Hz) Fast (lt 4 kHz)
29
(No Transcript)
30
Human versus Ferret Sensitivity to
Spectrotemporal Modulations
31
(No Transcript)
32
(No Transcript)
33
Auditory Scene AnalysisPitch Extraction
34
Relevance to Auditory Scene Analysis Streaming
and grouping
Rate (Hz)
Frequency
Working Hypotheses Streaming Any
consistently isolated feature in the multiscale
representation can be streamed e.g.,
spectral patterns (tones or average vocal
tract spectra) repetitive
temporal dynamics (modulated noise or sinusoidal
FM tones) - transients as segmenters Grouping
Harmonicity and its linearly interpolated
extensions (pitch extraction and segregation,
regular patterns) Shared dynamics (Common
onsets and modulations)
35
Cortical Representation of Harmonic Shifted
Spectra
Multiscale Representation
Auditory Spectrum
Scale
16
14
12
Reduced Representation
10
8
6
4
2
0
0
20
40
60
80
100
120
140
Shifted Spectra are also grouped although
they are inharmonic
Scale
Frequency
36
(No Transcript)
37
(No Transcript)
38
(No Transcript)
39
Voice Morphing
40
(No Transcript)
41
Morphing Voices
42
Acknowledgment
Cortical Physiology and Auditory
Computations Didier Depireux, Jonathan Fritz,
David Klein Jonathan Simon
Auditory Speech and Music Processing Tai Chi,
Mounya El-Hilali, Powen Ru
Supported by MURI N00014-97-1-0501 from the
Office of Naval Research NIDCD T32 DC00046-01
from the NIDCD NSFD CD8803012 from the National
Science Foundation