Correlating Consonant Confusability and Neural Responses: An MEG Study

1
Correlating Consonant Confusability and Neural
Responses An MEG Study

• Valentine Hacquard1
• Mary Ann Walter1,2
• 1Department of Linguistics and Philosophy,
  KIT-MIT MEG Laboratory, Massachusetts Institute
  of Technology, Cambridge, MA 02139, USA,
  2National Science Foundation Fellowship

2
Nasal Confusability

• Numerous behavioral studies have investigated
  error rates in identification of phonemes masked
  with noise.
• These show that nasal consonants are more
  confusable with each other than oral consonants
  e.g. m/n vs b/d (Miller Nicely 1955, Wang
  Bilger 1973).

3
Consequences

• Such perceptibility asymmetries motivate much
  current work in phonology and (its claimed)
  phonological processes (Hume 1998, Steriade
  1999).
• Unsurprisingly, given this stance, Mohanan (1993)
  finds that nasals are particularly susceptible to
  place assimilation.

4
An Acoustic Model

• These confusability and assimilation facts, as
  well as
• studies on offline similarity judgments
  (Hura et al.
• 1992), support a model of similarity based
  on
• acoustic, perceptual factors.
• context-dependent, but inventory-independent
• Although nasality itself is highly salient,
  persevering nasality masks the F2 transition into
  the following vowel, which is the primary cue for
  place of articulation.

5
Another Proposal
Frisch et al. (1997) and Frisch (1996) propose a
metric in which similarity is computed according
to natural classes Similarity
shared natural classes shared unshared
natural classes context-independent, but
inventory-dependent English b/d ( .29) m/n
( .28), where 1 is identity.So, nasals equally
or less similar to each other than orals.
6
The Question

• Does a difference in similarity correlate with a
  difference in an auditory brain response?
• If so, brain data can be used to substantiate
  proposals for similarity metrics, as well as
  their internal organization.

7
MAGNETOENCEPHALOGRAPHY (MEG)

• MEG measures the magnetic fields (B) generated
  by electrical activity in the brain
  specifically, by potentials in the apical
  dendrites of pyramidal cells in cortex

8
SCALE
Earth field
Intensity of magnetic signal(T)
The magnetic field of the brain, recorded with
MEG, is 100 billion times weaker than the earths
magnetic field!
Urban noise
Contamination at lung
Heart(QRS)
Fetal heart
Muscle
Spontaneous signal (a-wave)
Signal from retina
Evoked signal
Intrinsic noise of SQUID
9

• MMF mismatch field automatic, auditory brain
  response evoked by a deviant stimulus following a
  sequence of standards, peaking 180-250 ms
  post-stimulus onset.

MMF
M100
M100 automatic auditory evoked response that
peaks 100 ms post-stimulus onset
10
MMF LOCALIZATION
Origin of signal in auditory cortex
Note left-hemisphere concentration of the
mismatch field with linguistic stimuli.
11
Properties of the MMF
Sharma Dorman (1999) and Phillips et al. (2000)
show that the same VOT span crossing a phonemic
category boundary evokes a far greater MMF than
one that doesnt.
Näätänen et al. (1997) show that a small acoustic
difference crossing a phonemic category boundary
evokes a far greater MMF than a large one that
doesnt.
MMF A M P L I T U D E
10 ms VOT span within category
10 ms VOT spanacross category
50 HZ F2 spanwithin category
10 HZ F2 spanacross category
Phonological difference outweighs acoustic
difference. Do similarity distinctions matter
when category is kept constant?
12
Procedure

• Oddball paradigm Conditions (8x30240)
• 1) ba (400 ms) ba (400 ms) ba (400 ms) ba
  (400 ms) da deviant
• 2) da (400 ms) da (400 ms) da (400 ms) da (400
  ms) da standard
• 3) da (400 ms) da (400 ms) da (400 ms) da (400
  ms) ba deviant
• 4) ba (400 ms) ba (400 ms) ba (400 ms) ba (400
  ms) ba standard
• 5) ma (400 ms) ma (400 ms) ma (400 ms) ma (400
  ms) na deviant
• 6) na (400 ms) na (400 ms) na (400 ms) na (400
  ms) na standard
• 7) na (400 ms) na (400 ms) na (400 ms) na (400
  ms) ma deviant
• 8) ma (400 ms) ma (400 ms) ma (400 ms) ma (400
  ms) ma standard
• Subjects (n16) made same-different button-press
  judgments.
• Synthesized stimuli (4 tokens) were presented in
  six blocks of 40 trials, randomly ordered, with
  self-regulated breaks in between.

13
Predictions

• According to an acoustic-based similarity
  framework, the MMF-baseline gap should be larger
  for oral consonant pairs than for nasals.
• According to a natural-class-based one such as
  Frischs, it should be the opposite, or
  equivalent.
• If abstract phonological features are the only
  relevant factor in perceptibility at this stage,
  the gaps should be equivalent.

14
Behavioral Results Error Rate

• Deviants overall received significantly more
  errors than standards (p.0009).
• No effect for manner was observed (p.3538).
• ? Cf. lack of masking noise or filtering,
  comparatively small number of trials.

15
Behavioral Results Reaction Time

• RT to deviants overall was significantly faster
  than to standards (p
• (Some subjects reported a waiting strategy in
  standard trials.)

• RT to nasals was significantly faster than to
  orals (p.0197).

16
MMF Results
single subject
OralStandard
OralDeviant
17
MMF Results
Deviants have significantly greater amplitude in
MMF window than standards (psignificantly greater amplitude in MMF window
than nasals (psignificantly greater M100 amplitude than orals
(p
selected sensors RMS waves

18
MMF Comparison Nasal/Oral
single subject
NasalDeviant
OralDeviant
19
MMF Comparison
The MMF/baseline gap is significantly greater for
oral consonant pairs than for nasals (p.0399).
single subject
20
Conclusions

• Oral mismatches elicit a stronger MMF than nasal
  mismatches.
• So oral consonants are perceived as more
  different from each other than nasal ones at this
  stage.
• Phonological categories are not the only relevant
  factors in perception at this stage acoustic
  similarity also plays a role.
• Finally, it is an acoustic-based similarity
  metric that appears to be operating at this time
  period, rather than a natural-class one, or
  feature-counting.

21
For Future Research

• Our results suggest that the MMF can be used to
  test proposals about degree of perceptual
  distance in acoustic-based similarity frameworks.
• Our next study will isolate Frisch-style
  similarity as a variable, testing the same
  phonological contrast with speakers of languages
  that have the contrast, but whose inventories
  differ in other ways.

22
THANK YOU !
Alec Marantz Diana Sonnenreich Donca
Steriade Karen Froud Linnaea Stockall Pranav
Anand Ben Bruening, Elissa Flagg, Vivian
Lin you, the audience