Psycholinguistics Lecture 7

1
PsycholinguisticsLecture 7

• Speech Perception II

2
Announcements

• Midterm, Nov. 1st!
• Covers material up to October 25th
• October 25th (by 1159pm) submit potential exam
  questions.
• Form groups of 1-2 students
• list 5 major points covered in the course
• submit 5 short essay questions.
• the questions you submit will count for 1/3 of
  your final midterm grade.
• October 30th part new stuff, part review.

3
What is Categorical Perception?
Review
ba
da
ga
4
Methods for Testing Categorical Perception
Continuation on Categorical Perception

• Identification
• Randomly play the audio clips and asked to
  identify the phoneme
• Discrimination
• Randomly play pairs and asked to make
  Same-different Judgment
• Same pairs
• Different pairs

5
Identification
Continuation on Categorical Perception

• Identification
• Randomly play the audio clips and asked to
  identify the phoneme
• If there is CP, what should the graph look like?
  
• X-axis stimuli arranged in a continuum with very
  small incremental difference between the stimuli
• Y-axis Identification as the tested category

6
Identification(idealized results)
Continuation on Categorical Perception
100
80
60
Identification as Category X
40
20
0
2
3
4
5
6
1
7
Stimulus
7
Discrimination Study
Continuation on Categorical Perception

• Last lecture
• The ba/da/ga study varied transitional state (up,
  down of F2).
• In this example, Varying Voice Onset Time.

8
Voice Onset Time (VOT)
Continuation on Categorical Perception

• VOT time between consonant release and vocal
  cord vibration

b
p

• So what is the difference in VOT between
  VOICELESS b and VOICED p?
• SHORT VOT ? voiced
• LONG VOT ? voiced

9
Voice Onset Time (VOT)
Continuation on Categorical Perception

• Short VOT ?
• Long VOT ?
• Which one is /di/ and which one is /ti/?

di
ti
10
Discrimination Study
Continuation on Categorical Perception
Same/Different?
0ms 60ms
Same/Different?
Why is this pair difficult?
0ms 10ms
(i) Acoustically similar? (ii) Same Category?
Same/Different?
40ms 40ms
11
Discrimination
Continuation on Categorical Perception
A More Systematic Test
Same/Different
D
D
0ms 60ms
0ms
20ms
D
T
20ms
40ms
Same/Different
0ms 10ms
T
T
40ms
60ms
Same/Different
Within-Category Discrimination is Hard
40ms 40ms
12
Categorical Perception(Idealized Discrimination
Data)
Continuation on Categorical Perception
100
80
60
Correct Discrimination
40
20
0
1-2
2-3
3-4
4-5
5-6
Pairs by VOT
13
What researchers found
Contrast Major Acoustic Cue
Place in oral stops ba-da-ga Start and direction of the second formant
Word initial voicing in oral stops ba-pa Voice Onset Time
Place in fricative sa-ša Frequency of the turbulent noise
Word final voicing in oral stops ab-ap Duration of preceding vowel
Voice in final fricative as-az Duration of preceding vowel
Place in nasal stops ma-na Start and direction of the second formant
14
Where did our speech perception abilities come
from?
Motor Theory vs. Auditory Theory

• The Motor Theory
• perception is informed by our innate knowledge
  of articulation
• The Auditory Theory
• speech perception is based solely on auditory
  properties of speech

15
Contrasting the two theories
Motor Theory vs. Auditory Theory
The Motor Theory The Auditory Theory
Perception is based on production (i.e., understanding of articulatory gestures speakers make) Perception is based on genetic auditory mechanisms
Perception is species-specific Speech production and perception evolved together Perception is not species-specific Production system evolved by making use of existing auditory capacities
Perception is innate Tacit knowledge of articulation is given by evolution Perception could be innate or learned
(slides adapted from J. Snedeker, C. Phillips)
16
Questions to Ask
Motor Theory vs. Auditory Theory
The Motor Theory QUESTIONS
Perception is based on production (i.e., understanding of articulatory gestures speakers make) Q3. Is speech perceptionaffected by knowledge ofarticulation?
Perception is species-specific Speech production and perception evolved together Q2. Is speech perception species-specific?
Perception is innate Tacit knowledge of articulation is given by evolution Q1. Is speech perception innate?
(slides adapted from J. Snedeker, C. Phillips)
17
Question 1
Motor Theory vs. Auditory Theory Question 1

• Is speech perception innate?
• Do newborns have categorical perception?
• If CP requires exposure to language (e.g.,
  knowledge of minimal pairs in ones language),
  then NO.
• If CP is innate, then YES.
• How do we test newborns?

18
High Amplitude Sucking Procedure
Motor Theory vs. Auditory Theory Question 1

• Infant given a pacifier that measures sucking
  rate
• Habituation Infant sucks to hear sound (e.g.
  ba) until bored.
• Test Play sound (e.g., ba or pa). Is there
  dishabituation?
• Infants will suck to hear sound if the sound is
  no longer boring.

http//psych.rice.edu/mmtbn/language/sPerception/v
ideo/sucking_h.mov http//www.learner.org/vod/vod_
window.html?pid1620 (250 min. into videoclip)
19
Stimuli for Eimas et. als Study
Motor Theory vs. Auditory Theory Question 1

• BA vs. PA
• Vary Voice Onset Time (VOT) time btw consonant
  release and vocal cord vibration

BA
PA
VOT in milliseconds
20
Predictions
Motor Theory vs. Auditory Theory Question 1
Between Category BA1-PA Within Category BA1-BA2 Within Category Control BA1-BA1
Innate Categorical Perception dishabituate remain habituated remain habituated
Untuned Sensitivity dishabituate dishabituate remain habituated
Insensitive remain habituated remain habituated remain habituated
BA1 VOT 20ms BA2 VOT 0ms PA VOT 40ms
21
Results for Eimas et. als Study
Motor Theory vs. Auditory Theory Question 1
MEAN NUMBER OF SUCKING RESPONSE
22
Question 1 Answer
Motor Theory vs. Auditory Theory Question 1

• Q1 Is Speech Perception Innate?
• Many other studies since tested
• Infants (Neonates) on other contrasts.
• Consensus Yes to Innate Q.
• Infants do not discriminate all physically equal
  acoustic difference they show heightened
  sensitivity to those that are important for
  language.
• BUT there is language-specific fine-tuning

23
Motor vs. Auditory TheoryScore card
Motor Theory vs. Auditory Theory Question 1

• Q1 Is Speech Perception Innate?
• Answer YES.

• Consistent with Auditory Theory
• Speech perception due to innate structure of
  auditory system

• Predicted by Motor Theory
• Speech perception driven by innate knowledge of
  articulation

24
Question 2
Motor Theory vs. Auditory Theory Question 2

• Is speech perception species-specific?
• Do other animals show categorical perception on
  the same speech sounds?
• How do we test animals?

?
25
Avoidance Conditioning Procedure
Motor Theory vs. Auditory Theory Question 2

• Kuhl Miller (1978) test chinchillas and humans
  with identical stimuli
• Human Task identification (d or t)
• Chinchillas avoidance conditioning

26
Avoidance Conditioning Procedure
Motor Theory vs. Auditory Theory Question 2

• Speech sound at one end of the continuum paired
  with shock
• Other end paired with safety

Safety
Shock
VOT
27
Avoidance Conditioning Procedure
Motor Theory vs. Auditory Theory Question 2

• Animals learn to avoid shock.
• What will they do for between cases?

Safety
FLEE
?
?
?
?
?
?
Shock
STAY
VOT
28
Predictions
Motor Theory vs. Auditory Theory Question 2
Categorical Perception
100
Graded Perception
80
60
Identification as d
40
20
0
2
3
4
5
6
1
7
Stimulus
29
Kuhl Miller (1978)
Motor Theory vs. Auditory Theory Question 2
30
Motor vs. Auditory TheoryScore card
Motor Theory vs. Auditory Theory Question 2

• Q2 Is Speech Perception Species-Specific?
• Answer NO.

• Consistent with Auditory Theory
• General auditory abilities adequate for (some
  aspects of) speech perception

• Contrary to Motor Theory Claim
• That is, only humans have (innate or learned)
  knowledge of articulation

31
Some species-specific aspects of speech perception
Motor Theory vs. Auditory Theory Question 2

• Other primates (e.g., Macaques, vervet monkeys,
  chimpanzees)
• Boundary of ra/la not human like (Sinnott
  Brown, 1997)
• Fail to make use of vowel length in consonant
  discrimination (Sinnott, Brown, Borneman, 1998)
• Fail to use formant transitions alone in
  consonant discrimination (Sinnott Williamson,
  1999)
• Fail to categorize two different vowels in the
  same way infants learn to categorize those two
  vowels (Kuhl, 1991)
• Different similarity space for vowels (Sinnot,
  Brown, Malik, Kressley, 1997 Kojima
  Kiritani, 1989)

32
Question 3
Motor Theory vs. Auditory Theory Question 3

• Is speech perception based on our knowledge of
  articulation?
• Would knowledge of
• Visual information of mouth movements
• Coarticulation
• influence our speech perception?
• What experiments?
• McGurk Effects
• Co-articulation Experiments

33
McGurk Effect
Motor Theory vs. Auditory Theory Question 3
Audio Visual Your Perception
ba ba ba
ba va va
ba tha tha
ba ga da
http//www.faculty.ucr.edu/rosenblu/McGurkcompres
sB1.mov
34
McGurk Effect
Motor Theory vs. Auditory Theory Question 3

• McGurk Finding phoneme categorization is
  affected by visual information
• audio BA visual GA percept DA
• Q3 Is speech perception based on our knowledge
  of articulation?
• YES. Adult speech perception is affected by
  visual cues.
• Are infants?

35
Three Infant Studies
Motor Theory vs. Auditory Theory Question 3

• Do infants make use of visual cues to
  articulation?
• Kuhl Meltzoff (1982)
• Rosenblum, Lawrence, Schmuckler, Johnson (1997)
• Burnham Dodd (2004)

36
Kuhl Meltzoff (1982)
Motor Theory vs. Auditory Theory Question 3
Experiment 1
Preferential Looking Paradigm

• Familiarlization
• a and i face
• NO AUDIO
• Test Phase
• a and i face
• AUDIO of either a or i
• MEASURE looking time to appropriate face.

37
Kuhl Meltzoff (1982)
Motor Theory vs. Auditory Theory Question 3
Experiment 1

• Do infants know visual cues to articulation?
• Kuhl Meltzoff (1982)
• Show that 4-5 months-old infants can match sound
  with correct mouth shape
• But not evidence of visual cues influencing
  categorization

Kuhl Meltzoff, 1982
38
Rosenblum et al. (1997)
Motor Theory vs. Auditory Theory Question 3
Experiment 2

• Tested 5-months-old infants
• Habituated on audio VA visual VA
• Tested on
• (1) Audio BA Visual VA (adults perceive as VA)
• (2) Audio DA Visual VA (adults perceive as DA)
• Results
• Dishabituate to (1)? (2)?
• Infants dishabituate only to (2)
• Show?
• Infants are like adults!
• They are sensitive to visual cues to articulation.

39
Burnham Dodd (2004)
Motor Theory vs. Auditory Theory Question 3
Experiment 3

• Tested 2 groups of ½ months-old infants

Control Group habituation Audio BA Visual BA
Test Group habituation Audio BA Visual GA

• Both groups then tested on
• (1) audio BA (2) audio DA (3) audio THA
• Results Dishabituation? (1)? (2)? (3)?
• Control Group
• Test Group
• Show

Dishabituated to DA, THA, not BA
Dishabituated only to BA
Again show infants are show the Classic McGurk
Effect Audio BA, Visual GA ? DA Caveat DA THA
40
Coarticulation Revisited
Motor Theory vs. Auditory Theory Question 3
Coarticulation
41
Coarticulation Revisited
Motor Theory vs. Auditory Theory Question 3
Coarticulation
42
Motor Theory vs. Auditory Theory Question 3
Coarticulation
Coarticulation Revisited
43
Coarticulation Revisited
Motor Theory vs. Auditory Theory Question 3
Coarticulation

• šu shoe, ša shot
• u oo F1 310 F2 870
• a ah F1 710 F2 1100
• š has lower frication frequency with u
• š has higher frication frequency with a

44
Coarticulation Identification Curve
Motor Theory vs. Auditory Theory Question 3
Coarticulation

• What should the identification curve if there is
  categorical perception?

45
Coarticulation Identification Curve
Motor Theory vs. Auditory Theory Question 3
Coarticulation

• Contextual Effect? (consonant plus vowel)
• u
• a
• We know
• š lo freq w/ u
• š hi freq w/ a

46
Coarticulation Identification Curve
Motor Theory vs. Auditory Theory Question 3
Coarticulation

• Contextual Effect? (consonant plus vowel)
• u
• a
• We know
• š lo freq w/ u
• š hi freq w/ a

47
Motor vs. Auditory TheoryScore card
Motor Theory vs. Auditory Theory Question 3

• Q3 Is Speech Perception based on our knowledge
  of articulation
• Answer YES. Our compensation for Coarticulation
  Effects and response to McGurk Stimuli suggest
  that knowledge of articulation influences speech
  perception.

• Inconsistent with Auditory Theory
• Though perhaps perceptual learning might explain
  these effects

• Predicted by Motor Theory
• Innate connection btw production and perception

48
Provisional Conclusions
Motor Theory vs. Auditory Theory General
Conclusions

• Speech Perception makes use of some auditory
  mechanisms which evolved prior to language
• These abilities are innate
• Speech Perception also makes use of our knowledge
  of articulation
• These abilities are likely innate

49
Becoming a Native Listener
Language Specific Fine Tuning

• Languages differ in their inventories of
  phonemes.
• What develops or changes in our speech perception
  abilities?

50
Japanese vs. English(Miyawaki et al. 1975)
Language Specific Fine Tuning
RA
AMERICANS
LA
51
Language Specific Fine Tuning
Hindi (spoken in India) unvoiced unaspirated
retroflex vs. dental stop
Dental Stop tip of tongue touching
back of front teeth
Retroflex Stop tongue curled so tip is
behind alveolar ridge
(English /t/ is typically somewhere between the
two)
52
Can you hear the difference?
Language Specific Fine Tuning
dental
Hindi
retroflex
53
Language Specific Fine Tuning
Salish (Native North American language) glotaliz
ed voiceless stops
Uvular tongue is raised against the velum
Velar tongue is raised behind the velum
(they are actually ejectives - ejective is
produced by obstructing the airflow by raising
the back of the tongue against or behind the
velum)
54
When does changes in sensitivity occur?
Language Specific Fine Tuning
Infancy
Adulthood
And testing method?
55
Conditioned Head-Turn
Language Specific Fine Tuning

• Conditioning
• Child hears a string of sounds.
• Conditioned to turn head when detects a change
  (e.g., bell ? whistle) with reward
• Test
• Speech sounds (e.g., da, da, da, da, ta,)
• Does the child turn his or her head with changed
  from da to ta?

Werker http//www.learner.org/vod/vod_window.html
?pid1630 Kuhl http//www.learner.org/vod/vod_win
dow.html?pid1631
56
When does Change Occur?
Language Specific Fine Tuning
57
What is changing? Two contrasting views 1 or 2?
Language Specific Fine Tuning

• Maintenance or Loss
• If you dont use it, you lose it.
• Parallel aspects of early visual development.
• Functional Reorganization
• Existing architecture reorganized for higher
  level of processing.

58
What is changing? Two contrasting views
Language Specific Fine Tuning
59
What is changing? 1. Maintenance or Loss View
Language Specific Fine Tuning
Phonology
Phonetics
Structure-changing
Non-native boundaries disappear. Resulting in
native language phonetics
Acoustics
60
What is changing? 2. Functional Reorganization
Language Specific Fine Tuning
Phonology
Phonetics
Structure-building
Native language phonemes built from universal
phones
Acoustics
61
Which view?
Language Specific Fine Tuning

• Werker (1997) noted some problems for the
  maintenance or loss view.
• 1. Many of the uncategorized sounds do appear in
  the native language but just are not meaningful
  (e.g., as allophones), and speakers can be made
  aware of the difference.
• Example
• /p/ is only aspirated in pin and not spin
• /p/ in pin and spin are allophones in English
• But could be minimal pairs in some other
  languages.

62
Which view?
Language Specific Fine Tuning

• Werker (1997) noted some problems for the
  maintenance or loss view.
• 1. Many of the uncategorized sounds do appear in
  the native language but just are not meaningful
  (e.g., as allophones), and speakers can be made
  aware of the difference.
• 2. Children who fail to show categorical
  perception for non-native phonemes can acquire a
  new language without an accent.
• 3. Adults can be trained to make non-native
  distinctions.
• 4. Perceptual distinction is readily available
  for non-linguistic tasks.

63
Which model?
Language Specific Fine Tuning

• Werker (1997) The evidence that poses problems
  for maintenance or loss view supports the
  functional reorganization view.
• I.e., the view that
• Those perceptual categories which are meaningful
  in the native language become speech categories.
• The remainder are perceived but not recruited in
  speech perception.

64
Why functional reorganization?
Language Specific Fine Tuning

• Developmentally 10-12 months-olds lose
  non-native distinctions
• What is going on?
• Analyzing statistical regularities in the input
  language and working towards word learning?
• (more to come when we study language acquisition)

65
Innate components
Language Specific Fine Tuning

• Articulatory rudiments
• Infants babble

• Phonetics serves as a connection
• McGurk effect

Articulatory
Phonetics
innate
66
Constructing phonology
Language Specific Fine Tuning
Syntactic
Lexical
Phonological
constructed
Articulatory
Phonetics
Auditory
innate