Language Perception

1
Language Perception
ABRALIN 22-FEB-05

• Eva M. Fernández
• Queens College Graduate Center
• CUNY

2
Language Is

PERCEPTION
grammar lexicon
SIGNAL
MEANING
PRODUCTION
knowledge about the real world
logic
3
Production

• Ill give you my undevoted attention!
• Youll earn her eternal grapefruit.
• This restaurant hasnt been awake very long.
• Put the oven on at a very low speed.

• We have a lot of churches in our minister.
• They roasted a cook.
• If you give the nipple an infant
• You ordered up ending.

SIGNAL
MEANING
Lexical Retrieval
Structural Assignment
Phonological Encoding

• phonological fool
• a glear plue sky
• spattergrain

4
Perception
Structure Building
Lexical Access
Phonological Decoding
SIGNAL
MEANING
Lexical Retrieval
Structural Assignment
Phonological Encoding
5
Not Present in (Speech) Signal

• phonemes
• word boundaries
• clause boundaries
• location of empty categories
• intended attachments for locally or globally
  ambiguous strings
• hidden intents of the speaker!
• SO HOW COME WERE SO GOOD AT DECODING?

6
Visual Illusions

• when the experiences people report dont
  correspond to physical properties of the stimulus
• very cool but also very informative about the
  way the visual / perception system works
• (which is modularly)

7
The Hermann Grid Illusion
How many grey dots do you see at the
cross-roads? Source http//dragon.uml.edu/psych
/illusion.html A great page to visit for many
more visual illusions.
8
A Face Cant Be Hollow!
A face is always perceived as convex not
concave. http//www.kyb.tuebingen.mpg.de/de/bu/dem
o/ Max Planck Institut für Biologische Kybernetik
9
Perceptual Illusions

• also very cool and also very informative about
  the way the language perception system works
• ?
• its modularity ensures its speed and accuracy,
  which are both in turn compromised when the
  signal is
• AMBIGUOUS

10
McGurk Effect

• by Arnt Maasø, of the University of Oslo
  http//www.media.uio.no/personer/arntm/McGurk_engl
  ish.html

11
Perceptual Displacement andPhonemic Restoration

• The state governors met with their respective
  legislatures convening in the capital city.

12
legislatures, with cough! ( 145 msec)
spliced in
13
legislatures, intact --- s 145 msec
14
Bottom ? Top ?

• study with Brocas patients (Pollack Picket,
  1964)
• The apple the boy is eating is red.
• The girl the boy is chasing is tall.
• bait, date, gate study (Garnes Bond, 1976)
• Heres the fishing gear and the ___.
• Check the time and the ___.
• Paint the fence and the ___.

15
Structure Building The Parser

• its input is a string of lexical items
• its job is to build syntactic structure
• its output is sent to a mechanism that decodes
  meaning
• it probably has limited access to information
  thats not in the grammar or in the lexicon
• it probably operates following a very small set
  of strategies, grounded on limitations imposed by
  working memory

16
RSVP Paradigm

• Center-screen, word-by-word display
• Timing N ms per word (here N 500 ms)
• Sentence-recall task

17
?
The
beautiful
black
cat
chased
the
colorful
ball
.
The beautiful black cat chased the colorful ball.
18
?
Black
colorful
the
ball
chased
cat
beautiful
the
.
Black colorful the ball chased cat beautiful the.
19
The Garden Path Sentence

• The soldiers marched into the desert surprised
  the Persian forces.
• Since Joel always jogs a mile seems like a short
  distance to him.
• Carmela put the candy on the table in her mouth.
• Konstantin understood the problem had no
  solution.
• Everybody at the party knew Anns date was a
  total fool.
• Local ambiguity
• Disambiguation downstream,which goes against
  parsers preferences
• Reanalysis or meltdown!

20
The Garden Path Theory

• Lyn Frazier Janet Fodor, late 1970s
• Minimal Attachment build the simplest tree
• Late Closure attach locally
• Minimal Chain Principle / Active Filler Strategy
  posit shortest possible chain / posit gaps for
  fillers ASAP
• (the parser is lazy)

21
Minimal Attachment
building complex structure processing cost

• The soldiers marched into the desert surprised
  the Persian forces.
• Since Joel always jogs a mile seems like a short
  distance to him.
• Carmela put the candy on the table in her mouth.
• Konstantin understood the problem had no
  solution.
• Everybody at the party knew Anns date was a
  total fool.

22
Late Closure
attaching non-locally processing cost

• John said Mary will arrive last night.
• Physicists are thrilled to explain what they are
  doing to people.
• Under the glistening tree there was a gift for a
  boy in a box.
• Professor Humperdinck artfully avoided looking at
  the exams of the students that were sitting in
  his office ungraded.
• Two sisters reunited after 18 years in check-out
  counter!

23
Late Closure
attaching non-locally processing cost

• Mary saw a gift for a boy

Mary saw a gift for a boy in a box.
NP
PP
P
NP
for
a boy
24
Late Closure
attaching non-locally processing cost

• John said Mary will arrive last night.
• Physicists are thrilled to explain what they are
  doing to people.
• Under the glistening tree there was a gift for a
  boy in a box.
• Professor Humperdinck artfully avoided looking at
  the exams of the students that were sitting in
  his office ungraded.
• Two sisters reunited after 18 years in check-out
  counter!

25
The RC Attachment Ambiguity
N1
N2
The plot concerns the guardian of the
prince who was exiled from the country for
decades
RC
La trama es sobre el guardián del
príncipe que fue exiliado del país por décadas
26
Cross-Linguistic Differences

• N1 attachment rates (), in studies using
  questionnaire instruments where
• RC was long
• N1/N2 were equal in animacy
• Complex NP was in canonical object position for
  the language

27
Cross-Linguistic Differences
As in previous table, for languages other than
English Spanish,listed (for lack of a better
strategy!) in alphabetical order
28
Cross-Linguistic Differences

• could be driven by
• genetic relationship?
• syntactic properties?
• existence of unambiguous alternatives?
• distribution of unambiguous strings in input?
• prosody?

PROSODY (phonology)
PARSER
PRAGMATICS
29
Pragmatics?

• Grices Cooperative Principle
• Make your conversational contribution such as
  is required, at the stage at which it occurs, by
  the accepted purpose or direction of the talk
  exchange in which you are engaged
• The plot concerns
• the guardian of the prince who was exiled
• the princes guardian who was exiled
• the princes guardians who was exiled

30
Long RCs are Informationally Heavy

• The plot concerns the guardian of the prince
• who was exiled.
• who was exiled from the country for decades.
• Long RC has more lexical content, so its more
  informative.
• Does informativeness influence attachment?
• RC length effect, confirmed in
• English, Spanish Arabic, Bulgarian, Croatian,
  French, German, Hebrew, Hindi, Portuguese

N1 INTERP MORE LIKELY
31
Long RCs are Prosodically Heavy

• The plot concerns the guardian of the prince who
  was exiled.
• The novels plot concerns the guardian of the
  prince who was exiled.
• The plot concerns the guardian of the prince who
  was exiled from the country for decades.
• The novels plot concerns the guardian of the
  prince who was exiled from the country for
  decades.

32
Elicited Production
Fernández, Bradley Taylor, in prep

• N 8 native US English speakers 5F, 3M
• N 6 ? 4 sentences,RC Length ? Matrix-Subject
  Weight
• RC 1 versus 3 prosodic words who was exiled (
  from the country for decades )
• MX 1 versus 2 prosodic words The ( unusual )
  plot

33
1
The unusual plot concerns the guardian of the
prince. The prince was exiled from the country
for decades.
34
Acoustic Analysis Regions
Fernández, Bradley Taylor, in prep
Duration Uniform acoustic signature of phrasal
break
Wt
S
V
N1
N2
RC1
RC3
The( unusual )
plot
concerns
theguardianof
theprince
who wasexiled
( from the countryfor decades )
35
Acoustic Analysis Regions
Fernández, Bradley Taylor, in prep
S V
V N1
N1 N2
N2 RC
36
Fernández, Bradley Taylor, in prep
N2 RC
37
Region N2
Fernández, Bradley Taylor, in prep
38
Elicited Production Summary
Fernández, Bradley Taylor, in prep

• MX F1(1,7) 2.80, p.138 RC F1(1,7)
  11.46, p 9.96, p
• Interaction MX x RC F1 .25
• N2 RC and nowhere else
• Likelihood of break grades with RC length and
  matrix weight, additively, i.e., with sentence
  length

39
Questionnaire Procedure
Fernández, Bradley Taylor, in prep

• Reading comprehension test
• 36 targets, 108 fillers (13 ratio)
• Comprehension question after each sentence

Example of target The plot concerns the guardian
of the prince who was exiled from the country for
decades. Who was exiled? the guardian the
prince Example of filler The sneaky burglars
took all the stereo equipment but overlooked the
computer system. What was stolen? the stereo
the computer
40
Questionnaire Participants
Fernández, Bradley Taylor, in prep

• N 44, Queens College students
• US English speakers
• Language-history questionnaire,
  non-nativespeakers excluded/replaced
• Rejected/replaced for errors 15 in fillers

41
Questionnaire Results
Fernández, Bradley Taylor, in prep
Relative Clause Length F1(1,40) 24.95,
pSubject Weight F1(1,40) 5.51, pF2(1,32) 9.43, p F2 42
The Implicit Prosody Hypothesis (IPH)

• In silent reading, a default prosodic contour is
  projected onto the stimulus, and it may influence
  syntactic ambiguity resolution
  (Fodor 1998, 2002)
• the brother of the bridegroom who snores
• the brother of the bridegroom who snores

43
Prosody and Syntax Align
Selkirk, 1986
the brother of the bridegroom who often
unknowingly snores the brother of the bridegroom
who snores
prosodic discontinuity
el hermano del novio que a menudo
inconscientemente roncaba el hermano del novio
que roncaba
syntactic discontinuity
44
Empirical Support for the IPH

• Behavioral evidence on how RCs are interpreted
  during silent reading
• existing dataset Hemforth et al. (submitted)
• Evidence on how the N-of-N-RC construction is
  produced in discourse-neutral speech
• elicited production experiment
• ?Do the patterns in the two datasets match up?

45
Behavioral Evidence
Hemforth et al. (submitted)

• Materials in English and Spanish
• with short and long RCs
• N1-N2-RC placed post- and pre-verbally

The guest impressed X. X impressed the
guest. El invitado impresionó a X. X impresionó
al invitado. X the brother of the
bridegroom who (often unknowingly)
snores el hermano del novio que (a menudo
inconscientemente) roncaba
46
Behavioral Evidence
Hemforth et al. (submitted)
Who snores? The brother (N1)

• Post-Verbal Objects
• Cross-linguistic difference
• RC length effect
• Pre-Verbal Subjects
• RC length effect reduced
• Cross-linguistic difference reduced

47
ENGLISH
SPANISH
The guest impressed the brother of the bridegroom
who often unknowingly snores.
El invitado impresionó al hermano del novioque a
menudo inconscientemente roncaba.
The brother of the bridegroom who often
unknowingly snores impressed the guest.
El hermano del novio que a menudo
inconscientemente roncaba impresionó al invitado.
48
Experiment Elicited Production
Fernández, Bradley, Igoa Teira, 2003 Fernández
Bradley, 2004

• Participants, N 8 per language
• English ? New York
• Spanish ? Madrid
• Materials, N 8 ? 4 per language(selected from
  Hemforth et al.s 32 ? 4)
• Post- and pre-verbal of identical length
• RCs right boundary with same lexical content,
  whether short or long

The guest impressed X. X impressed the
guest. X the brother of the
bridegroom who (often unknowingly) snores
49
Analyses N2 RCs Verb
Fernández, Bradley, Igoa Teira, 2003 Fernández
Bradley, 2004

• Duration Presence of Boundary
• Pitch movement Type of Boundary

The guest impressed the brother of the
bridegroom who snores. N2RC RC. The
brother of the bridegroom who snores
impressed N2RC RC.V the guest.
50
Monolinguals N2 Durations

• Placement Length InteractionF1(1,14) 5.77, p
  
• 123 ms Post-Verbal
• 68 ms Pre-Verbal

RC-Length ?
ENGLISH
SPANISH
Post-VerbalObjects
Pre-VerbalSubjects
51
Monolinguals RC Vb Durations

• Placement Length InteractionF1(1,14) 6.38, p
  
• 10 ms Post-Verbal
• 35 ms Pre-Verbal

RC-Length ?
ENGLISH
SPANISH
Post-VerbalObjects
Pre-VerbalSubjects
52
Monolinguals N2 Pitch

• Placement Language InteractionF1(1,14)
  16.56, p
• ?0.4 Hz/200 ms English
• 23.6 Hz/200 ms Spanish

Placement ?
ENGLISH
SPANISH
53
Monolinguals RCVb Pitch

• Interaction Placement LanguageF1(1,14)
  6.05,
• ?8.7 Hz/200 ms English
• ?38.6 Hz/200 ms Spanish

Placement ?
ENGLISH
SPANISH
54
Duration Pitch Monolinguals
ENGLISH
SPANISH
N2RC
RC.
Post-Verbal Objects
N2RC
RC.
Post-Verbal, Short
Post-Verbal, Short
Post-Verbal, Long
Post-Verbal, Long
RCV
N2RC
RCV
Pre-Verbal Subjects
N2RC
Pre-Verbal, Short
Pre-Verbal, Short
Pre-Verbal, Long
Pre-Verbal, Long
55
Summary of Data Outcomes
Fernández, Bradley, Igoa Teira, 2003 Fernández
Bradley, 2004

• Pitch Movements Type of Boundaryand
  Cross-Linguistic Differences
• Spanish N2 falls pre-verbally, rises
  post-verbally
• English N2 uniformly falls, pre- and
  post-verbally
• Duration Presence of Boundary and
  Cross-Linguistic Similarities
• In both languages Likelihood of breaks before
  RC is modulated by position

56
Conclusions and Speculations

• Behavioral similarities and differences are
  indexed in the prosodic patterns of Spanish and
  English
• But what is the source for the contrasting
  sentence-medial tunes in Spanish?
• Are such patterns projected entirely within the
  syntax-prosody interface?
• Or are such patterns the result of an interplay
  of syntax, prosody, and information structure?