Liina Pylkknen

1
V55.0660 Societies and the Social Sciences
Linguistic perspectives, April 24 2003
Neurolinguistics

• Liina Pylkkänen
• Department of Linguistics/ Center for
  Neuromagnetism
• New York University

2
Whats the biological instantiation of language
in the brain?

• Theoretical linguistics What is language?
• Primary data intuitions about well-formedness.
• Psycholinguistics How is language processed?
  What makes language processing hard or easy?
• Primary data behavioral reaction times, error
  rates.
• Neurolinguistics How is language processed? What
  are the neural correlates of linguistic
  operations?
• Primary data some more or less direct measure of
  neural activity.
• Aim to manipulate linguistic operations in a
  controlled experimental setting in order to find
  out what neural activity is affected by the
  manipulation.

3
Why do we need to find neural correlates of
linguistic operations?

• Because we want to understand the mind/brain
  (duh)
• Clinical applications
• Establishing new dependent measures for
  investigating the question what language is.
• This lecture using a neural correlate of lexical
  access as a tool to ask questions about
  morphology.

4
Same vs. similar
5
Same vs. similar
TEACHER vs. TEACH BROTHEL vs. BROTH SORCERY
vs. MAGIC
6
Morphological decomposition
vs. TEACH BROTHEL vs.
BROTH SORCERY vs. MAGIC
TEACH ER
7
Alternative(?) emergent morphology

• Morphology is similarity at the extreme.
• Effects of morphology should reduce to combined
  effects of semantic and phonological/formal
  similarity. (Seidenberg and
  Gonnerman, 2000)

Gonnerman and Plaut (2000)
8
To test the theories

• What are the effects of phonological and semantic
  similarity?
• Do effects of morphology reduce to combined
  effects of semantic and phonological similarity?

9
Dependent measures

• Behavioral lexical decision times to visually
  presented words
• Measurements of brain activity using
  MagnetoEncephaloGraphy (MEG)
• In particular the M350 -- an MEG index of
  automatic lexical activation

10
Magnetoencephalography (MEG)
EEG
http//www.ctf.com/Pages/page33.html
11
Magnetoencephalography (MEG)
EEG
MEG
http//www.ctf.com/Pages/page33.html
12
Magnetoencephalography (MEG)
Distribution of magnetic field at 93 ms (auditory
M100)
Averaged epoch of activity in all sensors
overlapping on each other.
13
Magnetoencephalography (MEG)
14
What happens in the brain when we read words?
Letter string processing (Tarkiainen et al. 1999)
Lexical activation (Pylkkänen et al. 2002)
15
M350
(i) 1st component sensitive to
lexical factors (such as lexical frequency)
(ii) not affected by competition
16
M350
(i) 1st component sensitive to
lexical factors (such as lexical frequency)
(ii) not affected by competition
17
Phonotactic probability/density early
facilitation

• Same/different task (low-level)
• RTs to nonwords with a high phonotactic
  probability are speeded up.

RT
Sublexical frequency effect
RT
(Vitevich and Luce 1998, 1999)
18
Phonotactic probability/density later inhibition

• Lexical decision (high-level)
• RTs to nonwords with a high phonotactic
  probability are slowed down.

Competition effect
RT
High probability
MIDE
RT
YUSH
Low probability
(Vitevich and Luce 1998, 1999)
19
High phonotactic probability/density
induces intense competition
20
If M350 Selection
21
If M350 Activation
22
Materials (visual)

• Four categories of 70 stimuli

• Lexical decision.

(Pylkkänen, Stringfellow, Marantz, Brain and
Language, 2002)
23
Effect of probability/density (single subject)
RT 640.36
24
Effect of probability/density (single subject)
RT 640.36
RT 620.03
25
Effect of probability/density (n10)
(Pylkkänen, Stringfellow, Marantz, Brain and
Language, 2002)
26
M350
(i) 1st component sensitive to
lexical factors (such as lexical frequency)
(ii) not affected by competition
27
Earlier effect of probability/density on M250
amplitude (n10)

(Pylkkänen, Stringfellow, Marantz, Brain and
Language, 2002)
28
To test the theories

• What are the effects of phonological and semantic
  similarity?
• Behaviorally?
• On the M350?
• Do effects of morphology reduce to combined
  effects of semantic and phonological similarity?

29
Crossmodal priming (materials adapted from
Gonnerman (1999))

• SOA
• Duration of prime
• Task
• Lexical decision
• 21 subjects

30
Phonological similarity behavioral inhibition

• In longer SOA priming words tend to be harder to
  recognize when they are preceded by similar
  sounding words (e.g. Soto-Faraco,
  Sebastián-Gallés Cutler, 2001)

slower when preceded by
SPINACH
SPIN
than when preceded by
MUFFLER
31
Phonological similarity behavioral inhibition
slower when preceded by
SPINACH
SPIN
32
Inhibited activation
activation level
time
time
RT
SPIN
s p i n a c h
PRIME
TARGET
33
Alternative Inhibited recognition
activation level
time
RT
SPIN
s p i n a c h
PRIME
TARGET
34
Mechanisms of recognition

• Inhibited activation
• Mismatching candidates are suppressed below their
  resting level
• Inhibited recognition
• Mismatching candidates are rejected simply
  because they receive less excitation from the
  input
• BUT make similar behavioral predictions

35
Timing of activation
INHIBITED ACTIVATION
INHIBITED RECOGNITION
36
M350
a tool for investigating inhibitory mechanisms
INHIBITED ACTIVATION
INHIBITED RECOGNITION
37
Materials

• Two types of phonological similarity
• (embedded in a larger experiment)

1. ONSET-MATCHING
2. NON-ONSET-MATCHING
38
Materials

• Two types of phonological similarity
• (embedded in a larger experiment)

• Would the M350 show inhibition or priming?
• If inhibition, activation is inhibited.
• If priming, RT inhibition originates in
  competition.

1. ONSET-MATCHING
2. NON-ONSET-MATCHING
39
Materials

• Two types of phonological similarity
• (embedded in a larger experiment)

1. ONSET-MATCHING
2. NON-ONSET-MATCHING
40
Results
n21
(Pylkkänen, Stringfellow Marantz, submitted)
41
Results
n21
(Pylkkänen, Stringfellow Marantz, submitted)
42
Results
n21
(Pylkkänen, Stringfellow Marantz, submitted)
43
Results
n21
(Pylkkänen, Stringfellow Marantz, submitted)
44
Results
n21
(Pylkkänen, Stringfellow Marantz, submitted)
45
Same behavior but different neurophysiological
effects
(Pylkkänen, Stringfellow Marantz, submitted)
46
Same behavior but different neurophysiological
effects

• Not all competitors are treated the same
• Some undergo complete deactivation

(Pylkkänen, Stringfellow Marantz, submitted)
47
Semantic similarity

• Behaviorally facilitory
• NURSE primes DOCTOR
• Would the M350 show semantic priming?

48
Results

• M350 First component affected by semantic
  relatedness

(Pylkkänen, Stringfellow, Gonnerman, Marantz, in
prep.)
49

• Phonological and semantic relatedness affect the
  same component, the M350
• Consistent with recent ERP results showing that
  phonological and semantic relatedness affect the
  same ERP component, the N400 (Radeau et al. 1998)

50
So far

• Onset matching phonological similarity and
  semantic similarity have opposite effects

51
What about TEACHER-TEACH?

• Decomposition view
• Relationship is one of identity.
• TEACHER contains TEACH
• Morphemes are emergent (e.g. Seidenberg and
  Gonnerman 2000)
• Relationship is one of similarity.
• TEACHER and TEACH are only semantically and
  phonological similar

52
What about TEACHER-TEACH?

• Decomposition view
• Relationship is one of identity.
• Morphemes are emergent (e.g. Seidenberg and
  Gonnerman 2000)
• Relationship is one of similarity.

• M350 RT should show repetition priming

• M350 RT should show added effects of
  phonological and semantic similarity

53
Materials (crossmodal) (part of previous
experiment)
AUDITORY PRIME VISUAL TARGET RELATED teacher
teach UNRELATED ocean teach
54
Results
Repetition priming (Pylkkänen et al 2000)
55
Results like repetition priming, not additive
similarity effects
56
What about TEACHER-TEACH?

• Decomposition view
• Relationship is one of identity.
• Morphemes are emergent (e.g. Seidenberg and
  Gonnerman 2000)
• Relationship is one of similarity.

• M350 RT should show repetition priming

• M350 RT should show added effects of
  phonological and semantic similarity

57
Possible objection

• Phonological similarity is only inhibitory in the
  absence of semantic similarity.
• Prediction ritzy glitzy should prime very much
  like morphologically related pairs.

58
Behavioral data from Gonnerman (1999)
EXPERIMENT 1 Prime-target example Priming

• Low sem, no morph spinach-spin -19
• Low sem corner-corn -24
• Mid sem dresser-dress 19
• High sem teacher-teach 40
• Hi sem, no phon idea-notion 13

Morphological priming exceeds semantic priming
EXPERIMENT 4 Prime-target example Priming
Psychology undergrads

• High sem phon ritzy - glitzy -16
• mid sem phon dismal-dismay -12
• low sem phon rankle-rank 12
• High sem, no phon idea - notion 21
• Hi sem, no phon pumpkin-pump -19

semantic priming exceeds ritzy glitzy priming
(Gonnerman, 1999, PhD thesis, USC)
59
Behavioral data from Gonnerman (1999)
EXPERIMENT 1 Prime-target example Priming

• Low sem, no morph spinach-spin -19
• Low sem corner-corn -24
• Mid sem dresser-dress 19
• High sem teacher-teach 40
• Hi sem, no phon idea-notion 13

Morphological priming exceeds semantic priming
EXPERIMENT 4 Prime-target example Priming
Honors students

• High sem phon ritzy - glitzy 24
• mid sem phon dismal-dismay -5
• low sem phon rankle-rank 19
• High sem, no phon sorcery-magic 54
• Hi sem, no phon pumpkin-pump -39

semantic priming exceeds ritzy glitzy priming
(Gonnerman, 1999, PhD thesis, USC)
60
(No Transcript)
61
Possible objection

• Phonological similarity is only inhibitory in the
  absence of semantic similarity.
• Makes the wrong predictions

62
Effect of lexical frequency

• High frequency words are processed faster than
  low frequency words.
• Prediction of decompositional theories of
  morphology cumulative root frequency effects.

63
Effect of lexical frequency

• High frequency words are processed faster than
  low frequency words.
• Prediction of decompositional theories of
  morphology cumulative root frequency effects.

Same number of derivates
High frequency derivatives
Low frequency derivatives
- ist ize -ism
- ic ize ism
terror
magnet
Matched for surface frequency
64
Cumulative root frequency effects for inflection

• Response times to a noun depend on the cumulative
  frequency of the singular and plural (Schreuder
  Baayen, JML, 1997)
• CAT
• CATS

65
But NO cumulative root frequency effects for
derivation
Schreuder Baayen (1997)

• Family frequency

HIGH
LOW
Family frequency does not affect lexical decision
times.
- ic ize ism
- ist ize -ism
terror
SB Therefore, no decomposition in derivation.
magnet
High family size speeds up lexical decision times.
SB this is a late post-lexical effect.
66
Alternative explanation for lack of cumulative
root frequency effects in derivation

• High morphological family frequency speeds up
  root activation
• BUT
• this facilitation is cancelled out by subsequent
  competition between the highly frequent
  morphological family members.
• Hypothesized affix-competition in priming
  (Marslen-Wilson, et al. 1994)
• In crossmodal priming,
• NO PRIMING FOR
• government governor
• ALTHOUGH ROBUST PRIMING FOR
• government govern
• (Marslen-Wilson, W. D., Tyler, L., Waksler,
  R., Older, L. (1994). Morphology and meaning
  in the English mental lexicon. Psychological
  Review 101, 3-33.)

67
Alternative explanation for lack of cumulative
root frequency effects in derivation

• High morphological family frequency speeds up
  root activation
• BUT
• this facilitation is cancelled out by subsequent
  competition between the highly frequent
  morphological family members.
• This hypothesis can be tested with the M350

68
Hypothesis

• Effect of high phonotactic probability/ high
  neighborhood density

M350
RT
- slow-down due to competition
- speed-up due to sublexical frequency
69
Materials (from Baayen, R. H., Lieber, R.,
Schreuder, R. (1997). Linguistics 35, 861-877)

• Four categories of visual words, all nouns

• Contrast 1 Family frequency

HIGH
LOW

• Matched for
• Length
• Freq. of the sg,
• Cumulative freq. of the sg. pl. forms
• Family size
• Mean bigram frequency

- ic ize ism
- ist ize -ism
terror (n18)
magnet (n18)

• Contrast 2 Family size

LOW

• Matched for
• Length
• Freq. of the sg,
• Cumulative freq. of the sg. pl. forms
• Family frequency (not perfectly)
• Mean bigram frequency

HIGH
- ic ity ify head test washed
- ist
acid (n21)
diary (n21)
70
Behavior
(Pylkkänen, Feintuch, Hopkins Marantz,
Cognition, to appear)
71
MEG data, single subject
(Pylkkänen, Feintuch, Hopkins Marantz,
Cognition, to appear)
72
MEG data, n 10
LATENCY
INTENSITY
n.s.
n.s.
n.s.
n.s.
M250
P0.006
n.s.

P0.03
n.s.
M350

(Pylkkänen, Feintuch, Hopkins Marantz,
Cognition, to appear)
73
MEG data, n 10

• High family size speeds up the M350, just like it
  does RT
• ? Family size affects processing early.

• Contrary to the hypothesis from decomposition,
  high family frequency has an inhibitory effect on
  M350 amplitudes

P0.006
n.s.

P0.03
n.s.
M350

(Pylkkänen, Feintuch, Hopkins Marantz,
Cognition, to appear)
74
Why?
75
1. Difference in the time course of competition
High frequency morphological family
High density phonological neighborhood
(frequency-weighted)

• Relationship between target and competitors
  qualitatively different difference is due to
  morphology.

DECOMPOSITION

• Difference is due to the different phonological
  and/or semantic properties of the competitors.

terrorism
TERROR
NO DECOMPOSITION
terrorist
terrorize
76
1. Difference in the time course of competition

• Non-decompositional account also predicts
  interference effects in priming for pairs such as
  TERRORISM TERROR.
• BUT this is completely unsupported by data
  effect is robustly facilitory (e.g. a-d).

• Difference is due to the different phonological
  and/or semantic properties of the competitors.

terrorism
TERROR
NO DECOMPOSITION
terrorist
terrorize

• (a) Marslen-Wilson, W. D., Tyler, L., Waksler,
  R., Older, L. (1994). Morphology and meaning in
  the English mental lexicon. Psychological Review
  101, 3-33.
• (b) Pylkkänen, L. Stringfellow, A., Gonnerman,
  L., Marantz, A. 2002. Magnetoencephalographic
  indices of identity and similarity in lexical
  access. In preparation.
• Gonnerman, L. 1999, Morphology and the lexicon
  exploring the semantics-phonology interface, PhD
  thesis, University of Southern California.
• Rastle, K., Davis, M., Marslen-Wilson, W.,
  Tyler, L.K. (2000). Morphological and semantic
  effects in visual word recognition A time course
  study. Language and Cognitive Processes, 15,
  507-538.

77
1. Difference in the time course of competition
High frequency morphological family
High density phonological neighborhood
(frequency-weighted)
DECOMPOSITION

• Competition between morphological family members
  appears to precede competition between
  phonological neighbors.
• There are currently no models capturing this
  effect but what does seem clear is that an
  account of the phenomenon needs to make a
  distinction between morphological and
  phonological competitors.

78
2. High family size has an early facilitory effect

• One possibility
• Effect is semantic in nature and is related to
  effects of polysemy.
• Heavily polysemous words (such as belt) are
  processed faster than words that only have few
  senses (such as ant).
• (Rodd, Gaskell Marslen-Wilson (2002) Making
  Sense of Semantic Ambiguity Semantic Competition
  in Lexical Access. Journal of Memory and
  Language 46, 245266)
• Different morphological environments induce
  different senses of the root and therefore nouns
  with large morphological families have more
  senses than nouns with small morphological
  families.
• Prediction semantically opaque morphological
  family members should contribute to the family
  size effect the most, as those would involve the
  most sense-switching.
• BUT there is at least some evidence that the
  family size effect is in fact mostly carried by
  the semantically transparent members of the
  family.
• (De Jong NH, Feldman LB, Schreuder R, Pastizzo
  M, Baayen RH (2002) The processing and
  representation of Dutch and English compounds
  peripheral morphological and central orthographic
  effects. Brain Lang 2002 Apr-Jun81(1- 3)555-67.
  )

79
2. High family size has an early facilitory effect
Alternatively The family size effect is not a
facilitory effect of high family size, but an
inhibitory effect stemming from more potent
competitors in the low family size condition.

• (See Perea and Rosa (2000) for a review of
  studies indicating that the important
  neighborhood variable in visual word recognition
  is not the number of neighbors per se, but the
  frequency of a word's neighbors relative to its
  own frequency. Perea M. and E. Rosa (2000)
  Psicologica, 21, 327-340)

80
Conclusion

• Evidence for decomposition (although somewhat
  indirect).
• Evidence for the existence of morphological
  competition (cf. Marslen-Wilson 1994).
• Identification of a neural correlate of the
  morphological family size effect.

81
Take-home message

• An important part of cognitive neuroscience is to
  use our existing knowledge about cognition in
  order to understand the brain.
• Brain activity doesnt come tagged for
  cognitive functions and so research needs to be
  hypothesis-driven.
• But once youve gained some understanding about
  the neural correlates of a function, it is
  possible to look into the brain and find out new
  things about language, and sometimes you can be
  taken by surprise!