Cognitive Neuroscience 2002

1
Conceptual Hierarchies Arise from the Dynamics of
Learning and Processing Insights from a Flat
Attractor Network
Christopher M. OConnor Ken McRae George S.
Cree University of Western Ontario University of
Western Ontario University of Toronto at
Scarborough London, Ontario, Canada London,
Ontario, Canada Toronto, Ontario,
Canada cmoconno_at_uwo.ca mcrae_at_uwo.ca gcree_at_utsc.ut
oronto.ca
Acknowledgements NSERC grant OGP0155704 NIH
grant R01-MH6051701 to Ken McRae
Superordinate Basic-level Representations

• Introduction
• peoples conceptual knowledge structure for
  concrete nouns traditionally viewed as
  hierarchical (Collins Quillian, 1969)
• superordinate concepts (vegetable) represented at
  a different level in hierarchy than basic-level
  concepts (carrot, or pumpkin)
• flat attractor networks i.e., models with a
  single layer of semantics have provided insight
  to a number of phenomena regarding basic-level
  concepts
• semantic priming
• statistically-based feature correlations
• concept-feature distributional statistics
• unclear how these networks could learn and
  represent superordinate concepts
• can such a network account for established
  results and provide novel insights?
• Goals
• demonstrate that a flat attractor network can
  learn superordinate concepts
• simulate typicality ratings to show model
  accounts for graded structure

• Superordinate Priming Temporal Dynamics of
  Similarity
• both spreading activation (Collins Loftus,
  1975) and attractor networks predict that
  magnitude of semantic priming is determined by
  degree of semantic similarity
• supported experimentally using basic-level
  concepts (McRae Boisvert, 1998)
• simulated using feature-based attractor nets
  (Cree, McRae, McNorgan, 1999)
• therefore, the degree that an exemplar target is
  primed by its superordinate should vary as a
  function of typicality
• high typicality gt medium typicality gt low
  typicality
• however, Schwanenflugel and Rey (1986) found that
  short SOA superordinate priming does not vary as
  a function of target exemplar typicality
• replicated and simulated their experiment
• Experiment
• 72 superordinate-exemplar pairs, e.g., vegetable
  paired with peas, turnip, garlic
• 12 superordinate primes with 2 exemplars each of
  low, medium, and high typicality
• 200ms superordinate prime, 50ms ISI, exemplar
  target until response (concrete object?)
• Results replicated Schwanenflugel and Rey (1986)
• main effect of relatedness, F1(1, 42) 8.09, p lt
  .01, F2(1, 66) 3.52, p lt .07

• activation of features influenced by
• Feature Frequency
• if many exemplars possess a feature, it is
  strongly activated
• Category Cohesion
• degree of featural overlap of exemplars
  determines activation of superordinate features
• more overlap more activation
• Feature Correlations
• activate one another during the computation of
  meaning

• Basic-level representations
• all features have activations close to 1 (on)

• Superordinate representations
• most features have intermediate activations

• Simulation
• superordinate prime wordform presented to model
  for 15 ticks
• exemplar target presented for 20 ticks
• cross entropy error recorded over last 20 ticks
• Results
• typicality relatedness did not interact, F lt 1
• main effect of relatedness, F(1, 66) 187.27, p
  lt .001
• related lower than unrelated for ticks 1 to 13

Category N Cosine/ Fam Res/ Cosine/ Typicality T
ypicality Fam Res
Feature Verification

• Typicality Ratings
• important for any semantic memory model to
  simulate graded structure
• Experiment
• collected behavioral typicality ratings for all
  20 categories (7-point scale)
• Simulation
• superordinate wordform presented representation
  recorded
• basic-level wordform presented representation
  recorded
• computed cosine similarity between each
  superordinate exemplar
• computed correlation between typicality ratings
  cosines for each category
• correlation between typicality ratings family
  resemblance served as baseline
• Results

• Model
• Structure
• input 30 wordform units representing
  spelling/sound of a word
• output 2349 semantic feature units representing
  features taken from McRae et al.s (2005) feature
  production norms
• e.g., lthas wingsgt, ltmade of metalgt, ltis redgt,
  lthas seedsgt
• single layer of semantics taxonomic features
  removed all semantic features were
  interconnected
• thus, no hierarchy built into the model
• Training
• model learned to map random 3-unit wordform for
  each concept to semantic features for that
  concept
• basic-level concepts trained in 1-to-1 manner
• 3-unit wordform paired with same set of semantic
  features on every learning trial
• superordinate concepts trained in 1-to-many
  manner
• wordform paired with semantic features of one of
  its exemplars on each trial

furniture 17 .76 .62 .78 fruit 29 .71 .69
.91 appliance 14 .61 .73 .89 weapon 39 .5
8 .70 .76 utensil 22 .57 .52 .68 bird
29 .57 .49 .69 insect 13 .52 .69 .77 ca
rnivore 19 .52 .45 .83 container 14 .46 .50
.51 vegetable 31 .45 .50 .90 musical
instrument 18 .44 .54 .94 clothing 39 .43 .5
0 .73 tool 34 .41 .38 .65 fish 11 .41 .36
.93 animal 133 .18 .12 .55 pet 22 .15 -.01
.86 herbivore 18 .04 .21 .78 predator 17 -.14
.06 .60 mammal 57 -.03 .14 .64 vehicle 27 -.14
.18 .72 p lt .05, p lt .01 Fam Res
Family Resemblance

• Explanation
• why is priming from superordinate to exemplar
  different than priming between basic-level
  concepts?
• superordinate features have intermediate
  activations, which (due to the sigmoid activation
  function) require less change in net input to be
  turned on or off

• basic-level priming features in prime but not in
  target relatively difficult to turn off
• prime target must have high degree of featural
  overlap to produce priming
• superordinate priming activation of prime's
  features more easily changed
• priming still results (vs. unrelated
  superordinate), but less sensitive to similarity
• therefore, same amount of facilitation for
  exemplars of all typicality levels

• Conclusions
• semantic memory can be represented as a single
  layer of semantics
• without a transparent hierarchical structure
• accounts for graded structure of categories
• predicts online superordinate verification
  latencies novel result
• due to the temporal dynamics of similarity,
  accounts for counterintuitive and seemingly
  inconsistent results regarding basic-level vs.
  superordinate priming
• results counter to hierarchical spreading
  activation theories

• models predicts typicality ratings at least as
  well as family resemblance
• therefore, the model was successful in simulating
  graded structure

• Feature Verification
• similar flat attractor networks have simulated
  basic-level feature verification
• model can also simulate verification of
  superordinate features
• Experiment
• 54 superordinate-feature pairs such as furniture
  ltmade of woodgt fruit lttastes sweetgt
• superordinate name for 400 ms, feature name until
  participant responded
• "Is the feature characteristic of the category?"
• Simulation
• present superordinate wordform and record
  feature's activation over 20 time ticks
• correlated model's feature activation with human
  verification latency
• feature activation in model predicts human
  verification from ticks 6 - 20

Semantic Features (2349 units)
References Collins, A. M., Quillian, M. R.
(1969). Retrieval time from semantic memory.
Journal of Verbal Learning and Verbal Behavior,
8, 240-247. Collins, A. M., Loftus, E. F.
(1975). A spreading activation theory of semantic
processing. Psychological Review, 82,
407-428. Cree, G. S., McRae, K, McNorgan, C.
(1999). An attractor model of lexical conceptual
processing Simulating semantic priming.
Cognitive Science, 23, 371-414. McRae, K.
Boivert, S. (1998). Automatic semantic similarity
priming. Journal of Experimental Psychology
Learning, Memory and Cognition, 24,
558-572. McRae, K., Cree, G. S., Seidenberg, M.
S., McNorgan, C. (2005). Semantic feature
production norms for a large set of living and
nonliving things. Behavior Research Methods, 37,
547-559. Schwanenflugel, P. J., Rey, M. (1986).
Interlingual semantic facilitation Evidence for
a common representational system in the bilingual
lexicon. Journal of Memory and Language, 25,
605-618.
Wordform (30 units)