Development and Disintegration of Conceptual Knowledge: A Parallel-Distributed Processing Approach

1
Development and Disintegration of Conceptual
KnowledgeA Parallel-Distributed Processing
Approach

• James L. McClelland
• Department of Psychology andCenter for Mind,
  Brain, and ComputationStanford University

2
Parallel Distributed Processing Approach to
Semantic Cognition

• Representation is a pattern of activation
  distributed over neurons within and across brain
  areas.
• Bidirectional propagation of activation mediated
  by a learned internal representation underlies
  the ability to bring these representations to
  mind from given inputs.
• The knowledge underlying propagation of
  activation is in the connections, and is acquired
  through a gradual learning process.

3
A Principle of Learning and Representation

• Learning and representation are sensitive to
  coherent covariation of properties across
  experiences.

4
What is Coherent Covariation?

• The tendency of properties of objects to co-occur
  in clusters.
• e.g.
• Has wings
• Can fly
• Is light
• Or
• Has roots
• Has rigid cell walls
• Can grow tall

5
Development and Degeneration

• Sensitivity to coherent covariation in an
  appropriately structured Parallel Distributed
  Processing system creates the taxonomy of
  categories that populate our minds and underlies
  the development of conceptual knowledge.
• Gradual degradation of the representations
  constructed through this developmental process
  underlies the pattern of semantic disintegration
  seen in semantic dementia.

6
Some Phenomena in Development

• Progressive differentiation of concepts
• Overextension of frequent names
• Overgeneralization of typical properties

7
(No Transcript)
8
The Rumelhart Model
9
The Training Data
All propositions true of items at the bottom
levelof the tree, e.g. Robin can grow, move,
fly
10
Target output for robin can input
11
Forward Propagation of Activation
12
Back Propagation of Error (d)
aj
wij
ai
di Sdkwki
wki
dk (tk-ak)
Error-correcting learning At the output
layer Dwki edkai At the prior layer Dwij
edjaj
13
(No Transcript)
14
(No Transcript)
15
Early Later LaterStill
Experie nce
16
(No Transcript)
17
What Drives Progressive Differentiation?

• Waves of differentiation reflect coherent
  covariation of properties across items.
• Patterns of coherent covariation are reflected in
  the principal components of the property
  covariance matrix.
• Figure shows attribute loadings on the first
  three principal components
• 1. Plants vs. animals
• 2. Birds vs. fish
• 3. Trees vs. flowers
• Same color features covary in
  component
• Diff color anti-covarying
  features

18
Properties Coherent Incoherent
CoherenceTraining Patterns
is can has is can has
Items
No labels are provided Each item and each
property occurs with equal frequency Coherently
co-varying inputs are not presented at the same
time!
19
Effect of Coherence on Representation
20
Overextension of A Frequent Name to Similar
Objects
Goat
Oak
goat
tree
dog
21
(No Transcript)
22
Overgeneralization of typical properties

• Rochel Gelman found that children think that all
  animals have feet.
• Even animals that look like small furry balls and
  dont seem to have any feet at all.

23
A typical property thata particular object
lacks e.g., pine has leaves
An infrequent, atypical property
24
Development and Degeneration

• Sensitivity to coherent covariation in an
  appropriately structured Parallel Distributed
  Processing system underlies the development of
  conceptual knowledge.
• Gradual degradation of the representations
  constructed through this developmental process
  underlies the pattern of disintegration seen in
  semantic dementia.

25
Disintegration of Conceptual Knowledge in
Semantic Dementia

• Progressive loss of specific knowledge of
  concepts, including their names, with
  preservation of general information
• Overextension of frequent names
• Overgeneralization of typical properties

26
Picture namingand drawing in Sem. Demantia
27
(No Transcript)
28
Grounding the Model in What we Know About The
Organization of Semantic Knowledge in The Brain

• Specialized areas for each of many different
  kinds of semantic information.
• Semantic dementia results from progressive
  bilateral disintegration of the anterior temporal
  cortex.
• Destruction of the medial temporal lobes results
  in loss of memory for recent events and loss of
  the ability to form new memories quickly, but
  leaves existing semantic knowledge unaffected.

language
29
Proposed Architecture for the Organization of
Semantic Memory
name
action
motion
Temporal pole
color
form
valance
30
Rogers et al (2005) model of semantic dementia

• Trained with 48 items from six categories (from a
  clinical test).
• Names are individual units, other patterns are
  feature vectors.
• Features come from a norming study.
• From any input, produce all other patterns as
  output.
• Representations undergo progressive
  differentiation as learning progresses.
• Test of picture naming Present vision input.
• Most active name unit above a threshold is chosen
  as response.

31
Errors in Naming for As a Function of Severity
Simulation Results
Patient Data
Fraction of Connections Destroyed
Severity of Dementia
32
Simulation of Delayed Copying

• Visual input is presented, then removed.
• After three time steps, the vision layer pattern
  is compared to the pattern that was presented.
• Omissions and intrusions are scored for
  typicality.

33
Omission Errors
34
Intrusion Errors
35
Development and Degeneration

• Sensitivity to coherent covariation in an
  appropriately structured Parallel Distributed
  Processing system underlies the development of
  conceptual knowledge.
• Gradual degradation of the representations
  constructed through this developmental process
  underlies the pattern of semantic disintegration
  seen in semantic dementia.

36
(No Transcript)
37
A Hierarchical Bayesian Characterization

• Initially, assume theres only one kind of
  thing in the world.
• Assign probabilities to occurrence of properties
  according to overall occurrence rates.
• Coherent covariation licenses the successive
  splitting of categories
• Probabilities of individual features become much
  more predictable, and conditional relations
  between features are available for inference.
• Overgeneralization and overextension are
  consequences of implicitly applying the kind
  feature probabilities to the item and depends
  on the current level of splitting into kinds.
• This process occurs
• By an on-line, incremental learning process.
• In a gradual and graded way.
• Without explicit enumeration of possibilities.

38
A Hierarchical Bayesian Characterization (Contd)

• Effect of damage causes the network to revert to
  a simpler model.
• Perhaps the network can be seen as maximizing its
  accuracy in explaining properties of objects
  given
• Limited training data during acquisition
• Limited resources during degredation

39
Thanks for your attention!
40
Sensitivity to Coherence Requires Convergence
A
A