The ICSI/Berkeley Neural Theory of Language Project

1
The ICSI/Berkeley Neural Theory of Language
Project
ECG
2
(No Transcript)
3
Connectionist Model of Word Recognition
(Rumelhart and McClelland)
4
Constraints on Connectionist Models

• 100 Step Rule
• Human reaction times 100 milliseconds
• Neural signaling time 1 millisecond
• Simple messages between neurons
• Long connections are rare
• No new connections during learning
• Developmentally plausible

5
Can we formalize/model these intuitions

• What is a neurally plausible computational model
  of spreading activation that captures these
  features.
• What does semantics mean in neurally embodied
  terms
• What are the neural substrates of concepts that
  underlie verbs, nouns, spatial predicates?

6
Abstract Neuron
7
Computing with Abstract Neurons

• McCollough-Pitts Neurons were initially used to
  model
• pattern classification
• size small AND shape round AND color green
  AND location on_tree gt unripe
• linking classified patterns to behavior
• size large OR motion approaching gt
  move_away
• size small AND direction above gt move_above
• McCollough-Pitts Neurons can compute logical
  functions.
• AND, NOT, OR

8
Distributed vs Localist Repn
John 1 1 0 0
Paul 0 1 1 0
George 0 0 1 1
Ringo 1 0 0 1
John 1 0 0 0
Paul 0 1 0 0
George 0 0 1 0
Ringo 0 0 0 1

• What are the drawbacks of each representation?

9
Distributed vs Localist Repn
John 1 1 0 0
Paul 0 1 1 0
George 0 0 1 1
Ringo 1 0 0 1
John 1 0 0 0
Paul 0 1 0 0
George 0 0 1 0
Ringo 0 0 0 1

• What happens if you want to represent a group?
• How many persons can you represent with n bits?
  2n

• What happens if one neuron dies?
• How many persons can you represent with n bits? n

10
Sparse Distributed Representation
11
Visual System

• 1000 x 1000 visual map
• For each location, encode
• orientation
• direction of motion
• speed
• size
• color
• depth
• Blows up combinatorically!

12
Coarse Coding

• info you can encode with one fine resolution unit
  info you can encode with a few coarse
  resolution units
• Now as long as we need fewer coarse units total,
  were good

13
Coarse-Fine Coding
Coarse in F2, Fine in F1
G
G

• but we can run into ghost images

Coarse in F1, Fine in F2
Feature 2e.g. Direction of Motion
14
Connectionist Models in Cognitive Science
Structured
PDP
Hybrid
Neural
Conceptual
Existence
Data Fitting
15
Models of Learning

• Hebbian coincidence
• Recruitment one trial Lecture 14
• Supervised correction (backprop)
• Reinforcement delayed reward
• Unsupervised similarity

16
Computing other relations

• The 2/3 node is a useful function that activates
  its outputs (3) if any (2) of its 3 inputs are
  active
• Such a node is also called a triangle node and
  will be useful for lots of representations.

17
Triangle nodes and McCullough-Pitts Neurons?
A
B
C
18
They all rose

• triangle nodes
• when two of the neurons fire, the third also
  fires
• model of spreading activation

19
Spreading activation and feature structures

• Parallel activation streams.
• Top down and bottom up activation combine to
  determine the best matching structure.
• Triangle nodes bind features of objects to values
• Mutual inhibition and competition between
  structures
• Mental connections are active neural connections

20
Representing concepts using triangle nodes
21
Feature Structures in Four Domains
Barrett Ham Container Push
deptCS Color pink Inside region Schema slide
sid001 Taste salty Outside region Posture palm
empGSI Bdy. curve Dir. away

Schneider Pea Purchase Stroll
deptLing Color green Buyer person Schema walk
sid002 Taste sweet Seller person Speed slow
empGra Cost money Dir. ANY
Goods thing
22
(No Transcript)
23
5 levels of Neural Theory of Language
Spatial Relation
Motor Control
Pyscholinguistic experiments
Metaphor
Grammar
Cognition and Language
Computation
Structured Connectionism
abstraction
Neural Net and learning
SHRUTI
Triangle Nodes
Computational Neurobiology
Biology
Neural Development
Midterm
Quiz
Finals
24
Categories and concepts- introduction

• CS182/Ling109/CogSci110
• Spring 2008

25
Lecture Outline

• Categories
• Basic Level
• Prototype Effects
• Neural Evidence for Category Structure
• Aspects of a Neural Theory of concepts
• Image Schemas
• Description and types
• Behavioral Experiment on Image Schemas
• Event Structure and Motor Schemas

26
Embodiment
Of all of these fields, the learning of
languages would be the most impressive, since it
is the most human of these activities. This
field, however, seems to depend rather too much
on the sense organs and locomotion to be
feasible. Alan Turing (Intelligent Machines,1948)
27
The WCS Color Chips

• Basic color terms
• Single word (not blue-green)
• Frequently used (not mauve)
• Refers primarily to colors (not lime)
• Applies to any object (not blonde)

28
Concepts

• What Concepts Are Basic Constraints
• Concepts are the elements of reason, and
• constitute the meanings of words and linguistic
  expressions.

29

• Concepts Are
• Universal they characterize all particular
  instances e.g., the concept of grasping is
  the same no matter who the agent is or what the
  patient is or how it is done.
• Stable.
• Internally structured.
• Compositional.
• Inferential. They interact to give rise to
  inferences.
• Relational. They may be related by hyponymy,
  antonymy, etc.
• Meaningful.
• Not tied to the specific word forms used to
  express them.

30
Concepts Traditional Theory

• The Traditional Theory
• Reason and language are what distinguish human
  beings from other animals.
• Concepts therefore use only human-specific brain
  mechanisms.
• Reason is separate from perception and action,
  and does not make direct use of the sensory-motor
  system.
• Concepts must be disembodied in this sense.

31
The neural theory

• Human concepts are embodied. Many concepts make
  direct use of sensory-motor, emotional, and
  social cognition capacities of our body-brain
  system.
• Many of these capacities are also present in
  non-human primates.
• Continuity Principle of Am. Pragmatists

32
Classical vs prototype model of categorization

• Classical model
• Category membership determined on basis of
  essential features
• Categories have clear boundaries
• Category features are binary
• Prototype model
• Features that frequently co-occur lead to
  establishment of category
• Categories are formed through experience with
  exemplars

33
Prototype theory

1. Certain members of a category are prototypical
   or instantiate the prototype
2. Categories form around prototypes new members
   added on basis of resemblance to prototype
3. No requirement that a property or set of
   properties be shared by all members
4. Features/attributes generally gradable
5. Category membership a matter of degree
6. Categories do not have clear boundaries

34
Prototype theory

• Certain members of a category are prototypical
  or instantiate the prototype
• Category members are not all equal
• a robin is a prototypical bird, but we may not
  want to say it is the prototype, rather it
  instantiates (manifests) the prototype or ideal
  -- it exhibits many of the features that the
  abstract prototype does
• It is conceivable that the prototype for dog
  will be unspecified for sex yet each exemplar is
  necessarily either male or female. (Taylor)

35
Prototype theory

• Categories form around prototypes new members
  can be added on the basis of resemblance to the
  prototype
• Categories may also be extended on the basis of
  more peripheral features
• house for apartment

36
Prototype theory

• 3. No requirement that a property or set of
  properties be shared by all members -- no
  criterial attributes
• Category where a set of necessary and sufficient
  attributes can be found is the exception rather
  than the rule
• Labov household dishes experiment
• Necessary that cups be containers, not sufficient
  since many things are containers
• Cups cant be defined by material used, shape,
  presence of handles or function

37
Prototype theory

• Wittgensteins examination of game
• Generally necessary that all games be amusing,
  not sufficient since many things are amusing
• Board games, ball games, card games, etc. have
  different objectives, call on different skills
  and motor routines
• -? categories normally not definable in terms of
  necessary and sufficient features

38
Prototype theory

• What about mathematical categories like odd or
  even numbers? Arent these sharply defined?
• (Armstrong et al.) Subjects asked to assign
  numbers a degree of membership to the categories
  odd number or even number
• ? 3 had a high degree of membership, 447 and
  91 had a lower degree (all were rated at least
  moderately good)

39
Categories - who decides?

• Embodied theory of meaning- categories are not
  pre-formed and waiting for us to behold them.
  Our need for categories drives what categories we
  will have
• Basic level categories - not all categories have
  equal status. The basic level category has
  demonstrably greater psychological significance.

40
Basic-level categories
41

• chair desk chair
• easy chair rocking
  chair
• furniture lamp desk lamp
• floor lamp
• table dining room table
  coffee table
• Superordinate Basic Subordinate

42
Categories Prototypes Overview
Furniture
Superordinate
Sofa
Desk
Basic-Level Category
leathersofa
fabricsofa
L-shapeddesk
Receptiondisk
Subordinate

• Three ways of examining the categories we form
• relations between categories (e.g. basic-level
  category)
• internal category structure (e.g. radial
  category)
• instances of category members (e.g. prototypes)

43
Basic-level -- Criteria

• Perception
• overall perceived shape
• single mental image
• fast identification

44
Basic-level -- Criteria

• Perception
• Function motor program for interaction

45
Basic-level -- Criteria

• Perception
• Function
• Words
• shortest
• first learned by children
• first to enter lexicon

46
Basic-level -- Criteria

• Perception
• Function
• Communication
• Knowledge organization
• most attributes are stored at this level

47
Basic-Level Category
What constitutes a basic-level category?

• Perception
• similar overall perceived shape
• single mental image
• (gestalt perception)
• fast identification
• Function
• general motor program

• Communication
• shortest
• most commonly used
• contextually neutral
• first to be learned by children
• first to enter the lexicon
• Knowledge Organization
• most attributes of category members stored at
  this level

48
Other Basic-level categories

• Objects
• Colors
• Motor-routines

49
Concepts are not categorical
50
Mother

• The birth model
• The person who gives birth is the mother
• The genetic model
• The female who contributes the genetic material
  is the mother
• The nurturance model
• The female adult who nurtures and raises a child
  is the mother of the child
• The marital model
• The wife of the father is the mother
• The genealogical model
• The closest female ancestor is the mother

(WFDT Ch.4, p.74, p.83)
51
Radial Structure of Mother
Geneticmother
Stepmother
Unwedmother
Adoptivemother
CentralCase
Birthmother
Surrogatemother
Naturalmother
Biologicalmother
Fostermother

• The radial structure of this category is defined
  with respect to the different models

52
Marriage

• What is a marriage?
• What are the frames (or models) that go into
  defining a marriage?
• What are prototypes of marriage?
• What metaphors do we use to talk about marriages?
• Why is this a contested concept right now?

53
Concepts and radial categories

• Concepts can get to be the "prototype" of their
  category in various ways.
• Central subcategory (others relate to this)
• Amble and swagger relate to WALK
• Shove relates to PUSH
• Essential (meets a folk definition birds have
  feathers, beaks, lay eggs)
• Move involves change of location.
• Typical case (most are like this "sparrow")
• Going to a conference involves air travel.
• Ideal/anti-ideal case (positive social standard
  "parent") anti-ideal case (negative social
  standard "terrorist")
• Stereotype (set of attributes assumed in a
  culture "Arab")
• Salient exemplar (individual chosen as example)

54
Category Structure

• Classical Category
• necessary and sufficient conditions
• Radial Category
• a central member branching out to less-central
  and non-central cases
• degrees of membership, with extendable boundary
• Family Resemblance
• every family member looks like some other family
  member(s)
• there is no one property common across all
  members (e.g. polysemy)
• Prototype-Based Category
• Essentially-Contested Category (Gallie, 1956)
  (e.g. democracy)
• Ad-hoc Category (e.g. things you can fit inside a
  shopping bag)

55
Prototype

• Cognitive reference point
• standards of comparison
• Social stereotypes
• snap judgments
• defines cultural expectations
• challengeable
• Typical case prototypes
• default expectation
• often used unconsciously in reasoning

• Ideal case / Nightmare case
• e.g. ideal vacation
• can be abstract
• may be neither typical nor stereotypical
• Paragons / Anti-paragons
• an individual member that exhibits the ideal
• Salient examples
• e.g. 9/11 terrorism act
• Generators
• central member rules
• e.g. natural number single-digit numbers
  arithmetic

56
Neural Evidence for category structure

• Are there specific regions in the brain to
  recognize/reason with specific categories?

57
Category Naming and Deficits

• People with brain injury have selective deficits
  in their knowledge of categories.
• Some patients are unable to identify or name man
  made objects and others may not be able to
  identify or name natural kinds (like animals)

58
A PET Study on categories (Nature 1996)
59
Study

• 16 adults (8M, 8F) participated in a PET
  (positron emission tomography) study.
• Involves injecting subject with a positron
  emitting radioactive substance (dye)
• Regions with more metabolic activity will absorb
  more of the substance and thus emit more
  positrons
• Positron-electron collisions yield gamma rays,
  which are detected
• Increased rCBF (regional changes in cerebral
  blood flow) was measured
• When subjects viewed line drawings of animals and
  tools.

60
The experiment

• Subjects looked at pictures of animals and tools
  and named them silently.
• They also looked at noise patterns (baseline 1)
• And novel nonsense objects (baseline 2)
• Each stimulus was presented for 180ms followed by
  a fixation cross of 1820 ms.
• Drawings were controlled for name frequency and
  category typicality

61
medial
lateral
62
(No Transcript)
63
(No Transcript)
64
Left middle temporal gyrus
ACC
Premotor
65
Calcarine Sulcus
66
Conclusions

• Both animal and tool naming activate the ventral
  temporal lobe region.
• Tools differentially activate the ACC, pre-motor
  and left middle temporal region (known to be
  related to processing action words).
• Naming animals differentially activated left
  medial occipital lobe (early visual processing)
• The object categories appear to be in a
  distributed circuit that involves activating
  different salient aspects of the category.