Connectionist Modelling Summer School

1
Connectionist Modelling Summer School

• Lecture Two

2
The Mapping Principle

• Patterns of ActivityAn input pattern is
  transformed to an output pattern.
• Activation States are Vectors
• Each pattern of activity can be considered a
  unique point in a space of states. The activation
  vector identifies this point in space.

• Mapping Functions
• T F (S)
• The network maps a source space S (the network
  inputs) to a target space T (the outputs).The
  mapping function F is most likely complex. No
  simple mathematical formula can capture it
  explicitly.
• HyperspaceInput states generally have a high
  dimensionality. Most network states are therefore
  considered to populate HyperSpace.

T
S
3
The Principle of Superposition
Matrix 1
Composite Matrix
Matrix 2
4
Hebbian Learning

• Cellular AssociationWhen an axon of cell A is
  near enough to excite a cell B and repeatedly or
  persistently takes part in firing it, some growth
  process of metabolic change takes place in one or
  both cells such that As efficiency, as one of
  the cells firing B, is increased. (Hebb 1949,
  p.50)
• Learning Connections
• Take the product of the excitation of the two
  cells and change the value of the connection in
  proportion to this product.

• The Learning Rule
• e is the learning rate.
• Changing ConnectionsIf ain 0.5, aout 0.75,
  and e 0.5then ?w 0.5(0.75)(0.5)
  0.1875And if wstart 0.0, then wnext 0.1875
• Calculating Correlations

2
0
1
5
Nature of mental representation

• Mind as a physical symbol system
• Software/hardware distinction
• Symbol manipulation by rule-governed processes

6
Nature of mental representation

• Mind as a parallel distributed processing system

• Representations are coded in connections and node
  activities

7
Evidence for rules

• Regular and Irregular Morphology
• talk gt talked
• ram gt rammed
• pit gt pitted
• hit gt hit
• come gt came
• sleep gt slept
• go gt went

8
Evidence for rules

• Errors in performance
• hitted
• sleeped
• goed, wented
• U-shaped development
• Recovery from errors

9
Evidence for rules

• Rote Learning Processes
• Initial error-free performance
• Rule Extraction and Application
• Overgeneralisation errors
• Speedy learning of new regular past tense forms
• Rote plus Rule
• Continued application of regularisation process
• Recovery from regularisation of irregulars

10
Models of English past tense

• Dual mechanism account
• Rule-governed component deals with regular
  mappings
• Separate listing of exceptions
• Blocking principle
• Imperfect retrieval of irregular past tense
  representations result in overgeneralisation
• Pinker Prince 1988

Output Inflection
Exceptions
Rule
Input Stem
11
Models of English past tense

• PDP accounts
• Single homogeneous architecture
• Superposition
• Competition between different different verb
  types result in overregularisation and
  irregularisation
• Vocabulary discontinuity
• Rumelhart McClelland 1986