The Symbolic vs Subsymbolic Debate

1
The Symbolic vs Subsymbolic Debate

• H. Bowman
• (CCNCS, Kent)

2
Disclaimer

• Serious simplification of a complex debate.
• Emphasize extreme positions to clarify basic
  points of controversy.
• What I present is not necessarily what I
  personally believe!!

3
The Mind-Body Problem

• symbolic
• putative characteristics of cognition
• information processing metaphor

• subsymbolic
• inspired by neurobiology
• how cognition emerges from neurobiology

4
The Symbolic Tradition
5
The Computer Metaphor

• Takes inspiration from,
• programming languages computational logic
• data structures knowledge representation
• link to programming langs. such as Lisp Prolog
• computer architectures
• Von Neumann architectures centralized processing
  
• computability theory
• the Church Turing hypothesis

6
Key Assumptions

• Symbols are available to the cognitive system
• Symbol processing engine, characteristics,
• Systematic, i.e. combinatorial symbol systems and
  compositionality
• Recursive knowledge structures

7
Syntax Grammars 1
A set of rules
Sentence S NP VP Noun phrase
NP det AL N N Verb phrase VP V
NP Adjective list AL A AL A
S, NP, VP, AL molecules det, N, V, A atoms

• recursion

8
Syntax Grammars 2
S
A Tree Data Structure
9
Compositionality

• Plug constituents in according to rules
• Structure of expressions indicates how they
  should be interpreted
• Semantic Compositionality,
• the semantic content of a (molecular)
  representation is a function of the semantic
  contents of its syntactic parts, together with
  its constituent structure
• Fodor Pylyshyn,88

10
Compositionality in Semantics 1

• Meanings of items plugged in as defined by syntax

M X denotes meaning of X
M John loves Jane M John M loves
M Jane composed together appropriately
11
Compositionality in Semantics 2

• Same example in more detail

M John loves Jane . M loves
....
M John
M Jane
12
Compositionality in Semantics 3

• Meanings of atoms constant across different
  compositions

M Jane loves John . M loves
....
M Jane
M John
13
Compositionality in Semantics 4

• Also, meanings of molecules constant across
  different compositions

M Jane loves John and Jane hates James
...... M and ...
M Jane loves John
M Jane hates James
14
Compositionality in Semantics 5
M Jane hates James and Jane loves John
...... M and ...
M Jane loves John
M Jane hates James
15
Caveat

• Compositionality of course not absolute, e.g.
  Idioms kicked the bucket

16
Compositionality non-linguistic examples

• Not just an issue for language
• Reasoning / planning / deductive thought
• Representation of knowledge
• hierarchical / superordinate structures

17
Representation of Visual Objects
From Marrs theory of object Recognition
18
Whole-part Hierarchies
S S S
S S S S S S S S S S S
S S S S
S S S
19
Production System Architectures of the Mind

• Most detailed and complete realisation of
  symbolic tradition, e.g.
• SOAR (Unified Theories of Cognition) Newell
• ACT-R Anderson
• EPIC Kieras
• GOFAI (Good Old Fashioned Artificial
  Intelligence)
• Based upon expert systems technology

20
Symbol Systems and Nature vs Nurture

• Learning theories of symbolic architectures are
  rather limited
• although, chunking-based theories do exist
• Where does the symbolic processing engine come
  from?
• THEREFORE
• Evolutionary explanations, e.g.
• Chomskys Universal Grammars

21
Symbol Systems and the Brain

• For symbolists, the algorithmic / specification
  levels are critical, the implementation level is
  insignificant (using Marrs terminology)
• for a Symbolist, neurons implement all
  cognitive processes in precisely the same way,
  viz., by supporting the basic operations that are
  required for symbol-processing i.e. all
  that is required is that you use your neural
  network to implement a Turing machine
  FodorPylyshyn,88
• A sort of compilation step.

22

• Computers used as an analogy, where software is
  the interesting thing and the hardware mapping is
  fixed and automatic.
• one should be deeply suspicious of the heroic
  sort of brain modelling that purports to address
  the problems of cognition. We sympathize with the
  craving for biologically respectable theories
  that many psychologists seem to feel. But, given
  a choice, truth is more important than
  respectability. FodorPylyshyn,88

23
The Sub-symbolic Tradition
24
Connectionism

• Inspiration from neurobiology
• Long tradition at least from 50s, e.g. Hebb,
  Rosenblatt, Grossberg, Rumelhart, McClelland,
  OReilly.
• Nodes, links, activation, weights, learning
  algorithms

25
Activation in Classic Artificial Neural Network
Model
output - yj
activation value - yj
node j
net input - hj
26
Sigmoidal Activation Function
Responsive around net input of 0 Unresponsive at
extreme net inputs Threshold unresponsive at
low net inputs
27
Connectionism and Nature vs Nurture

• Powerful learning algorithm
• directed weight adjustment
• extracting regularities (Hebbian learning)
• supervised learning (Back-propagation)
• Typically ascribe more to learning than to
  evolution

28
Example Connectionist Model

• Word reading as an example
• Orthography to Phonology
• Words of four letters or less
• Need to represent order of letters, otherwise,
  e.g. slot and lots the same
• Slot coding

29
A (Rather Naïve) Reading Model
PHONOLOGY
ORTHOGRAPHY
30
Connectionism Compositionality

• Highly non-compositional, e.g.
• a in ant and cat completely unrelated
  representations
• no sense to which plug in constituent
  representations
• maximally affected by context
• Same argument would generalise to semantic
  compositionality
• Alternative connectionist models do better, e.g.
  activation gradient models, but not clear that
  any model is truly systematic in sense of
  symbolic processing

31
Spectrum of Approaches
Systematic / symbolic
Centralised Production Systems Architectures,
e.g. SOAR Newell
Parallel Production Systems, e.g. EPIC Kieras
(Symbolic) Distributed Control, e.g. Actors
Hewett, Agents Kokinov, ICS Barnard,
Society of Minds Minsky
Hybrid Approaches, e.g. Gabbay
Localist Models with Serial Order, e.g. Solaris
Davis
Localist / Competitive Learning, e.g. IA model
McClelland
Distributed Representations / Back-prop., e.g.
Seidenberg
Unsystematic / subsymbolic
32
Possible Topics 1

• Introduction to connectionism
• OReilly Munakata, 2000
• Production system architectures
• ACT-R Anderson,93
• Connectionism Strengths and Weaknesses
• Fodor Pylyshyn, 88
• McClelland, 92 and 95
• Symbolic-like Connectionism
• Hinton, 90

33
Possible Topics 2

• Past tense debate
• Pinker et al, 2003
• Localist vs distributed debate
• Bowers, 2002 and Page, 2000.
• Dual process theory system 1 (neural) system
  2 (symbolic)
• Evans, 2003.

34
References

• Anderson, J. R. (1993). Rules of the Mind.
  Hillsdale, NJ Erlbaum.
• Bowers, J. S. (2002). Challenging the widespread
  assumption that connectionism and distributed
  representations go hand-in-hand. Cognitive
  Psychology., 45, 413-445.
• Evans, J. S. B. T. (2003). In Two Minds Dual
  Process Accounts of Reasoning. Trends in
  Cognitive Sciences, 7(10), 454-459.
• Fodor, J. A., Pylyshyn, Z. W. (1988).
  Connectionism and Cognitive Architecture A
  Critical Analysis. Cognition, 28, 3-71.
• Hinton, G. E. (1990). Special Issue of Journal
  Artificial Intelligence on Connectionist Symbol
  Processing (edited by Hinton, G.E.). Artificial
  Intelligence, 46(1-4).
• O'Reilly, R. C., Munakata, Y. (2000).
  Computational Explorations in Cognitive
  Neuroscience Understanding the Mind by
  Simulating the Brain. MIT Press.
• McClelland, J. L. (1992). Can Connectionist
  Models Discover the Structure of Natural
  Language? In R. Morelli, W. Miller Brown, D.
  Anselmi, K. Haberlandt D. Lloyd (Eds.), Minds,
  Brains and Computers Perspectives in Cognitive
  Science and Artificial Intelligence (pp.
  168-189). Norwood, NJ. Ablex Publishing Company.
• McClelland, J. L. (1995). A Connectionist
  Perspective on Knowledge and Development. In J.
  J. Simon G. S. Halford (Eds.), Developing
  Cognitive Competence New Approaches to Process
  Modelling (pp. 157-204). Mahwah, NJ Lawrence
  Erlbaum.
• Page, M. P. A. (2000). Connectionist Modelling in
  Psychology A Localist Manifesto. Behavioral and
  Brain Sciences, 23, 443-512.
• Pinker, S., Ullman, M. T., McClelland, J. L.,
  Patterson, K. (2002). The Past-Tense Debate
  (Series of Opinion Articles). Trends Cogn Sci,
  6(11), 456-474.