Connectionism and LOTH

1
Connectionism and LOTH
Connectionism

Language of Thought
2
Quizzes

• Some common problems
• LOTH does not require mentalese.
• Face recognition is innate?
• Homuncular fallacy vs. homuncular functionalism.
• Folk psychology originated in Ancient Greece?

3
Grading

• 70 A
• 60-69 B
• 50-59 C
• 40-49 D
• Below 40 F

4

• Remember
• This quiz is only 25 of your final grade.
• 1st paper 25
• 2nd paper 40
• Tutorial performance 10
• So, if you did badly, dont be too discouraged.
• But, if you did well, dont be too cocky!

5
How is connectionism an alternative to LOTH?

• LOT usually represented as implemented by
  classical AI. (Also known as GOFAI good,
  old-fashioned AI.)
• Semantic symbols and syntactic rules are easy to
  represent in classic AI architecture.
• Connectionism does not require symbols, but
  representations can be symbolic.

6

• Types of Connection Representations
• 1) Local representation.

Meows Fur Pointed ears Whiskers
Output its a cat
This node is a local representation of cat.
7

• Local representations
• Individual nodes are symbols, and can be
  components of a language of thought.
• Not typical of connectionist networks.
• Not so biologically plausible
• -- Grandmother neurons

8

• 2) Distributed representations.
• e.g.

Cat Tiger Leopard Lion
See also www.mind.ilstu.edu/curriculum/nature_o
f_computers/computer_types.php
9

• Distributed representations
• Connectionist networks are typically distributed
  representations.
• Distributed representations are not necessarily
  symbolic.
• Distributed representations are more robust to
  damage than local representations.

10

• 3) No representation.
• More controversially, connectionist networks can
  have no representational properties.
• Note
• Output of connectionist network may be
  recognition of a concept, e.g. cat, mine, man,
  etc. but
• Output of connectionist network may be action,
  e.g. moving through space, reading aloud
• Rather than representing content, networks can
  just act.

11
Comparison

• What goes on in your mind when you
• decide to drink a glass of water that
• is in front of you?
• LOTH the action is the conclusion of a practical
  syllogism conducted through symbol manipulation
• Connectionism the action is output of a neural
  net responding to a certain set of inputs

12

• LOTH approach
• I am thirsty.
• There is a cup of water in front of me.
• I believe that drinking the water will relieve my
  thirst.
• (There are is no reason not to drink the water)
• Conclusion I drink the water.
• The conclusion is reached after manipulating the
  semantic symbols representing beliefs and desires
  in accordance with syntactic laws.
• Beliefs and desires give rise to action.

13

• Connectionist approach
• Inputs from body Inputs from
  environment

Output I drink the water. There
are no symbols involved.
14

• Connectionism makes eliminativism possible.
• Note in the connectionist/eliminativist
  approach, the mind concocts the belief-desire
  explanation, I drank the water because I was
  thirsty to explain its behavior.
• But the desire (thirst) and beliefs (the water
  is in front of me, the water is safe to drink,
  the water will relieve my thirst) are not
  literally part of the process whereby the mind
  decides to drink.
• In other words, the mind only uses symbolic
  representation when translating/explaining its
  thoughts in language (talking to oneself or
  talking to others).

15

• But how can thirst not play a role in deciding
  to drink? Isnt it part of the input from the
  body?
• Thirst is a feeling. What plays the functional
  role of thirst may be a mechanism to detect
  that the body is low on water, or is somewhat
  overheated, but this may not be recognized by you
  as a desire, until you try to explain your own
  behavior.
• Note imagine reaching unconsciously for a glass
  of water, and when someone asks, why are you
  drinking that?, you say, I guess I was
  thirsty.
• The explanation could be rather different than
  the cause.

16
Advantages of Connectionism

• 1) Biological plausibility
• Connectionist networks are deliberately analogous
  to neural processes in the brain
• Units neurons
• Connections synapses
• Activations neural signals

Neuron Connectionist unit
17

• 2) Fast Parallel Processing
• Neurons are slow.
• Neurons change state very slowly compared with
  computer computations. Neurons can only process
  100 changes a second, whereas computers can
  process a million. But the brain can solve many
  complex problems in less than a second, e.g.
  recognizing a face.
• 100 Steps Rule
• To mimic brain operations, computer programs
  should solve similar problems in less than 100
  steps.
• Connectionist programs are conducted through
  parallel processing, thus more can be done in 100
  steps.

18

• 3) Performance of connectionist networks
  resembles performance of human brains
• Connectionist networks are good at
• Pattern recognition networks can learn through
  examples
• Content-addressable memory items can be
  retrieved based on their meanings or properties
• Generalizations networks can generalize
  connections between characteristics or properties

19

• Connectionist networks exhibit
• Graceful degradation
• When a connectionist network has some incorrect
  input -- noisy input -- or is partially
  damaged, it stills performs more poorly, but
  doesnt completely break down.

20

• 4) Connectionism provides a naturalistic
  mechanism for creating concepts.
• No need to posit inborn concepts.
• Concepts can precede language without being
  inborn.
• Fodor once claimed that mentalese was the only
  game in town.
• Connectionism is a new game!

21
Criticisms of connectionism

• The advantages of connectionism revisited
• Biological plausibility
• 100-steps rule
• Pattern recognition and concept formation yes,
  but very slow

22

• Biological plausibility
• Networks arent really like neurons.
• No reverse connections (necessary for backward
  propagation) in the brain.
• Neurons only fire or not they cannot be both
  inhibitory and excitatory.
• Connectionist units are too fast, neurons are
  quite slow.

23
Biological plausibility (cont.)

• There are many different types
• of neurons in the brain, but
• connectionist units are meant
• to represent all neurons.
• In addition, role of
• neurotransmitters and
• hormones in thinking is ignored
• in connectionist models.
• Note most people admit that connectionist
  networks are still more biologically plausible
  than classical AI architectures.

Different types of neurons
24

• 2) The 100-steps rule
• Problem what is a step?
• Is, recognizing a color one step? Or does it
  break down into numerous steps?
• The 100-steps rule only works if each unit of a
  connectionist network corresponds to one neuron.
• If one unit corresponds to several neurons
  working together, the 100-steps constraint may be
  greatly exceeded.
• Also, the 100-steps argument assumes only
  connectionist architectures are parallel
  processors, and all classical architectures are
  serial. But it is possible to build parallel
  classical architectures.

25

• 3) Network learning is extremely slow
• Connectionist networks need a huge amount of
  explicit feedback to learn.
• The brain often can learn a new concept or
  pattern in one shot.
• One-shot learning is especially easy when
  information is gathered through language.
• Example think of teaching an intelligent chimp
  vs. a five-year-old child, to push the red button
  for food.

26
Another weakness of Connectionism

• Systematicity and productivity very difficult
  (impossible?) to implement in connectionist
  architecture.
• Connectionist responses
• Deny systematicity and productivity of the mind
• Is human thinking really systematic/productive?
• Do animals think systematically/productively?
• Maintain the ability of connectionist nets to
  generate systematicity and productivity

27
The Relationship between Connectionism and LOTH

• Three possibilities
• Connectionism implements LOTH
• Connectionism replaces LOTH
• Hybrid theory. Some mental processes are
  connectionist, some are conducted through LOT.

28

• Connectionism implements LOT
• Connectionist nets can be regarded
• as a lower-level implementation
• of LOT.
• Neural nets can represent semantic
• symbols which are then manipulated in accordance
  with language-like laws (also implemented by
  neural nets).
• Criticism if connectionist nets only implement
  LOT, many of the advantages of connectionism are
  lost.

29

• 2) Connectionism replaces LOT.
• Consequence all the advantages (e.g.
  systematicity and productivity) of LOT are lost.
• Can we do without them?

30

• Hybrid theory.
• Some mental processes are connectionist, some
  are conducted through LOT.
• e.g.
• Perception, pattern recognition and motor
  control handled by connectionist nets.
• Reasoning and language handled by LOT (and
  implemented by connectionist nets).

31
Connectionism and Modularity

• Connectionist networks can do simple, small
  tasks.
• In more complicated tasks, they are overwhelmed
  by the complexity (because the connections
  increase exponentially).
• Mind must be organized into simple units,
  connected up in an efficient way.
• Connectoplasm the mind an unorganized mess of
  connections. Not a viable idea.
• Mental modules some connections preset, some
  learned. A way to contain the complexity.

32
Readings for next week

• Required
• Thomas Nagel (1974), What is it like to be a
  bat?, The Philosophical Review, LXXXIII, 4
  (October 1974), 435-50
• at members.aol.com/NeoNoetics/Nagel_Bat.html
• Block (2002), Some Concepts of Consciousness,
  in David Chalmers (Ed.). Philosophy of Mind
  Classical and Contemporary Readings Oxford
  University Press
• at www.nyu.edu/gsas/dept/philo/faculty/block/pap
  ers/Abridged20BBS.htm
• Optional
• Gallup, Jr., Povinelli (1998). Can Animals
  Empathize? Scientific American - Exploring
  Intelligence (a debate), available on reserve at
  Philosophy Department