Three Kinds of Learning

1
Three Kinds of Learning

• Chrisantha Fernando
• Marie Curie Fellow, Collegium Budapest

2
Aim

• I present some examples of learning
• Associative learning
• Causal inference
• Insight based problem solving
• My aim is to understand the mechanisms underlying
  these learning behaviours.

3
Classical Conditioning
Sound of Metronome (CS)
Smell of Food (US)
Salivation (Response)
Ivan P. Pavlov (1927)
4
Light touch (CS)
Withdrawal (Response)
Electric Shock (US)
Hawkins et al, 1989
5
Coincidence detectors
Pre-Synaptic (Eccles)
Post-Synaptic (Hebb)
6
Paramecia
7
What could they learn?

• Temperature change precedes O2 change in marine
  ecosystems by 20 minutes.
• Photon flux may precede temperature changes.
• Aerobic to anaerobic respiration from mouth to
  gut (signaled by increasing temperature).

8
David S Goodsell, 1998 The Machinery of Life
9
Previous Work
na A AB
nb B AB
Gandhi et al, 2007
10
A Simple Learning Circuit

• In collaboration with molecular biologists, I
  have designed Hebbian learning circuits for
  plasmids carried by E. coli.

v w.u dwi/dt uiv
11
A Gene Regulatory Network
12
(No Transcript)
13
Output P
No pairing
Pairing
14
Artificial Evolution in Silico
SBMLEvolver Synthetic Biology Toolbox
15
(No Transcript)
16
Evolution in vivo What will evolve?
17
Fitness function
18
Presynaptic
Subtractive Norm
Oja (L2 Norm)
BCM
Weight consolidation
19
MAPK Implementation
20
Why?
21
Later.
22
A
R
A
R
C
C
B
B
Susan
Jeffery
Learns patient specific (contingent) associations
23
Similar Recent Work
Tagkopoulos et al, 2008 (in press)

• Prediction, but no associative learning.

24
Limitations of Associative Learning

• Can only learn associations between pre-specified
  stimuli
• Learns only associations, not cause and effect
  relationships.

25
A Concept of Cause and Effect

• My argument is that causal understanding gave
  rise to tool-making that was the evolutionary
  advantage. It's tool-making that's really driven
  human evolution. This is not widely accepted, I'm
  afraid, but there's no question about it. It's
  tools that really made us human. They may even
  have given rise to language.

Lewis Wolpert, 2007
26
What is causal Inference?

• Does dropping a coin into a tin of coins cause
  the number of coins in the tin go up?
• Can moving a piece on a chess-board cause the
  opponent's queen to be pinned?
• Can ignorance cause poverty?
• Can poverty cause crime?
• Can ignorance cause a TV set to be moved through
  a broken window?
• Can inserting a certain sort of twig in a certain
  way into a particular partly built nest cause the
  nest to become more rigid?
• Analysing the concept of causation is probably
  the hardest unsolved philosophical problem. It's
  at the root of problems about relations between
  mind and body (or relations between virtual and
  physical machines).

Sloman, 2008 (pc)
27
Causal Inference

• What is the difference between causal inference
  and associative learning?
• Weak To utilize more than pair-wise correlations
  (perhaps unconsciously).
• Strong Combining observation of conditional
  probability P(XY) with novel appropriate
  interventions
• i.e. why dont Pavlovs dogs spontaneously ring
  the bell when they are hungry? (without
  reinforcement).
• Humans do generate hypotheses based on CP and
  produce interventions to test causal models.
• Parties -gt Wine -gt Insomnia
• Wine lt- Parties -gt Insomnia

28
Structuring Interventions

• A --gt B --gt C --gt D
• Intervene at C A-gt B C--gtD
• A lt-- B lt-- C lt-- D
• Intervene at C A lt-- B lt-- C D

29
Algorithms exist to discover causal networks.

• Bayesian learning
• Know prior probability of causal graphs
• Know probability of observations given each graph
• Use Bayes theorum to calculate probability of
  graph given observations and priors
• Fined the best graph
• Constraint-based learning
• For each pair of variables a and b in V, search
  for a set Sab such that (a _ b Sab) holds in
  P, i.e. a and b should be independent in P,
  conditioned on Sab. Construct an undirected graph
  G such that vertices a and b are connected with
  an edge iff no set Sab can be found. Connect
  dependent nodes
• For each pair of non-adjacent variables a and b
  with a common neighbor c check if c is an element
  of Sab. If it is continue, if it is not then add
  arrow heads pointing at c i.e. a--gt c lt-- b.
• In the partially directed graph that results,
  orient as many of the undirected edges as
  possible subject to two conditions
• i. the orientation should not create a new
  v-structure,
• ii.the orientation should not create a directed
  cycle.

30
Crows

• What is the evidence for causal understanding in
  crows?

31
(No Transcript)
32
Im not convinced

• Tool use is innate in New Calodonian crows no
  social learning is required.
• Crows can make the right length and thickness of
  tool for the right hole, on the first trial. This
  could still be associative learning.
• Must exclude simple strategies, e.g. random
  search, win-stay loose shift, reinforcement
  learning, operant conditioning, etc
• Rather than giving subjects a defined set of
  choices, they are placed in a situation where
  they have a low probability of solving a task by
  chance alone (for example, in a hook making task
  an animal may be given a piece of pliable
  material that can be changed into an infinite
  number of shapes, but only a small subset of
  these shapes would be functional)
• How does one define the null-hypothesis, e.g.
  what is the probability of manufacturing a
  hook-shaped object by chance alone?

33
Both Betty and Bob use trial and error search.
34
Causal Inference in Rats

• What is the evidence for causal inference in rats?

35
Causal Inference in Children

• What is the evidence for causal inference in
  humans?
• Understanding interventions (monkey sneezing
  blickets, etc.)
• A --gt B- --gt AB --gt AB (Children choose A)
• A --gt A --gt A --gt B- --gt B --gt B (Choose
  randomly)
• Retrospective disambiguation (by children)
• e.g. AB --gt A-, AB --gt A

36
Gopnik Schultz, 2004
37
Gopnik Schultz, 2004
38
Gopnik Schultz, 2004
39
Our approach

• To study intra-brain causal inference.

40
Insight in Humans
41
How to Solve it?

• What is an insight problem? A problem that
  requires restructuring of the initial problem
  representation, e.g. goal states, operators,
  constraints.
• What kinds of algorithm are used to solve these
  and related problems? What determines the set of
  goal, operators, constraints?

42
Missionaries and Cannibals

• 3 missionaries and 3 cannibals must cross a river
  using a boat which can carry at most two people.
• For both banks, if there are missionaries present
  on the bank, they cannot be outnumbered by
  cannibals.
• The boat cannot cross the river by itself with no
  people on board.

43
Intermediate goals may be used

• Early moves balance number of M C on each side
  of river.
• Intermediate moves maximize progress from one
  side to other.
• Later moves avoid revisiting previous states.

44
There is some evidence

• Non-maximal moves that allow a subsequent move to
  make more progress are retained as promising
  states for future trials.
• Goal criteria are relaxed and changed based on
  the quality (immediate benefit) of generated
  solutions.
• Sometimes hill-climbing to a wrong goal
  criteria can get stuck in local minima.

45
3 moves

• 7426 legal 3-move sequences
• 2 reach ring solution
• 176 reach 2 group solution
• 7426 sequences are not eqiprobable under random
  selection assumption.
• lt 1/3 participants solve problem within 10
  minutes.
• Choice of the correct first move based on the
  improved goal scores available from the second
  move was crucial.
• Few subjects even conceived of a two group
  solution when asked to produce a shape where
  each coin only touches two others.

CHRONICLE, MACGREGOR, AND ORMEROD,2004
46
Brain damage helps some problem solving!
II III I Type A IV III - I Type B VI VI
VI Type C
Solutions here
Reverberi et al 2005
47
Our approach

• To study mechanisms for restructuring of problem
  representations.

Poelwijk et al 2007
48
Conclusions

• What neural mechanisms underlie causal inference,
  and solving insight problems?
• What changes allow humans to have these
  capacities but precludes other apes from having
  them?
• What algorithms can best predict human
  performance in such problems?

49
Thanks to

• Eors Szathmary
• Lewis Bingle
• Anthony Liekens
• Aaron Sloman
• Jon Rowe
• Dov Stekel
• Christian Beck Thorsten Lenser