Dreschers Schema Mechanism and Chaputs CLA

1
Dreschers Schema Mechanism and Chaputs CLA

• Presented by
• Jonathan Mugan

2
Constructivism

• Jean Piaget (1896-1980)
• Child starts out with almost no a priori
  knowledge
• Intelligence constructed bit by bit increasing in
  sophistication
• Compare with nativism

3
Two challenges

• Empirical learning the same action has different
  effects in different situations
• Concept invention how to define radically new
  concepts

4
The schema

• Schema ltcontext/action/resultgt
• Context and result consist of items
• Items are propositions that can be On or Off
• InFrontOfDoor/OpenDoor/DoorOpen

5
Empirical learning via marginal attribution

• Each primitive action is initially a bare schema,
  lt/OpenDoor/gt
• Find a result item that is relevant to an action,
  lt/OpenDoor/DoorOpengt
• Find context items that make the result reliable,
  ltInFrontOfDoor/OpenDoor/DoorOpengt

6
Result spinoff

• Each schema maintains an extended result
  consisting of all items
• For each item, maintain the statistic
  P(?itemaction)/P(?itemnot-action)
• If it goes above a certain threshold spin off a
  new schema with that result item

7
Context spinoff

• Each schema also maintains an extended context
  consisting of each item
• For each item, maintain the statistic
  P(successitem)/P(successnot-item)
• If it goes above a certain threshold spin off a
  new schema with that context item added

8
Composite actions

• Low level actions can be built up to high level
  actions, like a subroutine
• Each time a new result is obtained, add a
  composite action for it and set up a bare schema
  with that action
• This sets up a controller that does backchaining,
  closest applicable action is always chosen

9
Concept invention synthetic items

• If a schema is unreliable but locally consistent
  then a synthetic item is created
• When the synthetic item is On then the schema is
  likely to succeed
• Door unlocked InFrontOfDoor/OpenDoor/DoorOpen
  

10
The microworld
11
What it learns

• To grasp, lt/grasp/hclgt
• That moving the glance moves objects in visual
  field, ltvf21/eyer/vf11gt
• The network for moving the hand and eye,
  ltvp22/eyeb/vp21gt
• Intermodal coordination, schemas chain to hp22 so
  that the robot can suck its thumb,
  lthp22/handb/taste1gt

12
What it learns (2)

• Towards a concept of persistent objects,
  /hp23/tactl

13
Computationally intensive

• 16,384 SIMD processors, each 32KB memory (524 MB
  total memory)
• Ran out of space after 10,912 time units
• Ran for about a day
• 7,371 schemas
• 184 synthetic items
• 343 composite actions
• 7,371184k updates per action

14
The Constructivist Learning Architecture (CLA)

• A hierarchy of SOMs to model cognitive
  development
• A more efficient implementation of Dreshers
  schema mechanism using SOMs

15
SOM review

• A SOM is a grid with nodes, and each node has
  neighbors in the grid
• Each node has a weight vector
• For each training vector find the node with the
  closest weight vector, make the weight vector of
  that node closer to that of the training vector,
  and also do the same for the neighbors, to a
  lesser degree

16
Hierarchy of SOMs
17
Different configurations

• The configurations of the SOMs are created
  manually or by some other process
• They can be in a tree structure or a time delay,
  or whatever structure is desired.
• When input vector to a level doesnt match well
  with SOM, fallback to earlier level

18
CLA for the schema mechanism (CLASM)

• Each action associated with a SOM
• Node weight vectors consist of items
• Train
• Harvest schemas and get synthetic items and
  composite actions
• Repeat

19
Training

• Each primitive action associated with a SOM
• Each node has 2num-items weights
• Context items between 0.0 and 1.0
• Result items between -1.0 and 1.0
• Trained when action taken
• Training vector consists of items before action
  and ? results after action

20
Harvesting

• Context lt 0.1 neg item, gt 0.9 pos item
• Result ? gt 0.9 become result items
• Schemas with results are harvested

21
The next stage

• All novel results become composite actions
• All schemas become synthetic items
• New SOMs are created for each action (including
  composite actions)
• The weights at each node include all the items
• Train again

22
Beyond the microworld
23
Robot up close
24
fin