BUILDING A COGNITIVE SYSTEM BY GNOSYS

1
BUILDING A COGNITIVE SYSTEM BY GNOSYS

• Co-ordinator John Taylor (KCL)
•  Asst Co-Ordinator Stathis Kasderidis (FORTH)
• EC PO George Stork
• Start date Oct 1 Kick-off Oct 20/21
• gnosys_at_ics.forth.gr
• Web-site http//www.ics.forth.gr/gnosys/
• Department of Mathematics
• Kings College London, UK
• emails john.g.taylor_at_kcl.ac.uk

2
CONTENTS

1. Vision of GNOSYS
2. GNOSYS Partners
3. GNOSYS Prototypes
4. GNOSYS Tasks/Milestones
5. GNOSYS Summary

3
1. VISION OF GNOSYS

1. Embodied cognition (wheel-robot gripper)
2. Create concepts/rewarded-goals under attention
   control
3. Learns goal-directed tasks
4. Learns novel environments
5. Reasoning by forward models
6. Guidance from brain (animal/infant/adult)
7. Various memory types (STM/LTM/associative/error-ba
   sed)
8. Interdisciplinary Comp vision/ Cog NSci/ Neural
   Networks/ Robotics/ AI/ Maths

4
General Work Plan of GNOSYS
5
GNOSYS Cognitive Powers

• Feature-based perception (M1-16) WP2
• Concepts/Goals/Attention (Sensory Motor)
• (M6-18, 12-24) WP2/WP3
• Rewarded drive-based learning (M12-24) WP2
• Goal-based Global Computation (M6-18) WP2
• Abstraction Hierarchy (M12-24) WP3
• Reasoning/Action Planning by motor attention-base
  forward models(M18-33) WP3
• Robot Platforms _at_ 2 levels (M18/M30)

6
HOW GNOSYS WORKS

• v v
• ?
• ?
• ?
• ?
• ?

ANN Adaptive Stream (Concepts/ Goals/ Attenti
on/ Rewards/ Values/ Forward Models learnt as
NN predictors)
Symbolic Control Threads (5
components)
Linguistic Connections (Words/Fuzzy
rules/ Symbolisation)
Relate to COSPAR
7
Drives/Motivation/Rewards

• Assign values (in AMYG/OBFC) as direct input
  (learnt), or by DA modulation from primary
  rewards (satisfying basic drives)
• Basic drives for GNOSYS
• Energy level/ Curiosity/ Stimulation/ Minimum
  pain (touch/pressure)/ Approbation/ Motor
  activity
• Use value maps --gt assign value to stimuli

8
2. GNOSYS PARTNERS

• 1 Kings College London (KCL) Comp Nsci Grp NNs,
  concepts, attn control
• ZENON S.A., Greece (ZENON) robots
• 3 Foundation of Research Technology - Hellas
  Greece (FORTH) global comput/robots
• 4 Eberhard-Karls-Universität, Tübingen, Germany
  (UTUB) perception/reward/robots
• 5 Università di Genova, Dipartimento di
  Informatica, Sistemistica, Telematica, Italy
  (UGDIST) motor control/robots

-gt RobotCub
9
Attentional Agent Architect (EC FP5 DC,
2001-2003)

• Distributed entity with four layers (attentional
  multi-level agent)
• L1 Sensors
• L2 Pre-processing
• L3 Local decision
• L4 Global decision

10

• GLOBAL CONTROL ARCHITECTURE
• EXTENDED ATTENTION V EMOTION ARCHITECTURE (EC
  ERMIS, NF, 2002-4 BBSRC 2004-7)
• (extended Corbetta Shulman, 2002)

Endogenous goals
Excitatory/Inhibitory
Exogenous goals
Inhibitory Interaction through ACG
Excitatory
Excitatory interaction
Inhibition from DLPFC In emotion recognition
11
MOTOR CORTEX ACTION NETWORK (NT, MH, OM JGT)
(in NetSim for sequence learning tested in PDs
J NSci24702 )
FROM OTHER CORTEX OTHER THALAMUS
MOTOR CORTEX
TO OTHER CORTEX
FROM CEREBELLUM
STRIATUM
NUCLEUS RETICULARIS THALMUS
CENTROMEDIAN PARAFISCULAR NUCLEUS
SUB-THALAMIC NUCLEUS
THALAMUS
GLOBUS PALLIDUS EXTERNAL
GLOBUS PALLIDUS INTERNAL
SUBSTANTIA NIGRA PARS COMPACTA
SUBSTANTIA NIGRA PARS RETICULARIS
GLUTAMATERGIC INPUT
SIMILAR STRUCTURES MODEL OBFC, DLPFC, ACG AND
VLPFC
GABAERGIC INPUT
DOPAMINERGIC INPUT
12
Cerebellar Structure Associated Regions For
Insertions,by error-based learning (with teacher)
GrC granule cells GoC golgi cells BK basket
cells PK purkinje cells DCN deep cerebellar
nuclei (excit. inhib.) IO inferior
olive PONS pontine nuclei HIPP hippocampal
regions PFC pre-frontal cortex inhibitory
conns. excitatory conns.
HIPP
PFC
13
HIPPOCAMPUS AMYGDALA (in NetSim for sequence
learning, and x20 speed-up in SWS) (MH, NT
JGT) as teacher
14
EPSRC Ventral Dorsal Concept Learning (-gt
GNOSYS)
Ventral pathway
Dorsal pathway
TE
TEO
V4
LIP
V5
V2
V1
V1
LGN Input
Learning
Currently Hard-wired
Hard-wired
LGN Input
15
Architecture Details Percepts

• V1 4 excitatory inhibitory layers for bar
  orientations, hardwired (1414)
• V2 (2828) trained on reduced set of pairs of
  bars (6), start positions in retina 121
• V4 (2828)-gtTEO (2828/1414)-gtTE (77) trained
  on 2 different triangles (121 start positions)
• Now by cluster computing
• Next step to DL/VLPFC as goals-gt attention

16
ERMIS/BBSRC GLOBAL BRAIN CONTROL by ATTENTION

Fusiform Gyrus
VCX
PL
PFC
PL
ACG/TPJ
PL
-gt Simulated Attentional Blink NF/JGT -gt
Consciousness by CODAM (Prog Neurobiology 03)
17
Model of Visuo-Motor Attention Control System
(JGT NF, IJCNN03)
-gt MACS for Attention filtering
-gtMINDRACES for anticipation
18
AB extended by AMYG as bias ERPs for T2 in Lag3
when no amygdala
19
ERPs for T2 in Lag3 amygdala input from T2s
object rep, fed back to same site
20
UGDIST Biomimetic trajectory formation via
artificial potential fields
the importance of smoothness and continuity
Tsuji T, Tanaka Y, Morasso P, Sanguineti V.
Kaneko M (2002) IEEE Trans SMC-C, 32,
426-439. Morasso P, Sanguineti V, Spada G (1997)
Neurocomputing, 15, 411-434
21
Real-time control of robot motion by
sub-symbolic neural activity
the importance of bidirectional communication
From the Neurobit project
22
Robotized haptic interface
the importance of softness and a soft touch
23
Computational Vision and Robotics Lab
(CVRL)Institute of Computer ScienceFoundation
for Research andTechnology Hellas (FORTH)
24
CVRL - FORTH

• Mission Study the mechanisms involved in the
  development of autonomous robotic systems

25
CVRL - FORTH

• Current RD activities  
• perceptual competences based on visual and range
  sensors and sensor fusion techniques
• coupling of perception and action
• autonomous navigation and control of complex
  robotic systems
• development of networked robotic systems
• content-based retrieval of images and video
• Future activities
• development of robotic behaviours that simulate
  corresponding behaviours of living organisms
• emergence of cognition in artificial systems
• complex heterogeneous robotic systems involving
  multiple robots

26
UTUB Experienced in robot movement and planning
Involved in GNOSYS perceptions
rewardsZENONRobotics Company in
AthensExperienced in robot applicationsTo
construct robot platforms (2)
27
3. GNOSYS PROTOTYPES

• PROTOTYPE I (M18) Attn control of sensory inputs
  response
• Learn concepts of simple shapes 3 rewarded
  actions, under attention
• Responses to commands/learn new goals as new
  actions on new objects
• PROTOTYPE 2 (M28) As above but more complex
  objects 3 sequences of action/object pairs
  in real scenes forward models for virtual goal
  seeking (reasoning)

28
4. GNOSYS TASKS, etc Reasoning
Domains/Environments (WP23)

• Three levels of environment
• Level 1 Learn shapes/colours move touch move
  pick up 2 3-D objects
• Powers Concept/Attn/Goals as actions on
  objects/Valence of objects in environment
• Level 2 Complex objects actions
• Powers ibid/manipulate to achieve goals
• Level 3 Hierarchy of objects run virtual
  object/action sequences to achieve goals
• Powers Reasoning/ novel objects/actions

29
Application to Patrolling, etc

• Construct loc/action and object/action map in
  patrol environment
• Reasoning tasks to discover actions (loc1,
  action)?loc2, (obj1,action)?obj2
• Meets barrier of boxes. Reasoning move box to
  pass through, instead of moving round barrier
• Over pond reasoning find plank to put across
  pond
• Plus many psychological tasks (WCST/Tower of
  London, etc, etc)

30
MILESTONES

• Level 1 Simple actions stimuli 2 (M6)
• Level 2 More complex actions stimuli
  3/colour/motion/audition/touch (M16)
• Level3 Real-world stimuli (M24)
• Prototype 1 (M18)
• Prototype 2 (M28)
• Assessment (M34)

31
5. GNOSYS SUMMARY

• Create concepts/goals by learning
• Can handle novel environments
• Embodied cognitive system
• Learning by infant-style development (by
  hierarchy of modules sequentially coming on line)
  
• Reasoning by forward models created by
  reward-based learning