Project ArteSImit Artefact Structural Learning through Imitation (TU M

1
Project ArteSImit Artefact Structural Learning
through Imitation(TU München, U Parma, U
Tübingen, U Minho, KU Nijmegen)Giorgio Panin -
TUM
2
Artesimit Project Partners

• Alois Knoll and colleagues (project coordinator,
  Robotics, Munich)
• Giacomo Rizzolatti and colleagues
  (Neurophysiology, Parma)
• Peter Thier and colleagues (Cognitive
  Neuroscience, Tübingen)
• Wolfram Erlhagen and colleagues (Cognitive
  Modeling, Minho)
• Harold Bekkering and colleagues (Cognitive
  Neuropsychology, Nijmegen)

3
The concept of goal-directed Imitation

• Central issue of imitation
• do we imitate movements (means) or consequences
  (goals) of the observed behavior?
• Movements somehow need to be directly mapped
  (via-point approach), whereas goals need to be
  represented.
• Movement mapping assumes homologue bodies, full
  access of the whole movement and stable
  environments.
• Goal representation assumes an association
  between an (observed) action and the (observed)
  consequence.

4
How to work together?

• Behavioral measurements (healthy and clinical
  patients)
• Brain research (single-cell, fMRI, MEG, EEG)
• Modeling (Dynamic neural fields)
• Robotics (Visual analysis, cognition, and
  artefact control)
• THE solution One common paradigm

5
Common paradigm
Movement 1
Starting position
Movement 2
A LED gives information about either the goal or
the means
6
Parietal Mirror Neuron findings
Motor responses of parietal neurons
Motor task
Visual responses of parietal mirror neurons
Observation task
7
Architecture of the dynamic model
Bridge Paradigm
PFC Goal representations Task input Object
properties (e.g. colour) STS Visual
desciption of grip and trajectory PF Sequence
of means F5 Movement primitives
8
Visual system in action Task recognition
Complete information acquisition for the decision
and planning tasks
9
Learning I Association between object properties
and goal
10
Learning II Copying the means of the teacher
11
Goal-directed imitation
12
Partial information Cueing of the goal
13
Partial information Cueing of the means
14
Summary

• The project combined ...
• the development of a new paradigm for
  goal-directed imitation with imaging studies
  across all modalities
• new findings in experimental neurophysiology and
  advanced neuromodelling of the structure of the
  PFC-(STS-PF-F5)-coupling to develop and implement
  computer-operational models for goal-directed
  imitation
• the development of a powerful model-based hand
  posture recogniser (using only one B/W camera)
• into ...
• A complete artefact (robot system) capable of
  mimicking imitation behaviour of creatures all
  project goals have been met!