Scenario Specification and Problem Finding

1
Scenario Specification and Problem Finding

• Luca Tummolini
• ISTC-CNR

2
Objective

• From the Annex 1 identify 6 scenarios
  corresponding to the situation in which cognition
  requires anticipation and implement 3 of them
• Scenarios are used for
• Understanding the role of anticipation in
  cognitive systems
• Identify the challenges
• Facilitate future integration

3
The scenarios

• Finding and Looking for
• IDSIA, IST, ISTC-CNR, NBU, UW-COGSCI
• Anticipation in a dynamic world
• LUCS, OFAI, UW-COGSCI
• Guards and Thieves
• ISTC-CNR, LUCS, NBU, OFAI
• The scenarios highlight the role of anticipation
  in
• Perception and categorization
• Selective attention
• Deliberation
• Action monitoring and control
• Skill learning and routinization
• Coordination
• Emotions

4
Scenario 1 Finding and Looking for

• Partners involved IDSIA, IST, ISTC-CNR, NBU,
  UW-COGSCI
• Two different assumptions
• the agent has a model of the environment
• the agent learns the model of the environment
• The agent
• a pan-and-tilt camera mounted on a fixed base
• a simulated fovea (higher resolution at the
  centre) on a mobile robot
• a robotic arm

5
Scenario 1 Finding and Looking for

• Task 1 Finding a specific object
• Find and reach a specific object in the
  environment (e.g. a green circle).
• This task involves
• Selective attention
• Anticipation based on memory
• Goal-based motor-primitives
• Detection of regularities in the spatial
  relations among objects

6
Scenario 1 Finding and Looking for

• Task 2 Finding members of a class of objects
• Find an object transferring a system of relations
  from one domain to another
• This task involves
• Anticipation by analogy
• Detection of relations among objects
• Relation extraction and encoding
• Categorization of observed objects

7
Scenario 1 Finding and Looking for

• Task 3 Looking for an object
• Look for an object that is hidden in one of the
  rooms in a believable way
• The target location can be probabilistically
  biased towards certain locations
• This task involves
• Prediction by analogy making
• Decision making about the rooms to visit
• Building models of the world structure and of the
  regularities between objects
• Emotional responses to expected and unexpected
  events

8
Scenario 2 Anticipation in a Dynamic World

• Partners involved LUCS, OFAI, UW-COGSCI
• This scenario involves prediction of objects with
  an intrinsic dynamics
• The first three tasks involve looking at two
  games with moving objects and the task is to
  predict the movement of one or several targets
• The last task deals with the prediction of a
  rolling ball from a developmental perspective

9
Scenario 2 Anticipation in a Dynamic World

• Task 1 The fish catching game
• Goal anticipate location and velocity of some
  targets in the scene
• The movement of the targets is very regular
• Prediction from different point of views
• This task involves
• Selective attention
• Target tracking based on anticipatory model
• Tracking in a fixed environment
• Quick relearning of the model when the viewing
  angle changes

10
Scenario 2 Anticipation in a Dynamic World

• Task 2 Marble run game
• Goal anticipate location and velocity of some
  targets in the scene
• Paths can be re-arranged
• Combination of the continuous dynamics of the
  ball with a compositional structure
• Generalization between different configurations
  of the set-up
• This task involves
• Selective attention
• Target tracking based on anticipatory model
• Tracking of partially occluded and moving targets

11
Scenario 2 Anticipation in a Dynamic World

• Task 3 Learning previous two games at the same
  time
• Context sensitive learning
• Priming effects biasing the predictions
• Detection of relevant context
• This task involves
• Generalization of target model from one viewing
  condition to another and between the different
  games
• Context dependent selection of target, predictive
  models and tracking strategy
• Simultaneous modelling, recognition, prediction
  and estimation of viewers position

12
Scenario 2 Anticipation in a Dynamic World

• Task 4 Basic how-to knowledge
• Initial basic interactions with the environment
  driven by basic instincts and motivations
• The robot learns through reinforcement which
  interactions are allowed by the objects in the
  environment
• This task involves
• The acquisition of object generalisations and
  concepts
• Learning of basic dynamical properties of objects
  (e.g. predict where a ball will reappear behind
  an obstacle)

13
Scenario 2 Anticipation in a Dynamic World

• Task 5 Generalisation
• If the ball moves slowly, the robot learns to
  look for the ball on the right side of the wall
• If the ball moves fast, the robot learns to look
  for the ball on the other end of the wall
• If the ball produces a noise, the robot goes to
  look for it behind the wall
• This task involves
• The development of more sophisticated concepts,
  such as object persistence
• Flexible how-to knowledge

14
Scenario 3 Guards and Thieves

• Partners involved ISTC-CNR, LUCS, NBU, OFAI
• Agents can have two different roles (guards or
  thieves)
• Thieves try to steal valuable resources
• Guards protect the valuables restricting access
  to them (e.g. blocking the entrance)
• In some variants of the task, only one of the two
  roles will be played by an anticipatory cognitive
  system
• In the more complex variants, both roles will be
  played by anticipatory agents

15
Scenario 3 Guards and Thieves

• Task 1 Conflict in accessing the valuables -
  simple
• Two agents one thief and one guard
• The valuables are hidden in at least two
  different places
• The session ends either when the thief has
  collected or found all the valuables or when the
  guard has blocked the thief.

• This task involves
• Recognition of the adversary among the moving
  objects
• Prediction of the adversary behaviour
  (avoiding/intercepting)
• Integrating different levels of action control
  (e.g. routinary, reasoning), based on different
  kinds of expectations (e.g. implicit, explicit)

16
Scenario 3 Guards and Thieves

• Task 2 Conflict in the access to valuables -
  complex
• A social task involving several agents several
  thieves and a guard
• The session ends either when all the valuables
  have been collected or found or when the guard
  has blocked all the thieves
• The thieves should be able to distinguish between
  guards (danger) and fellows (cooperation)
• This task involves
• Help and critical help by anticipating others
  needs, actions or capabilities, e.g. by removing
  obstacles or doing part of others work
• Delegating by trusting e.g. an agent can
  explicitly ask another one for help

17

• Thank you for the attention