Humanoid Robots as Cooperative Partners for People

1
Humanoid Robots as Cooperative Partners for People

• paper by Breazeal, C., et al.. (2003)
• MIT Media Lab, Robotic Life Group
• presentation by Kósa Máté Ágoston
• cognitive robotics _at_ Rijksuniversiteit Groningen
  2010

2
building socially intelligent robots

• important implications for how we will be able to
  communicate with, work with, and teach robots in
  the future
• it is a critical competence for robots that will
  play a useful, rewarding, and long-term role in
  the daily lives of people

3
socially intelligent robot

• robots that show aspects of human-like social
  intelligence, based on deep models of human
  cognition and social competence
• contrasted to socially evocative / receptive /
  situated / embedded
• brings research closer to the hard problem of
  artificial intelligence (in small steps)
• Fong, T., Nourbakhsh, I. Dautenhahn, K. (2002)

4
socially intelligent robot

• Why?
• We anthropomorphize by default
• Personality lends coherence and consistence to
  behavior (to know someone is to predict his
  actions)
• Natural learning
• Scalability reflects in trust and sincerity (for
  when it gets out of hand see Blade Runner, Ridley
  Scott 1982)

5
theory of mind

• Assumption
• each participant has a set of mechanisms and
  representations for predicting and interpreting
  others actions, emotions, beliefs, desires, and
  other mental states
• Derived models
• joint attention, representation, empathy,
  intersubjectivity, reason (mental states to
  behavior), inference, social reference etc.

6
Collaborative approach vs. ML

• supervised learning techniques the learning
  algorithm has no a priori knowledge about the
  structure of the state and action spaces, must
  discover any structure that exists on its own
• needs data, time, relatively stable enviroment
• problems with generalizing
• hard to guide for the laic
• bridges machine learning with HMC

7
Collaborative approach vs. Humans

• we are innate teachers
• we have a well-established social signaling ?
• we have infrastructure
• we have an affinity towards interdisciplinarity

8
Social Skills

• reciprocal cooperation is achieved with the goal
  to
• help the instructor maintain a good mental model
  of the learner
• help the learner leverage from instruction and
  guidance to build the appropriate task models,
  representations, associations, etc.
• test of abilities the button task

9
Social Skills

• Communication skill
• Deictic reference
• Joint attention
• Mutual beliefs

10
Communication

• Conversational policies
• Cohen et. al. (1990) argue that much of
  task-oriented dialog can be understood in terms
  of Joint Intention Theory
• Modeled after analysis of master-novice task
• Turn-taking skills
• Modeled after human model, very robust
• Envelope displays (para-linguistic cues)

11
Communication

• same goal and the same plan of execution
• different abilities, tools, partial knowledge
  and different beliefs referring to the state of
  the goal
• communication is necessary to mobilize the
  potential

• Conversational policies
• Cohen et. al. (1990) argue that much of
  task-oriented dialog can be understood in terms
  of Joint Intention Theory
• Modeled after analysis of master-novice task
• Turn-taking skills
• Modeled after human model, very robust
• Envelope displays (para-lingvistic cues)

12
Communication

• same goal and the same plan of execution
• different abilities, tools, partial knowledge
  and different beliefs referring to the state of
  the goal
• communication is necessary to mobilize the
  potential

• Conversational policies
• Cohen et. al. (1990) argue that much of
  task-oriented dialog can be understood in terms
  of Joint Intention Theory
• Modeled after analysis of master-novice task
• Turn-taking skills
• Modeled after human model, very robust
• Envelope displays (para-lingvistic cues)

Organizational markers Elaborations Clarifications
Confirmations Referential elaborations
Confirmations of successful identification
13
Communication

• Conversational policies
• Cohen et. al. (1990) argue that much of
  task-oriented dialog can be understood in terms
  of Joint Intention Theory
• Modeled after analysis of master-novice task
• Turn-taking skills
• Modeled after human model, very robust
• Envelope displays (para-linguistic cues)

14
Deictic reference

• Estimating gaze via estimating head-pose
• pan / tilt / rotation
• objects in 3D spatial map projected on gaze
  vector
• camera on the wall (panoramic view)
• Pointing
• background and depth map extraction
• candidates fit to ellipse, then presence of
  pointing finger is analyzed (kurtosis)
• stereo camera ceiling-mounted (birds eye view)

15
Deictic reference

• Estimating gaze via estimating head-pose
• pan / tilt / rotation
• objects in 3D spatial map projected on gaze
  vector
• camera on the wall (panoramic view)
• Pointing
• background and depth map extraction
• candidates fit to ellipse, then presence of
  pointing finger is analyzed (kurtosis)
• stereo camera ceiling-mounted (birds eye view)

16
Deictic reference

• Estimating gaze via estimating head-pose
• pan / tilt / rotation
• objects in 3D spatial map projected on gaze
  vector
• camera on the wall (panoramic view)
• Pointing
• background and depth map extraction
• candidates fit to ellipse, then presence of
  pointing finger is analyzed (kurtosis)
• stereo camera ceiling-mounted (birds eye view)

17
Joint Attention

• seeing vs. attending (in baby humans 7-9 months)
• referential looking (in baby humans 6-18 months)
• proto-declarative pointing (in b.h. 9-12 months)
• exploiting all these at 14 months of age (in
  b.h.)
• two entities looking at the same thing is not
  necessarily joint attention (necessary-not-suff)
• updating mutual belief with a common referent is
  closer to the human-human model

18
(No Transcript)
19
(No Transcript)
20
Joint Attention

• To keep in mind
• Attention focus (what gets the attention)
• Referent focus (the subject of communication)
• Saliency determines a list, not a particular
  object
• perceptual/internal/socially cued saliency
• Decay of saliency
• Leonardos model of own foci
• Leonardos model of instructors foci

21
Beliefs

• humans around the age of 3 note difference
  between perception and belief
• temporal integration of perceptual input
• (composite instances of real-world objects)
• percept tree gt snapshot gt belief .
• classification gt data structure gt create/update

22
Beliefs

• humans around the age of 3 note difference
  between perception and belief
• temporal integration of perceptual input
• (composite instances of real-world objects)
• percept tree gt snapshot gt belief .
• classification gt data structure gt create/update

• when the robot shares a belief with a human, the
  belief gets labeled as mutual belief
• humans attentional and referent focus are
  updated for the belief concerned

23
Learning

• From internal demonstration
• telemetry suit ?
• robot interpolates exemplars using a dynamically
  weighted blend of the recorded button pressing
  trajectories
• Names of things
• social cue feedback

24
Learning

• Task structure
• task is either a (sub)task or an action,
  hierarchically organized
• constraints exist as actions (currently used for
  sequential constraints but are expandable)
• task goals are more than the sum of (sub)task
  goals
• a goal can be either a
• state-change in world (attain a state)
• performance (just do it)
• Natural instruction

25
Performing in collaboration

• possible because of the goal-oriented approach
  (and the turn-taking implementation)
• communication of robots perceived SoW and
  intention leads to common ground which is the
  basis of joint intention/attention/planning
• knowledge of own abilities, negotiation of task
  with human
• importance of gestural cues during collaboration

26
Video time
27
Discussion

• Knowing what matters
• restraining search-space by saliency
• temporal cues joint attention
• Knowing what to try
• collaboration contrasted with imitation and
  experiment
• Knowing how to recognize success/faliure
• goal types change desired/performance, goal
  progress
• Knowing how to explore
• Knowing how to leverage the provided structure
• experienced demonstration, mo generally social
  context