Comparative Approaches

1

• Comparative Approaches
• to Emotion-Oriented Architectures
• (WP 7 Emotion in Cognition and Action)
• Last Plenary
• -/ -)
• Lola Cañamero (UH)

Plenary 3, 4-6 June 2007, Paris, France
2
WP7 The area

• Scope investigating computational models of
  emotional influences in cognition and action
• Enhance behavior interactions of
  emotion-oriented systems
• Feedback to emotion theorists (synthetic
  approach, operationalize)
• Exemplar
• Comparative approaches to emotion-oriented
  architectures assumptions, integration
  challenges, and guidelines for future research gt
  Output edited collection
• Divided in 4 elements
• Emotion in embodied cognition and action
• Emotion in reflective cognition and action
• Emotions in bridging the gap between embodied and
  reflective CA
• Emotions in social cognition and interaction
• Groups UH, OFAI, Bari, Paris8, DIST, GERG, HW,
  EMPL38, CNR, USC, ICCS, KCL, UM, INESC-IST, EFPL,
  Miralab, FT-RD, UOXF, USFD

3
WP7 Exemplar the four elements
4
Where are we? exemplar timeline

• Stage 1 Critical analysis of state of the art
  and needs
• Months 1 - 18 iterations to define problems and
  exemplar uncover assumptions and needs
• Stage 2 Integration challenges and key
  development goals
• Month 19 workshop
• Months 19-39 theoretical and practical work on
  integration challenges
• Stage 3 Conclusions and guidelines for future
  research
• Month 40 start developing guidelines for
  future research chapter proposals and abstracts
  gt moved to earlier date (D7e, Month 35)
• Month 42 (JUNE 15!!!!!) drafts of chapters due
• Months 43-48 chapters reviewed and revised
• Month 48 book to publisher

5
Months 1-5 (D7b) state of the art, analysis (1)

• Review of key achievements
• Emotion-based architectures (action selection,
  learning, memory)
• Appraisal and cognitive systems
• User modeling
• ECAs and virtual environments
• Key conceptual problems
• Mechanisms underlying the involvement of emotions
  in cognition and action
• Emotion elicitors (which factors activate those
  mechanisms?)
• Emotions as cognitive modes
• Relations among emotion, value systems,
  motivation and action

6
Months 1-5 (D7b) state of the art, analysis (2)

• Key integration challenges
• Problems arising from theories and models
  (diversity, poor understanding)
• Diversity of computational frameworks modeling
  approaches
• Embodied AI, dynamical systems
• Symbolic AI
• Hybrid systems
• Social simulation
• Key development goals
• Grounding problem of artificial emotions
• Dissolving the mind-body problem
• Untangling the knot of cognition links emotion
  intelligence
• Measuring progress the contributions of
  emotions to our systems

7
Months 6-11 (D7c) approach to exemplar

• Best approach to fulfill horizontal goals in our
  area
• Comparative approaches to emotion-oriented
  architectures assumptions, integration
  challenges, and guidelines for future research
• Key ideas
• Comparative approaches
• welcome the diversity of conceptual and
  computational models and frameworks
• de-emphasize idea of a unified model for an
  emotion-based architecture (misleading goal at
  this point) -gt complements blueprint
• Assumptions, integration challenges and
  guidelines for future research stresses the
  nature of our principled integration effort in
  setting sound grounds

8
Months 6-11 (D7c) elements of exemplar

• Focused working groups, integration at various
  levels
• WG1 Emotion in lower-lever cognition and
  action
• UH, GERG, CNRS EPML 38, KCL
• WG2 Emotion in higher-level cognition and
  action
• Uni. Bari, France Telecom RD, GERG, CNR-ISTC,
  QUB, UA
• WG3 Bridging gap between lower- and
  higher-level C A
• OFAI, HW, INESC-ID, IST, EPFL, USC
• WG4 Emotion in Social Cognition and Interaction
• OFAI, UH, Paris8, MIRALab, DIST, DFKI
• Output
• Edited book
• Reflection based on proof-of-concept designs
  and implementations

9
Support to Network activities

• Presentations (posters) at Plenary 1
• Presentations at WP3 workshop Contributions
  from robotic models of emotions several
  hands-on demonstrations
• Presentations at WP4 workshop links wp7-wp4,
  Markov-based analysis
• Presentations posters at WP6 workshop (ToM,
  affect-based imitation)
• Cross-WPs links (meetings in Saarbruecken,
  Geneva, Santorini, Paris)
• WP3, blueprint conceptual clarification
• WP4, constraints from cognition-action to
  signals/signs processing
• WP6, integration internal models-expressive
  behaviors
• WP8, mental states underlying external
  manifestations of persuasion
• Co-organization (with WP3) of symposium on
  architectures of computational models at Plenary
  2, May 2005
• Working visits (UH OFAI to GERG MIRALab, etc)
• Support actions (sessions) to WPs 3, 6, 8 planed
  at workshop (July 2005)

10
Dissemination external activities

• Symposium Architectures for Modeling Emotion,
  AAAI Spring Symposium, Stanford, March 2004
• Symposium Dimensions of Sociality, Vienna, Nov.
  2004
• Symposium Motivational and Emotional Roots of
  Cognition and Action, AISB05 _at_ UH, April 2005
• Symposium Mind-Minding Agents, AISB05
• Co-organization (with WP3) symposium
  Architecture of Computational Models at ISRE in
  Bari, July 2005
• Co-edition (with WP6) special issue Humanoid
  Robots
• Various press reports, numerous scientific
  articles published or submitted (e.g.
  contribution to special issue NNets)

11
Input from March 2005 project review

• No recommendations for WP7 but some comments
• C1 Establish more clearly synergies with other
  WPs
• WP3 interplay theories / implementations
  meetings
• WP6 (WP4) complementarity towards human-like
  capabilities (special issue) emotion-attention
  interplay for social interaction
• WP8 cognitive emotion models for dialog,
  communication, persuasion
• WP10 working towards standards (joint handbook
  chapter) ethics
• C2 Distinction between lower-level and
  higher-level misleading
• Elements renamed to make focus more precise and
  avoid confusion
• C3 Provide more details on plans to bridge the
  gap
• Element 3 re-structured and made more concrete
• C4 Robotic implementations shouldnt be toy
  demonstrations of problems
• Closer integration with emotion theory and formal
  analysis
• More prominent use of ECAs

12
Core achievements in 2005

• Definition of exemplar (D7d)
• WP 7 workshop, London, July 2005 (D7a)
• Other workshops to develop/support WP7 exemplar
  but not funded by HUMAINE
• New partner (CNRS-EPML38, development),
  involvement of other new partners (USC, CNR)
• Contribution to 2005 high quality dissemination
  deliverable and co-edition of 2006 one
• Joint conceptual, design, implementation work
• Publications
• Other dissemination activities and esteem factors

13
Feedback from March 2006 review

• Develop potential synergies with other projects
• IST06 Networking Session
• Further improve links with the other workpackages
  and provide clear assessment means of this
  progress
• Co-edition of special journal issue with WP6
• Cross-currents symposium
• ACE06 symposium
• Involvement of WP4 in follow-up proposal

14
Core achievements in 2006

• Good progress in the 4 elements of the exemplar
• Increased links with other WPs projects
• Sessions Cross-currents, Summer School, IST06,
  ACE06
• Co-edition of 2006 high quality dissemination
  deliverable with WP6 (IJHR special issue)
• Publications
• 9 joint (4 journal, 5 conf / wksp), 21 single
  institution (8 journal, 13 conf / w)
• Outline book submissions
• Other dissemination activities and esteem factors
• Edition (4), conf. organization (ACII, ACE,
  Ro-Man, EpiRob), inv. talks (10)
• Follow-up project merging E1 E4 ( WP4-WP6)
  FEELIX GROWING

15
E1 Emotion in embodied cognition and action

• Interactions between emotion cognition-action
  as occurring through the body
• UH, EPML38, Paris8, KCL , GERG
• Subtasks
• E1.1 Emotional modulation of perception-action
  in embodied agents proximal causes, development,
  evolution
• E1.2 Analysis of embodied emotion-oriented
  architectures and behavior of robots ethological
  mathematical
• E1.3 Novelty detection and emotion-attention
  interactions (ECAs)

16
Modulation of Per-Ac loops
Evolution
Proximal causation
Development
17
E2 Emotion in reflective cognition and action

• Influence of emotions in cognition-action from
  the perspective of subjective perception and
  reasoning (introspection, linguistic accounts)
• Bari, CNR, FT-RD, UM, USC
• Subtasks
• Role of BDIE models and relation to rationality
  and psych. theory
• Emotional conflict, cognitive dissonance
• Emotion and anticipation
• Validation of cognitive models of emotion
  activation by means of sensitivity analysis
  their extension to the interpretation of
  emotional expressions displayed by the user
• Application comparison of models for emotion
  activation and recognition to dialogs

18
Emotional Mind
19
Emotional Mind in action
20
E3 Emotions in bridging gap

• Role of emotions in relating behavioral meaning
  and symbolic representations
• OFAI, HW, USC, INESC, IST GERG, KCL, UOXF, UM
• Subtasks
• E3.1 A scenario-based survey of bridging
  functions of emotions
• E3.2 Improving upon symbolic models of
  reflective cognition action
• E3.3 Improving upon embodied models of
  cognition action
• E3.4 Bridging the gap between micro-
  (individual-based) and macro- (social) views on
  social functions of emotion

21
Scenario-based evaluation and design

• Goal Understand state of modeling across
  different disciplines
• Challenge Substantial differences in concrete
  scenarios addressed (over 12). Different
  affect-related phenomena modeled at different
  granularities in settings of different complexity
• Approach to compare systems, the functional role
  of emotion (use of term) must be explicated
• Focus on architectural building blocks
• Data structures, processes, interactions between
  processes
• Fixed vs. dynamic/implicit paths of communication
• Explicit differentiation of contexts of
  information processing (modules, levels,
  stages,)
• Bridging between such contexts
• Derive best practices for the development of
  computational models of emotion

22
Improving upon symbolic reflective models

• Suitable building blocks to model emotional
  processes?
• Question foundations of symbolic architectures
  symbolic shortcuts need to be motivated
  explicitly
• Parallel embodied processes as basic behavioral
  components
• Challenge realize reflective and symbolic
  processes on top
• Concurrent processes and resource management

23
Improving upon embodied models

• Further extension of previous work on emergent
  affective and personality model that integrates
  perception, motivation, action selection,
  planning and memory
• Autobiographical memory
• Group level dynamics

24
Bridging micro-macro gap

• From inwards-oriented appraisal towards
  social-communicative behavior
• Social emotions as result of supra-individual
  process of co-regulated reactions
• Extension of appraisal theory model
• Assessment of sequential evaluation check model
• Improving sensing and rapidly reacting to human
  emotional signals
• Integration of socially situated theory
  emphasizing centrality of social goals and
  contingent behavior
• Planned workshop (October 2007, USC/ISI)

25
E4 emotions in social cognition and interaction

• Roles of emotions in social cognition and
  interaction emotions, cognition and action not
  modeled from the perspective of the individual
  but of the interaction itself.
• OFAI, MIRALab, EPFL, ICCS-NTUA, DIST, UH,
  EPML-38, UBari, CNR, U. Sheffield
• Subtasks
• E4.1 Towards socially meaningful emotional
  agents Closing the emotion recognition-generation
  -expression loop
• E4.2 Socially situated nature of emotions
  Socially situated affective dialogue

26
Closing the emotion rec-gen-exp loop
Emotion recognition from full body motion
ECA copying observed expressive gestures
27
The Premio Paganini experiment
28
Cross-currents symposium, June 2006

• Dynamical systems as a framework to bridge gaps
  in emotion research?
• L. Cañamero (UH, coord), R. te Boekhorst (UH),
  A. Flykt (Mid Sweden U), P. Gaussier (EPML38), N.
  Korsten (KCL)
• Closing the emotion recognition-generation-express
  ion loop
• J. Gratch (USC, coord), A. Blanchard (UH), G.
  Castellano (DIST), A. Egges (Miralab), K.
  Karpouizis (ICCS), C. Peters (Paris8)
• Beyond the blackbox vs process models
  alternative reflective emotion models in
  comparison, with their mental ingredients, grain
  size, application perspectives and limits
• F. de Rosis (Bari, coord), Peter Goldie (UM),
  Stacy Marsella (USC), Sabine Payr (OFAI),
  Isabella Poggi (CNR)
• Avenues to bridge gaps between "embodied" and
  "reflective" systems
• P. Petta (OFAI, coord), Nienke Korsten (KCL),
  Robert Marsh (UH), Sandy Louchart (HW), Fiorella
  de Rosis (Bari)

29
Networking Session _at_ IST06

• Embodied Emotion, Cognition and Action for
  Autonomous and Interactive Artifacts
• Aims
• provide framework to explore opportunities for
  interaction among projects with a common interest
  in embodied emotion and cognition
• draft a longer-term research agenda for this
  area
• Presentations FP6 projects HUMAINE, euCognition,
  ICEA, ENACTIVE, MindRACES, S2S2, TAI-CHI, CALLAS
• Challenges, needs and other projects identified
• Over 100 participants
• Follow-up session _at_ Plenary07

30
Book proposed submissions

• 4 chapters from E1
• (1) modulation Per-Ac loops (2) neuromodulation
  (3) dynamical systems analysis (4) novelty and
  attention
• 3 chapters E2
• (5) emotional conflict (6) empathic dialogue
  agent (7) emotion and anticipation
• 4 chapters E3
• (8) AS architecture for virtual humans (9)
  hybrid affective mind (10) improving upon
  symbolic models (11) improving upon embodied
  models
• 5 chapters E4
• (12) emotion sharing understanding (13) PerAc
  models of imitation (14) analysis of movement
  dynamics for emotion recognition (15) full-body
  motion and gesture analysis for recognition (16)
  socially situated affective dialogue
• External input to each section

31
Socio-emotional development
FEEL, Interact, eXpress a Global appRoach to
develOpment With INterdisciplinary
Grounding FP6-IST-045169, December 2006 May 2010
32
Objectives

• Identification of key evaluation scenarios (types
  of problems) in global socially situated
  development of autonomous agents
• identify cross-disciplinary benchmarks (scenarios
  and methods) for a comparative evaluation
• Investigation of the roles of emotion,
  interaction, expression and their interplays in
  bootstrapping driving socially situated
  development
• implementation and testing of robotic systems
  that improve existing work
• Integration of (a) the above capabilities in
  at least 2 different robotic prototypes, and (b)
  feedback across the disciplines involved
• platform for grounded long-term multidisciplinary
  research (roadmap)
• Identification of needs towards achieving
  standards in
• (a) design of scenarios and problem typologies,
• (b) evaluation metrics,
• (c) design of everyday robotic platforms and
  related technology.

33
WP4 Feel and development

• Tasks
• Cross-disciplin. training critical analysis
  models of emotion in development
• Sample key work in psychology to inspire /
  support robotic studies
• Emotion elicitation in spontaneous vs induced
  imitation
• Roles of - emotion in attachment and emotion
  regulation
• Implementing testing in robots selected key
  aspects
• Hedonic processes and their roles in motivation
  and emotion regulation in social interaction
• Selected mechanisms for the detection /
  recognition of emotions in social interactions
  (modal amodal)
• Attachment processes their roles in
  exploration, learning and adaptation to social
  environment

34
Interact and development

• Tasks
• Cross-disciplin. critical analysis models of
  interaction in development
• Sample key work in psychology to inspire /
  support robotic studies
• Emotions in social referencing
• Emotion in joint attention (chimps w differential
  rearing conditions)
• Implementing testing in robots selected key
  aspects
• Joint attention, particularly the role of gaze
  direction
• Task learning by observation/imitation effect of
  (emotional) user feedback
• Interaction and emotional resonance

35
Express and development

• Tasks
• Cross-disciplinary critical analysis models of
  expression in development
• Sample key work in psychology to inspire /
  support robotic studies
• Normal impaired development of emotional
  resonance recognition
• Perception of emotion in human vs robot
• Use of FACS for robots
• Implementing testing in robots selected key
  aspects
• Development of emotional expression related to
  social interaction
• Use of expression as signalling for communication
  (no link to internal emotional state)
• Use of expression as manifestation of an
  internal emotional state