ICat as a Companion Robot

1
ICat as a Companion Robot

• Siska Fitrianie
• Dragos Datcu
• Alin Chitu
• L.J.M Rothkrantz
• s.fitrianie, d.datcu, a.g.chitu,
  l.j.m.rothkrantz_at_ewi.tudelft.nl
• http//mmi.tudelft.nl

2
Human AIBO Interactionhttp//mmi.tudelft.nl/aibo
3
Requirements and Challenges

• Requirements
• Develop a system architecture for a companion cat
• Design and develop multimodal system to portray
  emotions in iCat as a result of interaction with
  humans.
• Design of a personality model adapted to iCat.
• Challenges
• Development of a complete set of an HCI system
  input sensing, input processor, input fusion,
  dialog management, output fission.
• Use of iCats specific software that is new and
  still under development
• Little is still known or standardized in the
  cognitive
• model for humans.

4
Requirements in other words
Today is Monday, 27 March 2006 .. It will be
bright sunshine but a bit cold
iCat! Whats today whether?
iCat .. Read me a story, please ..
Get a life! I am still sleeping!
Are you out of your mind? I am sleeping!!!
OK, ok! Ill do it. Dont cry ..
900 AM
800 AM
I feel so lonely!!! I wish I have a companion
iCat!, Lets play tic tac toe!
Hurray!! Finally I have a friend!!
What was that?? Hey .. where is everybody??
OK!! I do the first move!
1100 AM
1400 AM
1600 AM
5
Multimodal Human Computer Interaction
6
Fusion Input(Feature Level)

• Audio-Video Fusion for Automatic Speech
  Recognition

7
Emotion AnalysisEmotion Extraction from Text (1)
8
Emotion AnalysisEmotion Extraction from Text (2)

2D space of Affective Lexicon database based on
relative distance of words meaning from WordNet
and Multidimensional Scaling
9
Emotion AnalysisEmotion Extraction from Facial
Information
10
Emotion AnalysisEmotion Extraction from Speech
Prosody
11
Dialog Action Manager
12
Fission Output
13
Fission OutputLanguage Generator

• Template-based (includes dialog act and emotion
  as parameters)

ltaimlgt ltdialog-act name"DIALOG-ACT"gtlttopic
name"TOPIC"gt ltuser-group name"USER-GROUP"gtltapp
raisal name"APPRAISSAL"gt ltinfo
name"INFO-TYPE"gt ltcategorygtltthatgtTOPIC-HISTOR
Ylt/thatgtltpatterngtINPUTlt/patterngt
lttemplategtltrandomgt ltligtltsentence_latticegt ...
lt/sentence_latticegtlt/ligt ... //other sentences
lt/randomgtlt/templategt lt/categorygt
... //other categories lt/infogt ...//other
categories with less available info
lt/appraisalgt ... // other categories with
different appraisal type lt/user-groupgt ...
//other categories for different user-group
lt/topicgt ... //different topic lt/dialog-actgt
... // different dialog-act lt/aimlgt
14
Fission OutputEmotion Generator (nPME model) - 1
15
Fission OutputEmotion Generator (nPME model) - 2
OCEAN Model of Personality Openness,
Conscientiousness, Extraversion, Agreeableness,
Neuroticism
41 Emotion Types and 3 Activation Levels
Maslow Pyramid of Needs
Mood Level of valence and arousal.
Goals, Preferences, Standards (Rule-based)
16
The Point

• Symmetrical I/O modalities of user-system
  interaction
• Modular and general, as a starting point of any
  CHIM
• Possible applications
• supervising, entertainment, tutor or help desk
  robot
• Current project development milestone
• Input Sensing and Emotion Analysis