ICANN workshop, September 14,

1
Intelligent Affective Interactiontechnologies
and applications

• ICANN workshop, September 14,
• Athens, Greece

2
workshop focus

• present/discuss experiences from FP6 projects in
  the field of multimodal interfaces, as well as
  related fields (robotics, e-inclusion, cognition,
  etc.)
• discuss the position of the European Research
  Area within the state of the art
• how this position can be maintained and improved
  within FP7

3
workshop presentations

• Prof. C. Pelachaud, Univ. Paris 8, FR
• Prof. Ar. Pnevmatikakis, AIT, GR
• Dr. V. Moschou, AUTH, GR
• Dr. K. Moustakas, ITI-CERTH, GR
• Dr. D. Arnone, Engineering, IT
• Dr. M. Price, BBC RD, UK
• Dr. K. Karpouzis, ICCS/NTUA, GR
• Organized by Prof. Stefanos Kollias, ICCS/NTUA,
  GR

4

lessons learned from HUMAINE

• and the way forward

Kostas Karpouzis, ICCS-NTUA, Greece
4
5
so, what is HUMAINE?

• HUMAINE stands for human-machine interaction
  network on emotions
• EU FP6 Network of Excellence
• IST thematic priority Multimodal Interfaces
• EC financial contribution 4.95M
• duration 48 months (01/04-12/07)

6
so, what is HUMAINE?

• 33 partner institutions from many disciplines
• computer science, psychology, human factors,
  SMEs,

7
why HUMAINE?

• the FP6 work program indicates emotion as one of
  the key factors of natural HCI
• but, what is emotion?
• engineers do not touch this question
• psychologists do, but need to express answers
  with measurable, quantitative terms
• since its an IST project, we need to find a way
  to put concepts and findings to actual use (IPs)
• the multi-disciplinary nature of HUMAINE shows
  that all are needed!

8
thematic areas

• theories and models of emotion
• signals to signs of emotion
• data and databases
• emotion in interaction
• emotion in cognition action
• emotion in communication persuasion
• usability of emotion-oriented systems
• ethics and good practice

9
HUMAINE main deliverables

• as described in the original Technical Annex
• a community of researchers/developers/users/
• joint awareness of suitable methods
• scientific basis for making research cumulative

10
the HUMAINE Research Area
11
HUMAINE in the state of the art

• leading European academia in the fields of
  emotion representation, ECAs and autonomous
  robots
• successful groups in the fields of innovative
  multimodal signal analysis and understanding
• leaders in naturalistic data capture and
  annotation
• application-related groups and SMEs
• design, evaluate, test, use

12
the real HUMAINE deliverables

• scientific community
• bringing together teams from diverse background
  in joint projects (exemplars)
• Europe still has the lead in emotion-related
  research
• mainly thanks to FP5 and FP6
• but others (US, Japan) are catching up fast!
• the leading portal on emotion-research
  (http//emotion-research.net)

13
the real HUMAINE deliverables

• again, what is emotion?
• in the context of everyday HCI
• and in the context of IST
• bits and bytes in related fields
• analysis and synthesis tools (visual, aural,
  physiological, etc.)
• specific applications (call centers, cocktail
  party, helping the disabled and elderly)
• deployed now

14
possible related applications

• ECAs driven by outside, text sources (e.g.
  weather reports or traffic announcements) to
  produce sign language for the deaf
• immersion in virtual or augmented environments
  for interaction entertainment
• autonomous machines (robots) that learn and adapt
  to user traits and assist everyday life

15
how to keep the advantage

• HUMAINE is an IST network of excellence
• contradiction in terms?
• IST is about applied technologies
• NoEs are research incubators
• ethics is a major issue
• hence, all steps need to be careful
• NoEs are the perfect place to build prototypes
• to be polished and deployed by Integrated
  Projects (IPs)

16
how to keep the advantage

• putting technological excellence into practice
• IPs in the multimodal interfaces thematic
  priority need to build on the concepts provided
  by the NoEs
• the CALLAS IP is one of these outlets
• facial and speech analysis, emotion recognition
• augmented reality, ECAs
• adaptation, personalization

17
how to keep the advantage

• FP7 draft work program uses terms like
  multimodal, emotion, affect in a number of
  diverse concepts
• interfaces, robotics, cognition, assisted living
• careful about using the word emotion in
  applications
• sometimes tends to move focus on representation
  and ethics, instead of interfaces and HCI
• small concepts from research areas ready to use
  in applications