VICTEC and Empathic Interfaces

1
VICTEC and Empathic Interfaces

• Ruth Aylett
• Ana Paiva

2
What does VICTEC stand for?

• V Virtual
• ICT Information Communication
• Technology
• E Empathic
• C Characters

3
Inspired by theatre

• Use of performance to dramatise the problem
• Theatre in Education
• Linked workshops
• But is not a cheap option
• In every group some will be bullying others
• Interaction is limited in conventional performance

4
Forum theatre (Boal)

• An interactive form of theatre
• Sections of audience own a character
• The actor meets them in role and discusses
  actions
• Can only act in character
• Episodic performance
• Actor can halt an episode to ask for more advice
• Hard to manage in school environment

5
Implementing a Virtual Forum Theatre

• 3D interactive graphics and virtual actors -
  synthetic characters
• Individual interaction based on creating empathy
  with the characters
• Computer game technology - but NOT a game!
• Provide a safe environment for children to
  explore different perspectives on bullying
  behaviour.

6
What are Intelligent SyntheticCharacters?

• A synthetic character is a digital or physical
  entity with personality that can interact with
  people in real time.
• Computer games characters are not very
  intelligent
• Do not remember anything about interaction
• Hard to believe in them as personalities
• Thus hard to create empathy with them

7
How realistic should they be?

• We want them to be believable
• Is believability the same as realism?
• We believe in Mickey Mouse
• Characters that are NEARLY realistic may be
  creepy
• We have high expectations
• And then they clash with them

8
Empathic Characters

• Already much experience in animation

9
Mediating Empathy

• (1) via the situation- the observer concludes the
  emotional state of the target from the situation
  the target is dealing with.

(2) via emotional expressions - the observer
interprets the behaviour of the target for
example, when a target cries this means that
he/she is probably sad.
10
Empathic Synthetic Agents
Characters show empathy towards users and
other characters
Users experience Empathic emotions
"empathic synthetic agent" as "an agent that is
able to perceive and have an internal
representation of the other agent's emotions
and/or experiencing an appropriate emotion as a
consequence
"empathy triggering synthetic agent" as "an
agent that is able to, by its behaviour and
features, allow the users to build an
empathic relation with it"
11
Mediation by situation

• Depends on narrative engagement
• Collection of narratives from children
• Using Kar2ouche story-boarding tool
• Observation of dramatic performance
• Scripting of dramas by drama students
• Performances by Uni Herts students
• Narratives should be believable, dramatic,
  somewhat unpredictable

12
Mediation by emotional expression

• Expressive behaviour linked to internal state
  modelling characters affect
• Facial Expression (expressive facial expressions
  that include emotions and communication signals)
• Voice
• Body expression, posture and gestures

13
Designing Characters in VICTEC Mediating
empathy through facial expression
14
Designing Characters in VICTEC Mediating
empathy though body expression
15
Designing Characters in VICTEC Mediating
empathy through exaggeration

• Voice
• Current Text-to-Speech technology is inadequate
• Real Voice must be used to emphasize emotional
  content

Through body expression and animation
Through posture and facial expression
16
Engaging the user

• The character must be believable
• No rerun - what happens counts
• Different scenarios and characters
• Emerging story as in improvisational drama, not
  pre-scripted
• Acting as an invisible friend.
• Tamagotchis seemed to evoke empathy
• Give advice about dealing with the situation and
  see what happens

17
A Trailer

• Not the finished software, but how we think it is
  going to look

18
Capturing affective responses

• Creating an affective loop
• FearNot! Attempts to evoke empathy in the user
• How could one produce an empathic response to the
  user?
• Requires ability to detect affective state of the
  user
• Non-invasively

19
Interaction using Sentoy

• Intelligent Physical Interface
• Aims at making the bridge between the real and
  virtual worlds, extending and enhancing the
  child-system interactions

20
Hardware of SenToy

• Magnet (on off)
• Pressure sensors
• (4 positions each)
• Accelerometer
• (x, y, z angles)

21
Embodiment

• Restrictions
• Sensors size / number
• Sensors capabilities
• Battery vs Autonomy
• Has to handled by children (not too heavy but
  also robust)

22
Validation of Embodiment

• A Teddy, a puppet and a Barbie were tested
• - They had to express emotions and actions with
  every dolls
• - The subjects preferred a neutral expression
• (Barbie
  and Teddy)
• - The doll had to be malleable (Teddy)
• The teddy was easier to handle, more flexible and
  allowed the free expression of emotions / actions

23
Built-in patterns for emotion recognition
24
Built-in patterns
25
Software

• The Data from the sensors is passed through radio
  into a receiver connected with the computer.
• Then some signal processing occurs that filters
  the information, detects patterns and send it to
  a specific application integration module that
  provides the semantic.

26
Sentoy in Victec

• Roles of SenToy in the Interaction
• Out-of-scenario interaction
• As an embodiment of the user (toy as best
  friend). Number of toys is big restriction for
  evaluation
• In-scenario
• As a form of transparently enriching the
  experience.
• As inspirational input to the character (eg.
  cheering)
• As a non-intrusive recording device

27
Virtually friends

• Event for 400 children in June 2004 at University
  of Hertfordshire
• Virtually friends Social role-play with robots
  and agents
• Questionnaire and interaction with the software
• Activities with the robots used in research

28
Challenges - 1

• There IS no easy answer
• No strategy always works
• Tension between realism and a straightforward
  message
• Do we have to present a happy ending?
• Producing believable characters and varying
  scenarios
• Pushing the technology here
• Especially language for unscripted interaction

29
Challenges - 2

• Cultural differences
• There is no word exactly equivalent to bullying
  in German or Portuguese
• School uniform and other differences
• Where bullying takes place
• No school dinners in Germany
• Children unaccompanied to school
• Now extending work to China...