Advanced HCI Affective Interaction

1
Advanced HCIAffective Interaction

• Affect is dealt with in
• Preece, Sharp and Rogers (Chapter 5)
• Benyon, Turner and Turner (Chapter 17)
• This lecture reviews explanations of why and how
  interactive products influence users emotions,
  and through this, their attitudes and behaviour.

2
Performance Emotion

• Early cognitive contributions to HCI focus on
  improving users performance
• Usability
• Making tasks easier (preventing error, providing
  help etc)
• Lightening the cognitive load
• Making tools intuitive
• Main aim in the late 80s to improve design of
  computer-based work
• Users emotions only viewed as byproduct of
  unhindered cognitive processing
• Software/Hardware works user satisfaction/accept
  ance
• Focus of computing changing from workplace to
  lifestyle
• iPod, Xbox, Palm Pilot, games, chat, shopping,
  entertainment, leisure
• Activities associated with pleasure (emotion)
• Not just productivity

CHILL OUT!
3
Performance Emotion

• What are emotions?
• Why, and how, do they affect our performance?
• In what ways can emotion be taken into account
  when designing programs or devices?
• 3 main research areas
• Computer recognition of human emotion
• Computer adapts its behaviour based on what it
  knows about user
• Computer simulation of human emotion
• Computer appears to display human characteristics
• Computer evocation of human emotion
• Computer targets human emotions and encourages
  particular responses

4
What is an emotion?

• A neural impulse that moves an organism to action
• We cannot choose our emotions (see below)
• Emotions happen spontaneously in response to
  events in the environment
• Causes physiological changes visible in behaviour
• Facial expression, faster heartbeat,
  perspiration, aggression, crying etc.
• Deeply rooted survival mechanism enabling
  organism to adapt to environment
• cf. Method Acting
• The actor replicates the emotional conditions
  of the character in order to add realism to a
  dramatic performance
• Actor draws on own emotional memory

Pairs of emotions (Plutchik, 1980)Primary
emotions can be combined to form secondary
emotionsDisgust Sadness Remorse
5
Where do emotions come from?

• Cerebral Cortex (Man)
• Planning, language etc.
• Limbic System (other mammals)
• Emotions
• Basal Ganglia
• 2-way messaging system between Cerebral Cortex
  Limbic System
• Brain Stem (present in reptiles)
• Basic body functions
• Hunt, kill, mate, defend territory

Evolution of the brain

• Common belief was that rationality emotion are
  separate cf. early cognitive view that treats
  user as an information processor
• Emotion plays some part in decision-making etc.
  Therefore affects performance, response etc.

6
Don Norman Revised model of cognition

• Visceral level
• Fast
• Makes rapid judgements
• Good/bad safe/dangerous
• Signals motor system alerts brain
• Inhibited/enhanced by control systems
• Behavioural level
• Learned behaviour coming from experience
• Largely automaticised
• Can inhibit/enhance visceral layer
• Inhibited/enhanced by reflective layer

• Reflective level
• No direct access to sensory input or control of
  behaviour
• Watches over, reflects upon, tries to influence
  behaviour level
• Responses slow based on assessment, judgement
  decision-making
• Norman, 2004

7
Emotional responses to stimuli
8
Emotional response to stimuli

• Ekman, Friesen Ellsworth (1972)
• identified 6 basic emotions
• recognised expressed in same way in all cultures

anger
fear
disgust
surprise
happiness
sadness
Which emotions are these people displaying?
(Ekman, Friesen Ellsworth, 1972)
9
Relationships with Computers

• Humans respond to computers like real people
• We prefer computers that flatter us
• We prefer computers which have a similar
  personality to our own
• We are polite to computers
• Therefore, we seem to have social relationships
  with computers
• Emotions are an important component of social
  relationships
• If computer has a friendly face, will we like
  it more?

10
Facial Action Coding System (FACS)

• Ekman and Friesen (1978) develop FACS
• Taxonomy of expressions
• Defines expressions as 1 of 46 Action Units
• Contraction or relaxation of one or more muscles
• Cowe Johnston (Psychology Dept. University
  College, London)
• system automatically generates a virtual puppet,
  or avatar, then drives it by mimicking the facial
  movements of an actor.
• Applications?
• Recognising user expression
• Generating appropriate responses for avatars

actor
avatar
11
Recognizing emotions

• Tele-healthcare (Lisetti, et al. 2003)
• System models patients affective state using
  multiple inputs from wearable sensors and other
  devices such as camera
• Detected emotions mapped onto intelligent agents
  embodied as avatars
• Avatar able to chat to patient to confirm emotion
• 90 success recognizing sadness
• 80 success for anger
• 70 success for frustration

12
Expressing emotion affective input to
interactive systems

• Can humans express emotions to computers?
• What if human wants to explicitly identify an
  emotion to influence the behaviour of a character
  in a game?
• SenToy
• Doll-based interface to game
• Gestures indicate emotions
• Emotions affect game strategy

happiness
sadness
anger
fear
surprise
gloating
13
Expressing emotion affective input to
interactive systems

• Users a little confused
• Hard to understand opponents emotions
• BUT many related to doll (projection)
• Higher satisfaction

Bullock, A. and Gambäck, B. (2003)
14
The tenuous area of Affective Computing

• Rash of projects exploring Affective Computing
• Example Gustbowl project (Keller, van der Hoog,
  and Stappers, 2004)
• Digital communication technology is increasingly
  affecting the way people organize their social
  contacts. Product designers therefore must
  understand users needs, not only on a functional
  level (what information they share) but also on
  an affective, experiential level (what emotions
  are involved). Technological communications
  solutions can easily fail because they reduce
  affective interactions to functional ones. Our
  team from Delft University of Technologys
  ID-Studiolab (http//studiolab.io.tudelft.nl)
  designed the Gustbowl to promote and support
  informal, unobtrusive interactions in families
  whose members live apart. The Gustbowl helps
  families keep in touch, rather than just exchange
  information, by letting members be a part of each
  others daily routines.

• Use scenario son throws keys into the Gustbowl
  mother notices the wobble and sees the picture
  appear in her Gustbowl.
• Can such projects really be classed as affective
  computing?

15
The tenuous area of Affective Computing

• Example A Foundation for Emotional Expressivity
  (Stahl et al. )
• To express emotions to others in mobile text
  messaging in our view require designs that can
  both capture some of the ambiguity and subtleness
  that characterizes emotional interaction and keep
  the media specific qualities. Through the use
  of a body movement analysis and a dimensional
  model of emotion experiences, we arrived at a
  design for a mobile messaging service, eMoto.
  The service makes use of the sub-symbolic
  expressions colors, shapes and animations, for
  expressing emotions in an open-ended way.

8 backgrounds each expressing a different
emotion?
16
Subjective association between colour emotion?

• Can we agree on emotion - colour linkage?
• What would you say your emotional state is at the
  moment?
• Dont say boredom!

17
Computers as Persuasive Technology

• Captology (Fogg, 2002)
• Interactive technology to persuade people to
  change ideas behaviour
• Computing technologies have some advantages over
  human persuaders. For example, computers can
• Be more persistent than humans
• Offer greater anonymity
• Manage huge volumes of data
• Go where humans cannot go or may not be welcome
•  Computing technologies persuade in different
  ways, depending on their role
• Tool
• Social actor
• Medium

18
Computers as Persuasive Technology

• A tool can be persuasive by
• Making target behaviour easier to do
• Leading people through a process
• Performing calculations or measurements that
  motivate people
• A social actor can be persuasive by
• Rewarding people by giving positive feedback
• Modelling a target behaviour or attitude
• Providing social support
• A medium can be persuasive by
• Allowing people to explore cause and effect
  relationships
• Providing people with various experiences that
  motivate them
• Helping people to rehearse a behaviour

19
Persuasive Technology some difficulties

• Ethical issues
• When and how am I being persuaded? Do I know?
• Intrusive?
• Research
• How easy is it to study persuasive technology?
• If your bank is doing this, will they tell you?
• How can you find out the effect?
• Even if you can find out
• Arent people who have been persuaded likely to
  be defensive?

20
References

• Bullock, A. and Gambäck, B. (2003). Evaluating
  affective interaction in gaming. Presented at the
  8th European Conference on Computer Supported
  Cooperative Work (ECSCW 03), 14th-18th September
  2003, Helsinki, Finland.
• Ekman, P., Friesen, W.V. and Ellsworth, P.
  (1972). Emotion in the Human Face. Pergamon, NY.
• Ekman, P. and Friesen, W.V. (1978). The Facial
  Action Coding System. Consulting Psychologists
  Press, Palo Alto, CA.
• Fogg, B.J. (2002). Persuasive Technology Using
  Computers to Change What We Think and Do. Morgan
  Kaufmann.
• Keller, I., Van der Hoog, W. and Stappers P.J.
  (2004). Gust of Me Reconnecting Mother and Son.
  IEEE Pervasive Comp. 3,1(2004), 22-28.
• Lisetti, C., Nasoz, F., LeRouge, C., Ozyer, O.
  and Alvarez, K. Developing multimodal intelligent
  affective interfaces for tele-home health care.
  Int. J. Human-Computer Studies 59 (2003),
  245-255.
• Norman, D. (2004). Emotional Design Why we love
  (or hate) everyday things. Basic Books, NY.
• Plutchik, R. (1980). Emotion A
  Psychobioevolutionary Synthesis. Harper and Row,
  NY.
• Reeves, B. and Nass, C. (1996) The Media
  Equation How people treat computers, television,
  and new media like real people and places.
  Cambridge University Press, New York.
• Ståhl, A., Sundström, P., and Höök, K. (2005). A
  Foundation for Emotional Expressivity. In
  Designing For User Experience, DUX 2005, 3-5 Nov,
  2005, Fort Mason, San Francisco, CA.