CS228 HumanComputer Interaction

1
CS228 Human-Computer Interaction
Can software feel?
Date Fri, 23 Feb 2007 120501 -0500 120501
PM ESTFrom Erin Fitzgerald lterin.fitzgerald_at_uvm.
edugt To PURCARDERS_at_LIST.UVM.EDU Reply-To Erin
Fitzgerald lterin.fitzgerald_at_uvm.edugt
Subject purchasing card software
brokenHeaders Show All Headers Greetings,You
have not received the email notification of
purchasing card transactions (PVS reports) since
February 6.  The software is currently broken. 
We're working on it.  When it is feeling better,
we'll send out all of the delayed
reports.Thanks, ErinErin M. Fitzgerald       
                      802-656-0885Assistant
Director, Procurement Services       
erin.fitzgerald_at_uvm.eduUniversity of Vermont   
                       fax  [phone]
Roosevelt Highway, Suite 120Colchester, VT  05446
2
CS228 Human-Computer Interaction
Affective computing

• Getting computers to recognize emotion
• 2. Enabling technologies to give the impression
  of having emotion
• 3. Designing interactive systems which evoke
  positive human emotions

3
CS228 Human-Computer Interaction
Affective computing

• Getting computers to recognize emotion
• 75 of people surveyed admitted to swearing at
  their computers.
• Q Why enable technologies to recognize emotion?

4
CS228 Human-Computer Interaction
Affective computing

• Getting computers to recognize emotion
• 75 of people surveyed admitted to swearing at
  their computers.
• Q Why enable technologies to recognize
  emotion?
• A Adaptive interfaces could
• Become passive, rather than active, if anger or
  frustration is detected,
• Change tactics if negative emotions are
  detected,
• provide more information if wonder is
  expressed,
• provide less information if confusion/frustratio
  n is expressed,
• or remember the current interaction if it
  induces positive emotion.

5
CS228 Human-Computer Interaction
Affective computing

• Getting computers to recognize emotion
• Q What are emotions?

6
CS228 Human-Computer Interaction
Affective computing

• Getting computers to recognize emotion
• Q What are emotions?
• Emotions have three components
• Physiological changes
• (trembling with fear, blush with embarassment,
  etc.)
• Behavioral response
• Retreating (fear), approaching (anger) ? fight
  or flight
• Subjective experience / cognitive interpretation
  (cognitive labeling)
• We interpret the emotion based on the stimulus,
• our physiological and behavioral responses
• and the larger context
• Pulse jumps, we move quickly backward in
  response to image
• Image was unexpected (I was surprised)
• Image was of something disturbing (I was
  scared)

7
CS228 Human-Computer Interaction
Affective computing

• Getting computers to recognize emotion
• Q What are emotions?

FACS (Facial action coding scheme) Ekman
Ellsworth (1972) Face muscles tensed to express
emotion Is relatively uniform over culture (few
cultural differences in the way emotions are
expressed) therefore facial expression of
emotion must be evolved Leads to the
quantification of emotion ? Emotions can be
measured by analyzing images of faces
The emotion wheel (Plutchik, 1980)
Disgust
Anger
Sadness
Anticipation
Surprise
Joy
Accept- ance
Fear
8
CS228 Human-Computer Interaction
Affective computing

• Getting computers to recognize emotion
• Q What are emotions?
• The emotion wheel
• (Plutchik, 1980)

Disgust
Anger
Sadness
Anticipation
Surprise
Joy
Accept- ance
Fear
9
CS228 Human-Computer Interaction
Affective computing

• Getting computers to recognize emotion
• Q How could we enable technologies to recognize
  emotion?

10
CS228 Human-Computer Interaction
Affective computing

• Getting computers to recognize emotion
• Q How could we enable technologies to recognize
  emotion?
• Input
• Pattern Recognition
• Reasoning
• Learning

11
CS228 Human-Computer Interaction
Affective computing

• Getting computers to recognize emotion
• Q How could we enable technologies to recognize
  emotion?
• Input Collect sensor data images of faces,
• accelerometers / electromyograms measure body
  movements,
• record respiration rates, heart rate, skin
  conductance

12
CS228 Human-Computer Interaction
Affective computing

• Getting computers to recognize emotion
• Q How could we enable technologies to recognize
  emotion?
• Input Collect sensor data images of faces,
• accelerometers / electromyograms measure body
  movements,
• record respiration rates, heart rate, skin
  conductance
• Pattern recognition feature extraction measure
  amount of furrowing on the brow,
• curvature of lips up or down (smile/frown),
  etc.

13
CS228 Human-Computer Interaction
Affective computing

• Getting computers to recognize emotion
• Q How could we enable technologies to recognize
  emotion?
• Input Collect sensor data images of faces,
• accelerometers / electromyograms measure body
  movements,
• record respiration rates, heart rate, skin
  conductance
• Pattern recognition feature extraction measure
  amount of furrowing on the brow,
• curvature of lips up or down (smile/frown),
  etc.
• Reasoning transform features into a prediction
  of what emotion is being expressed

If (teethDetected true ) if ( lipCurvature
UPWARD ) then predictedEmotion JOY else
if ( browFurrowed true ) then
predictedEmotion ANGER
14
CS228 Human-Computer Interaction
Affective computing

• Getting computers to recognize emotion
• Q How could we enable technologies to recognize
  emotion?
• Input Collect sensor data images of faces,
• accelerometers / electromyograms measure body
  movements,
• record respiration rates, heart rate, skin
  conductance
• Pattern recognition feature extraction measure
  amount of furrowing on the brow,
• curvature of lips up or down (smile/frown),
  etc.
• Reasoning transform features into a prediction
  of what emotion is being expressed
• Learning adapt predictions to match actual user
  emotions
• Stimulus1 ? Prediction JOY ? User indicates
  they were angry
• Stimulus2 ? Prediction JOY ? User indicates
  they were pleased
• (restructuring of decision tree learning )
• Stimulus1 ? Prediction ANGER ? User
  indicates they were angry
• Stimulus2 ? Prediction JOY ? User
  indicates they were pleased

15
CS228 Human-Computer Interaction
Affective computing

• Getting computers to recognize emotion
• Enabling technologies to give the impression of
  having emotion
• Can we enable technologies to do this? Why would
  we want to do this?
• People respond very strongly to objects that seem
  to express emotions
• Anthropomorphization attributing human qualities
  to non-human objects

16
CS228 Human-Computer Interaction
Affective computing

• Getting computers to recognize emotion
• Enabling technologies to give the impression of
  having emotion
• Can we enable technologies to do this? Why would
  we want to do this?
• People respond very strongly to objects that seem
  to express emotions
• Anthropomorphization attributing human qualities
  to non-human objects.
• By expressing emotions, technologies can help
  create an emotional bond
• between the user and technology Q Why would
  this be useful?

17
CS228 Human-Computer Interaction
Affective computing

• Getting computers to recognize emotion
• Enabling technologies to give the impression of
  having emotion
• Can we enable technologies to do this? Why would
  we want to do this?
• People respond very strongly to objects that seem
  to express emotions
• Anthropomorphization attributing human qualities
  to non-human objects.
• By expressing emotions, technologies can help
  create an emotional bond
• between the user and technology
• deepens and strengthens the human-technology
  bond increases interaction
• people are willing to continue with the
  interaction
• Educational software requires the user to
  persevere
• (I know this is difficult for you, but)
• Emotion-aware teaching software tracks
  student attention
• New Scientist Tech, January 5, 2007
• Therapeutic software may cause the user
  necessary pain
• (I know this is emotionally difficult,
  but)
• Virtual reality therapy for combat stress,
  NPR

18
CS228 Human-Computer Interaction
Affective computing

• Designing interactive systems which evoke
  positive human emotions
• Designing for pleasure
• Four dimensions to pleasure how to maximize them
  using technology?
• Physio-pleasure Arises from the observation or
  handling of technology
• Examples of technologies that maximize this?
  iPods,
• Socio-pleasure Arises from relationships with
  others.
• What technologies facilitate social activity
  create new ways to connect
• with one another improve relationships with
  others?
• Psycho-pleasure Cognitive or emotional
  satisfaction
• Learning something challenging (a new
  programming language)
• getting things done efficiently, etc.
• Ideo-pleasure Ideological pleasure
• witnessing / experiencing something that
  conforms to our core values.
• Using open-source rather than commercial
  software,