Affective Computing: Machines with Emotional Intelligence

1
Affective Computing Machines with Emotional
Intelligence

• Hyung-il Ahn
• MIT Media Laboratory

2
doesnt notice you are annoyed. Doesnt
recognize your emotion You express more
annoyance. He ignores it. Stupid about
handling your emotion He winks, and does a happy
little dance before exiting. Stupid about
expressing emotion.
3
Skills of Emotional Intelligence

• Expressing emotions
• Recognizing emotions
• Handling anothers emotions
• Regulating emotions \
• Utilizing emotions /
• (Salovey and Mayer 90, Goleman 95)

if have emotion
4
Research Areas

• Robotic Computer
• - Recognizing anothers emotions
• - Expressing emotions
• - Handling anothers emotions
• Affective and Cognitive Decision Making
• - Regulating and utilizing emotions
• - Affect as a self-adapting control system
• Affect changes the operating characteristics
  of other three domains
• (cognition, motivation, behavior)

5
Recognizing Emotions
6
Recognition of three basic states
7
Future teacher for every learner
8
Can we teach a chair to recognize behaviors
indicative of interest and boredom? (Mota and
Picard)
9
Boredom
Interest
10
What can the sensor chair contribute toward
inferring the students state Bored vs.
interested?
Results (on children not in training data, Mota
and Picard, 2003) 9-state Posture Recognition
89-97 accurate High Interest, Low interest,
Taking a Break 69-83 accurate
11
Detecting, tracking, and recognizing facial
expressions from video (IBM BlueEyes camerawith
MIT algorithms)
12
Autism Spectrum Conditions

• Center for Disease Control and Prevention (2005)
• 1 child in 166 has ASC

13
Mind-Read gt Act gt Persuade
hmm Roz looks busy. Its probably not a good
time to bring this up
Inference and reasoning about mental states
Modify ones actions Persuade others
Analysis of nonverbal cues
14
Real time Mental State Inference
El Kaliouby and Robinson (2005)
Facial feature extraction
Head facial action unit recognition
Head facial display recognition
Mental state inference
Head pose estimation
Feature point tracking
hmm Let me think about this
Nevenvision face-tracker
15
Affective-Cognitive Mental States
16
Physically animated Robotic Computer (joint with
Prof. Cynthia Breazeal) Goal increase user
movement without distraction and annoyance,
further social-rapport building
17
Robotic Computer (RoCo) A physically animated
computer
Learning the user can guide RoCos behavior by
explicit and implicit rewards and
punishments (Reinforcement Learning)
18
RoCos postures congruous to the user affect
Stoop to Conquer Posture and affect interact
to influence computer users comfort and
persistence in problem solving tasks
People tend to be more persistent and feel more
comfortable when RoCos posture is congruous to
their affective state
N(17)
19
Procedure and Tasks
Tracing Task a solvable and an unsolvable puzzle
Decision-making Task (in Experiment 2) to make
subjects keep the target posture longer
20
Affective Cognitive Decision Making
21
(Example 1) Two-armed bandit gambling tasks
Inspired by Bechara Damasios IOWA gambling
tasks (Bechara et al. 1997)
The left arm has Negative Valence Arousal
(uncertainty) as feeling uneasy
The right arm has Positive Valence Arousal
(uncertainty) as feeling lucky
22
(Example 2) Decision making under risk

• Loss aversion People strongly prefer avoiding
  losses than acquiring gains

• Risk-Averse choices in the domain of Likely
  Gains

gt
Option 1
Option 2
lt
Expected value 3000 (Gain)
Expected value 4000 0.8 0 0.2 3200
(Gain)

• Risk-Seeking choices in the domain of
  Likely Losses

lt
Option 1
Option 2
gt
Expected value - 3000 (Loss)
Expected value - 4000 0.8 0 0.2 -
3200 (Loss)
23
The PT (Prospect Theory) value function

• Diminishing sensitivity
• less sensitive to outliers for both gains and
  losses
• Loss aversion the function is steeper in the
  negative (loss) domain

(Tversky Kahneman)
24
Endowment Effect

• people place a higher value on objects they own
  relative to objects they do not.
• In one experiment, people demanded a higher price
  for a coffee mug that had been given to them but
  put a lower price on one they did not yet own.
• The endowment effect was described as
  inconsistent with standard economic theory which
  asserts that a person's willingness to pay (WTP)
  for a good should be equal to their willingness
  to accept (WTA) compensation to be deprived of
  the good. This hypothesis underlies consumer
  theory and indifference curves.
• The effect is related to loss aversion and status
  quo bias in prospect theory.

25
(Example 3) Effects of mood on decision making
(Lerner Keltner 2000, 2001, 2004)
Happiness
Anger
Optimistic about judgments of future events
Optimistic judgments of future events,
Risk-Seeking choices
Reverse Endowment Effect
Pessimistic judgments of future
events, Risk-Aversive choices
Sadness
Fear
26
Subjective Value Function(mood influences
decision making)
27
Affective Cognitive Learning and Decision Making

• A new computational framework for learning and
  decision making
• inspired by the neural basis of motivations and
  the role of emotions in
• human behaviors
• A motivational value (reward)-based learning
  theory
• decision value extrinsic (cognitive) value
  intrinsic (affective) value
• extrinsic value from the cognitive
  (deliberative and analytic) systems
• intrinsic value from multiple affective
  systems such as Seeking, Fear, Rage, and other
  circuits.
• Probabilistic models Cognition (cognitive state
  transition), Multiple affect circuits (Seeking,
  Joy, Anger, Fear, ), and Decision making model
• Any prior and learned knowledge can be
  incorporated for expecting the consequences of
  decisions (or computing the cognitive value)

28
To destroy the ring in Mordor with less effort
Choice 1 Effort (r -80)
Prob
Fearless/ Neutral / Fearful Mood Incidental
Emotions
Reward
0
70
-30
20
Expected Values Cognitive Expectations choice 1
20, choice 2 20
Choice 2 Effort (r -30)
Valenced Uncertainty Values Anticipatory
Emotions from the Seeking Circuit choice 1
positive, choice 2 negative
Pr 0.5

• Success (r 100)
• Fail (r 0)

Fear Anticipatory Emotions from Other Circuits