Emotion%20Recognition%20from%20Electromyography%20and%20Skin%20Conductance

1
Emotion Recognition from Electromyography and
Skin Conductance
Arturo Nakasone (University of Tokyo) Helmut
Prendinger (National Institute of Informatics,
Tokyo) Mitsuru Ishizuka (University of Tokyo)
6. Architecture of Emotion Recognition Component
1. Emotion Recognition - Introduction
Embodied Conversational Agents (ECA) are being
developed to enhance communication in a natural
way between humans and computer applications. In
this context, emotions are considered one of the
key components to increase the believability of
ECAs. By analyzing emotional state inputs, ECAs
may be able to adapt their behaviors, allowing
users to experience the interaction in a more
sensible way. Applications like Affective Gaming
are making use of emotion recognition through
physiological signal analysis in order to control
several aspects of the gaming experience
2. Objective of Research

• Develop a real time Emotion Recognition Component
  (ERC) based on the analysis of two physiological
  signals
• Electromyography
• Skin conductance

3. Experimental Gaming Environment

1. The ERC was integrated to a game where the user
   plays a card game called Skip-Bo against the
   ECA Max.
2. The perceived emotion from the ERC allows Max to
   adapt his own emotional behavior expressed by his
   facial expressions and game play

1. Initialization parameters are provided to control
   data sampling rates, data file storage and queue
   sizes for retrieved values.
2. The Device Layer retrieves the data from the
   Procomp Infiniti unit and store them in separate
   queues corresponding to each of the sensors
   attached to the unit.
3. Prompted by the ECA Max, the mean of the current
   values stored in the queues are calculated and
   compared to the baselines in order to search for
   meaningful changes in the valence/arousal space.

happy
surprised
7. Emotion Resolution through Bayesian Networks
reproach
angry
EMG and SC signal values
4. Relation between Emotions and Physiological
Signals

1. In his research, P.J. Lang claimed that emotions
   can be characterized in terms of judged valence
   (pleasant or unpleasant) and arousal (calm or
   aroused)
2. The relation between physiological signals and
   arousal/valence is established due to the
   activation of the autonomic nervous system when
   emotions are elicited

1. Meaningful changes in EMG and/or SC are
   categorized into discrete levels in the
   Categorization Layer
2. In our network, the value from the categorized
   skin conductance signal is used to determine
   arousal directly.
3. Since the value from the categorized
   electromyography signal cannot completely
   determine the sign of the valence component, a
   non-physiological node was introduced to
   discriminate this value based on the current
   outcome of the game (i.e. game status).
4. Probability values have been set according to
   psychophysiology literature.

8. Conclusions and Future Work

• Skin Conductance (SC) and Electromyography (EMG)
  have been chosen because of their high
  reliability.
• Skin Conductance determines arousal level through
  linear relation
• Electromyography has been shown to correlate with
  negatively valenced emotions

1. Emotions are key components in the development of
   truly believable ECAs. Even if people do not
   perceive them as humans, some suspension of
   disbelief is possible when emotions come into
   play.
2. Empathic behavior contributes to a better
   interaction in terms of user experience.
3. In some cases, the use of only two signals may
   not be enough to properly handle the emotion
   recognition process. Therefore, other kind of
   information like gaze and pupil dilation will be
   included in our ERC to further enhance the
   emotion recognition network.

5. Issues in Real Time Assessment of
Physiological Data

1. Baseline values calculations were performed by
   inducing an initial relaxation period on the
   subject of approx. 3 minutes. These values were
   used for comparison purposes.
2. Properly detection of emotional activity required
   sampling every 50 milliseconds and using a 5
   second window of data values