APML, a Markup Language for Believable Behavior Generation

1
APML, a Markup Language for Believable Behavior
Generation

• Soft computing Laboratory
• Yonsei University
• October 25, 2004

2
Contents

• Introduction
• Expressing believable behaviors
• MagiCster architecture
• A markup language for behavior specification
  APML
• Overview of existing markup languages for
  expressing human-like behavior
• Defining APML tags
• Facial description language
• An example
• Conclusion

3
Introduction

• Humans communicate using verbal and non-verbal
  signals
• Body posture, gestures, facial expressions, gaze,
  intonation and prosody, and words and sentences
• Embodied conversational agents
• Virtual body that interacts with another agent
• Human-like manner
• Believable way
• Express emotion
• Exhibit a given personality
• Two approaches on Body and Mind
• Strictly and necessarily interdependent
• Mainly independent from each other

4
EU project MagiCster

• Mind and Body are interfaced by a language
  based on XML
• During the conversation
• Mind decides what to communicate considering
  different factors that trigger the goal of
  communicating and influence the contents to
  communicate
• Body reads what the Mind decides to communicate
  and interprets and renders it at the surface
  level according to the available communicative
  channels
• Define a set of language for specifying the
  format of dialogue moves at different abstraction
  levels
• APML (Affective Presentation Markup Language)
• Express the content of the dialogue move at the
  meaning level

5
Chapter Overview

• Describe the main features of the underlying
  architecture
• Present the APML language and how it has been
  used in the context of the MagiCster project
• Show hot it has been interfaced with a 3D
  realistic face call Greta and a synthetic voice
• Use an example in the medical domain

6
Expressing Believable Behaviors

• Communication A means to influence others
• Beliefs forming the content of a communicative
  act
• Information about the world
• Deictics, adjectival
• Information about the speakers identity
• Information about the speakers mind
• Speakers beliefs Degree of certainty,
  metacognitive information
• Goals Performativity of the sentence,
  topic-comment or theme-rheme distinction,
  rhetorical relations, turn-taking and backchannel
• Emotions Affective words, gestures, intonation,
  facial expression, gaze, and posture

7
MagiCster Architecture (1)

• An example of advice-giving dialogue in the
  medical domain
• The agent (Gi) A doctor
• The Interlocutor (Uj) A patient
• Coordination of the speech with various
  expressions
• In move G1, she manifests her empathy with the
  User
• In move G2, Greta indicates her chest while
  saying a spasm of chest
• In move G3, she looks at the User while saying
  your problem

8
MagiCster Architecture (2)

• The MagiCster system
• Mind component A content planner, a dialogue
  manager, and an affective agent modeling module
• Body component A 3D face/avatar with a speech
  synthesizer
• A plan enricher An interface between the Mind
  and Body components

9
MagiCster Architecture (3)

• The affective agent modeling module Decide
• Whether a particular affective state should be
  activated
• Whether the felt emotion should be displayed in a
  given context
• The content planner
• Generate the discourse plan appropriate to the
  context
• DPML DTD Achieve the goal in that piece of
  conversation

10
MagiCster Architecture (4)

• The dialogue manager
• Top of the TRINDI architecture
• Compute dialogue moves and a space in which
  information relevant to the move selection
• The plan enricher
• Translate the symbolic representation of a
  dialogue move into an Agents behavior
  specification at the meaning level
• Translate the DPML-based tree structure into APML
• The face and body animation
• Interpret the APML-tagged dialogue move
• Decide how to convey every meaning

11
A Markup Language for Behavior Specification APML

• High-level primitives for specifying behavior
  acts
• Express agent behavior at different levels of
  abstraction
• Control easily the behavior of ECAs independently
  of the body
• APML (the Affective Presentation Markup Language)
• Specification of the agent behavior at the
  meaning level
• Affective aspect of the communication between the
  agent and the user

12
Overview of Existing Markup Languages for
Expressing Human-Like Behavior
APML

• Human markup language (HML)
• Provide a very abstract level language
• Difficult for controlling specific agent bodies
• Require developing complex interpreters
• Enhance the fidelity of human communications
• Allow the representation of physical, cultural,
  social, kinetic, psychological, and intentional
  features
• VHML Provide several languages for acting on
  different modalities
• MPML (Multimodal Presentation Markup Language)
  Enable authors of web pages to add agents for
  improving human-computer interaction (MS-Agent)
• BEAT (Behavior Expression Animation Toolkit)
  Generate embodied agents animation from textual
  input
• Avatar markup language (AML) Represent a new
  high-level language to describe avatar animation

13
Defining APML Tags
APML

• APML DTD Communication function (A
  meaning-signal pair)

14
Facial Description Language (1)

• Describe facial expressions as (meaning, signal)
  pairs
• Define expressions to capture a slight variation
• At a high level A facial expression is a
  combination of other facial expressions already
  defined
• At a low level A facial expression is a
  combination of facial parameters
• Combine facial expressions due to distinct
  co-occurring communicative acts using a Bayesian
  network
• Facial basis (FB) a basic facial movement
• Facial display (FD) a set of FBs
• surprise raised_eyebrow raised_lid
  open_mouth

15
Facial Description Language (2)
16
Facial Description Language (3)
17
An Example Medical domain application (1)
DPML
APML
18
An Example Medical domain application (2)
19
Conclusion

• Describe the architecture of the behavior
  generator of a believable conversational agent
• Focus on the importance of Mind-Body separation
• Define two XML-like markup languages to represent
  the Mints output and the Bodys input
• APML Not available in public