Dialogue Models for Virtual Humans

1
Dialogue Models for Virtual Humans

• David Traum
• traum_at_ict.usc.edu
• http//www.ict.usc.edu/traum

2
Outline

• Goals, methodology, and approach for vhuman
  dialogue
• Brief overview of Virtual Human dialogue work at
  ICT
• Sgt Blackwell
• Mission Rehearsal Exercise (MRE)
• Minor character conversation
• Stabilization and Support Opperations, Simulation
  and Training(SASO-ST)

3
NL Dialogue Overview

• Communication involving
• Multiple contributions,
• Coherent Interaction
• More than one participant
• Interaction modalities
• Input Speech, typing, writing, menu, gesture
• Output Speech, text, graphical
  display/presentation, animated body
• Types of Dialogue Agents
• Information provider
• Service provider
• Instruction-giver
• Advisor/Critic
• Tutor
• Collaborative partner
• Conversational partner

4
Our focus on-line Interaction

• Characters dont know the script
• Must react to current stimulus
• Interpret stimulus
• Update internal state
• produce contextually appropriate behaviors
• Initiatives
• responses

5
Believable from what perspective?

• External view (black box)
• Surface behavior
• Holistic performance/acceptability
• internal view (glass box)
• internal coherence/representational fidelity
• fidelity of a subsystem

6
Complexity of Behavior

• Simple isolated phenomenon or function (e.g.
  gaze or backchannel)
• Toy domains
• Simple tasks
• More complex tasks
• Extended interaction/multiple tasks

7
Degrees of robustness type of user

• Demo
• Trained user
• Motivated user
• General populace
• Red team

8
Spiral methodology

• For a given system, start with simple version
• Then Add
• more robustness,
• more accurate model of phenomena,
• more complex phenomena handled,
• more complex tasks handled

9
Methodology
Computational Model
Implement
Artificial Systems
Simplify, Abstract, extend, operationalize
Evaluate
Learn
Corpus
Verify
NATURAL
CONTROLLED
Training Set
Test Set
Annotation Scheme
Theory
Motivate
Annotate
CONTEXT
Introspect
People
Act
Record
Behavior
Data
Analyze
Observe
10
Forming Computational Dialogue Models

• Machine Learning Approach
• Use annotated data
• Learn to recognize moves from features

• Symbolic Approach
• Use theory
• Explicit representations
• Inference, grammar, etc

• Hybrid Approach
• Use both

11
Parsimonious Dialogue Modelling

• What should go in computational dialogue model?
• Not full theory
• too complex
• Hard to calculate
• Too slow
• not needed
• Only some aspects will come up in any interaction

12
Which Razor?

• History of Shaving
• Represent only with evidence from data

• Represent only if functional consequence

• Represent only if simplest way to achieve
  consequence

• Represent only if necessary function for task

13
Information-State Approach to Computational
Dialogue Modeling

• (Larsson Traum 2000, Traum Larsson 2003)
• Information Specification
• Dialogue Moves for Update
• Behaviors Dialogue
  moves
• Toolkits for easy implementation (TrindiKit,
  dipper, Midiki, USC Steve/Austin)
• Modular approach to sub-components
• More direct relation to theory
• Modular approach to theory

14
Dialogue Processing
Utterance
Dialogue Acts
Recognition Rules
Information State
Update Rules
Components
Selection Rules
Dialogue Acts
Realization Rules
Output Utterance (verbal and nonverbal)
Dialogue Manager
15
Example 1 Sgt Blackwell

• Focus technology demo
• Highlights
• Life-sized, mixed reality
• Trans-screen
• High-production quality
• Rendering (gt 60K polygons)
• Voice
• Authored Text
• Robust responsiveness
• Speech recognition and speech and non-verbal
  reply
• Limited domain of interaction responding to
  interview/QA

16
Sgt Blackwell Video
17
Sgt Blackwell Dialogue Model

• Set of pre-constructed answers
• In domain
• Off-topic
• Prompt
• Local history
• IR-based classification
• Given possibly unseen question, map to best answer

18
Sgt Blackwell Evaluation

• Questions
• What are the best classification techniques?
• How much do speech recognition errors affect
  performance?
• Metrics
• Accuracy of Speech recognizer classifier
• Appropriateness of replies (including to unseen
  and out of domain questions)
• Answers rated for relevance (scale from 1-6)
• Experiment 20 users, asking 20 questions 10
  given, 10 user-defined (Leuski et al IUI 2006, to
  appear)

19
Gandhe et al 2004 Response coherence coding
20
Sgt Blackwell Evaluation Results
21
Beyond Sgt Blackwell Dialogue models for rich
interaction

• To allow more flexible interaction in more open
  and complex domains need
• Internal models of domain problems and solutions
• Models of dialogue structure interaction
• Models of agent personality

22
Case Studies Virtual Human Dialogue _at_ ICT

• MRE

• SASO-ST

23
Immersive Training Environment

• Mission Rehearsal Exercise (Swartout et al 01)
• Human lieutenant (student) faces peacekeeping
  dilemmas
• Appears in video offsceen
• Artificial agents interact with user
• Mentor (e.g., sergeant, front left)
• Teammates (e.g., medic, front right)
• Locals (e.g., mother, front center)

• VR Theatre
• 8 150 Curved Screen,
• Multiple Projectors
• 10-2 3-d spatialized sound

24
Mission Rehearsal Exercise Video
25
MRE Spoken Language Processing
Virtual Humans
Sergeant
Logging
Corpus Creation
MRE System
Emotion
NLG
Task Reasoning
Dialogue
Body Control
Perception
TTS
Combat Lifesaver
ASR
NLU
Emotion
NLG
Task Reasoning
West Point Cadet Trainee
Dialogue
Body Control
Perception
26
Dialogue ApproachLayered Information State

• Layer captures coherent aspect of communicative
  interaction (e.g., turn, grounding, obligations)
• Layer consists of
• Information State components (state of
  interaction)
• Dialogue Acts (Packages of changes to information
  state)

Input Utterance
Recognition Rules
Update Rules
Info State Components
Selection Rules
Output Utterance (verbal and nonverbal)
Realization Rules
Dialogue Manager
27
MRE Dialogue Layers (Traum Rickel AAMAS 2002)

• Social
• Obligations-Commitments
• Negotiation-Collaboration
• Social Roles
• Individual
• Perception
• Rational
• belief,desire, intention,..
• Emotional
• Coping strategies

• Contact
• Attention
• Conversation
• Participants
• Turn
• Initiative
• Grounding
• Purpose
• Rhetorical

28
Social Roles

• IS
• Status (e.g.,Military Rank)
• Superior
• equal
• subordinate
• Activity roles (e.g., forward observer, pilot)
• Action-performance roles
• Actors of parts of complex actions
• Responsibility (team leadership)
• Authority
• Action Effects
• Authorize action
• Perform action
• Take-up, drop role

29
Social Commitments (Traum Allen94, Allwood 94,
Matheson et al 00)

• IS
• Obligations, Social Commitments to Propositions
• Actions
• Order, Request, Suggest
• Promise, Offer
• Statement, Question
• Accept,..
• Effects are to Obligations Commitments
• Belief updates based on inference, not speech act
  effects

30
Team Negotiation (Traum et al AAMAS 2003)

• IS task (CGU) annotated with negotiation
  objects
• Components Agent, Action, Stance, audience,
  reason
• Stances Committed, endorsed, mentioned, not
  mentioned, disparaged, rejected
• Action effects
• Suggestion mentioned
• command, promise, request, or acceptance
  committed
• Rejection rejected
• Counterproposal disparaged1 endorsed2
• Justification endorsed or disparaged (depending
  on direction)
• Offer mention (conditional commitment)
• Retract stance
• Factors
• Relevant Party Authorizing or Responsible Agent
• Dialogue State who has discussed
• Plan State how do I feel about it

31
MRE Team-Negotiation Example
32
Sgts Negotiation Behavior

• Focus1
• Lt U9 secure a landing zone
• Committed(lt,7,sgt), 7 authorized, Obl(sgt,U9)
• Sgt U10 sir first we should secure the assembly
  area
• Disparaged(sgt, 7,lt), endorsed(sgt,2.lt),
  grounded(U9)
• Lt U11secure the assembly area
• Committed(lt,2,sgt), 2 authorized,
  Obl(sgt,U11),grounded(U10)
• Sgt U12understood sir
• Committed(sgt,2,lt), grounded(U11), Push(2,focus)
• Goal7Announce(2,1sldr,2sldr,3sldr,4sldr)
• Goal8 Start-conversation(sgt, ,1sldr,2sldr,,2)
  
• Goal8 -gt Sgt U21 Squad leaders listen
  up!
• Goal7 -gt Sgt U22 give me 360 degree
  security here
• Committed(sgt,2,1sldr,2sldr,3sldr,4sldr)
• Push(3, focus)
• Goal9authorize 3
• Goal9 -gt SgtU231st squad take 12-4
• Committed(sgt,3, 1sldr,2sldr,3sldr,4sldr),
  3 authorized
• Pop(3), Push(4)

1
Decomposition
Area Secure
Squads in area
ALt, RSgt
ALt ,RS
2
7
Decomposition
3
4
ASgt,R1sldr
ASgt,R2sldr
5
6
ASgt,R3sldr
ASgt,R4sldr
33
MRE Evaluation(Traum et al LREC 2004)

• No standard dialogue evaluation measures exist
  for this kind of task
• Use combination of measures
• User satisfaction survey
• Task completion (subjective objective
  intentions fulfilled)
• Understanding accuracy (F-score)
• Response appropriateness (new coding scheme)

34
Measuring Performance

• A
• Lt

ASR score P0.666 R0.666 F0.666
NLU output s.mood imperative s.sem.type
action s.sem.event send s.sem.patient
4th-sqd s.sem.time present s.sem.dest
celic s.addressee sgt s.sem.event move
Speech Acts Speech-act ltA734gt type csa
action order actor lt addresseesgt  
content lt V152gt event move
patient 4th-sqd time present
type act reference-name
one-squad-fwd

• Utterance
• sergeant lt,gt send fourth squad to celiclt.gt

ASR output S sergeant send fourth squad child
NLU score P0.6 R0.5 F0.55
Precision words found correct / words found
Recall words found correct / words
spoken F-score harmonic mean of P and R
35
Evaluations (MRE) Mar vs Dec 2003(Traum et al
LREC 2004)

• Recognition how well did system understand?
• Speech recognition
• Language understanding
• Speech Act
• Addressee

• Appropriateness how correct was system response?
  
• High Inter-rater reliability 0.9? (for 4 raters)

36
Appropriateness Coding Scheme (Traum et al 2004)
37
Dialogue Simulation for Background Characters
(Jan Traum IVA 2005, based on Padilha
Carletta 2002, OSullivan et al 2002, Patel et al
2004)
Characters not too far away to ignore completely,
not too close to be the lead characters of the
scenario
38
Character Information State

• Characters
• Position
• Orientation
• Conversations
• Participants
• Turn
• TRP
• Utterances
• Speaker
• Addressee
• feedback

39
Character Basic Actions

• Speech
• Begin speaking
• End speaking
• Pre-TRP signal
• TRP signal
• Addressee selection
• Positive negative feedback
• Non-verbal
• Walk - change location
• Turn body - change orientation
• Nodding
• Gestures
• Posture shifts
• Gaze

40
Character Behavior Control

• Set of adjustable parameters controlling
  behavior
• talkativeness likelihood of wanting to talk.
• transparency likelihood of producing explicit
  positive and negative feedback, and turn-claiming
  signals.
• confidence likelihood of interrupting and
  continuing to speak during simultaneous talk.
• interactivity the mean length of turn segments
  between TRPs.
• verbosity likelihood of continuing the turn
  after a TRP at which no one is self selected.
• Probabilistic Algorithm
• Test parameters
• Perform actions

41
Culture-specific attributes

• Gaze probabilities
• Target Speaker, Addressee, random, away
• Depends on self role
• Multiplier for mutual gaze
• Silence/overlap
• offset mean for turn-transition
• Variance
• Proxemics
• Intimate zone
• Personal zone
• Social zone

42
Example
43
Minor Character Evaluation(Jan and Traum 2005)

• 3 tests
• Judgments of believability (comparison to random
  behaviors)
• Qualitative personality judgments of characters
  (comparisons to parameters)
• Judgments of conversation participation- who is
  talking to whom, how many conversations
  (comparisons to internal models)

44
SASO-ST Beyond Team negotiation
45
Negotiation Strategies (Traum et al IVA 2005)

• Result from orientations toward negotiation
• Avoidance
• avoid
• Distributive
• attack
• Integrative
• negotiate
• Govern choice of dialogue move, posture, and
  interpretation

46
Modelling Trust

• Represented as Variable
• 0 (no trust) to 1 (full trust)
• Initial value can be set
• Updated as a result of interaction
• Linear combination of three components
• Familiarity (observance of polite social norms)
• Solidarity (same goals)
• Credibility (shared beliefs)
• Used to update beliefs based on reports
  (commitments, promises, belief attribution)
• Used in assessing proposals

47
SASO-ST

• Doctor Perez runs NGO clinic
• Doctor values Neutrality
• No prior relationship with Doctor
• Recently
• Rise in insurgent activity
• More casualties in clinic
• Planned operations

• Mission
• Convince Doctor to move (but dont give op
  details)
• Gain working relationship with Doctor

48
SASO-ST video
49
Summary

• 4 virtual human dialogue projects at ICT
• Sgt Blackwell (simple QA, robustness)
• MRE main characters (multiparty teamwork)
• Minor characters (group conversation)
• SASO Doctor (adversarial negotiation)
• Working toward generic, parameterizable dialogue
  models
• Support Face to face interaction
• Multi-party and multi-conversation
• Maintain immersion in task and virtual world
• Support other cognitive modelling (emotion)

50
Thanks to

• Anton Leuski, Bilyana Martinovski, Susan
  Robinson, Jens Stephan, Ashish Vaswani, Sudeep
  Gandhe, Dusan Jan, Ronak Patel, Ed Hovy, Shri
  Narayanan, Rahul Bhagat, Dagen Wang, Jigish
  Patel, Michael Fleischman,Yosuke Matsusaka

• Jeff Rickel, Jon Gratch, Stacy Marsella, Bill
  Swartout, Lewis Johnson, Patrick Kenny, Jarrell
  Pair, Pete McNerney, Ed Fast, Arno Hartholt,
  Andrew Marshall Marcus Thiebaux, Diane Piepol,
  Ernie Eastland, Justine Cassell, Matthew Stone