Toward Virtual Humans

1
Toward Virtual Humans

• Bill Swartout
• Director of Technology

2
The ICT Mission

• The ICT is intended to serve as a bridge between
  the entertainment and simulation communities
• Focus on cognitive decision-making under stress

3
To the Holodeck
4
Virtual Humans

• Our goal Virtual humans to simulate the human
  element
• Behaviors not pre-scripted
• Behave by understanding situation and reasoning
  about possibilities
• Communicate in natural language
• Can explain actions coach
• Coordinated gestures and non-verbal communication
  
• Understand social situation
• Respond emotionally to situation

5
Major Goals for Virtual Humans

• Create rapidly reconfigurable Vhumans that can be
  easily retargeted to different training
  applications
• Push the boundary of what can be taught in
  simulation (e.g. negotiation and conflict
  resolution)

6
Key Idea Model-Based Approach

• Traditional approach Pre-compiled animations and
  behaviors worked out in advance (e.g. branching
  story line)
• Model-based approach Virtual human derives
  gestures, interaction, behaviors from reasoning
  about model of simulated world and what others
  say and do

7
Virtual Humans Mission Rehearsal Exercise
Project
Holodeck version 0.001 alpha
8
(No Transcript)
9
MRE Accomplishments

• Integrated Virtual Human Architecture
• Most comprehensive integration in world today
• Basic science advances
• Emotion modeling
• Multi-party dialogue model
• Speech recognition in noisy environments
• Natural language pragmatics
• Social reasoning
• Negotiation about tasks

10

• Emotions Project How can games and simulations
  influence our ability to successfully navigate
  high-stakes emotional encounters?
• leadership, psychotherapy, conflict
  management

11
Child HealthyFalse
Eagle 1-6 Assist Desire LT Satisfied False
Assist Eagle 1-6False
Accident Blame unresolved
Child-Healthy Desire SGT Satisfied False
Appraisal
Sgts Appraisal of Accident from his perspective
12
Eagle 1-6 Assist Desire LT Satisfied False
Child HealthyFalse
Assist Eagle 1-6False
Accident Blame unresolved
Get Medevac ResponsibilityLT
Child-Healthy Desire SGT Satisfied
False Probability 75?
Make Amends
Distress 80
Problem-Focused Coping Form intention to help
Boy
13
Eagle 1-6 Assist Desire LT Satisfied False
Child HealthyFalse
Assist Eagle 1-6False
Accident Blame MOM
Get Medevac ResponsibilityLT
Child-Healthy Desire SGT Satisfied
False Probability 75?
Make Amends
Shift Blame
Distress 80
Emotion-Focused Coping Blame Mother
14
Eagle 1-6 Assist Desire LT Satisfied False
Child HealthyFalse
Assist Eagle 1-6False
Accident Blame unresolved
Child-Healthy Desire SGT Satisfied
False Probability 75?
Get Medevac ResponsibilityLT
Shift Blame
Make Amends
Distress 80
Personality
15
Training Negotiation Skills

• Pedagogical goals for successful negotiation
• DO
• Demonstrate shared goals
• Create interdependence
• Build and maintain trust
• DONT
• Threaten, bully or attack
• Virtual humans
• Can understand negotiating statements and
  relationship to own beliefs and goals
• Three phases of negotiation
• Avoidance
• Unreasonable demands
• Negotiation

16
Demo Dealing with Doctors

• Background Operations are planned in area of NGO
  clinic
• Your mission Persuade doctor to re-locate clinic
  (without revealing operational plans)
• Resources Medical supplies and transportation
• Demo built in about 90 days
• Intended as integrated research prototype, not
  full training system

17
Avoidance phase
18
Attack phase
19
Negotiation Resolution
20
After Action Reviewwith Explainable AI
21
Summary

• Virtual humans are becoming a reality
• Possible medical uses
• Emergency medicine
• Doctor/patient interactions
• Pt History Present Complaint
• Pt compliance
• Cultural sensitivity

22
ICT Approach to Research
Basic Research
23
MRE Spoken Language Processing
Virtual Humans
Sergeant
Logging
Corpus Creation
MRE System
Emotion
NLG
Task Reasoning
Dialogue
Body Control
Perception
TTS
Combat Lifesaver
ASR
NLU
Emotion
NLG
Task Reasoning
West Point Cadet Trainee
Dialogue
Body Control
Perception
24
(No Transcript)