Title: Error Awareness and Recovery in TaskOriented Spoken Dialogue Systems
1Error Awareness and Recovery in Task-Oriented
Spoken Dialogue Systems
- Thesis Proposal
- Dan Bohus
- Carnegie Mellon University, January 2004
- Thesis Committee
- Alex Rudnicky (Chair)
- Roni Rosenfeld
- Jeff Schneider
- Eric Horvitz (Microsoft Research)
2Problem
Lack of robustness when faced with understanding
errors
- Spans most domains and interaction types
- Has a significant impact on performance
3An example
- S Are you a registered user ?
- U No I'm not. No NO I'M NOT NO
- S What is your full name ?
- U Adam Schumacher ATHENS IN AKRON
- S There is an Athens in Georgia and in Greece.
Which destination did you want ? - U Neither NEITHER
- S I'm sorry, I didn't catch that. There is an
Athens in Georgia and in Greece. Which
destination did you want ? - U Georgia GEORGIA
- S A flight from Athens... Where do you want to
go ? - U Start over START OVER
- S Do you really want to start over ?
- U Yes YES
- S What city are you leaving from ?
- U Hamilton, Ontario HILTON ONTARIO
- S Sorry, I'm not sure I understood what you
said. Where are you leaving from ? - U Hamilton HILTON
- S I'm still having trouble understanding you. To
go on, I need you to answer the following
question. What city are you leaving from ? - U Toronto TORONTO
4Some Statistics
- Corrections Krahmer, Swerts, Litman, Levow
- 30 of utterances correct system mistakes
- 2-3 times more likely to be misrecognized
5Significant Impact on Interaction
40
Failed
26
33
sessions
Contain understanding errors
- Multi-site Communicator Corpus Shin et al
Failed
37
63
sessions
6Outline
- Problem
- Approach
- Infrastructure
- Research Program
- Timeline Summary
problem approach infrastructure indicators
strategies decision process summary
7Increasing Robustness
- Increase the accuracy of speech recognition
- Assume recognition is unreliable, and create the
mechanisms for acting robustly at the dialogue
management level
- ASR performance increases / demands increase
- More general
problem approach infrastructure indicators
strategies decision process summary
8Snapshot of Existing Work Slide 1
- Theoretical models of grounding
- Contribution Model Clark, Grounding Acts
Traum - Practice heuristic rules
- Misunderstandings
- Threshold(s) on confidence scores
- Non-understandings
Analytical/Descriptive, not decision oriented
Ad-hoc, lack generality, not easy to extend
problem approach infrastructure indicators
strategies decision process summary
9Snapshot of Existing Work Slide 2
- Conversation as Action under Uncertainty Paek
and Horvitz - Belief networks to model uncertainties
- Decisions based on expected utility, VOI-analysis
- Reinforcement learning for dialogue control
policies Singh, Kearns, Litman, Walker, Levin,
Pieraccini, Young, Scheffler, etc - Formulate dialogue control as an MDP
- Learn optimal control policy from data
Do not scale up to complex, real-world domains
problem approach infrastructure indicators
strategies decision process summary
10Research Program Goals Approach
A task-independent, adaptive and scalable
framework for error recovery in task-oriented
spoken dialogue systems
A task-independent, adaptive and scalable
framework for error recovery in task-oriented
spoken dialogue systems
Approach
- Decision making under uncertainty
problem approach infrastructure indicators
strategies decision process summary
11Three Components
0. Infrastructure
- 1. Error awareness
- 2. Error recovery strategies
- 3. Error handling decision process
Develop indicators that ? Assess reliability
of information ? Assess how well the dialogue is
advancing
? Develop and investigate an extended set of
conversational error handling strategies
? Develop a scalable reinforcement-learning
based approach for error recovery in spoken
dialogue systems
problem approach infrastructure indicators
strategies decision process summary
problem approach infrastructure indicators
strategies decision process summary
12Infrastructure
- RavenClaw
- Modern dialog management framework for complex,
task-oriented domains - RavenClaw spoken dialogue systems
- Test-bed for evaluation
Completed
Completed
problem approach infrastructure indicators
strategies decision process summary
13RavenClaw
Dialogue Task (Specification)
Domain-Independent Dialogue Engine
problem approach infrastructure indicators
strategies decision process summary
14RavenClaw-based Systems
- RoomLine
- CMU Lets Go!! Bus Information System
- LARRI Symphony
- TeamTalk 11-741
- Eureka 11-743
problem approach infrastructure indicators
strategies decision process summary
15Three Components
- 0. Infrastructure
- 1. Error awareness
- 2. Error recovery strategies
- 3. Error handling decision process
Develop indicators that ? Assess reliability
of information ? Assess how well the dialogue is
advancing
? Develop and investigate an extended set of
conversational error handling strategies
? Develop a scalable reinforcement-learning
based approach for error recovery in spoken
dialogue systems
problem approach infrastructure indicators
strategies decision process summary
16Existing Work
- Confidence Annotation
- Traditionally focused on speech
recognizerBansal, Chase, Cox, and others - Recently, multiple sources of knowledgeSan-Segun
do, Walker, Bosch, Bohus, and others - Recognition, parsing, dialogue management
- Detect misunderstandings 80-90 accuracy
- Correction and Aware Site DetectionSwerts,
Litman, Levow and others - Multiple sources of knowledge
- Detect corrections 80-90 accuracy
problem approach infrastructure indicators
strategies decision process summary
17Proposed Belief Updating
- Continuously assess beliefs in light of initial
confidence and subsequent events
S Where are you flying from? U
CityNameAspen/0.6 Austin/0.2 S Did you
say you wanted to fly out of Aspen? U No
CityNameBoston/0.8
initial belief
system action
user response
updated belief
CityNameAspen/? Austin/?
Boston/?
problem approach infrastructure indicators
strategies decision process summary
18Belief Updating Approach
- Model the update in a dynamic belief network
t
t 1
C
C
C
C
initial belief
updated belief
system action
system action
User response features
CurrentTop
Current2nd
Current3rd
Confidence
Yes
No
Positive Markers
Negative Markers
Utterance Length
problem approach infrastructure indicators
strategies decision process summary
19Three Components
- 0. Infrastructure
- 1. Error awareness
- 2. Error recovery strategies
- 3. Error handling decision process
Develop indicators that ? Assess reliability
of information ? Assess how well the dialogue is
advancing
? Develop and investigate an extended set of
conversational error handling strategies
? Develop a scalable reinforcement-learning
based approach for error recovery in spoken
dialogue systems
problem approach infrastructure indicators
strategies decision process summary
20Is the Dialogue Advancing Normally?
- Locally, turn-level
- Non-understanding indicators
- Non-understanding flag directly available
- Develop additional indicators
- Recognition, Understanding, Interpretation
- Globally, discourse-level
- Dialogue-on-track indicators
- Summary statistics of non-understanding
indicators - Rate of dialogue advance
problem approach infrastructure indicators
strategies decision process summary
21Three Components
- 0. Infrastructure
- 1. Error awareness
- 2. Error recovery strategies
- 3. Error handling decision process
Develop indicators that ? Assess reliability
of information ? Assess how well the dialogue is
advancing
? Develop and investigate an extended set of
conversational error handling strategies
? Develop a scalable reinforcement-learning
based approach for error recovery in spoken
dialogue systems
problem approach infrastructure indicators
strategies decision process summary
22Error Recovery Strategies
- Identify
- Identify and define an extended set of error
handling strategies - Implement
- Construct task-decoupled implementations of a
large number of strategies - Evaluate
- Evaluate performance and bring further refinements
23List of Error Recovery Strategies
problem approach infrastructure indicators
strategies decision process summary
24List of Error Recovery Strategies
problem approach infrastructure indicators
strategies decision process summary
25Error Recovery Strategies Evaluation
- Reusability
- Deploy in different spoken dialogue systems
- Efficiency of non-understanding strategies
- Simple metric Is the next utterance understood?
- Efficiency depends on decision process
- Construct upper and lower bounds for efficiency
- Lower bound decision process which chooses
uniformly - Upper bound human performs decision process (WOZ)
problem approach infrastructure indicators
strategies decision process summary
26Three Components
- 0. Infrastructure
- 1. Error awareness
- 2. Error recovery strategies
- 3. Error handling decision process
Develop indicators that ? Assess reliability
of information ? Assess how well the dialogue is
advancing
? Develop and investigate an extended set of
conversational error handling strategies
? Develop a scalable reinforcement-learning
based approach for error recovery in spoken
dialogue systems
problem approach infrastructure indicators
strategies decision process summary
27Previous Reinforcement Learning Work
- Dialogue control Markov Decision Process
- States
- Actions
- Rewards
- Previous work successes in small domains
- NJFun Singh, Kearns, Litman, Walker et al
- Problems
- Lack of scalability
- Once learned, policies are not reusable
S2
A
S3
S1
problem approach infrastructure indicators
strategies decision process summary
28Proposed Approach
- Overcome previous shortcomings
- Focus learning only on error handling
- Reduces the size of the learning problem
- Favors reusability of learned policies
- Lessens the system development effort
- Use a divide-and-conquer approach
- Leverage independences in dialogue
problem approach infrastructure indicators
strategies decision process summary
29Gated Markov Decision Processes
RoomLine
Login
Welcome
GreetUser
Gating Mechanism
AskRegistered
AskName
- Small-size models
- Parameters can be tied across models
- Easy to design initial policies
- Decoupling favors reusability of policies
- Accommodate dynamic task generation
- Independence assumption
problem approach infrastructure indicators
strategies decision process summary
30Reward structure learning
- Rewards based on any dialogue performance metric
- Atypical, multi-agent reinforcement learning
setting
- Multiple, standard RL problems
- Model-based approaches
problem approach infrastructure indicators
strategies decision process summary
31Evaluation
- Performance
- Compare learned policies with initial heuristic
policies - Metrics
- Task completion
- Efficiency
- Number and lengths of error segments
- User satisfaction
- Scalability
- Deploy in a system operating with a sizable task
- Theoretical analysis
problem approach infrastructure indicators
strategies decision process summary
32Outline
- Problem
- Approach
- Infrastructure
- Research Program
- Summary Timeline
problem approach infrastructure indicators
strategies decision process summary
33Summary of Contributions
- Overall Goal develop a task-independent,
adaptive and scalable framework for error
recovery in task-oriented spoken dialogue systems - Modern dialogue management framework
- Belief updating framework
- Investigation of an extended set of error
handling strategies - Scalable data-driven approach for learning error
handling policies
problem approach infrastructure indicators
strategies decision process summary
34Timeline
data
indicators
strategies
decisions
proposal
now
Misunderstanding andnon-understandingstrategies
Investigatetheoreticalaspects
ofproposedreinforcementlearningmodel
end ofyear 4
Evaluatenon-understandingstrategies
developremaining strategies
Data collection forbelief updating andWOZ study
milestone 1
Develop andevaluate thebelief updatingmodels
Implementdialogue-on-trackindicators
milestone 2
Error handling decision process reinforcement lea
rning experiments
Data collection forRL training
Data collection forRL evaluation
end ofyear 5
milestone 3
Contingency data collection efforts
Additional experiments extensions
or contingency work
defense
5.5 years
problem approach infrastructure indicators
strategies decision process summary
35Thank You!
- Questions Comments
- committee members,
- then floor
36Indicators Goals
- Goal Increase awareness and capacity to detect
problems - Develop indicators which can inform the error
handling process about potential problems
37 problem approach support work indicators
strategies decision process summary
38Three Desired Properties
- Task-Independence
- Reuse the proposed architecture across different
spoken dialogue systems with a minimal amount of
authoring effort - Adaptability
- Learn from experience how to adapt to the
characteristics of various domains - Scalability
- Applicable in spoken dialogue systems operating
with large, practical tasks
39ExplConf
ExplConf
ExplConf
ImplConf
ImplConf
ImplConf
HC
LC
MC
NoAct
NoAct
NoAct
NoAct
0
40Belief Updating Approach
- Model the update in a dynamic belief network
- Top-N values
- Fixed structure
- Learn parameters
- Data collection
- Evaluation
- Accuracy
- Soft-error
t
t 1
C
C
C
C
System Action
System Action
CurrentTop
Current2nd
Current3rd
CurrentTop
Current2nd
Current3rd
Confidence
Confidence
No
No
Yes
Yes
Negative Markers
Utterance Length
Negative Markers
Utterance Length
Positive Markers
Positive Markers
User response features
problem approach infrastructure indicators
strategies decision process summary
41Gated Markov Decision Processes
RoomLine
Login
Welcome
GreetUser
Gating Mechanism
AskRegistered
AskName
- Issues
- Structure of individual MDPs
- Gating mechanism
- Reward structure and learning
problem approach infrastructure indicators
strategies decision process summary
42Structure for individual MDPs
- State-space
- informative subset of corresponding indicators
- Concept-MDPs confidence / beliefs
- Topic-MDPs non-understanding, dialogue-on-track
indicators - Action-space
- corresponding system-initiated error handling
strategies
problem approach infrastructure indicators
strategies decision process summary
43Gating Mechanism
- Heuristic derived from domain-independent
dialogue principles - Give priority to topics over concept
- Give priority to entities closer to the
conversational focus
problem approach infrastructure indicators
strategies decision process summary