Title: Generic dialogue modeling for multiapplication dialogue systems
1Generic dialogue modeling formulti-application
dialogue systems
- Trung H. Bui
- Job Zwiers
- Anton Nijholt
- Mannes Poel
2Outline
- Introduction
- 3-step approach
- Conclusions
- Future work
3Introduction Motivation
- Example (extracted from the ICIS scenario)
- S1 You have found the route to the tunnel.
What else do you want? - U2 Umm, I want to have some information about
a patient. - S3(App.-specific) Sorry, the system only
supports route navigation. - S3 (Multi-application) Seeking for the
patient information. What is the patients name? - Multi-application dialogue system
- A dialogue system which allows transparent
switching between a set of applications
A1 Route navigation
A2 Patient information search
4Introduction Goal
Generic Dialogue Modeling for the efficient
production of interfaces for multi-application
dialogue systems
53-step approach
- Producing finalized dialogue models for each
application using the Rapid Dialogue Prototyping
Methodology (RDPM) - Designing an application interaction hierarchy to
connect the applications - Navigating between the applications
6Step 1 Producing finalized dialogue models using
the RDPM
- Main idea of the RDPM Rapid design and
production of a deployable frame-based dialogue
model and its improvement through an iterative
process - Producing finalized dialogue model
- 1. Produce a task model for the targeted
application - 2. Derive an initial dialogue model and the
associated multimodal dialogue-driven interface
from the produced task model - 3. Carry out Wizard-of-Oz experiments to improve
the dialogue model - 4. Carry out an Internal Field-test to further
refine the dialogue model - 5. Carry out an External Field-test to evaluate
the final dialogue model
7RDPM Task modeling
- The task is described in the form of a set of
relational tables (frames), where the rows
correspond to the possible functions (also called
solutions or targets) and the columns are the
attributes needed to uniquely identify each of
the functions and to invoke it - Example
Patient information search
select_patient(Name, Date of birth, Address, )
8RDPM Finalized dialogue model
- Local Interaction
- Help
- Repeat
- NoInput
- NoMatch
- Global Dialogue Strategies
- Branching Logic
- Name and Date of birth ? Address
- Dead-end
- No patient found
- Confirmation
- Tue, 10/2, 10.2 ? 10/2/2004
- Termination
- Result lt 5 patients
- Incoherencies
- Postcode conflicts with Address
multimodal Generic Dialogue Node (mGDN)
9RDPM System architecture
(solution table, mGDN config)
10Step 2 Designing an application Interaction
Hierarchy
- Goal
- Connecting the different applications
- Related work
- Call-routing dialogue system using vector space
model techniques (Carroll Carpenter 1999) - Application-independent Knowledge processing
using ontology descriptions (Vrugt Portele
2004) - Our approach
- Vector space model
- Hierarchical clustering
11Vector space model
- Finalized dialogue model M1,M2,,Mn ? Textual
description di,d2,,dn - Textual sources from solution table, mGDN
config, NLU mapping, - Produce index terms t1,t2,,tk from the textual
descriptions - NLP pre-processing
- Construct occurrence matrix F
- Lexical profile representation
- F n x k matrix
- Compute the score
- E.g. Cosine similarity
Finalized dialogue model
Textual description
Index terms
Occurrence Matrix
Description similarity
12Hierarchical clustering algorithm
- Consider each di as single cluster
- Find the most similar pair of clusters
- Compute distances between the new cluster and
each of the old clusters - Repeat steps 2 and 3 until all items are
clustered into a single cluster, size n. - Transform to the application interaction hierarchy
13Step 3 Navigating between applications based on
users application of interest
- Start
- Active node determination
- Score computation
- Upward propagation
- Downward traversal to determine the active node
- Response generation
- Case 1 Active node is the root or an internal
node - Case 2 Active node is a leaf
14Active node determination (1/2)
Score computation and upward propagation
0.9
M0-9
Users query
0.9
0.4
M0-4
M5-9
0.15
0.4
0.9
0.7
M5-7
M8-9
M0-2
M3-4
0.3
0.25
0.1
0.15
0.7
0.5
0.9
0.85
0.8
0.4
M1
M2
M3
M4
M5
M7
M9
M0
M8
M6
Threshold 0.15
15Active node determination (2/2)
Downward traversal to determine the active node
- Mactive root
- Compute the difference between two highest score
child nodes of Mactive ? Mi gt Mj - If (Mi Mj) lt ts then STOP, else Mactive Mi,
and return to ii.
16Response Generation
- Case 1 Active node is a non-leaf
- Application selection process
- List processing commands next, previous, stop,
up, down, select - Case 2 Active node is a leaf
- The application takes control and interacts with
the user as an application-specific dialogue
system - If the users request is out of the applications
domain ? go back to Start
17Complete example
Start
M0-9
S1 What can I do for you? U2 Give me the
direction to the tunnel. S3 Please select your
application from the list (1) car route
navigation, (2) air route navigation, (3) traffic
lane. U4 One. S5 First, you need to go from
Twente airport, Sk-1 What else do you
want? Uk Umm, I want to have some information
about a patient. Sk1 Seeking for the patient
information. What is the patients name?
1
k
k
2
2
3
M0-2
4
4
5
M2
M1
M0
M8
M9
k-1
Car route navigation
Patient information search
Air route navigation
k
k1
18Conclusion
- A 3-step framework for the development of the
interfaces for multi-application dialogue
systems. - RDPM toolkit is available and has been used in
several research projects InfoVox, INSPIRE,
IM2.MDM and is being extended for ICIS. - Application independent dialogue strategies in
RDPM can be re-used for the development of
multi-application dialogue systems.
19Future work
- Evaluation of the approach in the ICIS project
through the integration of 10 applications (car
route navigation, air route navigation, traffic
lane, ) - Crossing-application
- concurrent applications and tasks
- Task selection
- application interaction hierarchy ? task
interaction hierarchy - task-sharing
20Thank you!