Speech-to-Speech MT Design and Engineering

1
Speech-to-Speech MTDesign and Engineering

• Alon Lavie and Lori Levin
• MT Class
• April 5 2000

2
Outline

• Design and Engineering of the JANUS/C-STAR
  speech-to-speech MT system
• The C-STAR Travel Domain Interlingua (IF)
• Evaluation and User Studies
• Open Problems, Current and Future Research

3
Overview

• Fundamentals of our approach
• System overview
• Engineering a multi-domain system
• Evaluations and user studies
• Alternative translation approaches
• Current and future research

4
JANUS Speech Translation

• Translation via an interlingua representation
• Main translation engine is rule-based
• Semantic grammars
• Modular grammar design
• System engineered for multiple domains
• Incorporate alternative translation engines

5
The C-STAR Travel Planning Domain

• General Scenario
• Dialogue between one traveler and one or more
  travel agents
• Focus on making travel arrangements for a
  personal leisure trip (not business)
• Free spontaneous speech

6
The C-STAR Travel Planning Domain

• Natural breakdown into several sub-domains
• Hotel Information and Reservation
• Transportation Information and Reservation
• Information about Sights and Events
• General Travel Information
• Cross Domain

7
Semantic Grammars

• Describe structure of semantic concepts instead
  of syntactic constituency of phrases
• Well suited for task-oriented dialogue containing
  many fixed expressions
• Appropriate for spoken language - often disfluent
  and syntactically ill-formed
• Faster to develop reasonable coverage for limited
  domains

8
Semantic Grammars

• Hotel Reservation Example
• Input we have two hotels available
• Parse Tree
• give-informationavailabilityhotel
• (we have hotel-type
• (quantity (two)
• hotel (hotels)
• available)

9
The JANUS-III Translation System
10
The JANUS-III Translation System
11
The SOUP Parser

• Specifically designed to parse spoken language
  using domain-specific semantic grammars
• Robust - can skip over disfluencies in input
• Stochastic - probabilistic CFG encoded as a
  collection of RTNs with arc probabilities
• Top-Down - parses from top-level concepts of the
  grammar down to matching of terminals
• Chart-based - dynamic matrix of parse DAGs
  indexed by start and end positions and head cat

12
The SOUP Parser

• Supports parsing with large multiple domain
  grammars
• Produces a lattice of parse analyses headed by
  top-level concepts
• Disambiguation heuristics rank the analyses in
  the parse lattice and select a single best path
  through the lattice
• Graphical grammar editor

13
SOUP Disambiguation Heuristics

• Maximize coverage (of input)
• Minimize number of parse trees (fragmentation)
• Minimize number of parse tree nodes
• Minimize the number of wild-card matches
• Maximize the probability of parse trees
• Find sequence of domain tags with maximal
  probability given the input words P(TW), where
  T t1,t2,,tn is a sequence of domain tags

14
JANUS Generation Modules

• Two alternative generation modules
• Top-Down context-free based generator - fast,
  used for English and Japanese
• GenKit - unification-based generator augmented
  with Morphe morphology module - used for German

15
Modular Grammar Design

• Grammar development separated into modules
  corresponding to sub-domains (Hotel,
  Transportation, Sights, General Travel, Cross
  Domain)
• Shared core grammar for lower-level concepts that
  are common to the various sub-domains (e.g.
  times, prices)
• Grammars can be developed independently (using
  shared core grammar)
• Shared and Cross-Domain grammars significantly
  reduce effort in expanding to new domains
• Separate grammar modules facilitate associating
  parses with domain tags - useful for multi-domain
  integration within the parser

16
Translation with Multiple Domain Grammars

• Parser is loaded with all domain grammars
• Domain tag attached to grammar rules of each
  domain
• Previously developed grammars for other domains
  can also be incorporated
• Parser creates a parse lattice consisting of
  multiple analyses of the input into sequences of
  top-level domain concepts
• Parser disambiguation heuristics rank the
  analyses in the parse lattice and select a single
  best sequence of concepts

17
Translation with Multiple Domain Grammars
18
A SOUP Parse Lattice
19
User Studies

• We conducted three sets of user tests
• Travel agent played by experienced system user
• Traveler is played by a novice and given five
  minutes of instruction
• Traveler is given a general scenario - e.g., plan
  a trip to Heidelberg
• Communication only via ST system, multi-modal
  interface and muted video connection
• Data collected used for system evaluation, error
  analysis and then grammar development

20
System Evaluation Methodology

• End-to-end evaluations conducted at the SDU
  (sentence) level
• Multiple bilingual graders compare the input with
  translated output and assign a grade of Perfect,
  OK or Bad
• OK meaning of SDU comes across
• Perfect OK fluent output
• Bad translation incomplete or incorrect

21
August-99 Evaluation

• Data from latest user study - traveler planning a
  trip to Japan
• 132 utterances containing one or more SDUs, from
  six different users
• SR word error rate 14.7
• 40.2 of utterances contain recognition error(s)

22
Evaluation Results
23
Evaluation - Progress Over Time
24
Alternative Approaches SALT

• SALT - Statistical Analyzer for Lang. Translation
• Combines ML trainable and rule-based analysis
  methods for robustness and portability
• Rule-based parsing restricted to well-defined set
  of argument-level phrases and fragments
• Trainable classifiers (NN, Decision Trees, etc.)
  used to derive the DA (speech-act and concepts)
  from the sequence of argument concepts.
• Phrase-level grammars are more robust and
  portable to new domains

25
Alternative Approaches Pangloss

• Glossary-based Translation
• Translates directly into target language (no IF)
• Based on Pangloss translation system developed at
  CMU
• Uses a combination of EBMT, phrase glossaries and
  a bilingual dictionary
• English/German system operational
• Good fall-back for uncovered utterances

26
Current and Future Work

• Expanding the travel domain covering descriptive
  as well as task-oriented sentences
• Development of the SALT statistical approach and
  expanding it to other domains
• Full integration of multiple MT approaches SOUP,
  SALT, Pangloss
• Task-based evaluation
• Disambiguation improved sentence-level
  disambiguation applying discourse contextual
  information for disambiguation

27
Students Working on the Project

• Chad Langley improved SALT approach
• Dorcas Wallace DA disambiguation using decision
  trees, English grammars
• Taro Watanabe DA correction and disambiguation
  using Transformation-based Learning, Japanese
  grammars
• Ariadna Font-Llitjos Spanish Generation

28
The JANUS/C-STAR Team

• Project Leaders Lori Levin, Alon Lavie, Monika
  Woszczyna, Alex Waibel
• Grammar and Component Developers Donna Gates,
  Dorcas Wallace, Taro Watanabe, Boris Bartlog,
  Marsal Gavalda, Chad Langley, Marcus Munk, Klaus
  Ries, Klaus Zechner, Detlef Koll, Michael Finke,
  Eric Carraux, Celine Morel, Alexandra Slavkovic,
  Susie Burger, Laura Tomokiyo, Takashi Tomokiyo,
  Kavita Thomas, Mirella Lapata, Matthew Broadhead,
  Cortis Clark, Christie Watson, Daniella Mueller,
  Sondra Ahlen