Speech Translation CMU Consortium

1
Speech TranslationCMU Consortium

• Babylon Kickoff Meeting
• 6.6.02
• Alex Waibel
• Carnegie Mellon University
• http//www.c-star.org

2
Overview

• Background
• Goal
• Organization and Team
• Technical Status
• PDA Integration
• Speech Recognition
• Synthesis
• MT
• Plans for the coming Year
• Interlingua A Tutorial

3
C-STAR, now
LH
IRST
CLIPS Limsi
Karlsruhe EML DFKI Siemens
SRI
ATR-ITL Sony
UPC- Barcelona
ETRI
CMU MIT Lincoln Labs ATT Fonix
HKU
IIT
4
Interlingua Approach

• C-STAR Partners Developed Common Interlingua 6
  languages
• Need only N parser/generators instead of N2
• Rapid Addition of New Output Language
• Can generate culturally / contextually
  appropriate interpretation
• Eliminate Disfluencies, Clean-Up Language
• Generate Paraphrase in Own Language for
  Verification

5
Speech Translation Effort

• JANUS Speech Translator
• Since 1989, First US System, ICASSP91
• Interlingua Based
• Spontaneous Speech (93)
• C-Star Demonstrations 1993, 1999
• Approaches
• MT Grammar Based, EBMT, Statistical MT (Direct,
  Interlingua), Hybrids
• Recognition HMM, NN Multilingual Recognition
  and Portability
• Synthesis Concatenative Synthesis
• Projects
• C-STAR (English,German,Japanese,Korean,Italian,Fre
  nch,Chinese,Spanish)
• ATR (English,Japanese,German)
• Verbmobil (English, German, Japanese)
• Nespole (English, German, Italian, French)
• Egypt (English, Arabic)
• Fame (English, Spanish)

6
JANUS System Overview
7
Telephone / Video Conferencing

• Real-time requirement
• Cooperative user online
• Example Task Domains
• appointment scheduling
• travel planning
• hotel reservation
• telephone shopping
• Problems
• Bandwidth
• Language

8
Web-Based Travel Assistance
9
Wearable Language Assistants
navigation
information
translation
pictures
10
Project Goals

• Two-Way Speech-to-Speech Translation
• Languages English-Arabic, (Option Chinese,
  Others)
• Platform Laptop and/or PDA
• Approach
• Interlingua Based
• Learning Approaches as Much as Possible
  (Portability, Coverage)
• Data Collection
• Use Data from Babylon Partners (MITRE, IBM, CMU)
• Collect in-House, Produce Translation/Paraphrasing
  

11
Team

• Carnegie Mellon University
• Machine Translation
• Interlingua Design
• Semantic Grammar Development
• Data Collection and Translation
• Synthesis and Recognition, Portability
  Experiments
• Multimodal Technologies
• PDA Based Speech Recognition
• Cepstral Technologies, LLC
• PDA Based Speech Synthesis
• Mobile Technologies, LLC
• PDA Based System Integration