clause: weather_event

1
Mokusei Multilingual Conversational Interfaces
S. Seneff, J. Glass, T.J. Hazen, J. Polifroni,
and V. Zue MIT Laboratory for Computer Science
Y. Minami NTT Cyberspace Laboratories
Language as Interface
Research Objectives

• Language Understanding (TINA Seneff, Comp Ling,
  92 )
• Japanese grammar contains gt900 unique
  non-terminals
• Translation file maps Japanese words to English
  equivalent
• Produces same semantic frame as for English
  inputs
• Left recursive structure of Japanese requires
  look-ahead to resolve role of content words
• Parse each content word into structure labeled
  object
• Drop off object after next particle, which
  defines role and position in hierarchy
• Language Generation (GENESIS, Glass et al., ICSLP
  94)
• Used English language generation tables as
  template
• Modified ordering of constituents
• Provided translation lexicon for words
• Many language specific challenges, including
  constituent ordering, quantifier translation, and
  multiple meanings

• Speech Recognition (SUMMIT Glass et al., ICSLP
  96)
• Lexicon gt2,000 words with phonemic
  pronunciations
• Phonological modeling
• Japanese specific phonological rules, e.g.,
• desu ka ? /d e s k a/
• Japanese phonetic units mapped into English ones
• Acoustic modeling
• Used English models to generate forced
  transcriptions utterances
• Retrained acoustic models to create hybrid models
• Language modeling
• Class n-gram using 60 word classes. trained on
  3,500 read spontaneous sentences
• Also exploring a class n-gram derived
  automatically from TINA
• Speech Synthesis
• NTT Fluet text-to-speech system

• Explore language-independent approaches to speech
  understanding and generation
• Port human-language technologies for English
  conversational interfaces to Japanese
• Use existing Jupiter domain as test case
• A telephone-only conversational interface for
  weather information
• More than 500 cities worldwide (350 in US)
• On-line information from four Web sites
• Use the Galaxy client server architecture

Lessons Learned
Future Plans
Language as Content

• Our approach to developing multilingual
  interfaces appears feasible
• A top-down approach to parsing can be made
  effective for left-recursive languages
• Word order divergence between English and
  Japanese motivated a redesign of our language
  generation component
• Novel technique of generating a class n-gram
  language model using the NL component appears
  promising
• Involvement of Japanese researcher is essential

• Additional data collection from native Japanese
  speakers
• Nearly 1000 sentences were collected in December
• Improvement of individual components
• Vocabulary coverage, acoustic and language models
• Parse coverage
• Continued development of a more sophisticated
  language generation component
• Expansion of weather content for Japan

• Use the same internal representation for Japanese
  and English
• Update from Web sites and satellite feeds at
  frequent intervals
• Parse all data into semantic frames to capture
  meaning
• Scan frames for semantic content and prepare new
  relational database table entries

English Some thunderstorms may be accompanied by
gusty winds and hail Japanese
rain/storm

• clause weather_event
• topic precip_act, name thunderstorm, num pl
• quantifier some
• pred accompanied_by
• adverb possibly
• topic wind, num pl, pred gusty
• and precip_act, name hail

Frame indexed under weather, wind, rain, storm,
and hail
weather
Note Sample sentences from Japanese speakers can
be played from PC
wind
hail