OpenSALT Design Discussion

1
OpenSALT Design Discussion

• Y. Sun and J. Sherwani

2
What is SALT

• A set of tags that allow browsers to interact
  with speech.
  
• SALT, NLSML, Stochastic LM
• Independent of browser implementation
• Independent of speech engine implementation.

3
What is SALT (cont.)

• A complete voice browser

SALT
browser
4
What is SALT (cont.)

• FSG/CFG and/or N-gram speech decoding
• Semantic and context interpretation (optional)
• Less concern for speech output at the moment.

5
What is OpenSALT

• We dont know all the details yet.
• Our design goals are
• Open-source over Firefox/Mozilla browser
• Engine-independent but with Sphinx III and
  Festival variants as initial engines

6
Engine Independence

• Need an intermediate interface between the speech
  engines and the browser.
  
• SAPI over Sphinx III and Festival?
• In-house interface?

7
Firefox/Mozilla

• There are several implementation options under
  Firefox/Mozilla.
  
• Combination of XSL transforms and XBL bindings.
• Plug-in.
• Changes to Mozilla codebase.

8
Implementation Issues

• Common requirements
• Event exposure
• Initialization
• Access to browser features (parser, http)
• Other SALT conformance
• Distribution

9
Implementation XSL and XBL

• Advantage
• Easy to distribute
• Easy to setup and modify
• Disadvantage
• Less or even no control over events
• Less or even no tools available from browser

10
Implementation Plug-in

• Advantage
• Very easy to distribute
• Access to browser tools and DOM
• Dynamic initialization
• Disadvantage
• Yet to be seen
• Need implementation practice!

11
Implmentation Mozilla Codebase

• Advantage
• Powerful. We have the power of C/C, parser,
  http getter, script engine, on our side
  
• Disadvantage
• Hard to distribute
• Hard to incorporate into main Mozilla code-base.

12
Features of Sphinx, Festival

• Specifying timeouts to decoder
• External CFG/FSG Parser
• Semantic interpretation
• Output markup