speech in, speech out

1
speech in, speech out
2
speech-in components

• Nuance server
• compiled recognition grammar, master language
  package, licence manager
• Nuance client

3
recognition grammar

• anticipate users responses
• what pieces of information are needed to
  complete the dialog?
• in what order will they be requested?
• one piece of information at a time in
  particular order (directed dialog), several
  pieces at once, in any order, and prompt for
  missing items (mixed initiative)?

4
recognition grammar

• syntax
• Nuance Grammar Specification Language (GSL)
• Diamant Speech Recognition Grammar Format
  (SRGF)

5
recognition grammar

• GSL grammar doc in a file with .grammar
  extension e.g. mygram.grammar (mygram will
  be the resulting package name)
• contents GrammarRuleName   GrammarDescription
• GrammarRuleName at least one uppercase
  character
• GrammarDescription sequence of words, grammar
  names, and operators that define a set of
  recognizable word sequences
• words (terminals) in lower-case
• operators

() concat (A B C ... Y) A and B and ...
disjunction A B C ... Y either A or B or...
? optional ?Y Y is optional
positive closure Y at least one Y
Kleene star Y zero or more Y
6
recognition grammar

• GSL grammar example expressions
• morning afternoon evening
• morning, afternoon, evening
• (good morning afternoon evening)
• good morning, good afternoon, good
  evening
• (?good morning afternoon evening)
• good morning, good afternoon, good
  evening, morning, afternoon, evening
• (thanks very much)
• thanks very much, thanks very very much,
  ...
• (thanks very much)
• thanks much, thanks very much, thanks
  very very much, ...

7
recognition grammar
.GO_FLOOR FLOORf (?the FLOORf floor)
(?the FLOORf please) (?Filler ?the FLOORf
floor ?please) ltfloor fgt Filler (i
would like to go to) (i want to go to)
(uh) FLOOR first return("1")
second return("2") third return("3")
fourth return("4")

• example GSL grammar
• .grammar file
• .slot_definitions file

floor
8
recognition grammar

• another option SRGF and export as Nuance GSL
• GrammarTest.bat

9
recognition grammar

• compiling the package (compile-package.bat)
• set PKGHOME path to your gsl file (w/o
  extension)
• nuance-compile PKGHOME English.America.1.3.0

10
recognition grammar

• testing the grammar (text)
• parse-tool -package path_to_your_model
• nl-tool package path_to_your_model grammar
  grammar_in_your_model

11
speech recognition

• running Nuance
• licence manager lm.bat
• recognition server rs.bat
• set PKGHOME path to your compiled model
• recserver -package PKGHOME
  lm.Addresseslocalhost config. ...
• testing the grammar (speech)
• xapp -package path to your compiled model
  lm.Addresseslocalhost

12
Diamant with speech-in

• running nuance client
• edit Diamant config file Clients.ini
• NuanceClient.bat
• (btw, have the licence manager and the server
  running too... duh!...)

13
Diamant with speech-in

• adding speech-in
• add device as usual
• activate recognition output ltstringgt start
  (start command) to nuance client
• read (speech) input from nuance
• client into variable as usual
• access recognition confidence
• (of type Real) like this varconfidence

14
speech-out components

• Mary server
• online at DFKI...
• Mary client
• MaryClient.bat

15
Diamant with speech-out

• adding speech-out
• add device as usual
• optionally, set format format ltstringgt
  (default plain text) and voice
• voice ltstringgt
• in output node, output ltstringgt to Mary client
  as usual

16
speech-enabled dialogs

• recognition tends to be imperfect...
• if recognition confidence low, then, for example
  (btw, think grounding)
• repeat question
• ask for confirmation (did you say blah?)
• inform user what they can say (you can say
  blah, bloo, and blee, please try again)
• but... dont let user get stuck in endless
  clarification dialog either!