Creating User Interfaces

1
Creating User Interfaces

• Discussion current speech reco products.
• VoiceXML
• Homework register as developer at
  studio.tellme.com. Do tutorial. Come to class
  with phones (prepare to share).

2
Discussion

• Reports on current speech products

3
Telephone

• Caller to system speech recognition,
• using grammars (limited vocabulary, general
  audience, no training)
• optional use of touch tones (numbers)
• System to caller recorded audio (wav files) plus
  TTS (text to speech)
• Limited bandwidth, in comparison to other
  applications, but very familiar, ubiquitous
  medium
• 800 long distance, some airline information
  systems, others?

4
studio.tellme.com

• Company that provides engine for applications
• Provides developing environment
• We are doing the tellme version of VoiceXML, but
  it appears to be standard.
• Register as a developer
• Provide your own id assigned a PIN
• Scratchpad for quick testing
• Put VoiceXML in ScratchPad place (no audio files)
• 1-800-555-VXML (8965)
• SAY id and then PIN.
• Application URL for projects with multiple files
• To look at someone else's project, you change
  your Application URL
• called pointing your account to a new source.

5
(No Transcript)
6
VoiceXML

• XML document (VXML header)
• VoiceXML has tags for flow-of-control and
  calculations.
• Also can use ltscriptgt for JavaScript
• Grammars come in different varieties. We will
  use the tellme way.
• Grammars are included in CDATA tags to prevent
  XML interpretation.
• Many grammars constructed for you.
• ltfield name"answer" type"boolean" gtwill
  listen for yes or no. ltfield name"price"
  type"currency" gt will listen for currency.
• ltmenu gt ltchoice gt ltchoicegt for list

7
example

• This is current URL that my MyStudio points to.
  You can set yours to point to it.
• http//newmedia.purchase.edu/Jeanine/interfaces/
  rps1.xml
• NOTE this is not particularly appropriate but

8
Very brief overview

• ltvxmlgt document contains ltformgt and/or menu
  elements.
• ltformgt can contain ltblockgt, ltfieldgt
• ltblockgt can contain ltaudiogt or do its own audio
• ltfieldgt can contain ltpromptgt, ltgrammargt,
  ltnoinputgt, etc.
• NOTE certain types of ltfieldgt elements use
  built-in grammars, for example, boolean
• Can have a child node ltfilledgt that indicates
  what to do if there is a match
• ltmenugt is a compressed way use a simple grammar

9
Very brief, cont.

• Logic can be done using a ltscriptgt element that
  contains a variant of JavaScript and/or
• vxml logic elements, including
• ltvargt
• ltifgt, ltelsegt ltelseifgt
• other
• These may be part of a ltfilledgt element

10
Audio

• Tellme studio provides way to record your
  speech as a wav file to upload to a website.
  Sends it to your email address
• You upload your VoiceXML file plus any wav files
  (and anything else)ltaudio src"mygreeting.wav"gtWe
  lcome to my site lt/audiogtIf tellme can't find
  the mygreeting.wav file, it uses its Text to
  Speech on the string "Welcome to my site".
• Note you also can use a full URL
  http//....
• You put in the URL for the voicexml file into
  your Tellme studio account, called pointing to
  the URL.
• TEST

11
VoiceXML basics, continued

• ltformgt element can contain
• ltblockgt elements, which can contain ltaudiogt,
  ltgogt, other
• ltfieldgt which can contain
• ltpromptgt
• ltgrammargt (if not one of built-in grammars)
• ltfilledgt
• ltvargt tags can be at different levels (for
  example, document, block, or higher levels)
• ltifgt ltelseifgtltelsegt tags
• ltscriptgt elements for JavaScript (which can also
  appear in expressionsgt

12
VoiceXML basics typical case

• a form element
• ltfieldgt
• ltpromptgt, made up of ltaudiogt, with reference to
  recorded wav file and backup text
• ltgrammargt, if NOT using built-in grammars
  designated by type attribute of field. This is a
  CDATA section.
• ltfilledgt with (follow-on) code using field
• ltcatchgt for nomatch, noinput cases

13
Caution

• A form contains various elements,
• including
• a field.
• If a field has a grammar and the grammar is
  satisfied, control goes to a
• filled tag

14
obligatory

• lt?xml version"1.0"?gt
• ltvxml version"2.0"gt
• ltformgt
• ltblockgt
• ltaudio src"prompt1.wav"gtHello, world lt/audiogt
• lt/blockgt
• lt/formgt
• lt/vxmlgt

recorded using tellme studio
backup using TTS, just in case src file missing
15
Preparation objects

• JavaScript (and other languages) use classes and
  objects
• Objects (aka object instances) are declared
  (created, instantiated) as members of a class
• Objects have
• properties ('the data')
• methods (functions that you can use 'on' the
  objects)
• static methods
• Math.random

16
Example tm_date

• var dt new tm_date creates a date/time
  object.
• Use methods to extract/manipulate information
  held 'in' dt.
• var day dt.get_day()
• Use static methods supplied to do common tasks
• var dntm_date.to_day_of_week_name(day)
• or directly
• var dntm_date.to_day_of_week_name(dt.get_day())

17
outline

• Header stuff
• script with external reference
• script (code) encased in CDATA notation
• Form/Block, with text to speech using value
  produced by script
• Closing stuff

18

• lt?xml version"2.0"?gt
• ltvxmlgt
• ltscript
• src"http//resources.tellme.com/lib/code/tm_date.
  js"/gt

Will make use of data functions
19

• ltscriptgt lt!CDATA
• var dt new tm_date()
• var monis tm_date.to_month_name(dt.get_month
  ())
• var dateis dt.get_date()
• var dayis tm_date.to_day_of_week_name(dt.get
  _day())
• var yearis tm_date.to_year_name(dt.get_full_
  year())
• var houris dt.get_hours() - 4
• var minutesisdt.get_minutes()
• var whole 'The date is ' monis'
  'dateis'. It is ' dayis'. The time is '
  houris ' ' minutesis
• gt lt/scriptgt

brute force correction from GMT
20

• ltformgt
• ltblockgtHello.
• ltvalue expr"whole"/gt
• Good bye.
• lt/blockgt
• lt/formgt
• lt/vxmlgt

Can use block for audio
21
Example my family

• Directed responses to 3 family members
• Daniel,
• question/response on activities
• Aviva,
• question/response on number of cranes
• Esther
• response
• Calculations (arithmetic) done using variables
• if tags
• The cond attribute is a condition test.
• limited error handled exit on no-match event
• alternative is to repeat prompt, generally using
  count attribute

22

• ltvxml version"2.0"gt
• ltformgt
• ltfield name"childid"gt
• ltpromptgt
• ltaudio src"whosthis.wav"gtHello. Who is
  calling?lt/audiogt
• lt/promptgt

23

• ltgrammar type"application/x-gsl" mode"voice"gt
• lt!CDATA
• dan daniel (daniel meyer) (dan meyer) ltchildid
  "daniel"gt
• aviva (aviva meyer) ltchildid "aviva"gt
• esther (esther minkin) ltchildid "esther"gt
• gt
• lt/grammargt

24

• ltcatch event"noinput nomatch"gt ltaudio
  src"sorry.wav"gtSorry. I didn't get that.lt/audiogt
  ltexit/gt lt/catchgt
• ltfilledgt
• ltif cond"'daniel'childid"gt
• ltgoto next"danfollowup"/gt
• ltelseif cond"'aviva'childid"/gt
• ltgoto next"avivafollowup"/gt
• ltelseif cond"'esther'childid"/gt
• ltgoto next"estherfollowup"/gt
• ltelse/gt
• ltreprompt/gt
• lt/ifgt
• lt/filledgt
• lt/fieldgt
• lt/formgt

never happens
Note inner, single quote marks. Note double 's
25

• ltform id"danfollowup"gt
• ltfield name"today" gt
• ltpromptgt
• ltaudio src"congratsdan.wav"
  gtCongratulations on the new job. Did you work on
  your thesis, or do aikido or jo today?lt/audiogt
• lt/promptgt
• ltgrammar type"application/x-gsl" mode"voice"gt
• lt!CDATA
• aikido (i key dough) lttoday "aikido"gt
• thesis (work) lttoday "thesis"gt
• jo (joe) lttoday "jo"gt
• both (all) (everything) ((i key dough)
  jo)lttoday "both"gt
• none nothing (sort of) lttoday "nothing"gt
• gt
• lt/grammargt
• ltcatch event"noinput nomatch"gt ltaudio gtI didn't
  quite understand. Call or send e-mail.lt/audiogt
  ltexit/gt lt/catchgt

26

• ltfilledgt
• ltif cond"today'aikido'" gt
• ltaudiogtSome aikido is fine. lt/audiogt
• ltelseif cond"today'thesis'" /gt
• ltaudiogtGood, but do other things
  also.lt/audiogt
• ltelseif cond"today'jo'" /gt
• ltaudiogtdon't get hit in the
  head.lt/audiogt
• ltelseif cond"today'both'" /gt
• ltaudiogtDoing some of everything is best.
  lt/audiogt
• ltelseif cond"today'nothing'"/gt
• ltaudiogt You deserve a break, but
  remember you want to be done by September.
  lt/audiogt
• ltelse/gt
• ltaudiogt See you soon.lt/audiogt
• lt/ifgt
• lt/filledgt lt/fieldgt
• ltblockgt
• ltaudiogt Good bye lt/audiogt lt/blockgt
  lt/formgt

27

• ltform id"avivafollowup"gt
• ltvar name"rest" expr"1000"/gt
• ltfield name"bcount" type"number"gt
• ltpromptgt
• ltaudio src"howmanycranes.wav"gtHello,
  Aviva. How many cranes have you made? lt/audiogt
• lt/promptgt
• ltgrammar type"application/x-gsl" mode"voice" gt
• lt!CDATA
• NATURAL_NUMBER_THRU_9999
• gt
• lt/grammargt
• ltcatch event"noinput nomatch"gt ltaudio
  src"sorry.wav"gtSorry. I didn't get that.lt/audiogt
  ltexit/gt lt/catchgt

28

• ltfilledgt
• ltassign name"rest" expr"1000-bcount"/gt
• ltaudiogt ltvalue expr"rest" /gt lt/audiogt
• ltaudio src"togo.wav"gt to go. lt/audiogt
• ltif cond"restlt200" gt
• ltaudio src"homestretch.wav"gtYou're in the
  home stretch lt/audiogt
• ltelseif cond"restlt500" /gt
• ltaudio src"morethanhalf.wav"gtMore than
  half way lt/audiogt
• ltelseif cond"restlt800" /gt
• ltaudio src"goodstart.wav"gtOff to a good
  start lt/audiogt
• ltelse/gt
• ltaudiogt Get a move on lt/audiogt
• lt/ifgt
• ltaudio src"goodbye.wav"gtGood bye.
  lt/audiogt
• lt/filledgt
• lt/fieldgt
• lt/formgt

can't use lt
29

• ltform id"estherfollowup"gt
• ltblockgt
• ltaudio gtHello, Mommy. This is all I can do
  now. lt/audiogt
• lt/blockgt
• lt/formgt
• lt/vxmlgt

30
again Application logic

• Implicitly in way menus and grammars work
• VoiceXML elements (for example, ltifgt and ltvargt.
• JavaScript code in attributes (for example, cond,
  expr)
• JavaScript code in ltscriptgt lt/scriptgt
• Encase in CDATA to avoid problems with certain
  characters
• external JavaScript code, cited using ltscript
  srcfile address /gt

31
Class work if time

• EVERYONE (who hasn't already) signup
  studio.tellme.com
• Design SIMPLE application (you may work in
  groups)
• Ask one question
• Detect and respond to each of 2 or 3 answers
• Use examples here for models
• All text to speech
• Pick (at least) one and implement.

32
Homework

• Go to studio.tellme.com
• signup as developer
• try examples (using scratch pad)
• record some voice samples
• Study tellme tutorials!!!!
• Note final project will be a tellme application,
  may be done in teams of 2 or 3.