Video Interactive Services

1
Video Interactive Services with
VoiceXML SpeechTEK 2006, San Francisco
Dave Burke PhD Chief Technology Officer, Voxpilot.
2
Introduction

• Question
• Whats beyond Interactive Voice Response?

3
Introduction

• Answer
• Interactive Voice and Video Response (!)
• queue video

4
Contents

• Modern Video Telephony
• Video Media in VoiceXML
• Summary

5
Modern Video Telephony

• New developments in handsets and networks are
  enabling a new world of video interactive
  services
• Example applications
• Video mail
• Video entertainment services
• Video call center / coaching
• Video blogging
• And more

6
Modern Video Telephony

• Video adds a new channel and dimension to
  humanmachine interaction
• New possibilities
• Richer interface
• More intuitive
• Experience of immediate real-time interaction may
  be likened to interactive TV

7
Modern Video Telephony

• Major developments fueling video telephony
• 1. 3G mobile video telephony (circuit-switch)
• Available across Europe and Asia
• Available in Japan (NTT FOMA)
• 2. Video-over-IP telephony

8
Modern Video Telephony Circuit Switch

• Circuit-switch telephony based on H.324M
  technology from ITU and adopted by 3GPP
• H.324M (3G-324M)
• Runs over 64 kbps channel
• H.223 multiplexes audio/video/data
• H.245 provides control
• Capability exchange
• Media channel management
• DTMF
• Complex low-bit rate codecs, e.g.
• Video H.263 / MPEG-4
• Audio AMR / G.723

9
Modern Video Telephony Circuit Switch

• Several advantages to circuit-switch video
• Ubiquitous access (any 3G phone)
• No DRM issues
• Standard telephony revenue collection
• Fast, immediate interface

10
Modern Video Telephony Video-over-IP

• Video-over-IP is gaining momentum, e.g.
• Skype 2.0 (with video)
• SonyIVE
• 3GPP IP Multimedia Subsystem
• Signalling such as SIP/H.323 works unchanged
• RTP transports video (IETF defines payload
  formats for video codecs)

11
Video Media in VoiceXML

• VoiceXML was not designed originally for a video
  channel
• However, VoiceXML 2.0 / 2.1 is suitable for
  running video applications without change
• Key idea Work with multimedia container file
  formats, (e.g. .3GP, .MP4, .MOV)

12
Video Media in VoiceXML

• VoiceXML 2.x supports
• Video playback
• DTMF/ speech recognition on audio channel
• Video record
• POST video to Webserver
• Video call transfer

13
Video Media in VoiceXML - Playback

• ltvxml version2.0gt
• ltformgt
• ltblockgt
• ltpromptgt
• ltaudio srcwelcome.3gp/gt
• lt/promptgt
• lt/blockgt
• lt/formgt
• lt/vxmlgt

14
Video Media in VoiceXML - Recognition
Bargein occurs on audio channel causes video
frame freeze

• ltvxml version2.0gt
• ltformgt
• ltfield namepin typedigitsgt
• ltprompt bargeintruegt
• ltaudio srcpasswd.3gp/gt
• lt/promptgt
• lt/fieldgt
• lt/formgt
• lt/vxmlgt

15
Video Media in VoiceXML - Record

• ltvxml version2.0gt
• ltformgt
• ltrecord namevideomsg
• typevideo/3gpp
  Codecss263,samr/gt
• lt/formgt
• lt/vxmlgt

16
Video Media in VoiceXML - POST

• ...
• ltfilledgt
• ltsubmit namelistvideomsg
• nextsave_message.pl
• enctypemultipart/form-data
• methodpost/gt
• lt/filledgt
• ...

17
Video Media in VoiceXML - Future

• VoiceXML 2.x provides serendipitous and
  sufficient functionality for video applications
• But does it provide all necessary features?

18
Video Media in VoiceXML - Future

• W3C Voice Browser Working Group is adding
  features to VoiceXML 3.0 to support more advanced
  video applications
• Examples
• Audio / video mixing
• Detect media channel types
• Barge-in modes
• Add general ltmediagt for ltaudiogt
• Enable text overlay (e.g. W3C Timed Text)

19
Summary

• Briefly reviewed developments in video telephony
• Looked at how VoiceXML supports video
  applications today
• Discussed future developments in VoiceXML related
  to video