Nessun titolo diapositiva

1
The NESPOLE! Multimodal Interface for
Cross-lingual Communication - Experience and
Lessons Learned
Showcase scenario

• When the client wants to know more about a
  particular topic or prefers to have a more direct
  contact, the speech-to-speech translation service
  allows her to interact in her own language with
  an APT agent.
• A videoconferencing session can be opened
  clicking a button
• The dialog starts.

• The first NESPOLE! Showcase was developed for the
  following scenario
• The client aims at organizing a trip in Trentino.
  
• She starts by browsing APT web pages to get
  information.

AeWhiteboard
NESPOLE! Monitor

• Remote speech translation textual
  representation of the last translated utterance
  that arrived from the other party.
• Recognized hypothesis string recognized text
  representation of the last utterance spoken by
  the local user.
• String converted from IF textual
  representation resulting from the translation of
  the last utterance spoken by the local user back
  into their own language.
• Cancel Translation button generates a red
  flashing message on the monitor of the other
  party indicating that the incoming translated
  message should be ignored.

• image loading
• free-hand drawing
• rectangular or elliptical area selecting
• URL opening.

The user can share all these operations with the
remote user as they were sitting around a table
exchanging brochures and illustrative material.
Hallo I would like to take a trip to Trentino
Hallo I would like to visit Trentino
The demands for user-friendly interfaces to
interactive computer applications have
significantly grown in recent years. Moreover, it
has became common practice to carry out
psychological studies for investigating the
users cognitive skills and their preferences and
expectations concerning interfaces. Throughout
the first two years of the NESPOLE! project we
took advantage of many opportunities to collect
data concerning the system and the interface
usability with actual users most of the
interface improvements leading to the current
version were based on the comments and
suggestions of these users and on our observation
of their behavior.
Experimental interactions (Summer 2001)
User group meeting(September 2001)
Monolingual data collection(February 2002)
User Studies
Interface Improvements
Quick map savingthe user can save the map by
simply clicking a button
Live video transmission
Recognized hypothesis string changed in System
hears and made editable by the user in order to
manually correct recognizers errors.
Image scrolling and zoom
Audio Enable/Disable (push to talk) button
Dialog History Window it contains all feedback
text strings ordered on a temporal base
Progress bars and icons indicating system status
and audio flow
NESPOLE! system allows users speaking different
languages to interact in a natural way by means
of

• hearing their original voice and their translated
  speech
• seeing the face and expression of their partner

• exchanging data information through gestures on
  drawings and map selections
• monitoring the translation process and correcting
  it if and when something goes wrong.

What does the client need? A light PC with a
standard free software for videoconferencing, a
microphone and a camera common users should be
able to use NESPOLE! Service from their home PC.