Demonstrator 3 overview - PowerPoint PPT Presentation

1 / 10
About This Presentation
Title:

Demonstrator 3 overview

Description:

We have created a shared image database that is used by both Imogen and Vidiam ... For example, I tested imogen.gen's ability to distinguish the answers by feeding ... – PowerPoint PPT presentation

Number of Views:32
Avg rating:3.0/5.0
Slides: 11
Provided by: wwwhomeC
Category:

less

Transcript and Presenter's Notes

Title: Demonstrator 3 overview


1
Demonstrator 3 overview
  • Changes made since Demo 2
  • Image database
  • Future plans
  • Demo

2
Changes in demonstrator
  • Fixed minor bugs/loose ends
  • Minor architectural changes
  • New versions of Gui, Dam
  • New picture database for Imogen.gen

3
Changes architecture
  • Peninput is now a submodule of Dam
  • Uses annotations of visual elements within a
    picture (if present)
  • Handles encircles, underlines, taps
  • New QA module vidiam.qa (which is meant to
    control picture retrieval)
  • vidiam.qa simply uses tfidf on the associated
    texts in the picture database. The result is
    that the correct picture is always selected.

4
Changes user interface
  • Based on usability tests made by students at
    Twente. This includes user interviews.
  • Some prompts and button labels have been changed
  • users thought some of the prompts and buttons
    were confusing
  • The old answer is no longer displayed when
    waiting for a new answer
  • some users confused the old answer with a new one
  • Erase mouse strokes button

5
Changes dialogue behaviour
  • Support for keyword queries
  • Users liked to type keywords. The system used to
    misinterpret this as an inform act rather than a
    question.
  • Basic support for hello/goodbye
  • Error prompts and interpretation of user answer
    were improved
  • Users misinterpreted some prompts as yes/no
    questions, and the system sometimes did not react
    to valid yes/no answers.

6
Shared image database
  • We have created a shared image database that is
    used by both Imogen and Vidiam
  • Contains about 800 pictures from the encyclopedia
    and the web
  • Annotated with
  • associated text
  • (optional) caption
  • (optional) visual element annotation
  • (future) modality profiles (Yulia Bachvarova)

7
Shared image database usage
  • Imogen.gen now includes the full picture database
  • performance seems much better
  • captions used by the dam to give general picture
    information when the user asks about a picture
  • visual element annotations used to recognise
    pointing at VEs and identifying them.
  • still few visual element annotations

8
Future plans
  • No major changes announced by any of the parties,
    minor updates still possible.
  • Suggestions module was mentioned, but...
  • Vidiam project is now at its end.
  • New version of Paradime?

9
Future plans
  • What about ASR? We have 3 ASRs available, but we
    need
  • large vocabulary
  • proper grammar or language model

10
Future plans
  • Make use of the Hooijdonk student answer
    database?
  • contains 32 questions with about 50 alternative
    answers for each, and a total of 412 pictures
  • Useful for performance tests? For example, I
    tested imogen.gen's ability to distinguish the
    answers by feeding it one answer, and ranking all
    1600 answers to see if the 50 right ones came
    up on top.
  • Add the pictures to the picture database?
Write a Comment
User Comments (0)
About PowerShow.com