IMIX Future Research Lou Boves On behalf of the Programme Committee

1
IMIXFuture ResearchLou BovesOn behalf of the
Programme Committee
2
Points of Departure

• Progress in individual projects is according to
  expectations
• Basic versions of all components are operational
• Second version of demonstrator is operational
• Thanks to contributions of co-ordinator and
  system integrator
• Essential for collaborative research
• And for research in dialogue management and
  multimodal interaction
• Also useful as vehicle for demonstrating the
  concept of interactive information extraction to
  companies and general public
• Provided that the demonstration is given by an
  expert
• STEVIN Programme as context for ongoing and
  future research in IMIX

3
Two possible perspectives

• Where will the seven projects go?
• Covered by the individual presentations
• To what extent will the strategic and thematic
  priorities be satisfied?
• Focus on issues that alle projects have in common
  and on the second perspective

4
Individual Projects

• Scientific and strategic objectives will be met
• Scientific output will be large relative to the
  level of funding
• Mainly incremental contributions
• Breakthroughs?
• PhD theses are likely to be completed in time
• Quite exceptional, given that only 50-60 of
  Dutch PhD students completes thesis before the
  end of the contract
• All research lines are likely to be continued
  with funding from STEVIN, other national and
  international programmes and universities
• No loss of experience and knowledge

5
Strategic Priorities

• Improve visibility of Language and Speech
  Technology
• In academia
• In industry and society
• Strengthen multidisciplinary collaboration
• Strengthen position of Dutch as language for
  government and business communication

6
Visibility

• In academia
• Language and Speech Technology, as different from
  Computational Linguistics, will become solidly
  established in most universities
• Machine learning and engineering approach to NLP
  are gaining academic respectability in
  Linguistics
• Need to also inprove visibility of Language
  Speech Technology in Polytechnics and high
  schools
• Taken up in context of STEVIN Programme
• In industry and society
• Strengthen collaboration with companies
• Individual components
• Demonstrator
• The fact that the demonstrator runs on a laptop
  is important

7
Multidisciplinary Collaboration

• Several plans for collaborative research are in
  the making
• Many of which involve the Computer Science Group
  in Amsterdam and Dialogue Management projects
• Will result in closer collaboration between
  Linguistics and Computer Science Departments
• Continued collaborative work on the demonstrator
• Will help to promote engineering approach in Arts
  Faculties
• Unfortunately, no funds yet for human factors
  studies with multimodal interactive IE

8
Position of Dutch

• IMIX is integral part of the STEVIN Programme
• Several STEVIN Projects originated from
  collaborative research in IMIX
• Three Outreach Projects under way in STEVIN with
  close links to multimodal IE
• Ba, Ma/MsC, PhD students trained in Language and
  Speech Technology with focus on Dutch will enter
  the job market

9
Thematic Priorities

• Automatic Speech Recognition
• In the context of multimodal interaction
• Dialogue management and reasoning
• Information presentation
• Focus on language in multimodal settings
• Information Extraction

10
SPEECH

• Speech as an unsolved fundamental problem is
  still on the research agenda
• New speech-related corpora/databases in STEVIN
• Project on noise robustness in STEVIN
• ASR as crucial part of the common Demonstrator
• Word graphs as input for NLP
• Review meeting pointed towards need for renewed
  attention to TTS as research platform

11
Dialogue and Reasoning

• Collobaration and cross-fertilisation between
  rapid prototyping and formal specification
  approaches will continue
• Dialogue Management in IMIX Demonstrator is
  distributed
• Somewhat comparable with RITEL
• NLP
• Fusion
• Fission
• Experiments with the demonstrator will shed light
  on the role of conceptual reasoning in multimodal
  IE

12
Multimodal Information Presentation

• Is becoming solidly established in academic
  teaching and research
• Attention for the role of emotion, affect and
  personality in communication
• Human human
• Human system
• Prospects for better understanding of multimedia
  presentation and multimodality in interaction
  with information systems
• Future direction of research in multimodality in
  input and output will be influenced by the
  experience and results in IMIX

13
Information Extraction

• Research will continue in two directions
• Improve IE per se
• Participation in European and American
  competitions
• Further integration of automatic learning and
  knowledge-based NLP
• Improve IE through interaction
• Important role for Demonstrator
• Multimodal input nd presentation
• Experiments in universities and industry

14
Corpora and Annotation in IMIX

• Some corpus compilation work has been done
• Medical texts with semantic annotation from
  Rolaquad
• Medical questions
• Partly with follow-up questions in DM projects
• Factmine and QADR sets of facts and relations
• Very small speech corpus
• Planned corpus creation activities
• Real dialogues
• Spoken, typed, multimodal
• Small number of annotated pictures
• Corpus compilation projects in STEVIN
• From speech recordings to semantic adaptations