VOICE Project

1
VOICE Project

• THE VOICE PROJECT
• Accompanying Measure
• Presentation of Giuliano Pirelli for the
• Projects Annual Review
• Brussels, 14th April 2000

2
Overview

• To promote the use of voice to text
• recognition systems for the deaf.
• Example words may be recognised by a PC and
• converted into text, enabling a deaf person to
  read
• off a monitor what is being said.
• Subtitles for
• school
• conferences
• TV transmissions
• telephone conversations

TEXT
VOICE
3
Telematics Applications Programme

• VOICE Accompanying Measure
• Giving a VOICE to the deaf, by developing
  awareness
• of VOICE to text recognition capabilities
• Objectives
• develop prototypes of voice systems for the deaf
• demonstrate the prototypes to relevant
  organisations
• use a VOICE Forum on the Internet as a Project
  tool
• organise meetings / workshops using the
  prototypes
• Partners
• JRC-ISIS, Kepler University of Linz,
  SoftSol-FBL, ALFA, CECOEV, IHSB
• Duration 2 years

4
Objectives

• Prototype demonstrator(innovative, simple,
  inexpensive, design for all)Interfaces
• ConferencesSpreading information, developing
  awareness
• Web SiteInformation flow, reference discussion
  point, internet / e-mail for the hearing impaired
  (equal access as anyone else)

5
Prototype demonstrator

• Application of voice to text recognition to
  enable hearing impaired, elderly and
  disadvantaged people to read off a monitor what
  is being said to them, at home, at school and at
  conferences.
• Subtitling facilities in education,
  tele-education and live television broadcasts.
• Test of speech recognition in telephone
  communication.

6
Subtitling

• Conferences and television broadcasts subtitling

7
On the telephone line

• Special equipment is ONLY neededat the deaf
  persons end.

8
Design for all

• VOICE TO TEXT RECOGNITION
• Development of friendly interfaces for the use of
  commercial speech recognition systems.
• Prototypes developed in
• English, Italian, Spanish, German, French.
• ACCESS TO THE WEB
• General approach for specific problems (deaf,
  blind, limited hands mobility for using of mouse)

9
VOICE Web Forum

The partners exchange all information via a VOICE
Forum on the Internet and spread the results and
best practices via a VOICE Web Site
http//voice.jrc.it
10
Background demand

• 25 of EU elderly or disabled(hearing loss
  increasing with age)
• 5 of EU deaf or hard of hearing
• Just imagine being one of Europes...
• 300,000 deaf (0.1 of the population)
• 3,000,000 partially deaf (1)
• 30,000,000 hard of hearing, disavantaged and
  foreigners(10)
• and try to read my lips.

11
Deafness

• We may help a person with physical impairment by
  pushing,
• We may help a person with visual impairment in
  crossing the road,
• We, the able, help him in HIS disability!
• DEAFNESS a communication limit, a disability
  concerning the SPEAKER and his control of the
  communication (visual) channel.
• We may help a person with hearing impairment in
  understanding our message from spoken VOICE to
  visual TEXT.
• HIS communication problem becomes OUR disability!

12
European dimension

• There are technical solutions the Consortium is
  helping people persons with hearing impairment
• to get the best results of them and
• to discuss with the producers and services
  providers
• in the best situation, supported by a technical
  international team, and
• on time a few months before
• Prototype as demonstrator of a possible reality,
  not of a dream (Accompanying Measure)

13
School or University

A slide is projected on the left screen by the
speaker the text generated by the speech
recognition system is converted into subtitling
lines and projected on the right wall-screen,
under the speaker's image taken by a
video-camera.
14
Subtitled Videoconference

A deaf user communicates with a colleague the
text generated by the speech recognition system
is converted into subtitling lines and overlaid
onto the correspondents video image.
15
Presentations foreseen in the Project Programme

• 1st Year Hours Participants H x P
• ICCHP Special Section 2,0 100-150 250
• ICCHP Workshop 4,0 10-15 50
• HANDImatica Conference 0,5 100 50
• HANDImatica Workshop 4,0 10-15 50
• 2nd Year
• Experts Austria 0,5 100 50
• Experts Austria 4,0 10-15 50
• Experts Italy/Europe 12,0 30-50 500
• total 27 400 1.000

16
Presentations organised by VOICE

• Hours Participants H x P
• ICCHP Vienna Budapest 1,0 50
  50
• Austrian Day Vienna 1,0 50
  50
• Voice Workshop Vienna 8,0 60
  480
• Handim. Bologna Worksh. 8,0 50
  400
• Handim. Bologna Stand 0,5 500
  250
• ASL Pavia 16,0 60 960
• AFA Como 3,0 200 600
• RAI Bologna Tech. 3,0 150 450
• RAI Bologna Users 6,0 200 1200
• Videoconf. Chile 2,0 140 280
• Dir. Did. Arona 7,0 200 1400
• Other 65 events 162,8 3048 4523
• total 219 4.800 10.695

17
VOICE Presentations

• Presentations foreseen in the Project Programme
• Hours Participants H x P
• 27 400 1.000
• Presentations organised by VOICE
• 219 4.800 10.695
• rate 1000

18
Achieved results

• Presentations
• Orgnaised 1000
• Costs 300 no extra funds
• Prototype
• conversation 100
• conferences 100
• television 80
• telephone 60
• Web
• Web Site 80
• Web Forum (User Groups) 60
• Accessibility for all (extra) 40
• User needs
• Technical 80
• User reports 80
• Fundamental and survey 80

19
Next steps
Duration 2 years -------------------
VOICE New Logo spreading results

20
Project Programme Changes

• WP09 and WP10 conferences anticipated from 4th to
  3rd semester
• WP06 Web Site and Internet Forum delayed from 3rd
  to 4th semester
• continuation after Projects end
• Staff two auxiliary and some thesis students
  instead of a doc/postdoc
• contracts signature delay
• experience and motivation

21
Review Recommendations

• Red Flag
• contracts signature delay, new documentation,
• retroactive starting date, delays in deliverables
• earlier Reviews date, no answer to 3 letters
• Review Recommendations
• ...
• ...
• mainly be nearer to the users and their needs
• we did ...

22
Review Recommendations

• Make the demonstrator software available as
  source code and executable on the web, so the
  software can be freely downloaded and
  subsequently uploaded when the user has changed
  it.
• Attempt to put current contacts with IBM, Dragon
  and LH on a formal basis. And request that they
  make some attempt to promote the web site where
  the demonstrator software can be downloaded.
• Formulate a detailed plan of the specific
  awareness raising activities to be carried out
  up to the end of the project.
• Resubmit Deliverable 02.1 Pamphlet as an
  informational brochure promoting the work of the
  Voice consortium and their web site. Efforts
  should be made to produce a multi-lingual
  brochure of high quality
• Resubmit Deliverable 04.1 User needs analysis
  report giving attention to scientific
  state-of-the-art and reviewers comments in the
  previous section.
• Web Site follow-up after the Project's end.

23
Projects end

• Projects activities on time, but users need time
  to get familiar with new tools
• . . .
• . . .
• August-October 2000 continuation useful for both
  TAP and VOICE (without extra funding)
• transition phase for autonomous life of the Site
  and Forum,
• if no other problems (extra review, extra
  reports, delays, )

24
Work Packages

• WP01 Scientific coordination
• WP02 Activities with external groups
• WP03 Financial coordination
• WP04 Users' needs and validation
• WP05 hw/sw demonstrator
• WP06 www VOICE Forum
• WP07 ICCHP98 Vienna, etc.
• WP08 Handimatica98 Bologna, etc.
• WP09 Austrian meetings
• WP10 European Workshop

25
Deliverables

• D01.1 VOICE Project Final Report
• D02.1 VOICE Project Pamphlet - Video
• D04.1 User needs analysis Report
• D04.2 Demonstrators validation Report
• D05.1 Demonstrator
• D06.1 VOICE Forum on the Internet
• D07.1 ICCHP-98 sections proceedings
• D08.1 Italian experts meeting Report
• D09.1 Austrian experts meeting Report
• D10.1 European workshops programme
• D10.2 European workshops proceedings

26
Background my activity

• Photo from JRC Eurocourse 1993 programme
• Information Technology and Disability -
• the Assistive Technology Approach
• Tests at
• Policlinico
• Milano
• Phonos system
• developed by
• TXT
• (TIDE)

27
Conclusions

• Tests at Policlinico Milano using Phonos system
  developped by TXT with contacts with TIDE

28
Letteratura italiana il Leopardi

• Che fai tu, luna, in cielo ?dimmi, che fai,
  silenziosa luna ?Sorgi la sera, e
  vai,contemplando i deserti.
• dal Canto notturno di un pastore errante
  dellAsia, composto tra il 22 ottobre 1829 e il
  9 aprile 1830,pubblicato la prima volta nell
  edizione fiorentina del 1831.

29
Geografia spaziale
30
Navigando verso lignoto