Tanmoy Bhattacharya

1
ICT for PwDswith Special Reference to Indian
Sign Language

• Tanmoy Bhattacharya
• Coordinator
• Equal Opportunity Cell
• University of Delhi
• tanmoy1_at_gmail.Com

2
Equal Opportunity Cell University of Delhi

• Access Audit 140 digitised reports
• Services Reading, Braille printing, e-text
  conversion, recording for talking books,
  Transportation, ICT lab, assistive devices, Sign
  Language Interpreters
• Events Inclusive Chess tournament, Sports
  events, Cultural Festival, National Disability
  Conference Awareness Workshops in Colleges and
  Schools
• Skills Development Courses English
  Communication, ICT for Blind and Mobility
  Impaired, News Reading and Cinematography, Sign
  Language Interpretation A, B and C Levels,
  Disability and Human Rights. Short-term Summer
  Courses on Computer literacy

3
On Sign Language

• Sign languages create representations in the
  space in front of the signer (Signing Space)
• Due to the importance of vision, signed languages
  take advantage of spatial representations (he,
  she, it, etc.)
• The linguistic uniqueness of sign localisation is
  beyond doubt

4
The Signing Space
5
Time in Indian Sign Language An Example
6
SL Technology

• Text/ Speech-to-Sign
• Voice recognition module (Speech Recogniser)
• Conversion of sentence to fit the grammar of sign
  language (Inter-Language Translator)
• 3D Avatar Animation Module
• Sign-to-Text/ Speech
• Sign Recognition Video Signal input to large
  vocabulary speech recognition database
• Automatic machine translation system to create a
  spoken language translation
• Problems Simultaneousness 3D body-centred
  Signing Space Sub-word units

7
Status of SL Recognition Systems

• Todays Sign Language recognition is at about
  the stage where speech recognition was 20 years
  ago
• -- Thad Starner, Head of the Contextual Computing
  Group at the Georgia Institute of Technology

8
Challenges to ICT for Deaf

• The major barrier in using ICT for the Deaf is
  the assumption that English/ Hindi/ State
  language is their first language their
  capabilities are often measured against
  understanding the written word
• Most effective ICT for the Deaf is visual rather
  than based on the written word or sound
• Text messages are limiting since it doesnt
  convey emotions, voice inflections or body
  language (similarly Text-Speech systems)
• The smart phones with front-facing cameras for
  videoconferencing can be used for video chat but
  are too much of a bandwidth hog

9
Applications Currently Available or Under Research

• Video Chat Softwares
• Online Dictionaries
• Speech-Signal Translators
• Automated SL Generation System
• Smart Phone Applications

10
Video Chat Software
11
ASL On-line Dictionary
12
SiSi (Say It Sign It)

• IBM Research, Hursely, UK, 2007
• Voice-Text-Sign

13
NHK Science Technology Research laboratories,
Japan
14
Smart Phone Application
15
Existing Infrastructure

• Schools, Common Service Centres, Primary Health
  Centres, Panchayats, Womens Self Help Groups
  (SHGs), already covered under various USOF
  programmes
• Rural and Remote Areas Mobile Services of USOF
  500 districts in 27 states
• Rural Broadband Scheme 8,61,459 Broadband
  connections Scheme for Intra-District Networks
  on Bandwith Sharing with 2.5 Gb capacity

16
Future Directions

• Development of the ISL sign set
• Development of software for converting Hindi/
  regional language words to ISL through online
  dictionary
• Interactive learning software using the NBT book
  series for shapes, measures, colours, time, money
  for 2-3 year olds newspaper and adult education
  for Deaf adults
• Send video over both 3G and Wi-Fi networks at a
  very low bit rate
• Optimisation of compressed video signals by
  increasing image quality around the face and
  hands to bring data rate down
• Motion detection to identify whether a person is
  signing or not for extending battery life

17
Contact

• Tanmoy Bhattacharya
• Coordinator
• Equal Opportunity Cell
• DU-NTPC Foundation ICT Training Centre
• Tutorial Building, Arts Faculty
• University of Delhi
• Delhi 110007
• Phone [phone] (Office)
• Email tanmoy1_at_gmail.com eoc_at_du.ac.in
• Website http//eoc.du.ac.in