Semitransparent Video Assisting Deaf Persons in Meetings

1
Semi-transparent VideoAssisting Deaf Persons in
Meetings

• Department of Computer ScienceDorian Miller
• Karl Gyllstrom
• David Stotts
• James Culp

2
Overview

• Assistive technology for Deaf community
• Semi-transparent video applications

Opaque windows
Semi-transparent windows overlaid
3
Deaf Community

• Deaf population
• 20 million, 8.6 of total US population
• Deaf community lives with hearing world
• Education or work environment
• Unemployed or not in work force
• 16.5, 18-44 years
• 33.3, 45-64 years

http//gri.gallaudet.edu/Demographics/deaf-US.php
4
Assistive technology

• Video applications support collaboration between
  deaf and hearing
• Lecture scenario
• Deaf person in hearing lecture
• 1-on-1 collaboration
• Deaf person working with hearing person
• Video conferencing

5
Deaf Person Attending Presentation

• Audience member divides attention between
  simultaneous information.
• Attend to one information, miss other information

Watch presentation
and watch speaker
and watch interpreter
While taking notes
6
Facetop for Notetaking

• Deaf person uses computer to take notes
• While taking notes watch interpreter
• Looking at screen, it is easier to switch focus

Computer screen shows notes and interpreter
Web cam Connected to computer
Computer to take notes
Interpreter
7
Features

• Semi-transparent video
• Expand writing space
• Adjust transparency level
• Record video
• Review later
• Future feature
• Bookmark video

8
Considerations

• Student-interpreter interaction
• Eye contact used to set interpreting pace
• Interpreter agreement to be recorded

9
Suggestions from Deaf Community

• Problem Copy signing
• Blocked visibility
• Requires multiple signers
• Solution
• Watch video of signer

Desk
Signer
Blocked visibility
Copy Signer
10
Deaf Computer Classroom

• Problem
• Students face computer
• Students back to instructor
• Solution
• Video of instructor on screen

students
instructor
11
Deaf/ Low Vision

• Problem
• Low vision limits field of view
• Solution
• Position video in persons field of view

Limited field of view
Person in video
12
Personal Meeting Scenario

• Synchronous remote collaboration
• 1 deaf person
• 1 hearing person
• Conventional video conferencing
• Communicate through instant messaging (IM)
• Pointing with telepointer
• Shared workspace checkers

13
Facetop interface

• Semi-transparent video overlaid on workspace
• User point gesture at workspace
• User sees self for self registration

(Live video)(alphalt1)
(Workspace)
Facetop
(Live video)(alphalt1)



14
Communication

• Deaf-hearing pair
• Communicate through gestures
• Enhancement, but not complete solution
• Deaf-deaf pair
• Communicate fluently with sign language

15
Preliminary Result

• Participants successfully complete tasks
• Gesturing through video
• Signal acknowledgement
• Get other persons attention

16
Improvements

• Problem
• Participants dislike diffuse video workspace
  image
• Solution ACMSE Presentation
• Saturday 145350
• Computer Graphics and Information Visualization
• Improving the Visibility and "Sharability" of
  Semi-transparent Video in Shared Workspaces

17
Future Work

• Collaborate with Deaf community
• Formal evaluation of Facetop interface

18
Acknowledgements

• Feedback from Deaf community
• UNC Greensboro Disability Service Office
• Wilson Regional Center for the Deaf the Hard of
  Hearing (NC DHHS)
• Funding
• U.S. Environmental Protection Agency
  (R82-795901-3)
• National Library of Medicine
• IBM PhD fellowship