iGraph Helping People with Visual Impairments Gain Access to Graphical Information Through Natural L

1
iGraphHelping People with Visual
ImpairmentsGain Access to Graphical
InformationThrough Natural Language

• Shelley Roberts, Leo Ferres, Avi Parush Gitte
  Lindgaard
• Human-Oriented Technology Laboratory
• Carleton University,
• Ottawa, ON, K1S 5B6 - Canada
• smrobert_at_connect.carleton.ca

2
Introduction

• Reasoning is often easier when data is presented
  in graph form
• Graphs take advantage of the ability to recognize
  salient visual patterns
• Motivation
• Unfortunately not readily available to people who
  are blind or visually impaired.
• Objective
• To provide people with visual impairments
  non-visual access to graphical information.

3
Current Solutions

• Sonification Musical notes represent Y-axis
  values across time.
• Problems
• Hard to follow for non-musical ears, limited
  information (no legends, no information on the
  X-Axis,...), inconvenient for large datasets, .
• Screen Reader Reading numbers off of a
  spreadsheet
• Problems
• Inflexible (linearly reads numbers off of the
  spreadsheet), no interaction is possible,
  inconvenient for large datasets,

4
Our Solution iGraph

• Takes advantage of
• natural language flexibility
• richness of natural language to express complex
  concepts
• Requires less training
• Allows for real-time, graph exploration.

5
Development

• Currently provides information through short
  verbal descriptions (summaries of the data).
• The graph starts at 9 at X0. There is a increase
  at x1 to 16. There is an decrease at x2 to 14.
  There is an increase at x3 to 59. There is a
  decrease at x4 to 24. There is an increase at x5
  to 77. Finally, there is an decrease at x6 to
  40.

6
Tailoring iGraph

• iGraph is not only an engineering problem, we
  have to take into consideration two important
  issues
• The needs of people with visual impairments when
  interacting with graph and
• The language needed to used to convey visual
  information

7
User Needs Analysis

• User Needs Analysis
• Method used to gather needs and requirements of
  users
• Verify
• the need, functions and capabilities
• build a better language model
• validate the solution that iGraph currently
  provides

8
Method

• Semi-structured interview
• Demographics
• Previous experience
• User-centered system evaluation
• Information elicitation
• Wizard-of-Oz

9
Demographics

• 11 users
• Age range (19-56)
• All had some post-secondary education
• Visual conditions
• Congenitally blind to non-congenitally visually
  impaired
• 5 sighted through high school, and 6 were either
  born blind or lost their vision prior to high
  school

10
Previous Experience

• Previous experience
• High school, work, help kids with homework
• Current interaction
• Friends and family
• Magnifying glass
• Use fingers to draw on table
• Vision teachers
• Tactile graphs (home made)
• In general
• Horrible experience
• Try to avoid/nuisance but wish there was a better
  way
• Did not know of any technical aid available

11
Information Elicitation

• I have a graph in front of me. Please ask me as
  many questions as you need so that you obtain
  enough information in order to understand what
  the graph looks like
• Results helped to understand the type of language
  participants used.

12
Results Language Used
13
Wizard-of-Oz

• Immediate feedback regarding systems current
  function and usability
• Read out loud
• The graph starts at 9 at X0. There is a increase
  at x1 to 16. There is an decrease at x2 to 14.
  There is an increase at x3 to 59. There is a
  decrease at x4 to 24. There is an increase at x5
  to 77. Finally, there is an increase at x6 to
  40.
• Asked for their understanding of the graph and
  their feedback about the current system

14
Results Feedback

• Overall
• Able to visualize what the graph following the
  reading
• 9 of 11 liked how the graphs were described
• All reported that iGraph would be useful.
• 5 most requested requirements.

15
Conclusions

• People ARE able to understand verbal descriptions
  of graphs
• Overall iGraph was found to be useful
• But
• Users want context
• Needs to incorporate description
• General information needs to be followed by
  details
• Users want more interactivity

16
Our papers

• Parush, A., Ferres, L., Rasouli, M. Lindgaard,
  G. Impact of External Representation and Task on
  Graph Comprehension. To be presented at the
  Annual Cognitive Science Conference, July 2006,
  Vancouver, proceedings by Lawrence Earlbaum.
• Ferres, L., Parush, A., Roberts, S. Lindgaard,
  G. Helping people with visual impairments gain
  access to graphical information through natural
  language the igraph system. (Forthcoming) ICCHP,
  July 2006, Vienna, and Lecture Notes in Computer
  Science.
• Ferres, L., Parush, A., Li, Z., Oppacher, Y.
  Lindgaard, G. Representing and querying line
  graphs in natural language the iGraph system,
  (submitted for review) SmartGraphics06, July
  2006, Vancouver, Lecture Notes in Computer
  Science.

17
Credits

• Gitte Lindgaard (Director, HOTLab),
• Jennifer Woods (HTX),
• Bruce Tsuji, Cathy Dudek (HOTLab),
• Jing Liu (HOTLab)
• and the People who participated in the
  experiments.

18
Our Research Group

• Leo Ferres, Avi Parush (HOTLab, Carleton),
  Principal Investigators
• Stephan Jou (Cognos), Industry partner, iGraph
  architecture
• Shelley Roberts, Maria Rasouli (HOTLab,
  Carleton), UNA, WoZ, Behavioral Studies
• Zhihong Li, Yandu Oppacher, Petro
  Verkhogliad (SCS, Carleton), Data Structures,
  programming,
• Ricardo Tabone (Linguistics, University of
  Ottawa), NLU, NLG