THE INFLUENCE OF SCREEN READERS ON WEB COGNITION

1
THE INFLUENCE OF SCREEN READERS ON WEB COGNITION

• Tony Stockman Oussama Metatla Queen Mary,
  University of London

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
2
OVERVIEW

• Examine implications of SR technology for web
  cognition
• Report findings from survey and study of
  collaborative web use
• Propose draft taxonomy of errors in collaborative
  web interaction
• Examine potential role of non-speech sound in
  addressing identified problems

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
3
FEATURES OF SR WEB INTERACTION

• Default linear model of page presentation
• No ambient representation of location on page
• No representation of spatial layout
• No immediate indication of information density
• Little to assist formulation of mental model of
  page

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
4
CURRENT FUNCTIONALITY

• Focus Jaws, WE and VO.
• Cursor key navigation strength and weakness of
  analogy with other apps
• Other mechanisms
• By listing of navigating forward/back between
  links, headers, frames, tables, forms, text
  elements, markers etc.
• Forms mode

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
5
OVERVIEWS

• Important but often under supported/neglected
• Typically lists no. of links, frames, headers,
  and forms with reminder of related hot keys

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
6
CURRENT NON-SPEECH SOUND

• Jaws schemes
• Window-eyes events
• VoiceOver defaults

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
7
VoiceOver

• Embedded in OS
• Factors in switching
• Group v. DOM mode navigation
• Overview followed by interaction model
• Non-speech sound more visible

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
8
IMPLICATIONS FOR COGNITION AND INTERACTION

• Hindered by linear navigation EG pageing through
  search results
• Hot keys and markers help, but ignore density of
  info and spatial layout - consequences for
  collaboration
• Overviews neglect spatial layout, ordering and
  esthetics,
• Tables are navigable but lack overviews

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
9
DESCRIBING WEB PAGES

• Employed widely known pages
• Characteristics of sighted descriptions
• Relatively short but covering main features
• Column layouts, colours, mood, style, pictures,
  emotional response to message
• Characteristics of VI descriptions
• Longer, more factual and granular, more focus on
  function and usability
• Conclusion the two groups approach web tasks
  from widely differing contexts

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
10
COLLABORATIVE TASKS

• Involved pairs of sighted and visually impaired
  users
• Both could read the web pages used
• One gave instructions while the other performed
  the task
• The tasks involved
• Simple information searches
• Comparisons of data values
• Navigating pages and filling forms

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
11
VISUALLY IMPAIRED INSTRUCTOR

• These tasks were performed generally quite
  straightforwardly
• VI user was generally familiar with sites
• Sighted users perspective generally compensated
  for difference in views of each user
• Sources of problems
• Screen-readers focus unavailable to sighted user
• Sighted user referring to spatial layout
  unavailable to screen-reader

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
12
SIGHTED INSTRUCTOR

• Substantially more problems
• Point in task unclear because screen-reader focus
  unavailable to sighted user
• Sighted references to spatial layout
• CAPTCHEs
• Non-standard form controls
• Column headers not spoken on forms
• Dynamic updating of form fields

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
13
TOWARDS A TAXONOMY OF COLLABORATIVE ERROR 1

• Location disconnects
• Layout disconnects
• Missing objects
• Navigation disconnects
• Contextual disconnects

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
14
TOWARDS A TAXONOMY 2

• Affordance disconnects
• Modal disconnects
• Hollistic disconnects
• Multi-focus disconnects
• Esthetics disconnects

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
15
NON-SPEECH AUDIO

• Audio is inexpensive and widely used
• Screen-readers only gradually adopting limited
  non-speech sound, eg forms and progress bars
• Growing body of knowledge on how to design and
  use (www.ICAD.org)
• Range of techniques that could be examined as
  part or whole solutions to problems described

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
16
NON-SPEECH AUDIO 1

• Monitoring for dynamic changes
• The structure of Earcons could for EG reflect
  object type and nature of update
• Ambient sound might convey esthetics and/or
  interaction mode
• Auditory icons might signal affordance
  open/closed
• Spatial sound might convey overall layout,
  density, locations of users

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
17
NON-SPEECH AUDIO 2

• Example of non-speech auditory overviews compared
  with speech, more like a glance
• Spearcons for typical radio button options
• Crucial to avoid auditory overload, masking etc.

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc
18
CONCLUSIONS

• Speech-only model struggles to convey rich web
  content and enable increasingly complex
  interactions
• This is highlighted by gap in first impressions
  of common web pages
• Cross modal web collaboration os subject to a
  range of disconnects due to differences in
  presentation and interaction
• Non-speech audio is an under-used mechanism that,
  with careful design, could help to address some
  of the issues sited

Interaction Media Communication, Department of
Computer Science, Queen Mary University of
London http//www.dcs.qmul.ac.uk/research/imc