Evaluating Sonified Rapid Serial Visual Presentation: An Immersive Reading Experience on a Mobile De

1
Evaluating Sonified Rapid Serial Visual
Presentation An Immersive Reading Experience on
a Mobile Device

• Mikael Goldstein, Ericsson Research
• Gustav Öquist, Uppsala University
• Staffan Björk, Interactive Institute

2
Outline

• Motivation and background
• Dynamic text presentation
• Rapid Serial Visual Presentation
• RSVP with Sound
• Evaluation of Sonified RSVP
• Conclusions and further work

3
Motivation

• Small screens affect readability negatively today
• Screens on mobile devices are likely to remain
  small tomorrow
• Readability on small screens is crucial for
  mobile usability
• Enhanced readability means enhanced usability

4
Background

• Readability is often defined as the ease of
  which the meaning of a text can be comprehended
• On large screens readability improved with
  increased legibility
• But, traditional text presentation still requires
  space absent on small screens
• Trading a spatial dimension for a temporal may be
  one solution to get further

5
Dynamic text presentation

• Leading and Rapid Serial Visual Presentation
  (RSVP) are the two most commonly used dynamic
  formats

6
Dynamic text presentation

• Leading and Rapid Serial Visual Presentation
  (RSVP) are the two most commonly used dynamic
  formats

Lea
Leading
Leading scr
eading scrolls
ing scrolls the
crolls the text
the text horiz
text horizonta
horizontally a
ontally across
ly across the
ss the screen
e screen
en

• Leading scrolls the text horizontally across the
  screen

7
Dynamic text presentation

• Leading and Rapid Serial Visual Presentation
  (RSVP) are the two most commonly used dynamic
  formats

location
Rapid
Serial
Visual
Presentation
successively
presents
the text
at a
single
visual
location

• Rapid Serial Visual Presentation successively
  presents the text at a single visual location

8
Which format to choose?

• Comparisons between Leading and RSVP have so far
  been inconclusive
• Leading is more familiar and thus initially more
  acceptable to the users
• RSVP has, at least at normal reading speeds, been
  found to be more effective
• RSVP seem to suit the natural reading process
  better since the then text moves successively

9
Rapid Serial Visual Presentation

• Originated as a tool for studying reading
  behavior
• Has lately received more interest as a
  presentation technique in itself
• Has on large screens been found to be just as
  effective as a paper book

10
RSVP on mobile devices

• Reader, the first evaluation of RSVP on a PDA
• Compared to a paper book by 10 subjects
• No differences in reading speed or comprehension
• However, significantly higher task load for RSVP

11
Adaptive RSVP

• Attempts to mimic the readers cognitive text
  processing pace
• Can be seen as a form of predictive text output,
  just as T9 is a form of predictive text input
• Two adaptive algorithms were developed, and both
  were found to decrease task load for most factors
• However, even though task load may increase
  slightly, RSVP can give additional advantages

12
Sonified RSVP

• One of the less explored properties of the RSVP
  format is its capability to present other media
  simultaneously
• Adding sound to a text or a picture is generally
  considered not to impose any harmful effects.
• When watching a movie, sound is even considered
  to be a mandatory attribute
• Does the same line of reasoning apply for adding
  sound to the text being read?

13
Multimodal aspects

• Paivios dual coding theory states that
  information is processed through two independent
  channels
• Verbal information is processed in one channel
  and non-verbal in another
• The use of both channels simultaneously to convey
  information is called referential processing
• Sonified RSVP is intended to augment the visual
  channel of RSVP with non-verbal sounds

14
Sounds suitable for RSVP

• Auditory icons is one form of non-verbal sounds
  that may create intuitive links to objects or
  actions
• Gaver identified three types Symbolic,
  metaphorical and nomic
• Brewsters earcons, abstract and synthetic tones
  constructed from motives, were also an
  alternative
• However, nomic auditory icons had several
  attributes making them more feasible for linking
  to text

15
The Listen Reader

• An attempt to augment a paper book with sounds
• Tactile movements triggered the sounds
• Very popular among the visitors at the museum
• But, it was not possible to take with you..

16
The Bailando prototype

• RSVP reader implemented on a Compaq iPAQ
• iPAQ has a large screen, but is good for
  prototypes
• Presents the text via Adaptive RSVP
• Main difference compared to Reader is the GUI

17
The Bailando prototype

• RSVP reader implemented on a Compaq iPAQ
• iPAQ has a large screen but is good for
  prototypes
• Presents the text via Adaptive RSVP
• Main difference compared to Reader is the GUI

18
Sonification

• Bailando supports sonification by parsing
  embedded XML tags
• If a start sound tag is found, the sound linked
  to is played
• If a end sound tag is found all playing sounds
  are stopped

At a high speed a car speeded down the street,
as ltSOUND SRC"HONK.WAV"gt it honked its horn all
lt/SOUNDgt the children fled to the curb.
19
The experiment

• Performed by 12 subjects in a usability
  laboratory where each subject read with and
  without sound
• No differences in reading speed, comprehension,
  task load, or attitude was set as a null
  hypothesis

20
Resulting reading speed scores
Reading speed (wpm)
21
Resulting comprehension scores
Comprehension scores ()
22
Resulting task load ratings
23
Resulting attitude ratings

24
What did we learn?

• The addition of nomic auditory icons did not
  cause any difference in reading efficiency
• That the comprehension scores were equal seems to
  support the dual coding theory
• Unfortunately, the task load ratings remained
  high although reading speed was quite low
• The only significant difference in the experiment
  was the rating of immersion, in favor of
  sonification

25
A few remarks on sonification

• It seems that sonfication may give a better, or
  at least more immersive, reading experience
• However, it seems strange there was no positive
  co-dependence with other attitude ratings
• Maybe dramatic soundscapes could make the
  experience more exciting?
• Yet, that would probably require a professional
  sound producer, which may not be negative

26
Some conclusions

• The addition of sound to the text displayed using
  the RSVP format can improve the reading
  experience
• Maybe a more dramatic soundtrack could strengthen
  the other attitude values
• The most important finding is that it is possible
  to get an immersive experience on a small screen
• Adding more elaborate soundscapes to texts may
  bring forward applications that many users like

27
Future work

• Improve the sound playback capabilities
• Try out sonification with dramatic soundscapes
• Evaluate Sonified RSVP on smaller mobile devices
• Assess how it works in mobile environments

28
Questions and comments?

• Mikael Goldstein
• Usability Interaction Lab, Ericsson Research
• mikael.goldstein_at_era.ericsson.se
• Gustav Öquist
• Department of Linguistics, Uppsala University
• gustav_at_stp.ling.uu.se
• Staffan Björk
• PLAY Studio, Interactive Institute
• staffan.bjork_at_interactiveinstitute.se
• See also http//stp.ling.uu.se/gustav/bailando

29
Thank you!

• Mikael Goldstein
• Usability Interaction Lab, Ericsson Research
• mikael.goldstein_at_era.ericsson.se
• Gustav Öquist
• Department of Linguistics, Uppsala University
• gustav_at_stp.ling.uu.se
• Staffan Björk
• PLAY Studio, Interactive Institute
• staffan.bjork_at_interactiveinstitute.se
• See also http//stp.ling.uu.se/gustav/bailando