Defining Learning Outcomes for Students Using Screen Readers:

1
Defining Learning Outcomes for Students Using
Screen Readers
Presented by Ted Wattenberg, Ed.S, C.R.C. San
Joaquin Delta College Stockton, CA
2
Hello, my name is Mike. I am an ATT 16 MHz.
Normal synthesized voice. My designers think I am
a big improvement over my older cousin
synthesized voices that are hard to listen to. My
designers thought my more natural sounding speech
would help people like me and want to use me to
read to them. Standing next to me, is Ted
Wattenberg, an instructor who helps people to use
me to read their college textbooks, newspapers,
and other books for enjoyment. But, Ted didnt
believe my designers. Ted wanted to see if I
would really make it easier for people to
understand and to comprehend what I was reading
to them. Good bye for now. I think Ted is getting
impatient, and wants to say something.
3
Hey, there Mike! Are you trying to pretend that I
was not part of this evaluation too? Ted, why
arent you standing up to him?
My name is Crystal and I am also an ATT Natural
Voice. Both Mike and I were part of this test. In
fact, I think that I did much better than Mike.
Sorry, Crystal
4
Independent studies by Riemer-Reiss Wacker
(2002) and Scherer (2004) indicate that up to 70
of assistive technology is rejected by their
users within two years. The reasons are not due
to the inability of the technology to accomplish
its purpose. People reject their assistive
technologies because they have not been delivered
in a way that integrates them well with their
learning styles and social environments.
(Riemer-Reiss Wacker, 2002 Scherer, 2004)
5
Thats not very good! I rather think that I do a
good job at reading the text that is given to me.
A lot of it is really hard to read.
For once I agree with you Mike. Sometimes the
things that people want me to read is quite
difficult. Sometimes a little embarrassing. And
at other times there are a lot of text mistakes.
6
Participation of People With Disabilities
Advocacy Usability Learner-Centered Instruction
7

• This presentation will review
• A usability evaluation of the affectiveness of
  the new ATT 16 MHz. Normal voices, in helping
  students to improve their intelligibility and
  comprehension using synthesized voices with
  screen readers.
• Development of learning outcomes.
• Further research of the learnability of screen
  readers.

8
What Are Screen Readers? (Dutoit, 2001)

• Complex computer applications that convert e-text
  into human speech.
• Synthesized computer speech sounds like a machine
  and is difficult to listen to.
• Read any text that has been converted to e-text.

9
Who uses Screen Readers as Assistive Technology?

• Estimated that 10 to 20 of postsecondary
  students in the United States have learning
  disabilities and could benefit from assistive
  technology.
• (Wattenberg, 2004)

• Low-vision
• Learning disabilities
• Mobility impairments
• English as second language
• Elderly
• Reading delays

10
Major Problems Found with the Use of Screen
Readers

• Dislike of synthesized voices
• Comprehension of read material
• Reading fatigue
• Cost
• Ability to find prepared e-text copies of text
  materials

• Cognitive Overload

(Dutoit, 2001 van Oostendorp Goldman, 1999
van Oostendorp de Mul, 1996)
11
The Process of Reading
Social Learning Understanding happens within a
social context.
Learnability Cognition of information is
understood through language processes.
Mental Representations The Context and language
processes result in mental representations that
are stored and retrieved from long-term memory
12
Mental Representations

• Context and language processes result in mental
  images that are stored and retrieved from
  long-term memory.
• Simple explanation of mental representations

13
A fabric of varying levels of representational
detail with factors of time, context, and prior
experience.
14
This has to be the time when you talk about me.
It has been hypothesized that it will be easier
for humans to learn positive affect towards
synthesized speech that is considered to the
listener as sounding more normal or more human.
(Dutoit, 2001)
15
I think that they liked me more than Crystal.
Wow! What an ego. Ill take on that bet. I am
sure that they wanted me more than you, block
head.
Gender bias has been shown to transfer to
synthesized speech. People trust what a mans
voice says more than a womens. Likewise, people
feel closer, or more affect, with a womens voice.
16

• I wanted to find out
• If a less synthesized and more normal sounding
  voice would help students read with higher
  comprehension?
• If the improved voices would lesson student
  frustration and result with higher student
  retention of their assistive technology/
• How long it takes for a student to gain positive
  affect towards a new synthesized voice?
• What are affective teaching strategies for
  teaching students to use a screen reader
  resulting in higher reading comprehension?

17

• A full blown statistical study takes a long time,
  costs a lot of money, and necessitates many
  peoples work. Something that most schools do not
  have or want to provide teachers developing
  curriculum.

18
Nielsens Discount Usability Heuristic Method of
Evaluation

• Discount means that this type of evaluation can
  be done quickly, inexpensively, and with only a
  few people working on the project.
• Widely accepted within the computer/technology
  industry.
• 80 - 95 of problems identified.
• Emphasis is on the users ability to perform tasks
  as designed and expected.

19
Heuristics used in this Study

• Visibility of system status
• User control and freedom
• Consistency and standards
• Affect
• Intelligibility
• Comprehension
• Persistence of comprehension over time

20
This is where I came in. Ted and I worked with
students to see if I could help them understand
what they were reading better because of my more
natural sounding voice. I read to them selections
of written texts and books which Ted thought
would be interesting and effective in determining
how well the readers comprehended what I was
saying.
I remember me working harder than you.
21
Procedures/Tasks of the Usability Evaluation

• Introduction and training
• Intelligibility/vocabulary
• Comprehension of short phrases
• Persistent comprehension of phrases
• Heuristic evaluation

22
Rainbow vocabulary
Strikes Air Prism Division Long Apparently Pot
Centuries Various Causes Bows
Universal Phenomena Physicists Raindrops Size
Greeks Norsemen Passed Aristotle Refraction
23
Rainbow Passage Understanding Short Phrases and
Sentences
When the sunlight strikes rainbows in the air,
they acted as a prism and form a rainbow. The
rainbow is a division of white light into many
beautiful colors.
These take the shape of a long around arch, with
its path high above, and its two ends apparently
beyond the horizon. There is, according to
legend, a boiling pot of gold at one end. People
look, but no one ever finds it. When a man looks
for something beyond his reach, his friends say
he is looking for the pot of gold at the end of
the rainbow.
24
Passages Used for Persistent Comprehension
Evaluation
Aesops Fable The Fox and the Goat Aesops
Fable The Raven and the Swan Tom Sawyer,
Detective, Chapter One, by Mark Twain Tom Sawyer,
Detective, Chapter Two, by Mark Twain
25
Ted, this is getting a little boring. When will
we be getting to the part about me. Isnt that
what we are here for anyway? How is it that my
more natural sounding voice can help students to
read quicker or prevent them from rejecting using
an assistive technology that can actually help
them learn?
26
Synthesized speech and text to speech technology
can aid students with disabilities affecting
delays in reading by

• Increasing reading comprehension
• Improvement of neuro-cognitive processing

OK, now he becomes a convert. See how much I can
help others with learning.
27

• Students with disabilities can have one or all
  three of these stages disrupted.

• The use of screen readers can impact all three
  stages of learning.

• A student will reject their assistive technology
  if it appears to the student to further disrupt
  their learning process.

28
Results and Implications of the Study
So, Ted, how did I do? Were all of the students
absolutely impressed with my reading ability and
my naturally sounding voice? You can be honest.
They loved me didnt they?

The evaluators overall reported thirty usability
problems rated level 3 or greater.
The majority of these problems were found in
heuristic criteria of affect, consistency, and
persistence.
Individual evaluators expressed one-quarter of
the usability problems two or more evaluators
identified the rest
29
Summary of Results

• Most evaluators chose Crystal over Mike
• All evaluators did very well with intelligibility
• All evaluators that never or rarely use screen
  readers did poorly with comprehension and
  persistence
• All evaluators did better after reading the
  extended passages
• Only those evaluators with extended experience
  with screen readers felt positive affect towards
  the synthesized voice.

30
Most Significant Problems Reported

• Cognitive overloading
• Speech speeds
• Pausing
• Either too much or too little emotion
• Voice was distracting

31
This sounds bad for me. They like Crystal more
than me. And, some of the students had a lot of
difficulty understanding me anyway.
Ha, Ha, Ha. So, block head. Now, what do you say?
32
Significant Aids to Instruction

• The more natural sounding voices are easier for
  new users of screen readers to adjust to
• Positive affect supports greater intelligibility,
  comprehension, and persistence
• Affect increased with usage
• Cognitive overloading seemed to lesson when the
  speed of speech was increased
• Emotion seemed to be less of a concern when speed
  was increased

33
Suggested Instructional Considerations

• Dominant learning modalities and learning
  capabilities should be evaluated

• A learning curve of several weeks should be
  expected for all users

• Attention and memory problems need to be
  identified to determine if multi-modality input
  of information will only confuse the learner

34
Suggested Instructional Considerations

• Time allotted for student to become familiar with
  the synthesized voice.

• Initial oral vocabulary building is helpful.

• Practice with short phrases can help the student
  to strengthen working memory and other cognitive
  functionalities.

35

• Greater levels of affect might be associated with
  greater speeds.

• Teaching strategies should be focused on
  integrating oral learning into the students
  learning style and learning strategies.

36
How I have changed my instructional strategies

• The student is expected to take an active role in
  the conversion of text to e-text by themselves.

• All new users of screen readers begin by reading
  several novels of their choice.

• Work on comprehension of short and long phrases.

• Integrate social learning into the reading
  processReading clubs.

• Instruction on reading and learning strategies.

37
Accessible Learning Through Text-to-Speech.
Support the increased affectiveness and use of
technology by identifying learnability problems
of users of computer screen readers.
Involvement of actual users of screen readers
and other professionals responsible in the
delivery or instruction of assistive technology
in focus groups interviews. The focus groups
will attempt to identify learnability attributes
that can help in the development of successful
instructional and delivery strategies. 
38
Accessible Learning Through Text-to-Speech.
www.wattenberg.biz Web site for the Accessible
Learning Through Text-to-Speech
project. Potential participants can find
information and submit the participant's form
online. All focus groups will be online and
accessible. Results will be potentially be
published and presented at conferences.
39
For more information on the Accessible Learning
Through Text-to-Speech Project go
to www.wattenberg.biz
40
References
Cahn, J. E. (1990, July). The generation of
affect in synthesized speech. Journal of the
American Voice I/O Society, 8(1), 1-19. Chomski,
N. (2004). Radio interview. KPFA, 94.1. Berkeley,
CA. Dutoit, T. (2001). An Introduction to
Text-to-Speech Synthesis (1st ed.). Dordrecht,
Netherlands Kluwer Academic Publishers. Kitamura,
Y. Yamaguchi, Y. Imamizu, H. Kishino, F.
Kawato, M. (2003). Things happening in the brain
while humans learn to use new tools. In V.
Bellotti, T. Erickson, G. Cockton, P. Korhonen
(Eds.), Proceedings of the ACM CHI'2003
Conference on Human Factors in Computing Systems
(Ft. Lauderdale, Florida, April 5-10, 2003) (1st
ed., pp. 417-424). New York ACM Press. Lai, J.
Wood, D. Considine, M. (2000). The effect of
task conditions on comprehensibility of synthetic
speech. In (Ed.), Proceedings of the ACM
CHI'2000 Conference on Human Factors in Computing
Systems (The Hague, The Netherlands, April 1-6,
2000) (pp. 321-328). New York, NY ACM
Press. Riemer-Reiss, M. L., Wacker, R. R.
(2000, July/August/September). Factors associated
with assistive technology discontinuance among
individuals with disabilities. Journal of
Rehabilitation, 66(3), 44-50. Scherer, M.
(2004, March). The revised and improved matching
person technology assessment process and forms.
Paper presented at the 20th Annual International
Conference 'Technology and persons with
Disabilities", California State University
Northridge (CSUN), 1st. Retrieved April 10, 2004,
http//www.csun.edu/cod/conf/2004/proceedings/18.h
tm van Oostendorp, H. Goldman, S. (1999). The
Construction of Mental Representations During
Reading (1st ed.). Norwood, New Jersey Ablex
Publishing Corporation. van Oostendorp, H. de
Mul, S. (1996). Cognitive Aspects of Electronic
Text Processing (1st ed.). Norwood, New Jersey
Ablex Publishing Corporation.
41
Thank you for attending. A CD is available that
includes PowerPoint Presentation Extended
Research Articles Ted Wattenberg, Ed.S., C.R.C.