Augmentative and Alternative Communication AAC: Effects of Multimedia Integrated in a SpeechGenerati

1
Augmentative and Alternative Communication
(AAC)Effects of Multimedia Integrated in a
Speech-Generating AAC Device Amanda Lee Spear,
Joanna Beck, and Filip Loncke Communication
Disorders Program - University of Virginia
Results
Multimedia and AAC
The Project
The study consisted of ten, one-hour
sessions with an individual participant. This
participant was trained and observed using an
integrated communication device under two
conditions (1) the BASIC condition where the
communication device was used exclusively to
generate a spoken message through the use of text
input (a keyboard) and (2) the PLUS condition
where the communication device was used with the
added options which included the use of a
camera, games, juke box, jokes, and e-mail.
During each session, both conditions were
implemented for a half hour each. During use
of the BASIC condition, the participant was
encouraged to create sentences, participate in
conversations with the researcher and his family,
initiate conversations with the researcher, and
role play situations such as ordering from a
restaurant. During the use of the PLUS condition
the participant was encouraged to use the camera
to take pictures, communicate with acquaintances
through the use of email, play games which often
worked on therapy goals, use jokes to engage in
conversation, and use the juke box as a way to
express his emotions.
Augmentative and Alternative Communication
systems integrated in a multimedia format allow
the user to connect linguistic activities
(accessing words, constructing sentences,
generating speech) with pictorial
representations, telephone and internet
communication, music, and games. Our hypotheses
are that (1) such a multimedia environment
closely mirrors the working of the human mind
(where the use of language and communication is
embedded in a network of representational and
functional structures) and that (2) devices that
allow the user to use multiple media in an
integrated way are more conducive to learning.
The theoretical and applied possibilities of
multimedia have been discussed mainly from a
functional perspective (e.g., Hine, Arnott,
Smith, 2003 Hirotimi and Mirenkow, 2002), but
rarely from a linguistic or cognitive perspective
(with a few exceptions, e.g., Nelson, Heimann,
Tjus, 1997)

Discussion and Observations
AAC Devices More Can Be Better!
Prior to the study, the participant did not
appear to enjoy speech therapy, he constantly
watched the clock and couldnt wait for the fifty
minute session to be over. He also needed strong
encouragement to use his device. However, during
the study he appeared to be very excited to work
with the device and the researcher. He asked if
he could keep the device at his house. Then he
used the device to ask the researcher when she
would be coming back with the device. Throughout
the sessions, he frequently had a smile on his
face and even danced. Now that the has study
ended, he has been able to use the device during
therapy sessions at the University of Virginia
Speech and Hearing Clinic. His therapists have
reported that he appears to be much happier and
interactive. They have seen a great deal of
improvement in his communication skills since he
began using this integrated device. The
study clearly indicates an increased involvement
in the activities, which is reflected in the
increased intensity of language use, especially
in the PLUS condition. Not only did the
participant enjoy the multimedia/ multimodal
technical environment, his production and
engagement were also stronger. His mother
confirmed that he has never been so enthusiastic
about learning and using language in a multimodal
context.
Traditionally, AAC devices have been
conceived as facilitating direct communication
between people who have severe limitations in
their communication and their environment. The
latest generation of speech-generating devices
have incorporated functions that allow the user
to employ the device as much more than a speech
generator but also as a conversation illustrator
(through picture taking and displaying), music
player, distant communicator (through text
messaging and emailing) and information generator
(through internet access). This poster
reports on a comparative study of a single
AAC-users learning process in two conditions
pure speech-generating condition versus the
condition with added multimedia.Results will be
discussed in terms of (1) implications for
teaching (and therapy) and (2) our understanding
of mental processes by AAC-users.
Methodology
Each session was transcribed into a time line to
indicate (1) who was communicating (2) verbal
(spoken) utterances (3) gestural and non-verbal
communication (4) device-generated speech and (6)
which conditions were being used. Then 5 minutes
of each condition were randomly chosen to examine
how many utterances, turns, and words the
participant used.
Tellus
Intervention Schedule
The device used is the TELLUS 3. The Tellus3 is
a device that has been developed by Technology
Integration. The device consists of a tablet PC
(with touch screen) with added speech- generating
software, a camera function (that allows the user
to take pictures that will be included in
conversations), a music player (which allows the
user to play and refer to their favorite songs),
games (which can be used as a conversation-enhance
r), and email possibilities.
Conclusions and Further Plans
The results of this study are promising a
device that makes it easy for the user to link
multiple forms of representations has a higher
degree of correspondence with how real-time
mental functioning works in comparison to a
device that only allows the user to generate
language and produce speech. Further
research is needed to indicate how to optimally
use this cognitive multimodality in order to
maximize learning gains.
References
Acknowledgment
Hine, N., J. L. Arnott, et al. (2003). "Design
issues encountered in the development of a mobile
multimedia augmentative communication service."
Universal Access Information Society
2(255-264). Hirotomi, T. and N. Mirenkov (2002).
Multimedia communication environment for
children, handicapped, and elderly people.
Proceedings of the 22nd International Conference
on Distributed Computing Systems
Workshops. Nelson, K. E., M. Heimann, et al.
(1997). Theoretical and applied insights from
multimedia facilitation of communication skills
in children with autism, deaf children, and
children with other disabilities. Communication
and Language Acquisition. Discoveries from
Atypical Development. L. B. Adamson and M. A.
Romski. Baltimore, MD, Paul H. Brookes Publishing
Co. 295-325.
We would like to acknowledge Technology and
Integration (Gent, Belgium) for supporting this
study through a travel and training fund and for
providing a TELLUS 3 to conduct the
project. Thanks also to Becca Bishow and Katie
Earnshaw for the many hours transcribing the
sessions.