Alternative communication

1
Alternative communication access to
information
Alternative communication access to
information
Grigori Evreinov
Department of Computer and Information
SciencesUniversity of Tampere, Finland
Department of Computer and Information
SciencesUniversity of Tampere, Finland
www.cs.uta.fi/grse/
January March, 2003
2
ACAI
information may be defined as the characteristics
of the output of a process these characteristics
enhance knowledge, being informative about the
process and its input hierarchies of processes,
linked together, provide a communication channel
between each of the corresponding functions and
layers in the hierarchies fundamentally
information is not modality-specific (i.e. not
tied to its form of presentation, be it visually,
as sounds or whatever) but the end user
(destination, addressee) for whom the information
exists, is a person who is trained to manipulate
through modality-specific notions (which are
formed by percepts) therefore, within
information technology modal aspect of
information maybe more essential for development
of relations between person and computer 1
http//www.fst.ch/FST2/al/default.htm
TAUCHI MMIG Evreinov G. p 01_23
22.01.2003
3
ACAI
elements of communication process 3
communication is defined as the transfer of
information from a communicator-source
/originator to a communicator-receiver the
communication process allows people to share
information, ideas, and feelings this is the
transfer of meaning where no meaning is
transferred, no communication has taken place the
goal of communicators is to accomplish this
process efficiently (rationally) and effectively
TAUCHI MMIG Evreinov G. p 02_23
22.01.2003
4
ACAI
human communication has no other purpose than to
cause some kind of action - to direct, to inform,
to question, and to persuade 2 the challenge is
to come up with the right combination of codes,
media, and contexts in order to make the transfer
of information fast, cost effective, and
accurate ideas and feelings can be shared only if
they are represented by symbols symbols are
things that stand for something else signs are
literal symbols are not a symbol means more than
it literally says 4 all of our communication
messages are made up of two kinds of symbols
verbal and nonverbal symbols can have several
kinds of association like personal, cultural or
universal interpretation of symbols meaning
depends on the set of parameters, conditions and
features of elements involved in communication
process
TAUCHI MMIG Evreinov G. p 03_23
22.01.2003
5
ACAI
the ideal message is one that is received
interpreted as it is intended a receiver is a
one or more individuals for whom the message is
intended to be an effective, communicator first
gain and keep the receiver's attention communicat
ors must have similar knowledge /background with
regards to usage of the medium or get knowledge
before during learning communication conditions
noise is interference of signals with similar
parameters in analyzers that keeps a message from
being understood or accurately interpreted psychol
ogical noise occurs in the minds of the
communicators and the receivers when they are
distracted by something there are different
types of communication intrapersonal,
interpersonal, interviews, small group, and public
TAUCHI MMIG Evreinov G. p 04_23
22.01.2003
6
ACAI
intrapersonal communication occurs within us it
involves thoughts, feelings, and the way we look
at ourselves 2, 5
TAUCHI MMIG Evreinov G. p 05_23
22.01.2003
7
ACAI
communication messages are made up of two kinds
of symbols verbal and nonverbal voice set
(pitch, volume, rate and rhythm) is defined by
the speakers physical and emotional
condition certain kinds of voice sets are
strongly classified with gender and age there are
three types of nonverbal vocalizations 7
vocal characterizers (yawning or belching
) vocal qualifiers (momentary variations of
speech or volume, personality traits ) vocal
segregates (sounds or silences made between
expression of words, such as ahh, hah, mmmh, and
include pauses during speech, which can convey an
attitude of calmness and reflection) 6 however,
the information and richness contained in
paralanguage, music, and background sounds is
lost to the deaf viewer 10, 11
Demo 1
Demo 2
Demo 3
Demo 4
Demo 5
Demo 6
Demo 7
Demo 8
Demo 9
Demo10
9 http//www.esl-lab.com/para.htm
TAUCHI MMIG Evreinov G. p 06_23
22.01.2003
8
ACAI
Sonification of facial expressions
on the contrary, for blind viewer how to
transform emotional expression from the picture
into sound/earcon or whatever 12 there is not
any system to reveal an emotional access to
pictures for blind persons surveys show that it
is very important for blind to transport emotions
independently of the content especially for
modern images the sonification system can enable
an emotional access using the similarity of
feeling by watching a picture or hearing music
and transforming visual information in music
compositions results of psychological,
physiological and emotional acoustic researches,
methods of image analysis and the rules of music
composition are basis for this translation and
were describe by Schwende in Auditory Emotional
Access to Visual Information 33
on the contrary, for blind viewer how to
transform emotional expression from the picture
into sound/earcon or whatever 12 there is not
any system to reveal an emotional access to
pictures for blind persons surveys show that it
is very important for blind to transport emotions
independently of the content especially for
modern images the sonification system can enable
an emotional access using the similarity of
feeling by watching a picture or hearing music
and transforming visual information in music
compositions results of psychological,
physiological and emotional acoustic researches,
methods of image analysis and the rules of music
composition are basis for this translation and
were describe by Schwende in Auditory Emotional
Access to Visual Information 33
TAUCHI MMIG Evreinov G. p 07_23
22.01.2003
9
ACAI
Sonification of facial expressions
propose or design a system of very brief
eARmoticons which could describe a facial
expression and evoke the same reaction
http//www.newag.org/funpics/weard/html/imagepages
/image68.htm http//www.berghuis.co.nz/abiator/wei
rd/ears.html, http//www.velvet.net/poppy/kitch/,
http//www.domainnamesanity.com/default.php,
http//www.dogandponysound.com/ico.htm
TAUCHI MMIG Evreinov G. p 08_23
22.01.2003
10
ACAI
tactile diagrams are relatively easy to produce
yet there are no standard guidelines to assist
during the design process attempting to include
as much information as possible can lead to
diagrams that are simply too confusing to be of
use research into a project on non-visual access
to music notation (Challis, Weasel) has shown
that an interactive multimodal approach to
presentation could be of benefit in these
circumstances
Weasel, http//www.benchallis.com/research.htmWea
sel National Centre for Tactile Diagrams
http//www.nctd.org.uk/ GEWA Access Computer
Keyboards - Remote Computer Access
http//www.zygo-usa.com/accessir.html Tellus und
Mind Express http//www.weissenstein-bs.de/produkt
e/softw/mindexpr/mindexpr.htm
TAUCHI MMIG Evreinov G. p 09_23
22.01.2003
11
ACAI
Access to Mathematics
communication of mathematics is usually visual -
formulas, diagrams, graphs etc. that makes it
very difficult for blind and partially sighted
people /students to do mathematics and is one of
the biggest obstacles for them in school and at
the university 13-20 6-dot Braille can
represent 64 unique characters through the use of
special sequences Braille can support a much
larger set of characters, that is, the basic
characters can have different meaning in
different contexts while there is effectively no
limit to the number of characters one can
represent in the fashion, the reading and
understanding of these special sequences becomes
very involved 13 the more new characters one
needs, the more escape sequences one
requires e.g., the letter a could be an a, an
A, a 1, the first finger in music notation
and so on
http//presentations.animfactory.com/pmpjumbos.htm
l
TAUCHI MMIG Evreinov G. p 10_23
22.01.2003
12
ACAI
Access to Mathematics
most of the work reported in developing
techniques and deal with mathematics can be
presented through the next categories tactile as
in Braille and other raised representations audio
aids that read equations with tools to help in
the reading process tonal representations of
equations and graphs (sonification/audification) h
aptic or forced feedback devices represent shapes
of objects and curves integrated /multimodal
approaches
Y X
Y sin X
http//www.gfai.de/sachsen/blind_i.htm
http//www.seeingwithsound.com/winvoice.htm
TAUCHI MMIG Evreinov G. p 11_23
22.01.2003
13
ACAI
Access to Mathematics
the Mathtalk command language is based on the
concept of a action and a target that is the user
might choose to hear the (whole) current
expression, then the current term (i.e. the first
term of the expression), followed by the next
term (i.e. the second term of the expression) and
so on the command language is mnemonic, so that,
the command to read an expression is r e, to move
to the next term it is n t, etc. 19
TAUCHI MMIG Evreinov G. p 12_23
22.01.2003
14
ACAI
Stuttgarter Mathematikschrift für Blinde
the Stuttgarter Mathematikschrift für Blinde
(SMSB) is an 8-dot-Braille which includes small
and capital letters, the numbers and mathematical
symbols SMSB provides three alphabets, the Latin,
the old German and the Greek alphabet, to write
letters which are used in mathematics capital
letters differ from the small ones by the
supplemented dot 7 each character of SMSB has a
representation in Braille and one in print no
SMSB-character stands for a string, a group of
characters or even a word as it could be in
contracted Braille that means, that each
SMSB-character can be shown on a
Braille-output-device as well as on a screen and
may be printed 22, 23 SMSB provides tactile
characters for symbols like
Access to Mathematics
TAUCHI MMIG Evreinov G. p 13_23
22.01.2003
15
ACAI
Access to Mathematics
SMSB uses characters, which can be understood
intuitively by a person, who is familiar with
mathematics symmetric Braille symbols are
symmetric in meaning
TAUCHI MMIG Evreinov G. p 14_23
22.01.2003
16
ACAI
Access to Mathematics
DotsPlus is a tactile font, but unlike Braille,
it requires no translation from ordinary
literature 27, 28 any computer file that uses
fonts with screen characters having the right
size can be printed directly in DotsPlus on the
Tiger tactile graphics and Braille
embosser DotsPlus has many more symbols than
Braille DotsPlus and the Tiger can make graphics
such as maps, floor plans, pictures of animals,
etc. http//dots.physics.orst.edu/dotsplus.html
Demo 11
TAUCHI MMIG Evreinov G. p 15_23
22.01.2003
17
ACAI
Special Braille Technical Drawings
the domain of the technical drawings includes
electronics, architecture and mechanical eng.
they contain graphical items that follow
industrial norms, like ISO or other
standards transforming visual info in technical
drawings into an understandable form for blind
people is very important for vocational training
digitized technical drawings can be presented
and edited on standard PCs with appropriate
software blind people need to access a user
interface and presentation tool specially
tailored for this type of info
http//www.braille-scs.com/tech_code.html http//w
ww.tedub.org/Start_tab.html
TAUCHI MMIG Evreinov G. p 16_23
22.01.2003
18
ACAI
Access to Mathematics
Demo12
TAUCHI MMIG Evreinov G. p 17_23
22.01.2003
19
ACAI
Access to Mathematics
input graphics sonification based on numerical
keyboards paradigm
Demo13
TAUCHI MMIG Evreinov G. p 18_23
22.01.2003
20
ACAI
TL, a language for creating games for visually
impaired and blind children 17, 18 this
language is a part of the TiM (Tactile
Interactive Multimedia computer games for
visually impaired children) project whose overall
aim is to offer to young visually impaired
children the possibility to play with computer
games
TAUCHI MMIG Evreinov G. p 19_23
22.01.2003
21
ACAI
Gaze Control
http//www.metrovision.fr/mv-vi-notice-us.html
http//www.delta7.asso.fr/Deltavision202001/ecrir
e3.html the Camera Mouse http//www.cs.bc.edu/gi
ps/CM/ or http//www.cameramouse.com/
TAUCHI MMIG Evreinov G. p 20_23
22.01.2003
22
ACAI
References
1 Edwards, A. D. N, Carey, K., Evreinov, G. et
al. Information and Communication Technology in
Special Education Analytical Survey. Moscow
UNESCO, Institute for Information Technology in
Education, 2001 2 Effective Communication
Skills, http//www.srsd.org/jrotc/Text20Books/LEI
I_U1.doc 3 The Diffusion of Web Icons.
Communication, http//chrissnider.com/academic/ico
ns/Theories/Communication/communication.html 4
Definition of Symbols, http//web.umr.edu/gdoty/c
lasses/concepts-practices/def-symbols.html 5
Intrapersonal Communication Processes. Joan E.
Aitken Leonard J. Shedletsky (Eds.)
http//www.usm.maine.edu/com/intrap.htm 6 Non
Verbal Communication, https//webspace.utexas.edu/
QUAHS/WWW/ 7 Eisenberg, A.M., Smith, Jr.,
R.R. Nonverbal communication. N-Y The
Bobbs-Merrill Company, Inc., 1971 8 John Carey,
Paralanguage in Computer Mediated Communication,
http//acl.ldc.upenn.edu/P/P80/P80-1018.pdf 9
Paralanguage, http//www.esl-lab.com/para.htm 10
Towards Emotive Captioning for Interactive
Television, http//www.ryerson.ca/clt/towards_emot
ive_captionin_1.doc 11 Charles Silverman and
Deborah I. Fels, Emotive Captioning in a Digital
World, 2002, LNCS 2398, p. 292,
http//link.springer-ny.com/link/service/series/05
58/papers/2398/23980292.pdf 12 Frank Deconinck
and Patricia Verschueren, A model of the
understanding of graphical information by blind
people, TIDE103/VUB/930531-2, 1993
http//minf.vub.ac.be/fdconinc/underst.rtf 13
Karshmer, A.I., Gupta, G., Gillan, D.
Architecting an Auditory Browser for Navigating
Mathematical Expressions, ICCHP 2002, LNCS 2398,
p. 477 http//link.springer.de/link/service/serie
s/0558/papers/2398/23980477.pdf 14 Gaura, P.
REMathEx - Reader and Editor of the Mathematical
Expressions for Blind Students, 2002, LNCS 2398,
p. 486, http//link.springer-ny.com/link/service/s
eries/0558/papers/2398/23980486.pdf
TAUCHI MMIG Evreinov G. p 21_23
22.01.2003
23
ACAI
15 Fitzpatrick D. Speaking Technical Documents
Using Prosody to Convey Textual and Mathematical
Material, ICCHP 2002, LNCS 2398, p. 494,
http//link.springer.de/link/service/series/0558/p
apers/2398/23980494.pdf http//www.computing.dcu.
ie/dfitzpat/publications.html 16 Math project,
http//www.cs.york.ac.uk/maths/index.html 17
Prosody in Mathtalk http//www.cs.york.ac.uk/maths
/robert/prosody.html 18 Mathematical Access for
Technology and Science, http//www.papenmeier.de/
reha/research/mathe.htm 19 Edwards, A. D. N.,
Stevens, R. D. and Pitt, I. J. Représentation non
visuelle des mathématiques, (translated by A.
Assimacopoulos) in A. B. Safran and A.
Assimacopoulos (editors) Le Déficit Visuel,
Éditions Masson, pp. 169178 (1995),
http//www.cs.york.ac.uk/ftpdir/pub/alistair/publi
cations/ps/geneva.ps 20 Karshmer, A.I., Gupta,
G., Geiger, S., and Weaver, C. Reading and
Writing Mathematics The MAVIS Project, BIT
(Behaviour Information Technology), January
1999 21 Calle Sjöström, Using Haptics in
Computer Interfaces for Blind People,
http//www.certec.lth.se/doc/usinghapticsin/usingh
apticsin.pdf 22 W. Schweikhardt 8-Dot-Braille
for Writing, Reading and Printing Texts which
Include Mathematical Characters. In A.D.N.
Edwards, A. Arato, W.L. Zagler Computers and
Assistive Technology, ICCHP98, Proceedings of the
XV. IFIP World Computer Congress, Wien and
Budapest, 31. August - 4. September 1998 23 W.
Schweikhardt Requirements on a mathematical
Notation for the Blind. In R. Vollmar, R. Wagner
(eds), CompurtersHelping  Peoples with Special
Needs ICCHP 2000, pp. 663-670, July, 17-21.
2000 24 Typographic Semantics of Webpages
Accessible for Visually Impaired Users - Mapping
Layout and Interaction Objects to an Auditory
Interaction Space, in Proceedings ICCHP 2000 -
International Conference on Computers Helping
with Special Needs. Karlsruhe 17-21 July 2000,
http//www-cg-hci.informatik.uni-oldenburg.de/reso
urces/ICCHP2000-gorny.pdf
TAUCHI MMIG Evreinov G. p 22_23
22.01.2003
24
ACAI
25 Yu, W., Ramloll, R., Brewster, S.A. and
Riedel, B. Exploring computer-generated line
graphs through virtual touch. In Proceedings of
IEEE ISSPA 2001 (Kuala-Lumpur, Malaysia), IEEE ,
pp 72-75, http//www.dcs.gla.ac.uk/stephen/papers
/ISSPA2001.pdf 26 Oakley, I., Brewster S.A. and
Gray, P.D. (2001). Communicating with feeling.
Haptic Human-Computer Interaction. Brewster, S.A.
and Murray-Smith, R. (Eds.), Springer LNCS, Vol
2058, pp 61-68, http//link.springer.de/link/servi
ce/series/0558/papers/2058/20580061.pdf 27
Equalizing Information Access, http//www.viewplus
tech.com/ 28 Tactile Graphics, An Overview and
Resource Guide, http//dots.physics.orst.edu/tacti
le/tactile.html 29 TIM Project
http//inova.snv.jussieu.fr/tim/ 30 Tactile and
Multimedia Tools for Young Visually Impaired
People http//inova.snv.jussieu.fr/colloques/BNet2
001/uk/programme.php 31 Hammarlund, J. Computer
Play for Children who are Severely Visually
Impaired, TRC Rapport nr. 20 (1999) 32 Anita
Hildén MUSSE version 2.0 http//www.sih.se/pdf/mus
se2_lathund.pdf 33 Schwende, H., Auditory
Emotional Access to Visual Information,
http//link.springer-ny.com/link/service/series/05
58/papers/2398/23980445.pdf 34 Gips James,
DiMattia Philip A., Curran Marialice, Lees Debra,
Accessing Internet Courses by Eye Movement and
Head Movement, http//link.springer-ny.com/link/se
rvice/series/0558/papers/2398/23980236.pdf 35
World of Gestures, http//nonverbal.ucsc.edu/gest.
html 36 Language Invention Resources Links,
http//www.linguistics.ucla.edu/people/grads/jfore
man/collegium/resource.htm
TAUCHI MMIG Evreinov G. p 23_23
22.01.2003
25
ACAI
All references presentations are available
at http//www.cs.uta.fi/grse/ACAI_2003/
http//www.bjorkasen.vgs.no/bcd/bcd0-99/ears2.htm
TAUCHI MMIG Evreinov G. p 00_00
22.01.2003