The Tessa Virtual Human or Avatar

1
School of Information Systems The University of
East Anglia, Norwich NR4 7TJ 
The TESSA Project
TESSA, the TExt and Sign Support Assistant, has
been developed to assist people who are deaf or
hard of hearing in transactions at a Post Office.
The TESSA system combines speech recognition
technology and state of the art virtual human
animation to enable Post Office assistants to
communicate with deaf customers. The assistant
speaks into a microphone which is then recognised
by a computer speech recognition system. The
speech is converted to British Sign Language and
signed by the virtual human for the customer.
English text can also be displayed for those who
do not use sign language.


 

• The Tessa Virtual Human or Avatar

The movements of the virtual human are copies
of those of a native sign language user. Software
specially developed for the project captures the
signers hand, mouth and body movements using a
variety of electronic sensors. These movements
are then stored and used to animate the avatar
when required. The Royal National Institute for
Deaf People, the UKs largest charity
representing deaf people, are closely involved in
the project. Their advice concerning the quality
of the avatar and animation as well as
information concerning the needs of deaf people
have helped to shape the systems development.
During the summer of 2001, the system was on
display at the Science Museum in London and was
simultaneously on trial at the Post Office there.


 
 
The movements of a native sign language user are
captured and used to animate the Avatar.
2
with which new signing lexicons can be produced
means that the system may be easily adapted to
other areas. A recent development of TESSA has
been to move from simple speech recognition to
speech "understanding" so that the PO clerk is
not so constrained in what he or she can say to
the system. Investigations are also being
undertaken into the possibility of the system
recognising a limited number of signs made by a
deaf customer. TESSA is just part of the EU
funded ViSiCAST project aimed at benefiting deaf
citizens by allowing them access to information
and services in sign language. The ViSiCAST
partners are also developing the technology to
facilitate the use of sign language on television
and the Internet. TESSA has recently won both a
gold medal and the overall IT award at the
prestigious British Computer Society's
Information Technology Awards. TESSA is being
developed as a collaboration between The School
of Information Systems at the University of East
Anglia (UEA), Televirtual, Consignia (the UK Post
Office) and the Royal National Institute for Deaf
People. UEA carried out system design, speech,
linguistic and interface development, while
Televirtual developed the Avatar and animation.
Consignia and the RNID are involved in assessment
of the system.
The project uses commercially available
speech-to-text software, which has been optimised
to work in a noisy Post Office environment with
several different speakers. TESSA enables over
450 of the most frequent Post Office transactions
including those involving variable quantities
such as money and days of the week to be
completed more effectively. The system has been
developed around the principle that British Sign
Language is simply another language. Thus the
technology can also be used to translate into
other spoken languages French, German, Welsh and
Somali are currently being implemented. This
means the system is particularly useful in Post
Offices with large numbers of tourist
visitors. While the system is currently targeted
at transactions in the UK Post Office, the ease
Post Office assistant says into microphone You
need to send it special delivery
Speech recognition system determines which phrase
has been spoken by the assistant
University of East Anglia http//www.sys.uea.ac.u
k, contact Dr Stephen Cox, sjc_at_sys.uea.ac.uk Con
signia (the Post Office) http//www.postoffice.c
o.uk/ contact alan.kennedy_at_consignia.com Televirt
ual - the Avatar http//www.televirtual.co.uk/ co
ntact Mark Wells, Mark_at_televirtual.com RNID http
//www.rnid.org/ contact jayne.beresford_at_rnid.org.
uk ViSiCAST project www.visicast.co.uk/ BCS
Award (http//www1.bcs.org.uk/docs/01000/1021/00w
inners.htm).
Computer looks up the matching Avatar motion file
Customer display shows selected sign(s) using
TESSA avatar, together with a written version
TESSA development is funded by Consignia. Further
research into TESSA is also being funded by
ViSiCAST, a European Union Information Societies
Technology (IST) project.