Speechreading

1
Speechreading

• Based on Tye-Murray (1998) pp. 193-234

2
(No Transcript)
3
Speechreading Training

• Very popular in the first half of the 20th
  century.
• Sophisticated hearing instruments and questions
  about the efficacy of training have reduced the
  emphasis on speechreading in AR programmes.
• Now the emphasis is on maximising access to
  visual cues rather than training and practice in
  lip-reading.

4
Definitions

• Lipreading A person who relies only on the
  visual signal provided by the talkers face for
  recognising speech.
• Speechreading The person attends to both the
  tracker's auditory and visual signals as well as
  the talkers facial expressions and gestures, and
  any other available cues.
• (Tye-Murray, 1998)

5
Who will be a better speech-reader and why?

• Dr E. 42 year old pharmaceutical executive with a
  PhD in organic chemistry. He has had a moderate
  bilateral hearing impairment since he was 16
  years old.
• Mr C. 17 year old male with normal hearing.
  Required to undergo an audiological evaluation
  prior to beginning work at at factory

6
Performance in speech reading tasks cannot be
predicted from

• Intelligence.
• Educational level.
• Duration of deafness.
• Onset of hearing loss.
• Socio-economic status.
• Verbal abilities.
• Personality.
• (Summerfield, 1989)

7
What are predictors?

• Gender (Dancer et al., 1994).
• Age (Farrimond, 1959 Honnell et al, 1991).
• Linguistic and world knowledge in children.
• Jeffers and Barley (1971) also cite some
  characteristics that may relate to better
  performance such as ability to capitalise on
  contextual cues, willingness to guess, mental
  agility and willingness to revise previous
  interpretations.

8
The Difficulty of lipreading

• Visibility of sounds.
• Rapidity of speech.
• Coarticulation and stress effects.
• Talker effects.
• Visemes and homophemes.

9
Visibility of sounds

• 60 of speech sounds are not visible on the mouth
  or cannot be seen readily (Woodward Barber,
  1960).
• Consonants with high visibility are bilabials
  (/p, b, m, w/), labiodental fricatives (/f, v/)
  and linguadental fricatives (/, D/).
• Limited visibility involve consonants produced
  within the mouth (/k, g, t, n/)

10

• Some features like voicing are not visible at
  all.
• Vowels tend not to be visible despite some
  differences in lip spreading and lip rounding.
• But vowels are relatively salient even with large
  degrees of hearing loss.

11
Rapidity of Speech

• We speak at around 180-220 syllables per minute.
• That is around 15 phonemes per second.
• The human eye can only register 9 or 10 discrete
  mouth movements per second.
• Hence, the speech reader has little time to think
  about the identity of the word and may not even
  register the important movement.

12
Coarticulation and Stress Effects

• Phonemes change in visual appearance depending on
  the phonetic and linguistic context
• e.g. the /b/ in beet and boot
• or you in the following question
• What did ya do yesterday?
• What did YOU do yesterday?

13
Talker Effects

• The same sound often look different when spoken
  by different speakers.

14
Visemes

• Viseme Group of speech sounds that appear
  identical (e.g. /p, b, m/) (Fisher, 1968).
• Erber (1974)
• /p, b, m/ /l/
• /f, v/ /w, r/
• /, D/ /k, g/
• /n, d, t, s, z/ /h/

15
Homophemes

• Homophemes Words that look identical on the
  mouth.
• Often not related to phonetics.
• E.g. grade yes are homophemes but boon
  and doom are not.
• Between 40-60 of the words in English are
  homophenous (Berger, 1972).

16
Importance of Residual Hearing

• The whole is greater than the sum of the parts!
• Residual hearing and visual cues complement each
  other well.

17
Factors that affect the speechreading process

• The talker.
• The message.
• The environment and communication situation.
• The speechreader.

18
The talker

• Familiarity.
• Gender.
• females speech is more recognisable
• the female voice is harder for hearing impairment
  people to hear
• male talkers may have facial hair

19
The Message

• Structure.
• length, syntactic complexity, frequency of use,
  similarity to other words and linguistic context
• Frequency of usage.
• Neighbourhoods.
• Context.
• e.g. set the ________ vs buy the _______

20
The environment and communication situation

• Viewing angle.
• Distance.
• Room conditions (e.g. lighting).

21
Viewing angle
22
Distance
23
Lighting
24
The speechreader

• Innate skill.
• Hearing acuity.
• Emotional and physical state.

25
Speechreading training

• It is rare that adults in rehabilitation will
  receive specific speechreading instruction.
• The principles of speechreading will be
  incorporated within the larger communication
  management plans.

26
(No Transcript)
27
Ways to improve speechreading

• Watch the talkers lips.
• Provide information to the talker about how to
  communicate with you.
• Try to ensure that the room is well lit and you
  are in the optimum position.
• Try to minimise background noise.
• Know the topic of conversation.
• Pay attention to contextual cues.
• Keep a positive attitude.