On the use of Intonation in ASR: preliminary results

1
On the use of Intonation in ASR preliminary
results

• March, 13th 14th 2003
• Meeting COST 275
• Halmstad

2
OUR GROUP

• UPV EI Bilbao
• Inma Hernáez (leader), Eva Navas, Jon Sanchez
• UVA ETSII Valladolid
• Valentín Cardeñoso, Isaac Moro, Carlos Vivaracho,
  David Escudero.
• Involved in a CICYT Project of Biometrics with
  Javier Ortega (Madrid) and Marcos Faundez
  (Barcelona).
• Experience in ASR and in Modelling Intonation

3
OUR AIM

• To work in the CICYT Project with the rest of the
  groups.
• To apply our knowledge in Intonation to the field
  of ASR.
• Here we present our preliminary results.

4
INTRODUCTION

• Why to make use of intonation in ASR?
• It is a feature that characterize to the speaker
• Speakers of the same group have a similar
  prosody.
• Each speaker can have its own prosody.
• It is a very robust feature
• Different sessions
• Different microphones
• Other experiences in applying intonation to ASR
• SUPER SID very simple model of intonation.

5
INTRODUCTION

• Aim of this preliminary work
• To show the potential capabilities of intonation
  facing different sessions and microphones
• To show that it can be important to make use of
  sophisticated models for getting benefits in
  ASR.
• Overview
• Presentation of the model of intonation.
• The corpus.
• The experiment of speaker verification.
• Considerations about the robustness of the
  results.
• Consideration about the use model of intonation.
• Conclusions and future work.

6
Modelling Intonation
7
Modelling Intonation
8
Modelling Intonation
BASIC IDEA FOR ITS APPLICATION TO ASR TO
COMPARE THE MODELS OF DIFFERENT SPEAKERS
9
Modelling Intonation
10
The Corpus

• Recorded at EUPMT by Marcos Faundez
• One paragraph read by 16 speakers in 2 sessions
  with 3 microphones.
• Each paragraph 11 sentences, 106 stress groups.
  3816 intonation units.
• Speakers are male and in the same social group.
• The pitch was obtained automatically and
  segmented into intonation units by hand.
• Intonation was parameterised according to the
  intonation model.

11
The Experiment

• Speaker Verification.
• We have 6 recordings for each of the Speakers 5
  for modelling and 1 for testing.
• Each Speaker will have each Impostor. The
  impostor is modelled with the samples of the rest
  of speakers.
• We will repeat the experiment of verification six
  times (one for each of the possible set of tests)
  for each of the speakers.
• The classifier is based on Decision Trees C.45.
  Freeware WEKA.

12
Results
Low rates, except for some of the speakers
13
Results robustness
No significant changes when different test input
14
Results relevance of prosodic knowledge.
Some parts of the utterance are more relevant
depending of the speaker
15
Conclusions and future work

• Promising results some speakers are recognised
  with high rates. Results are robust to changes in
  the session and in the microphones.
• Future work To test the benefits of including
  this results in a ASR system.
• To explore the use of our methodology for
  modelling intonation in a more general way.
• Making use of more classes of intonation.
• Getting knowledge of which of the classes of
  intonation are more relevant for characterizing
  to the speaker.
• New corpura are welcome.

16
Stop the war

• Thank you