EE3P BEng Final Year Project

1

• EE3P BEng Final Year Project 1st meeting
• SLaTE Speech and Language Technology in
  Education
• Martin Russell
• m.j.russell_at_bham.ac.uk

2
Introduction

• SLaTE
• Resources
• Some possible projects
• Phase 1 of project
• Things that you need to know
• Any questions?

3
SLaTE

• Speech and Language Technology in Education
  (SLaTE)
• Concerned with any aspect of applying speech and
  language technology in education
• Most common applications are language related
• Interactive intelligent second language (L2)
  learning
• Interactive reading tutors
• Interactive pronunciation tutors (L1 or L2)

4
SLaTE

• But, SLaTE is not restricted to language
  learning
• Spoken language interaction with any educational
  system (e.g. dictating mathematical formulae)

5
Resources

• SLaTE is a relatively new topic, but there are
  some resources
• BEng FYP 2009-2010 web page
• http//www.eee.bham.ac.uk/russellm/ee3p-fyp09_file
  s/EE3P-BEng-FYP2009.pdf
• Proceedings of the SLaTE 2007 and 2009 workshops
• SLaTE 2007
• SLaTE 2009

6
Timetable

• Autumn term
• Group meetings
• Learn background
• Mini-project
• Choose main project
• First bench inspection (5 minutes)
• Spring term
• Individual project
• Individual meetings
• Final bench inspection (weeks 10/11)
• Final report
• Summer term
• Present poster at Project Open Day

7
Possible Projects

• Measuring Goodness of Pronunciation (GoP)
• Dictation of mathematical expressions
• Selection of appropriate audio material for SLaTE
  based on acoustic analysis
• Measuring reading fluency and its relationship
  with reading ability
• Selection of appropriate audio material for SLaTE
  based on lexical analysis

8
1. Measuring Goodness of Pronunciation (GoP)

• Use Automatic Speech Recognition technology to
  decide whether a pronunciation of a given word by
  a learner of English as a second language is
  acceptable. The learners utterance is compared
  with an acoustic statistical model and accepted
  or rejected based on a score. By restricting the
  vocabulary appropriately it will be possible to
  develop a simple GoP system using the HTK speech
  recognition toolkit. This project should include
  a serious evaluation of the system.

9
2. Dictation of mathematical expressions

• This project involves the use of automatic speech
  recognition technology to dictate mathematical
  equations. The project will involve collecting
  data to determine how engineers say mathematical
  expressions, and then developing techniques to
  parse this data. For example, do engineers
  always say when brackets are needed, or can the
  need for brackets be inferred from pause
  duration? A simple real-time demonstrator will
  be developed using a commercial speech
  recognition system and evaluated using carefully
  prepared test data.

10
3. Selection of appropriate audio material for
SLaTE based on acoustic analysis

• Different voices are appropriate for different
  applications. For example, a voice which is
  suitable for reading stories to children would be
  inappropriate for communicating instructions to
  soldiers on a parade ground! The goal of this
  project is to find out if it is possible to use
  techniques from automatic speech recognition to
  differentiate between audio material on the web
  which is suitable or unsuitable for SLaTE
  applications with children. The decision should
  be based on properties of the acoustic signal
  rather than lexical or syntactic content. An
  important part of the project is the evaluation
  and critical analysis of the system

11
4. Selection of appropriate audio material for
SLaTE based on lexical analysis

• This project is related to Project 3, but in this
  case the goal is to determine the suitability of
  a piece of audion for SLaTE applications with
  children using lexical (word content) and
  syntactic (grammatical) analysis. This will
  involve passing the audio through a commercial
  speech recognition system and analyzing the
  result. Important questions will include whether
  or not current speech recognition technology is
  sufficiently accurate to support this
  application, and the effect of speech recognition
  accuracy on system performance.

12
5. Measuring reading fluency and its
relationship with reading ability

• The goal of this project is to design, implement
  and test a system for measuring, automatically,
  reading fluency. The system might measure
  factors such as the number of words spoken per
  minute, or the average length of pauses. The
  system will be based on a commercial automatic
  speech recognition system. The student will
  collect a set of recordings of people reading
  with different levels of proficiency, and this
  data will be used to test the system. In
  addition, the student will obtain human judgments
  of the reading proficiency shown in each of the
  recordings and discover whether these subjective
  judgments correlate with the measures of fluency.

13
Phase 1 of project (autumn term)

• Understand basic speech and language technology
  used in SLaTE
• Implement a simple SLaTE system (see next slide)
• Choose individual project for the Spring term
• Deliverables
• Knowledge of speech and language technology
• Simple software demonstration (of something)
• Individual project specification

14
Phase 1 possible project

• Implement (and test) a system that takes texts
  from the internet and decides whether they are
  appropriate for a particular level of learner of
  English as a second language.
• See the paper by Heilman, Zhao, Pino and Eskenazi
  (FYP 2009/10 web page)
• ...or you could choose something which is the
  first phase of your own project

15
Things you need to know

• Automatic speech recognition
• How does a basic system work?
• What is a hidden Markov model (HMM)?
• What is HTK?
• We will discuss this at the next meeting

16
Any Questions?