Vocal Joystick A New Dimension in Human-Machine Interaction

1
Vocal JoystickA New Dimension in Human-Machine
Interaction

• ET 2 Presentation
• Group 3
• Jeremy Moody, Carrie Chudy

2
Introduction

• A new technology is emerging in the software
  area of information technology.
• This emerging technology is called Vocal
  Joystick and will bring new light to the way we
  operate computers.

3
What Is It?

• Vocal Joystick is software that will enable
  individuals with motor impairments to control
  objects on a computer screen and ultimately
  electro-mechanical instruments simply by using
  vocal parameters.
• www.sciencentral.com/articles/view.php3?typeartic
  learticle_id218393030

4
Who Uses It?

• Vocal Joysticks primary user is anyone with a
  physical impairment that limits the use of their
  arms and hands.
• There are many people who have perfect use of
  their voice who dont have use of their hands
  and arms. -Jeffrey Bilmes, Creator
• However, Vocal Joystick software will be
  increasingly useful for everyday computer
  operators.

5
How Does It Work?

• Vocal Joystick utilizes three main components
• Acoustic signal processing
• Pattern recognition
• Motion control
• Translates verbal cues into directional movements.

6
Acoustic Signal Processing

• The signal processing module extracts short-term
  acoustic features
• Energy
• Autocorrelation coefficients
• Linear prediction coefficients
• Mel frequency cepstral coefficients (MFCC)
• Signal conditioning and analysis techniques are
  needed for accurate estimation of these features

7
Pattern Recognition

• The features are then directed into the pattern
  recognition module where energy smoothing, pitch
  and formant tracking, vowel classification and
  discrete sound recognition take place.
• This stage involves statistical learning
  techniques such as neural networks and dynamic
  Bayesian networks.

8
Motion Control

• Energy, pitch, vowel quality and discrete sound
  become acoustic parameters to be transformed into
  direction, speed and other motion related
  parameters.
• These motion control parameters are then used to
  launch corresponding actions.

9
Engine Diagram
10
Vowel Corpus

• The Vocal Joystick vowel corpus represents the
  sounds used to control the movements of the
  mouse.
• Vocalic signals are used with manipulations of
  pitch and intensity for variation.

11
Vocal Corpus (contd)

• Detects sounds 100 times a second
• Users are able to transition smoothly from one
  vowel to another unlike in speech recognition.
• Louder sounds make the cursor move faster.

12
Applications

• Browsing the Web
• Drawing on a screen
• Controlling a cursor
• Operating a robotic arm
• In the future, Vocal Joystick is optimistic about
  its use to control an electronic wheelchair.

13
Conclusion

• Vocal Joystick is an amazingly beneficial
  Emerging Technology that will have a major
  impact on the Information Technology world.
• With simply a microphone, a computer with a
  standard sound card and a user who can vocalize,
  anyone can utilize Vocal Joystick!

14
References

• http//www.sciencentral.com/articles/view.php3?typ
  earticlearticle_id218393030
• http//seattletimes.nwsource.com/html/businesstech
  nology/2008231288_btjoystick06.html
• http//ssli.ee.washington.edu/vj/

15
References (contd)

• http//ssli.ee.washington.edu/vj/engine.htm
• http//ssli.ee.washington.edu/vj/corpus.html
• http//www.yubanet.com/artman/publish/article_6748
  1.shtml