HAND GESTURE BASED HUMAN COMPUTER INTERACTION

1
HAND GESTURE BASEDHUMAN COMPUTER INTERACTION
2
Hand Gesture Based Applications

• Computer Interface
• A 2D/3D input device (Hand Tracking)
• Translation of gestures to commands
• More natural interface for 3D design applications
• Aid for disabled and deaf people
• Sign language analysis, translation
• Sign language teaching
• Using multimodality to enable communication
  between disabled people
• Entertainment
• Applications involving hand gesture recognition,
  from computer games to sound and image design
  applications

3
HMM Based Hand Gesture Recognition Process

• The System

2D Kalman Filter
3D Reconstruction
Marker Segmentation
3D Kalman Filter
Vector Quantizer
Right Image
Left Image
Gesture Spotter
Application
4
Gesture Recognition System Setup

• Requirements
• 2D Gesture Recognition System
• A simple webcam capable of acquiring RGB images
• 3D Gesture Recognition System
• Two simple webcams capable of acquiring RGB
  images
• A mainboard capable of handling two webcams
• No special equipment required

5
Gesture Recognition System Setup
6
3D GESTURES
7
Capabilities

• Recognition of 2D and 3D gestures in real time
• Real time hand tracking controlling mouse
  motion
• Capability of triggering windows input device
  events controlling third party applications

8
Future Work

• Usage of IOHMM, HMM ANN hybrids
• A visualization module for the HMMs
• Combining with body and face analyzers
• Real time recognition of simple sign language
  gestures
• Inclusion of hand posture in recognition process

9
Demonstration
Finger Tracker.avi