GESTURE SYSTEM

1
GESTURE SYSTEM

• Technology Overview

2
OVERVIEW

• Background
• Design Rationale
• Hardware
• Software Architecture
• Work Progress
• Conclusions

3
BACKGROUND

• Design Considerations

4
Design Drivers

• Usability
• Highly robust recognition
• Latency refresh rate
• gt Use low-risk machine vision techniques
• Optimal Lighting
• Efficient deterministic algorithms

5
Operating Environment

• 'Normal environment
• caves / meeting rooms
• Cluttered environments
• Low general illumination levels
• Dynamic colour illumination from display

6
DESIGN RATIONALE

• Main Design Considerations

7
Imaging

• Colour
• v Skin tracking
• x Low illumination
• x Display pollution

• Near Infra-Red
• v Independent imaging
• channel

8
Camera Orientation

• Downwards
• x Dynamic image
• x Cluttered image

• Upwards
• v Static image
• v Simple background
• v Commercialisation

9
Illumination Subject

• Onto Hands
• v Good in open-air
• x Ceiling shadows
• x Poor fore/background contrast near ceiling

• Onto Ceiling
• x Requires ceiling
• v No shadows
• v Excellent contrast
• v Ignores skin ethnicity /clothing reflectivity

gtUse most appropriate
10
Tracking

• 3D modelling
• x Computational expense
• x Development risk
• Flesh-tone detection
• x Using NIR imaging
• v v Feature tracking

11
Tracking Feature Detection

• Potential features
• X Palm
• X Wrist
• ? Fingertips use restriction

12
Posture Recognition

• Silhouette method
• Fingertips already detected
• Palm is in vicinity create vectors
• Distance measure to exemplars
• x Restricted gestures (acceptable)

13
(No Transcript)
14
HARDWARE
15
Camera / Filter

• Watec-902DM
• - High NIR Sensitivity
• - DSP pre-processing
• - Cokin 89B NIR filters

16
Illumination

• Pro-Optocam Illuminator
• - 140 high power NIR LEDs
• - Cooling allows optimal illumination power
• - Optimized polar characteristics

17
(No Transcript)
18
SOFTWARE ARCHITECTURE
19
System Black Box View

• Two way traffic via CORBA
• OUTPUT Control events to application
• INPUT Spatial sound feedback from application

20
Event-Queue Architecture

• Hand Events
• Gestures in image coordinates
• ?
• Raw Events
• Gesture start/stop in 3D coordinates
• ?
• Application Control Events
• Rotate, translate, point

21
Software Architecture

• Eldritch scheduler
• RawEvents hand ? raw events
• Temporal smoothing maintains hand identity
• Rome raw events ? control events
• CORBA corba traffic ? control events

22
WORK PROGRESS
23
Work Progress

• Hardware Build 95
• CORBA 90
• Spatial Sound 70
• Eldritch/Rome 50
• Machine Vision 40
• Commercialisation 0

24
Open Hand (Translate)
25
Select
26
Point
27
CONCLUSIONS
28
Summary

• Work proceeding according to project schedule
• CORBA test-stub ready for integration
• Partial machine vision
• Software architecture complete
• Full spatial sound
• Within budget

29
(No Transcript)