Pen

1
Pen Gesture Input
2
Agenda

• Questions?
• Pen input
• Gesture recognition

3
Pen input

• Free-form ink
• Soft keyboards
• Recognition systems
• generalize to gesture-based systems

4
Free-form ink

• ink as data
• humans can interpret
• time-stamping
• implicit object detection
• special-purpose domain objects

5
Free-form ink examples

• Ink-Audio integration
• Tivoli (Xerox PARC)
• eClass (GT)
• FlatLand (Xerox PARC)
• Dynomite (FX-PAL)
• The Audio Notebook (MIT)

6
Soft Keyboards

• common on small mobile devices
• many varieties
• tapping interfaces
• Key layout (QWERTY, alphabetical, )
• learnability vs. efficiency

7
T9 (Tegic Communications)

• Alternative tapping interface
• phone layout plus dictionary
• Soft keyboard or mobile phone

8
Quickwrite (Perlin)

• Unistroke recognizer

9
Cirrin (Mankoff)

• Word-level unistroke recognizer

10
Watch input

• onHand PC joystick
• Casio bezel
• TrekTrak concentric circles

11
Recognizing pen input

• Graffiti
• unistroke alphabet
• Other pen gesture recognizers
• for commands
• measure features of strokes
• usually no good for complex strokes

12
Handwriting recognition

• Lots of resources
• see Web
• good commercial systems
• Two major techniques
• on-line
• off-line

13
Mixing modes of pen use

• Users want free-form and commands
• or commands vs. text
• How to switch between them?
• (1 mode) recognize which applies
• (2 modes) visible mode switch
• (1.5 modes) special pen action switches

14
Error correction

• Really slows effective input
• word-prediction can prevent errors
• Various strategies
• repetition (erase and write again)
• n-best list
• other multiple alternative displays

15
Other interesting applications

• Signature verification
• Note-taking
• group (NotePals by Landay _at_ Berkeley)
• student (StuPad by Truong _at_ GT)
• meetings (Tivoli and other commercial)
• Sketching systems
• early storyboard support (SILK, Cocktail Napkin)

16
Gesture Recognition

• Emerging area
• Perceptual User Interfaces
• mainly computer vision researchers

17
Advantages

• Using Wizard of Oz techniques, we find
• Consistent gestures for certain commands
• Easy to learn
• Remembered over time
• DFAB interaction framework

18
Issues with Gesture Recognition

• Does a gesture clearly indicate
• functional information
• scope
• target information
• Variability

19
Non-obvious gestures

• Biometric signals
• smart floor
• affective computing