Gestural Interaction in Multiuser Environment

1
Gestural Interaction in Multi-user Environment

• Weiwei Geng
• November 21, 2004

2
Agenda

• Background
• Gestural Interaction in multi-user environment
• Synchronous Gestures for Multiple Persons and
  Computers
• Pick-and-Drop
• RoomPlanner
• Discussion
• Questions

3
Background

• Definitions
• Inputting angular velocities or accelerations and
  converting human action into code data, based on
  relative values. 1
• Gesture Input refers to the drawing of shapes as
  a means of executing pre-defined commands. 2
• Interacting with computers using gestures of the
  human body movements. 3

4
Background

• Text editor based on proofreaders marks in 1969.
  13
• gestural interface to the LOGO programming
  language in 1984 14
• Spreadsheet application that combines gesture and
  handwriting in 1986 15
• Etc.

5
Background

• Does gestural interaction enhance usability?
• Sandrine Robbe, April 1998 18
• Michael Moyle, February 2003 17
• Antti Pirhonen, April 20027

6
Background

• Gestural interaction with mobile devices.
• 3D audio pie menu 5
• Rhythmic interaction 6
• Gestural interaction on tabletop displays. 11
• Other.

7
Interaction in Multi-user Environment

• Motivations
• Information sharing.
• Collaborations.

8
Synchronous Gestures for Multiple Persons and
Computers

• Bumping as a synchronous gesture
• Display Tiling

9
Synchronous Gestures for Multiple Persons and
Computers

• Mutual information sharing
• Other synchronous gestures
• Shaking 9
• Simultaneous button pressing 10

10
Pick-and-Drop

• Intuitive use of pen

11
Pick-and-Drop

• System Structure

12
Pick-and-Drop

• State transition diagram

13
Pick-and-Drop

• Application
• Information change
• Other

14
RoomPlanner

• Single finger
• Tapping and dragging
• Context menu (double tapping)
• Flick
• Catch

15
RoomPlanner

• Two fingers
• Rotate object
• Adjust parameters

16
RoomPlanner

• Single hand
• Flat hand
• Vertical hand
• Horizontal hand
• Tilted hand

17
RoomPlanner

• Two Hands
• Sweep two vertical hands
• Corner-shaped hands

18
RoomPlanner

• Multi-user

19
Discussion

• Different roles of the gesture
• A mean to help collaborations
• A replacement of traditional computing
• without gesture sensor
• domain-specific (transferring data)
• similar to hand-writing
• New interaction method
• with gesture sensor
• can define new commands

20
Discussion

• Awareness
• Device awareness
• User awareness
• Privacy and security
• Extendibility

21
Discussion

• Gestural input must be associated with certain
  output channel
• Visual
• Audio
• Haptic 4
• New hardware required
• Sensing accuracy

22
Bibliography

• 1 Improvements in Gesture Input,
  http//www.lancs.ac.uk/ug/foulk/msc/myweb/index.ht
  m
• 2 Webopedia, http//www.webopedia.com/TERM/G/ges
  ture_recognition.html
• 3 Patent EP666544 Gesture input method and
  apparatus, http//patdb.ffii.org/EP/6/6/EP666544.h
  tml
• 4 Novel, minimalist haptic gesture interaction
  for mobile devices, Jukka Linjama, Topi
  Kaaresoja, October 2004,  Proceedings of the
  third Nordic conference on Human-computer
  interaction.
• 5 Interaction techniques for constrained
  displays Multimodal 'eyes-free' interaction
  techniques for wearable devices, Stephen
  Brewster, Joanna Lumsden, Marek Bell, Malcolm
  Hall, Stuart Tasker, April 2003, Proceedings of
  the conference on Human factors in computing
  systems.
• 6 Rhythmic interaction with a mobile device,
  Vuokko Lantz, Roderick Murray-Smith, October
  2004, Proceedings of the third Nordic conference
  on Human-computer interaction.
• 7 Speech, Audio, Gesture Gestural and audio
  metaphors as a means of control for mobile
  devices, Antti Pirhonen, Stephen Brewster,
  Christopher Holguin, April 2002, Proceedings of
  the SIGCHI conference on Human factors in
  computing systems.
• 8 Hinckley, K., Synchronous Gestures for
  Multiple Users and Computers, UIST 2003, 149-158.
• 9 Holmquist, L., Mattern, F., Schiele, B.,
  Alahuhta, P., Beigl, M., Gellersen, H., Smart-Its
  Friends A Technique for Users to Easily
  Establish Connections between Smart Artefacts,
  Ubicomp, 2001 Springer-Verlag, 116-122.
• 10 Rekimoto, J., Ayatsuka, Y., Kohno, M.,
  SyncTap An Interaction Technique for Mobile
  Networking, Mobile HCI 2003, 2003 Springer,
  104-115.

23
Bibliography

• 11 Multi-finger and whole hand gestural
  interaction techniques for multi-user tabletop
  displays, Mike Wu, Ravin Balakrishnan, November
  2003, Proceedings of the 16th annual ACM
  symposium on User interface software and
  technology.
• 12 Specifying gestures by example, Dean Rubine,
  July 1991, ACM SIGGRAPH Computer Graphics ,
  Proceedings of the 18th annual conference on
  Computer graphics and interactive techniques
• 13 COLEMAN, M. L. Text editing on a graphic
  display device using hand-drawn proofreaders
  symbols. In Pertinent Concepts in Computer
  Graphics, Proceedings of the Second Uni\ersity
  of Illinois Conference on Computer Graphics, M.
  Faiman and J. Nievergelt, Eds. University of
  Illinois Press, Urbana, Chicago, London, 1969,
  pp. 283290.
• 14 RHYNE, J. R., AND WOLF, C. G. Gestural
  interfaces for information processing
  applications. Tech. Rep. RC 12179, IBM T.J.
  Watson Research Center, IBM Corporation, P.O. Box
  218, Yorktown Heights, NY 10598, Sept. 1986.
• 15 MINSKY, M. R. Manipulating simulated objects
  with real- world gestures using a force and
  position sensitive screen. Computer Graphics 18,
  3 (July 1984), 195 203.
• 16 Pick-and-drop a direct manipulation
  technique for multiple computer environments, Jun
  Rekimoto, October 1997, Proceedings of the 10th
  annual ACM symposium on User interface software
  and technology
• 17 The design and evaluation of a flick gesture
  for 'back' and 'forward' in web browsers, Michael
  Moyle, Andy Cockburn, February 2003, Proceedings
  of the Fourth Australian user interface
  conference on User interfaces 2003 - Volume 18
• 18 An empirical study of speech and gesture
  interaction toward the definition of ergonomic
  design guidelines, Sandrine Robbe, April 1998,
  CHI 98 conference summary on Human factors in
  computing systems

24
Fin

• Questions
• Comments