drag-and-pop a technique for accessing remote screen content on touch- and pen-operated systems

1
drag-and-popa technique for accessing remote
screencontent on touch- and pen-operated systems

• patrick baudisch
• ed cutrell, dan robbins, mary czerwinski,peter
  tandler, ben bederson, and alex zierlinger
• microsoft research
• visualization and interaction research

2
goals

• mixing touch screens/pen-input with large screens
  creates interesting new interface challenges
• bringing target icons to the user
  (drag-and-pop) allows users to complete drag
  interactions faster
• general theme limiting interaction space to the
  display space at the users location can solve
  problems

3
contents

• goals
• scenario touch/pen input meets large screens and
  multimon
• problems dragging across bezels and long
  distances
• drag-and-pop
• related work
• design, design, and more design
• user study drag-and-pop makes users faster
• conclusions

4
touch/pen-input multimon

• touch and pen input renaissance
• PDAs
• Tablets
• Liveboards / Smartboards

• multi-display systems
• DynaWall, iRoom Smartboard wall
• connect tablet to external screen

touch/pen input breaks
5
scenario 1 tables screen

• tablet users scribble with pen but filing
  iconsinto folder on external monitor requires
  mouse

6
scenario 2 dragging bezels

• dragging across bezels in display wall is no
  problem for the mouse
• but a big problem when using pen/touch input

7
scenario 3 long distances

• dragging is designed for small screens but
  becomes time-consuming on large screens

8
drag-and-pop

• users starts dragging icon towards a distant
  folder or application
• icons of compatible type come towards mouse
  cursor
• user drops icon with minimal motion
• targets retract

drag-and-pop works across bezels
9
demo
bringing target icons to the user allows users
tocomplete drag interaction at the users
current locations
10
scenario 1 tablet screen

• filing icons into folder on external monitor

11
scenario 2 dragging bezels

• dragging acrossbezels in display wall

12
scenario 3 long distances

• dragging on large screens

13
video
14
related work

• techniques for transferring information
• drag-and-drop avoids hidden clipboard (e.g. Xerox
  Star)
• hyperdragging (Rekimoto, 1999)
• pick-and-drop (Rekimoto, 1997) take-and-put
  (Streitz et al., 2001)
• overcome large distances
• magic pointing (Zhai et al., 1999) requires an
  indirect input device
• gesture input techniques (Rubine, 1991)
• throwing (Geißler, 1998) for reliable target
  acquisition?
• laser pointers to acquire targets on a Smartboard
  (Myers et al. 2002)
• mouse-based interaction techniques
• lodestones and lay lines (Jul, 2002)
• flick (Dulberg et al., 1999)
• sticky icons (Swaminathan and Sato 1997)

15

• design

16
selecting candidates

• initialize
• all icons are candidates
• filter
• eliminate icons with non-matching file types
• eliminate icons that are too close
• eliminate icons outside target angle
• if necessary, restrict to some hard limit

17
preserving layout

• snap to grid
• eliminate empty rows and columns
• translate back
• place center of bounding box in front of user
• closer for experts

18
the rubber band

• animationdid not work
• frozenmotion blur
• narrow midriff
• suggests elasticity
• clue for distance
• simplified version

19
getting it out of the way

• to rearrange icons on the desktop (overloaded)
• any mouse motion moving away from the popped-up
  icons de-activates drag-and-pop
• introduce flick gesture into mouse motion

20

• study

21
pre-study

• 15 single, 6 dual, and4 triple monitor users
• overall resolutions 800,000 pixels to 3,900,000
  pixels
• ( 66 more than the display wall used in the
  experiment).

• 3 layouts for study sparse (11), frame (28),
  cluttered (35)

22
user study

• participants 2 female, 5 male
• dynaWall
• 3 Smartboard
• 15 long (4.5m)
• 3 x 1024x768 pixels
• native code not stable enoughà Macromedia Flash
  version
• task drag icons into matching folder
• highlighting disappeared when started
• each desktop 11-35 icons 10 icons to be filed

23
results
Control
Drag-and-pop
3.7 timesspeedup

• faster with drag-and-pop
• error rate higher with drag-and-pop
• most of the effect caused by the bezels

24
subjective satisfaction

• gt 6 (out of 7)
• I liked using drag-and-pop
• I always understood what was happening when
  drag-and-pop was on,
• I would use drag-and-pop for large displays.
• lt 3 for
• It took a long time to get used to drag-and-pop
• It was hard to control what the targets did when
  drag-and-pop was on.
• drag-and-pop interface causes less manual stress
  and fatigue than the control interface

25
lesson learned

• the shortest connection between two points on a
  display wall is not a straight line
• (we fixed this by opening target sector towards
  top of display)

26

• the larger picture

27
general theme

• WIMP metaphor can breakon large screens with
  pen/touch input
• drag-and-pop generalizes direct manipulation
• bring content to the user
• let the user interact with it
• send content back
• à interaction space is not the same as display
  space anymore

28
drag-and-pick

• problem
• launch app or open file
• drag-and-pick
• user drags background
• all icons in that direction move to the cursor
• user drags releases mouse over it
• target is activated

29
what about dialog boxes?
30
inside applications

• drag-and-pop workseven if target is
• occluded
• clipped
• closed (folder)
• use the concept tofile emails?

31
goals revisited - conclusions

• mixing touch screens/pen-input with large screens
  creates interesting new interface challenges
• bringing target icons to the user
  (drag-and-pop) allows users to complete drag
  interactions faster
• general theme limiting interaction space to the
  display space at the users location can solve
  problems

32
thank you!

• try it out Google drag-and-pop
• thanks to the VIBE research group(mary
  czerwinski, george robertson)diane kelly,
  dieter boecker,lance good, amanda williams,