Interaction

1
Interaction

• Jing Li
• CPSC 533C
• March 3, 2003

2
Overview

• Toolglass and Magic Lenses The See-Through
  Interface (1993)
• by Eric A. Bier, Maureen C. Stone, Ken Pier,
  William Buxton, Tony D. DeRose
• SDM Selective Dynamic Manipulation of
  Visualizations (1995)
• by Mei C. Chuah, Steven F. Roth, Joe Mattis,
  John Kolojejchick

3
Toolglass and Magic Lenses

• The see-through interface that can appear as
  though on a transparent sheet of glass between
  the application and the traditional cursor.
• These tools create spatial modes that can replace
  temporal modes in user interface systems.

4
Introduction

• Toolglass widgets semi-transparent interactive
  tools that are used in an application work area,
  and appear on a virtual sheet of transparent
  glass called a Toolglass sheet.
• The user can line up a widget, a cursor and an
  application object in a single two-handed
  gesture.
• A widget that has been stretched across the
  entire work area creates a command mode.
• User can move sheet over application object or
  move the object to a widget (trackball can
  control both sheet and scrolling).

Figure 1. Click-through buttons. (a) Six wedge
objects. (b) Clicking through a green fill-color
button. (c) Clicking through a cyan outline-color
button
5
Introduction (Cont.)

• Many widgets can be placed on a single sheet
• Widgets and lenses can be composed to create a
  number of specialized tools from a basic
  set(e.g. an outline color palette over a
  magnifying lens, to point to individual edges)

Figure 2. A sheet of widgets. Clockwise from
upper left color palette, shape palette,
clipboard, grid, delete button, and buttons that
navigate to additional widgets
Figure 3. An outline color palette over a
magnifying lens
6
Related Work

• Multi-Handed Interfaces
• Movable Tools
• Menus that pop up at the cursor position
• Toolglass sheets can be moved without tying up
  the cursor
• Transparent Tools
• Make menus over the work area transparent
• Viewing Filters
• Magic lenses are visual filters that may be used
  to change type of views or reveal hidden
  information.

7
Examples

• Shape and Property Palettes

Figure 4. Shape palette. (a) Choosing a shape.
(b) Placing the shape.
Figure 5. Font face palette. The word "directly"
is being selected and changed to bold face.
8
Examples (Cont.)

• Clipboards

Figure 6. Symmetry clipboard. (a) Picking up an
object. (b) Rotated copies appear. (c) The
copies are moved and pasted.
Figure 7. Fill-color rubbings. (a) Lifting a
color. (b) Moving the clipboard. (c) Applying
the color.
9
Examples (Cont.)

• Previewing Lenses

Figure 8. An achromatic lens over a drop shadow
lens over a knotwork. (Knotwork by Andrew
Glassner)
10
Examples (Cont.)

• Selection Tools

Figure 9. Vertex selection widget. (a) Shapes.
(b) The widget is placed. (c) A selected
vertex.
Figure 10. The local scaling lens that shrinks
each object around its own centroid (Tiling by
Doug Wyatt)
11
Examples (Cont.)

• Grids

Figure 11. Three grid tools
12
Examples (Cont.)

• Visualization

Figure 12. Gaussian curvature pseudo-color lens
with overlaid tool to read the numeric value of
the curvature. (Original images courtesy of Steve
Mann)
13
Implementation of Toolglass Sheets

• Multi-Device Input and Screen Refresh
• Handles simultaneous input from two pointing
  devices and updates the screen after multiple
  simultaneous changes
• Filtering Input Through Lenses and Widgets
• Modifies pointing events as they pass through
  widgets
• Filtering Output Through Lenses and Widgets
• Modifies graphical output as it passes up through
  each widget

14
Implementation of Magic Lens Filters

• Recursive Ambush
• catch the low level drawing calls
• typically a wrapper around the graphics context
• and modify them, and then call the original
  graphics primitive
• e.g. modify DrawLine to set the color to red
• works transparently across all applications
• but complicated to make a new lens and debug

15
Implementation of Magic Lens Filters (Cont.)

• Model-In Model-Out
• create new objects and transform them
• can be saved and used by any lenses over it
• very powerful, but
• must allocate storage
• cross application is an issue

16
Implementation of Magic Lens Filters (Cont.)

• Reparameterize clip
• Modify global parameters to drawing
• Redraw the model, but clipped to the boundary
  shape of the lens
• Best example scaling

17
Implementation of Magic Lens Filters (Cont.)

• Multiple model types need to be supported for
  cross-application functionality. Achieved by
• Type conversion convert the model to be of the
  type required
• Type tolerance unchanged if does not understand
  a model

18
Toolglass Magic Lens Conclusions

• See-through interface based on spatial modes
• Structured well for two-handed interaction
• May be integrated into any screen-based
  applications and tools

19
SDM Selective Dynamic Manipulation of
Visualizations

• Mei C. Chuah, Steven F. Roth,
• Joe Mattis, John Kolojejchick
• (Carnegie Mellon University)

20
Motivation

• Need new interactive techniques for 2D and 3D
  visualizations. We want
• Selective a high degree of user control
• Dynamic interactions all occur in real time, and
  interactive animation
• Manipulation Users can directly move and
  transform objects to perform different tasks
• This set of techniques is called SDM

21
Current Static Visualization

• Users cannot focus on specific object sets in
  detail while keeping them in the larger context
• Clutter and occlusion (hidden data) in dense
  information space
• Difficult to give a sense of scale some objects
  may be dwarfed in the scale used for the entire
  data set (green objects in Figure 1)
• No tools provided to classify data sets and save
  those classifications
• Difficult to compare quantities when objects are
  not spatially contiguous (e.g. when objects are
  at different distances from the user)

22
SDM Sample

• SDM deals with these issues
• Selection and control methods rely on objects
  rather than spaces.

23
Sample Supply Distribution Network

• Cylinders (red) supply centres
• Rectangular bars (purple) shelters
• Lines (black) on the floor plane major routes
• Heights of cylinders and bars quantities
  available at supply centres and needed by
  shelters
• Selected objects green

24
SDM Components

• Object Centered Selection
• Select objects by clicking or using sliders to
  place data constraints
• Object sets may be named, saved and used later
  from a scrollable menu
• Object sets need not be made up of homogeneous
  types (e.g. a set may contain supply centers,
  shelters, and routes)

25
SDM Components (Cont.)

• Dynamic and Flexible Operations
• Directly manipulate through object handles
• Attach a handle to an object, and push or pull on
  it causes the object to expand, shrink or move
  continuously
• Objects change with motion of handles (A)
  controls the radius of cylinder object (B)
  controls object height(C) controls bar width
• Arrow handles shift

26
SDM Components (Cont.)

• Object Constraints
• Context persistence
• SDM maintains some relationship between the focus
  objects set and its environment.
• Set-wide operations
• If you can move or scale one object in a focus
  set, you can move or scale any other object in
  this set.

27
SDM Feedback Mechanisms

• Clearly Identify the Selected Set
• The selected set is painted differently (green)
• A white grid may be drawn beneath all selected
  objects to indicate positions and spread
• Maintain Scene Context
• Used to maintain context when objects are
  displaced
• Object body (green) and object shell (white)

28
Feedback Mechanisms (Cont.)

• Maintain Temporal Continuity
• Use animation to help users to perceive changes
  to the scene
• Maintain Relationships Between the Selected Set
  and the Environment
• Keep a scale of the differences on screen, for
  example, the ratio axes technique (next slide)
• Allow Objects to be Easily Returned to Their Home
  Positions

29
Ratio Axes Technique
Scale of the environment (black) the selected
set (red)The left is 11 the right is 18
30
SDM Solutions

• Focusing on a Select Set of Objects while Keeping
  Scene Context
• Objects can be painted distinctly and the width
  can be increased
• Users may shift all focus objects to the front of
  the scene

31
SDM Solutions (Cont.)

• Viewing and Analyzing Occluded Objects
• Selected object set is elevated to solve
  occlusion problem

32
SDM Solutions (Cont.)

• Reduce the heights of unconcerned objects to zero
• Make all unconcerned objects very thin

33
SDM Solutions (Cont.)

• Viewing Different Sets of Elements Based on
  Different Scales
• Ratio axes technique (mentioned above)
• Interactively Augmenting the Visualization with
  New Classification Infomation
• Add width and color, save as a set and apply them
  later (i.e. identify important shelters)

34
SDM Solutions (Cont.)

• Comparing the Patterns, Widths, and Heights of
  Objects
• Draw a line of reference in the scene plane and
  move any set of objects to the reference line
• Compare data trends among multiple sets

35
SDM System Architecture
36
SDM Conclusions

• Good interactive techniques deals with the issues
  from current static visualizations
• Visualization is clear, precise and easy to
  understand
• The goal of SDM is to provide users with enough
  tools and flexibility (multiple alternative
  solutions) that they can solve a wide spectrum of
  data analysis tasks.