Interaction Techniques and Beating Fitts

1
Interaction Techniques and Beating Fitts Law
2
Interaction techniques

• A method for carrying out a specific interactive
  task
• Example enter a number in a range
• could use (simulated) slider
• (simulated) knob
• type in a number (text edit box)
• Each is a different interaction technique

3
Interaction techniques in libraries

• Generally ITs now come in the form of Widgets,
  controls, components, interactors
• Typically in reusable libraries
• e.g. widget sets / class libraries
• Also need custom ones

4
Design of interaction techniques

• Just going to say a tiny bit
• Scotts Guidelines for IT design
• Affordance
• Feedback
• Mechanics (incl. performance)

5
Affordance

• Can you tell what it does and what to do with it
  by looking at it?
• Most important for novices
• but almost all start as novices
• if people dont get past being novices you fail

Knurling (affordance for grip)
6
Feedback

• Can you tell what its doing?
• Can you tell the consequences of actions?
• e.g. Folders highlight when you drag over them
  indicating that if you let go the file will go
  inside the folder
• very important to reliable operation
• important for all users

7
Mechanics feel difficulty

• Fitts law (next) tells us about difficulty
• predicts time to make a movement
• Feel is trickier
• Can depend on physical input dev
• physical movements, forces, etc.
• Really gets back to the difficulty of the
  movement, but harder to characterize
• Important for all, but esp. experts

8
Fitts law

• Fitts was studying pointing behavior.
• Move your hand to touch within a circular target
• Analog for press a button of a certain size
• Originally e.g., in aircraft controls
• Also turns out to model move mouse to screen
  position to e.g., press a virtual button very
  well
• Turns out that time to do this is related to
  distance to target and size of target

9
Fitts law

• Time A Blog2(Dist/Size 0.5)
• Time is linearly proportional to log of
  difficulty
• proportionality constants depend on muscle group,
  and device
• Difficulty controlled by distance and required
  accuracy (size of target)
• Empirically derived (will see theory a bit later)

10
Fitts law

• (True) expert performance tends to be closely
  related to time required for movements
• not well related to learning (or performance) of
  novices
• still need to consider cognitive load

11
Mini case study 1The original Macintosh 7

• Macintosh (1984) was first big success of GUIs
• Originally came with 7 interactors built into
  toolbox (hence used for majority)
• Most not actually original w/ Mac
• Xerox Star ( Smalltalk earlier)

12
The Macintosh 7

• Generally very well designed (iterated with real
  users!)
• very snappy performance
• dedicated whole processor to updating them
  (little or no OS)
• Huge influence
• These 7 still cover a lot of todays GUIs (good
  and bad to that)

13
Button

• Shaped as rounded rectangles
• (about modern square corners)
• Inverted for feedback
• Recall Mac was pure B/W machine
• Pseudo 3D appearance hard and hadnt been
  invented yet

14
Slider

• Used for scroll bars (but fixed size thumb)
• Knurling on the thumb
• Pogo stick problem

15
Aside a different scrollbar design

• Openlook scroll bar

16
Pulldown menu

• This was original with Mac
• Differs slightly from Windows version you may be
  familiar with
• had to hold down button to keep menu down (one
  press-drag-release)
• Items highlight as you go over
• Selected item flashes

17
Check boxes, radio buttons, text entry / edit
fields

• Pretty much as we know them
• Single or multi-line text supported from the
  beginning

18
File pick / save

• Much more complex beast than the others
• built from the others some
• e.g. no affordance, by you could type and file
  list would scroll to typed name

19
Original Mac also had others

• Window close and resize boxes
• Drag open file icons and folders
• Not made generally available
• not in toolbox, so not (re)usable by other
  programmers

20
Second major release of Mac added a few

• Lists
• single multiple selection
• from textual lists (possibly with icons)
• Hierarchical (pull-right) menus
• Compact (in-place) menus
• select one-of-N pulldown
• Window zoom box

21
Have seen a few more added since then

• Tabbed dialogs now widely used
• Hierarchical lists (trees)
• Combo boxes
• Combination(s) of menu, list, text entry
• A few more variations on things
• Typically dont see much more than that

22
Almost all GUIs supported with the above 12-14
interactor types

• Good ones that work well
• uniformity is good for usability
• But, significant stagnation
• dialog box mindset
• opportunities lost by not customizing interaction
  techniques to tasks

23
Mini case study 2 Menus

• Menu
• supports selection of an item from a fixed set
• usually set determined in advance
• typically used for commands
• occasionally for setting value (e.g., picking a
  font)

24
Design alternatives for menus

• Simple, fixed location menus
• (see these on the web a lot)
• easy to implement
• good affordances
• easy for novices (always same place, fully
  visible)
• Focus of attention problems
• Screen space hog

25
Popup menus

• Menu pops up under the cursor (sometimes via
  other button, e.g., right button on PC)
• close to cursor
• not under it, why?

26
Popup menus

• Menu pops up under the cursor (sometimes via
  other button , e.g., right button on PC)
• close to cursor
• What does Fitts law say about this?

27
Popup menus

• Menu pops up under the cursor (sometimes via
  other button , e.g., right button on PC)
• close to cursor
• Fitts law says very fast
• also focus not disturbed
• takes no screen space (until used)
• can be context dependent (!)
• poor (non-existent) affordance

28
Getting best of both Mac pulldown menus

• Menu bar fixed at top of screen, with pull-down
  submenus
• benefits of fixed location
• provides good affordance
• good use of space via partial popup
• but splits attention requires long moves

29
Fitts law effects

• Windows menus at top of windows, vs. Mac menus at
  top of screen
• Interesting Fitts law effect
• what is it?

30
Fitts law effects

• Windows menus at top of windows, vs. Mac menus at
  top of screen
• Interesting Fitts law effect
• thin target vertically (direction of move) ? high
  required accuracy
• hard to pick
• but (anybody see it?)

31
Fitts law effects

• With menu at top of screen can overshoot by an
  arbitrary amount
• (Example of a barrier technique)
• What does Fitts law say about that?

32
Fitts law effects

• With menu at top of screen can overshoot by an
  arbitrary amount
• very large size (dominated by horizontal which is
  wide)
• Original Mac had 9 screen so distance not really
  an issue
• very fast selection

33
Pie menus

• A circular pop-up menu
• no bounds on selection area
• basically only angle counts
• do want a dead area at center
• What are Fitts law properties?

34
Pie menus

• A circular pop-up menu
• no bounds on selection area
• basically only angle counts
• do want a dead area at center
• Fitts law properties
• minimum distance to travel
• minimum required accuracy
• very fast

35
Pie menus

• Why dont we see these much?

36
Pie menus

• Why dont we see these much?
• Just not known
• Harder to implement
• particularly drawing labels
• but there are variations that are easier
• Dont scale past a few items
• No hierarchy

37
Beating Fitts law (a hobby of mine)

• Cant really beat it
• property of being human
• but you can cheat!
• One approach avoid the problem
• use a non-pointing device
• shortcuts fixed buttons
• mouse wheel for scrolling

38
Beating Fitts law

• Not everything can be a shortcut
• Other major approach manipulate interface to
  reduce difficulty
• distance (put things close)
• but not everything can be close
• have to make them smaller!

39
Beating Fitts law

• Most ways to cheat involve manipulating size
• typically cant make things bigger w/o running
  out of screen space (but look at that as an
  option)
• but can sometimes make things act bigger than
  they are

40
Cheating on target size

• Consider targets that are not just passive
• not all movements end in legal or useful
  positions
• map (nearby) illegal or non-useful onto
  legal ones
• hit of illegal position treated as legal
• e.g. positions above Mac menubar
• effective size bigger

41
Snapping (or gravity fields)

• Treat movement to an illegal point as if it
  were movement to closest legal (useful /
  likely) point
• Cursor or other feedback snaps to legal
  position
• Drawn to it as if it has gravity

42
Snapping

• Simplest grids
• Constrained orientations sizes
• 90 45, square
• More sophisticated semantic
• only attach circuit diagram items at certain spots

43
Snapping

• Even more sophisticated dynamic semantics
• Check legality and consequences of each result at
  every move
• dont catch errors, prevent them!

44
Other approaches

• Area cursors
• Make the cursor (or active area) much bigger
• Any part of cursor over object allows selection
• Width in Fitts law sense is sum of cursor and
  object
• Issue
• What is it?

45
Other approaches

• Area cursors
• Make the cursor much bigger
• Any part of cursor over object allows selection
• Width in Fitts law sense is sum of cursor and
  object
• Issue
• Pick ambiguity what if two more objects under?
• Solution ?

46
Other approaches

• Area cursors
• Make the cursor much bigger
• Any part of cursor over object allows selection
• Width in Fitts law sense is sum of cursor and
  object
• Issue
• Pick ambiguity what if two or more objects
  under?
• Solution adaptive approach
• Reduce cursor size when ambiguous

47
More aggressive approaches

• Barriers
• Put a barrier behind the object
• Natural screen edge (but cant put all things
  there)
• Artificial anywhere
• Effectively very large size (like Mac menus)

• But how do you know where to put them?
• Incorrectly placed they can block interaction
• Needs to be behind the intended target only
• How do you know what the intended target is?

48
More aggressive approaches

• Sticky icons
• Adjust the mouse gain
• Ratio of mickeys (mouse rotation ticks) to
  pixels
• Make mouse slower over target
• Has effect of making target larger
• BUT
• If you do this everywhere it just slows
  everything
• How do you know you are over the target?

49
So how do we know which is the intended target?

• Note A number of possibilities if we knew the
  intended target in advance
• Above aggressive approaches to beating Fitts
• Also prefetch and/or precomputation
• May be able to do a more detailed analysis of the
  movement to figure this out

50
More detailed explanation of Fitts law

• Why is it a log relationship (not linear)?
• One explanation lies in micro-structure of the
  movement control (not only / best explanation)
• Movement made up of main move and a series of
  corrections each regulated by feedback
• Sub-movements get smaller (by fixed percent)but
  take constant time (one perceptual-motor cycle)
• Log number of expected correction steps

51
Ballistic motions

• Recall
• Velocity is change in position per unit time
• First derivative of position WRT time
• Acceleration is change in velocity per unit time
• Second derivative
• Jerk is change in acceleration per unit time
• Third derivative
• Snap is change in jerk per unit time
• One model of ballistic motion minimize jerk
• Min acceleration doesnt work, min snap collapses
  back to min jerk when zero vel / accel at ends of
  movement

52
Minimum jerk model

• Minimum jerk model says that a single ballistic
  (uncorrected) movement would look like

Velocity
Time
53
Real movement micro-structure

• Observed movements show corrective tail
• Corrective movements based on perceptual feedback
  cycles (sometimes see separate bumps but
  usually appear continuous)
• other explanations exist

Velocity
Time
54
Reading

• Andrea H. Mason, Masuma A. Walji, Elaine J. Lee,
  Christine L. MacKenzie, Reaching movements to
  augmented and graphic objects in virtual
  environments, Proceedings of CHI '01, 2001,
  Seattle, pp.426-433.
• http//portal.acm.org/citation.cfm?doid365024.365
  308

55
Predicting being near target

• Aileen Worden, Neff Walker, Krishna Bharat, Scott
  Hudson, "Making computers easier for older adults
  to use, Proceedings of CHI '97, March 1997,
  Atlanta, pp. 266-271.http//doi.acm.org/10.1145/2
  58549.258724
• Adaptive area cursors and sticky icons
• Predicts that we are near target when velocity
  falls to 30 of peak velocity

56
Idea for better predictions Minimum jerk model
again

• Equations for this
• Assuming zero velocity and acceleration at start
  and end of movement
• Start position X0,Y0 End position Xf, Yf
• Movement time MT Normalized time ? t/MT
• X(t) X0 (X0- Xf)(-10 ?3 15 ?4 -6 ?5)
• Y(t) Y0 (Y0- Yf)(-10 ?3 15 ?4 -6 ?5)

57
Minimum jerk model again

• Simplify a little more
• Assuming X0,Y0 0 (relative movement)
• X(t) Xf(10 ?3 -15 ?4 6 ?5)
• Y(t) Yf(10 ?3 -15 ?4 6 ?5)
• (Model cant be completely right. Why?)

58
Supplemental reading(more details on equations)

• Mary D. Dlein Breteler, Stan C. A. M. Gielen,
  Ruud G. J. Meulenbrock, "End-point constraints in
  aiming movements effects of approach angle and
  speed", Biol Cybern, v85, pp 65-75, 2001.
• http//www.mbfys.kun.nl/mbfys/people/stan/biocyber
  n85-1.pdf

59
(Untested) idea for prediction

• Collect points from beginning of motion
• For each known possible ending point
• Predict MT based on Fitts law
• Test fit of observed points to min-jerk equation
• Best fit is prediction
• Issues
• Probably need to model corrective tail also
• Dont know error / uncertainty / sensitivity

60
Another (untested) idea

• Detecting non-dexterous users
• Look at microstructure of movement to detect
  motor control deficiencies
• Normal aging
• Minor to moderate difficulty mostly with small
  targets
• More severe disabilities of various sorts
• May be able to classify and provide specific help
• E.g., tremor vs. other kinds of dysfunction
• E.g., find onset of difficulty or apply
  specialized filters
• Kalman filter based on expected dexterous
  movements(?)

61

• Trying one of these out is a possibility for
  project 2

62
(No Transcript)