Experimental Analysis of Mode Switching Techniques in Penbased User Interfaces

1
Experimental Analysis of Mode Switching
Techniques in Pen-based User Interfaces
Yang Li University of California, Berkeley Ken
Hinckley Microsoft Research Zhiwei
Guan University of Washington James Landay
Intel Research Seattle DUB, University of
Washington
2
Pen-based UIs Have Shown Promise

• Allow entry of raw ink
• Efficient for expressing both graphics text
• Useful for capturing ideas assisting abstract
  thinking
• Allow using gestures
• Efficient for issuing commands for manipulating
  data

A Cut Gesture
3
Modes Used to Switch Between Inking Gesturing

• Ink Mode
• Raw ink for interpretation by a person
• Gesture Mode
• Gestures for immediate interpretation by computer
  
• Mode switching technique allows users to switch
  between modes
• Modes are a significant source of errors,
  confusion, unnecessary restrictions, and
  complexity in interfaces
• - The Humane Interfaces by Jef Raskin

4
Modeless Interactions vs. Better Mode Switching
Techniques

• Modeless Interactions
• Eliminate modes by processing ink and gesture
  input in a single mode
• Hard to discern gestures from other ink
• Hard to apply to the general case
• Better Mode Switching Techniques
• Increase the mode visibility reduce the effort
  for switching
• Require a little amount of user effort
• Offer consistent mechanisms applicable across a
  wide variety of pen-based UI

5
Our Goal
Investigate mode switching techniques in
pen-based user interfaces

• Create new mode switching techniques
• Develop an experimental analysis methodology
• Conduct quantitative analysis on the performance
  of techniques

6
Outline

• Motivation
• Mode Switching Techniques
• Experimental Design
• Experimental Analysis
• Implications for Design
• Conclusion Future Work

7
Mode Switching Techniques

• Barrel Button
• Press Hold
• Using Non-Preferred Hand
• Pressure-Based Mode Switching
• Using the Eraser End of a Pen

8
Technique 1Barrel Button
9
Technique 2Press Hold

• Spatial constraint in the range of 1.5mm
• Temporal constraint 1 second

10
Technique 3Using Non-Preferred Hand
Mode Switching Button
Toshiba Protégé Tablet PC
11
Technique 4Pressure-Based Mode Switching
Normal pressure for inking

• Heavy pressure for gesturing

255
Red Element of Ink Color
0
255
160
190
Average Pen Pressure
12
Technique 5Using the Eraser End of a Pen
Using the eraser end for gesturing
Using the tip for inking
13
Outline

• Motivation
• Mode Switching Techniques
• Experimental Design
• Experimental Analysis
• Implications for Design
• Conclusion Future Work

14
Experimental DesignPie-Crossing Task
15
Experimental DesignBaseline Compound Tasks
Baseline Task (No Mode Switch)
Compound Task (Mode Switch Required)
16
Experimental DesignProcedure An Example with
Pressure
17
Experimental DesignProcedure Participants

• 15 participants
• For each participant
• Training phase for the baseline tasks
• 5 sessions for 5 techniques respectively (Using
  5x5 Latin Square)
• Training phase for a mode switching technique
• Experimental phase
• A post-study questionnaire

18
Experimental DesignProcedure Participants
15 participants x 5 mode switching techniques
x 9 blocks of trials x 8 screens (8
orientations) x 5 pie-crossing tasks
27,000 pie-crossing tasks with 4,800 mode
switches performed
19
Experimental DesignPerformance Measures
Baseline Task
Start Cycle
Compound Task
20
Experimental DesignPerformance Measures
Baseline Task
Start Cycle
Full Cycle
Compound Task
21
Experimental DesignPerformance Measures
Baseline Task
Start Cycle
Full Cycle
Full Cycle
Compound Task
22
Experimental DesignAverage Cycle Duration of a
Block
1
2
3
4
5
6
D Average Cycle Duration
7
8
23
Experimental DesignPerformance Measures

• Mode switching time
• Total number of errors in a compound task
• Subjective preference of participants

compound
baseline
baseline
Warming up
MSTj Dj (Dj-1 Dj1)/2
24
Outline

• Motivation
• Mode Switching Techniques
• Experimental Design
• Experimental Analysis
• Implications for Design
• Conclusion Future Work

25
Statistical Analysis Approach

• Repeated Measure Variance Analysis
• Mode switching time
• Subjective preferences
• Chi-Square Analysis
• The number of errors

26
Experimental AnalysisTime Performance
Mean Time / Mode Switch (ms.)
1500
1000
500
139ms
0
Mean Time / Mode Switch (ms.)
BarrelButton
NonPrefHand
Eraser
Hold
Pressure
Technique
27
Experimental AnalysisError Analysis Error
Classification

• Mode Error
• Mode-In Error
• Mode-Out Error
• Crossing Error
• Not crossing a target slice
• Has the wrong orientation
• Out-Of-Target Error

28
Experimental AnalysisError Analysis Mean Error
Rate on Each Pie-Crossing
Mode Error
6
Crossing Error
OutOfTarget Error
5
Error Rate ()
4
Most accurate
3
2
Error Rate ()
1
0
BarrelButton
NonPrefHand
Eraser
Hold
Pressure
Technique
29
Experimental AnalysisError Analysis Mean Error
Rate on Each Pie-Crossing
Mode Error
6
5
Error Rate ()
Crossing Error
OutOfTarget Error
4
3
2
Error Rate ()
1
0
BarrelButton
NonPrefHand
Eraser
Hold
Pressure
Technique
30
Experimental AnalysisAverage Subjective
Preferences vs. Time
4.5
NonPrefHand
BarrelButton
Mean Preference (worst 15 best)
Pressure
3.5
Eraser
Hold
2.5
100
600
1100
1600
Time Performance (ms)
31
Experimental AnalysisAverage Subjective
Preferences vs. Error Rate
4.5
NonPrefHand
Mean Preference (worst 1-5 best)
BarrelButton
Pressure
3.5
Eraser
Hold
2.5
1.5
2.5
3.5
4.5
5.5
Error Rate ()
32
Outline

• Motivation
• Mode Switching Techniques
• Experimental Design
• Experimental Analysis
• Implications for Design
• Conclusion Future Work

33
Implications for DesignBarrel Button
NonPrefHand

• Share the same temporal model
• NonPrefHand was faster due to temporal overlap of
  two subtasks Kabbash et al
• Improve synchronization by introducing a 37ms
  detection phase
• Can reduce 50 Mode-In errors for Barrel Button
• Can only reduce 14 for NonPrefHand

34
Implications for DesignPressure

• Personalized Pressure Spaces
• Participants who had a lower pressure space
  tended to make more Mode-In errors and less
  Mode-Out errors, and vice versa
• The high negative correlation between the number
  of Mode-In and Mode-Out errors
• Might be improved using a personalized pressure
  space

35
Implications for DesignHold

• Performed the worst in our experiment, but
  requires the least hardware support
• StrokeHold
• Allow user to perform a Hold at any point of a
  drawing rather than only at the starting point
• Easier to hold the pen still in the middle of a
  drawing than at the moment of touching the
  slippery tablet

36
Implications for DesignEraser

• Least extra effort required while drawing a
  gesture
• Appropriate for situations requiring less
  frequent mode switching

37
Future Work

• Study these techniques in a more natural setting
• NonPrefHand and StrokeHold techniques have been
  deployed in DENIM
• Improve the Pressure technique using a
  personalized pressure space
• Consider mode switching frequency as well as
  gesture complexity
• Investigate different mode feedback
• Such as audio

38
Conclusions

• Investigated five mode switching techniques
• The first quantitative analysis of techniques for
  switching between ink and gesture modes
• NonPrefHand performed the best while Hold was the
  worst
• Designed an experimental methodology for further
  exploring pen-based mode switching techniques
• Gave implications for design of mode switching
  techs
• How these techniques can be improved
• The tradeoffs to using these techniques in
  particular situations