Computational Modeling Approaches

1
Computational Modeling Approaches Help Guide
Early Design Efforts for Usability
by
Maria Vicente Bonto-Kane
Robert St. Amant (Adviser)
North Carolina State University
http//www.marivicbontokane.com/research/paperTapi
a2009-mabkstamant.ppt
Tapia 2009 Conference, April 1-4, Portland, OR
2
Highlights About the Author

• Name Maria Vicente (Mari-Vic) Bonto-Kane
• US Citizen Security Clearance (US DoD 2009-2019)
• Education
• MA in Psychology (Binghamton University)
• MS in Computer Science (Binghamton University)
• PhD Candidate, North Carolina State University
• Awards Fulbright Hays Scholarship Grantee
• Some Internships
• US Dept. of the Navy (Avionics software, flight
  data)
• Oak Ridge National Lab (Cyber Security Research)
• IBM (Human Factors Usability Lab), GE, Qualcomm
• Hobbies Guitar, Ballroom dancing

3
Some Publications

• Pervasive 2007 Its About the User
• by Maria Vicente A. Bonto-Kane, Alvin Chin,
  Sheila McCarthy, Mayuree Srikulwong, and Paul J.
  Timmins, IEEE Pervasive Computing, Vol. 6, No. 4,
  pp 95-97. (October-December, 2007)
• Engineering Models of Human Performance on the
  Design of Mobile Device Interfaces
• by Maria Vicente A. Bonto-Kane (PhD Written
  Preliminary Exams)
• NCSU Computer Science Technical Report,
  TR-2008-11.
• Handwriting Recognition and Soft Keyboard Study
• by Naomi A. Kleid and Marivic A. Bonto, IBM
  Technical Report (1995)
• The Effect of Speech Intelligibility on
  Concurrent Visual Task Performance
• by David G. Payne, Leslie J. Peters, Deborah P.
  Birkmire, Marivic A. Bonto, Jeffrey Anastasi, and
  Michael J. Wenger, Human Factors (1994).

4
Human-Machine Interfaces (HMI)
5
Usability

• ISO 9241-11 Guidance on Usability (1998)
• Usability is the extent to which a product can
  be used by specified users to achieve specified
  goals with effectiveness, efficiency, and
  satisfaction, in a specified context of use.

• Why the importance of Usability
• Increased user efficiency
• Reduced development and support costs
• Reduction in catastrophic errors
• User satisfaction
• Higher revenue and sales

6
Traditional Methods

• Task Analysis
• Detailed description of tasks (blueprint)
• Hierarchical layout of tasks and subtasks
• Live User Testing
• Users perform tasks on software application
• Often done in a laboratory environment
• Observations and evaluation done by skilled
  professionals
• GOMS Modeling Techniques
• Scripts simulating tasks and procedures
• Actual script of how a user would do a task on a
  software application
• Some estimate of task duration and task difficulty

7
Task Analysis
1.1.9 Develop program for assay method using HTS
line control software (e.g., Beckman-Coulter SAMI)
1.1.9.1 Facilitate plate labeling and reading
1.1.9.2 Facilitate sample plate preparation
(serial dilution test compounds)
1.1.9.3 Facilitate test plate preparation
1.1.9.4 Facilitate test plate incubation
1.1.9.5 Facilitate raw data collection
Integrate pipeting device into HTS line control
(SAMI) method Link pipetting device to other
devices to be used in automated process (e.g.,
Bioworks).
Integrate plate reader into method. Reference
plate reader method from Plate reader software
(e.g., Flustart software).
Integrate incubator into method.
Integrate bar coder and reader into method.
Determine functions of device to be used during
assay. identify stock solution (deep-well)
plates to be used as resources, sample plates to
be used as transports, and tip types and other
resources.
Source Entzian, K. and Kaber, D (2004). Goal
Directed Task Analysis of High Throughput
Molecular Compound Screening
8
Live User Testing
Task Scenario and Instructions You are a chemist
about to enter instructions for a chemical assay
procedure. Enter the steps below by clicking the
correct icon from the Function Palette
Begin. Bulk_Dispense 0.9ml A1 to A3 using
Wash8 Gripper_Move A4, Lid to B2 Gripper_Move B6,
Lid to B5 Pipette 10.00ml from A4 to B6 using
MP200-250 Barrier Tip_Change, dispose of prior
tip Pipette 10.00ml from A3 to B6 using
MP200-P250 Barrier . . . End.

• Observations
• Did user accomplish the task? How long did it
  take?
• What types of errors were made?
• Did the user express any questions, suggestions
  while doing the task?
• Did user recognize the icons for their functions?
• Was the Help documentation useful?
• What types of strategies seem to be repeated?
• Etc.

9
GOMS Modeling

• Goals
• Operators
• Methods
• Selection Rules
• Some GOMS techniques
• CMN GOMS
• KLM GOMS
• NGOMSL (EGLEAN)
• CPM-GOMS (CogTool)

10
GOMS Model Script

• Selection_rules_for_goal Enter
  Assay_Instructions
• If ltnext_stepgt of ltcurrent_taskgt is_equal_to
  "PipetteTransfer",
• Then Accomplish_goal Choose Pipette_Transfer.
• If ltnext_stepgt of ltcurrent_taskgt is_equal_to
  TipChange",
• Then Accomplish_goal Choose Tip_Change.
• .
• .
• .
• Return_with_goal_accomplished.
• Method_for_goal Choose Pipette_Transfer
• Step 10. Point_to Pipette_Transfer_PB.
• Step 20. Click Pipette_Transfer_PB.
• Step 30. Accomplish_goal Enter
  Pipette_Transfer_Parameters
• Step 40. Return_with_goal_accomplished.
• Method_for_goal Choose Tip_Change
• Step 10. Point_to Tip_Change_PB.

11
GOMS Scripts as State Machines
G2400 Enter Assay Instructions
G2400.200 Choose Tip_Change
G2400.100 Choose Pipette_Transfer
Other goals
Other goals
12
Limitations of Traditional Methods

• Initial design left to designers intuition,
  skills, and craft
• Trial and error method
• Costly and time consuming
• Limited modeling of actual task flow
• Limited attention to usage patterns
• Limited task/performance metrics

13
Research Goals

• Probabilistic modeling of usage patterns
• Extend GOMS to Stochastic GOMS based on
  probabilistic modeling of task operators
• Illustrate inexpensive method of modeling,
  designing and evaluating interfaces
• Empirical validation of probabilistic and formal
  modeling as a viable approach within HCI

14
Stochastic GOMS

• Rationale for extending GOMS
• Historical usage data largely ignored
• Usage data indicates users most common
  strategies
• Probabilistic rather than deterministic user
  choices (compare to traditional GOMS)
• Improved workflow representation
• Improved task and user statistics
• More accurate, realistic modeling

15
GOMS Structures as State Machines
p 0.05
G2400 Enter Assay Instructions
Other goals
p 0.6
p 0.3
p 0.05
G2400.200 Choose Tip-Change
G2400.100 Choose Pipette_Transfer
Other goals
16
Evaluating Proposed Method

• How can we decide whether a new HCI modeling
  approach is worthwhile?
• Based on well-understood theoretical foundations
• Improved explanatory and predictive capabilities
  (GOMS vs Stochastic)
• Effectiveness in guiding interaction design
• Empirical validation of method

17
Life Sciences Application
High Throughput Screening (HTS) environment
18
Task Analysis Assay Procedures
1.1.9 Develop program for assay method using HTS
line control software (e.g., Beckman-Coulter SAMI)
1.1.9.1 Facilitate plate labeling and reading
1.1.9.2 Facilitate sample plate preparation
(serial dilution test compounds)
1.1.9.3 Facilitate test plate preparation
1.1.9.4 Facilitate test plate incubation
1.1.9.5 Facilitate raw data collection
Integrate plate reader into method. Reference
plate reader method from Plate reader software
(e.g., Flustart software).
Integrate incubator into method.
Integrate pipeting device into HTS line control
(SAMI) method Link pipetting device to other
devices to be used in automated process (e.g.,
Bioworks).
Integrate bar coder and reader into method.
Determine functions of device to be used during
assay. identify stock solution (deep-well)
plates to be used as resources, sample plates to
be used as transports, and tip types and other
resources.
19
Function Palette Toolbar
20
Function Palette Toolbar
Modify toolbar based on historical usage patterns
21
Usage Patterns
Published assay procedures from Promega website
22
Usage Patterns
Reset Cursor Mix Pipette Transfer Aspirate Bulk
Dispense Serial Transfer Purge Tool Wash
Button HighDensity Replicate
Next Labware Plate Read Set Shelf End Loop User
Function Insert BioScript Clear
Marks Comment Shift Stack
System Pause Tip Change Reset Tip Pause
Labware Send to Device Device Control Begin
Loop Wait for Device Gripper Move
23
Index of Difficulty (ID)
Fittss Law
Where, D Displacement from source to target W
Width of target area a and b are constants
representing intercept and slope respectively a
50msec b 300msec (one user and mouse) (Raskin,
2000 The Humane Interface)
24
ANOVA on Index of Difficulty
25
Measure Task Difficulty
Combine Fitts Law and latency of GOMS operators
26
GOMS Modeling Script

• Selection_rules_for_goal Enter
  Assay_Instructions
• If ltnext_stepgt of ltcurrent_taskgt is_equal_to
  "PipetteTransfer",
• Then Accomplish_goal Choose Pipette_Transfer.
• If ltnext_stepgt of ltcurrent_taskgt is_equal_to
  TipChange",
• Then Accomplish_goal Choose Tip_Change.
• .
• .
• .
• Return_with_goal_accomplished.
• Method_for_goal Choose Pipette_Transfer
• Step 10. Point_to Pipette_Transfer_PB.
• Step 20. Click Pipette_Transfer_PB.
• Step 30. Accomplish_goal Enter
  Pipette_Transfer_Parameters
• Step 40. Return_with_goal_accomplished.
• Method_for_goal Choose Tip_Change
• Step 10. Point_to Tip_Change_PB.

27
Task Difficulty (Current Interface)
Total Task Duration for DNAIQ Assay Procedure
(from Promega)
28
Redesign of Function Palette
Modified palette layout showing placement of more
probable icons in easily accessible locations
29
Model Predictions for Task Difficulty
Entry of same assay script using modified
Function Palette
30
Task Difficulty (Modified Interface)
Total Task Duration for Same Assay Procedure for
DNAIQ (from Promega)
31
T-Test on Task Duration
Comparison between current and modified interfaces
32
Redesign of Function Palette

• Optimized palette layout showing placement of
  more probable icons in easily accessible
  locations
• Model prediction needs to be verified with
  empirical runs on users (next step in the
  research)

33
HTS Experiment

• Verify model predictions with actual user
  performance
• Hypotheses
• H0 There is no difference in performance
  between current and modified interface
• H1 There is a difference in performance between
  current and modified interface
• Design Two-level independent variable (current
  interface vs. modified interface) within subjects
• Materials Published assay scripts (Promega
  website)
• Assignment of scripts to current versus modified
  interface and presentation of current interface
  first followed by modified interface are all
  counterbalanced
• Performance data consisting of task accuracy and
  latency will be analyzed using ANOVA

34
Analysis of Results

• Derive means for correct responses
• Analyze variance around the means for raw data
  values
• Significant difference in variance indicates an
  effect of the experimental manipulation
• Materials Published assay scripts (Promega
  website)
• Assignment of scripts to current versus modified
  interface and presentation of current interface
  first followed by modified interface are all
  counterbalanced
• Performance data consisting of task accuracy and
  latency will be analyzed using ANOVA

35
Experiment results

• Experimental results and analyses should address
• Is there a difference in performance between the
  current and modified interface?
• Are model predictions verified by empirical data?
• Are formal models useful in predicting
  performance on a candidate interface?
• More generally, are formal models useful in
  guiding design of the interface?

36
Semi-Automation Transitions

• Possible transitions in states and actions
• (s0) Begin
• (s1) Bulk Dispense
• (s2) Gripper Move
• (s3) Pipette
• (s4) Tip change, etc.

37
GOMS Scripts as State Machines
G2400 Enter Assay Instructions
p 0.05
Other goals
p 0.3
p 0.6
G2400.200 Choose Tip-Change
G2400.100 Choose Pipette_Transfer
p 0.8
G2400.200 Choose Tip-Change
38
Probability Transition Matrix
s0 s1 s2 s3 s4
s0 s1 s2 s3 s4
P
39
Design of Semi-Automated Menu
Smart software generates a pop-up menu listing
the next top 3-5 operations following most
current operation
40
Model Predictions

• Semi-automated menu reduces MT and Index of
  Difficulty in Fitts Law
• Semi-automated menu has top 3-5 choices thus
  reducing selection difficulty (Hicks Law)
• May not be necessary to verify model predictions
  empirically
• Model predictions based on sound theoretical
  foundations

41
Conclusions

• Advantages of using historical data on usage
  patterns
• Probabilistic models can give guidance to
  interaction design (Stochastic GOMS)
• Ability of more formal approahces to make
  predictions, evaluate, and generate optimum
  interfaces
• Importance for software to be in compliance with
  ISO standards

42
Conclusions

• Advantages for extending GOMS
• Historical usage data largely ignored
• Usage data indicates users most common
  strategies
• Probabilistic rather than deterministic user
  choices (compare to traditional GOMS)
• Improved workflow representation
• Improved task and user statistics
• More accurate, realistic modeling

43
Future Directions of Research

• Utility of Stochastic GOMS for other devices and
  applications (i.e., consider ubiquitous devices
  like cellphones)
• Implications of Stochastic GOMS
• Probability combined with duration give better
  estimates for task metrics
• Examine individual and population variance of
  GOMS operators
• Address assumption that GOMS operators have
  statistical independence (i.e., keystroke is a
  keystroke everytime

44
Thank You!