Cognitive Modeling 1

1
Cognitive Modeling 1

• Bringing a user model into the realm of
  predictive evaluation

2
JDF Notes to myself

• Lots of DFAB references
• Need to add data and examples

3
Todays Agenda

• What exactly is cognitive modeling?
• Why do we care about it?
• What are some models?
• What are their relative strengths and
  shortcomings?
• How do we use these models in predictive
  evaluation?

4
Cognitive Modeling Explained
5
How do we view the user?

• Three prevalent models (not mutually exclusive!)
• A sensory processor (literally,
  stimulus-response)
• An interpreter or predictor (dont you ever
  think?)
• An actor in an environment (a slice of the big
  picture)

6
System PhilosophiesShape User Models

• Interaction with philosophy
• Software system as tool, interface
  usability-engineered membrane
• Human-as-processor / -interpreter
• Interaction through philosophy
• Software as communication medium, interface plays
  a role in social context
• Human-as-interpreter / -actor

7
User Models Shape HCI

• If a user is a sensory processor
• Your model is informed by experimental
  psychology, quantitative sensory results
• You strive to obey human limits
• If a user is an interpreter/predictor
• Your model is informed by cognitive psychology,
  possibly a touch of AI
• You strive to fit a system into the users
  knowledge base

8
User Models Shape HCI, cont.

• If a user is an actor in an environment
• Your model is informed by ecological
  psychology, ideas from anthropology (e.g.
  ethnographic field studies)
• You strive to fit a system into a task and a
  social context
• Use Situated Action, or Activity Theory, or
  Distributed Cognition (see Unit on Social
  Modeling)
• Roles imply frameworks for design and evaluation

9
So why bother?

• Idea If we can build a model of how a user
  works, then we can predict how s/he will interact
  with the interface
• Cognitive model ? predictive evaluation
• No mock-ups or prototypes
• Consider, as we go What do you actually need,
  and what do gaps you fill up/bridge with
  assumptions?

10
Model Components

• User qualities
• Understanding
• Knowledge
• Intentions
• Processing
• Levels of detail
• Plans (high-level)
• Motor actions(low-level)

11
Model Human Processor

• Perceptual, cognitive and motor processors
• Recognize-act cycle
• Contents of WM trigger actions held in LTM
• Predictive rules
• Power law of practice
• Rationality principles

12
Focus Two Types of Modeling

• Stimulus-Response
• Fitts law
• Hicks law
• Practice law
• Human as interperter/predictor - based on Model
  Human Processor (MHP)
• Key-stroke Level Model
• GOMS Models

13
1. Stimulus-Response (Sensory Processor)

• Power law of practice
• Tn T1n-a
• T on the nth trial is T on the first trial times
  n to the power -a a is about .4, between .2 and
  .6
• Fitts law
• Hicks law - Decision time to choose n equally
  likely alternatives
• T Ic log2(n1)

14
2. Keystroke-Level Model (KSLM)

• KSLM - developed by Card, Moran Newell, in
  their book and in CACM
• Skilled users performing routine tasks
• Assigns times to basic human operations -
  experimentally verified
• Based on MHP - Model Human Processor

15
KSLM Accounts for

• Keystroking TK
• Mouse button press TB
• Pointing (typically with mouse) TP
• Hand movement betweenkeyboard and mouse TH
• Drawing straight line segments TD
• Mental preparation TM
• System response time TR

16
KSLM

• Decompose task into sequence of operations - K,
  B, P, H, D, M, R
• Place M operators
• In front of all Ks that are NOT part of argument
  strings (ie, not part of text or numbers)
• Example - select a word and type new text
• Home on mouse H(mouse)
• Point to word P(word)
• Select word M2B(mouse button)
• Home on keyboard H
• Type new 5-letter word M5K

17
KSLM

• Now remove Ms according to the rules
• Anticipated by prior operation
• PMK -gtPK
• If string of MKs is a single cognitive unit (such
  as a command name), delete all but first
• MKMKMK -gt MKKK (same as M3K)
• Redundant terminator, such as )) or rtn rtn
• If K terminates a constant string, such as
  command-rtn, then delete M
• M2K(ls)Mrtn -gt M2K(ls)rtn

18
KSLM

• Apply rules to example
• Home on mouse H(mouse)
• Point to word P(word)
• Select word M2B(mouse button)
• Home on keyboard H
• Type new 5-letter word M5K
• T 5TK 2TB TP 2TH TM TR

19
KSLM

• Plug in real numbers from experiments
• K .08 sec for best typists, .28 average, 1.2 if
  unfamiliar with keyboard
• B down or up - 0.1 secs click - 0.2 secs
• P 1.1 secs
• H 0.4 secs
• M 1.35 secs
• R depends on system often less than .05 secs
• T 5TK 2TB TP 2TH TM TR
  5(.28)2(0.2)1.12(.4)1.35 .05 5.10 secs

20
KSLM ProblemMouse for menu selection?

• What is the right operator sequence?
• HmousePBleft-clickMPBleft-click
• Complicated rules for placing Ms but boils
  down to chunking (one M before each chunk of a
  task)
• Candidate Ms before each B, K, and P involved in
  specification or selection of a command
  eliminate the Ms that are fully anticipated or
  in a cognitive unit
• Textbook timings (all in seconds)
• H 0.40, P 1.10, B 0.20, M 1.35
• Total predicted time 4.35 s

21
KSLM Problem

• Consider a KSLM decomposition of selecting File
  / Print from a pull-down menu
• Now consider the same task using only the
  keyboard, with the ALT-F accelerator to open the
  File menu and then the P key to select the Print
  option
• Use texts operator timings for these scenarios
  assume hands start on keyboard

22
KSLM ProblemKeyboard for Menu Selection?

• Recall mouse operator sequence over two chunks
  (open File menu, select Print option) HPBMPB
• Assuming same two chunks, you have
  MKALTKFMKP
• Times for K based on typing speed
• Good typist, K 0.12 s, total time 3.06 s
• Poor typist, K 0.28 s, total time 3.54 s
• Non-typist, K 1.20 s, total time 6.30 s
• Possible moral Shortcut keys not necessarily
  faster than using the mouse

23
Modeling Cognitive Capabilities

• Performance
• Quantitative
• Time prediction
• Working memory constraints
• Competence
• Legal behavior
• Consistency (transfer)

24
Cognitive models

• Hierarchical
• GOMS, CCT
• Linguistic
• BNF, TAG, CLG
• Cognitive architectures
• GPS, PUM, ICS

25
Hierarchical decomposition

• Common thread - divide conquer
• Proper granularity
• When to start/stop
• What is the unit task?
• Does not model problem solving

26
3. GOMS

• A successful engineering model
• G
• O
• M
• S

27
What G-O-M-S Means

• Goals desired endstates
• Subdivide into lower-level operations
• Operators lowest-level task-oriented actions
  (move mouse, read dialog box)
• Methods sequence of operators for accomplishing
  a goal
• Selection rules to choose between multiple
  methods
• GOMS attempts to predict method choice when are
  multiple ways to perform task

28
GOMS

• Probably the most widely known and used technique
  in the human as information processor vein
• Heavy MHP influence
• Same authors
• Rationality, goal orientation assumed
• Idea Assign times to each subtask in a linear
  task decomposition

29
GOMS Example

• GOAL ICONIZE WINDOW
• select
• GOAL USE-CLOSE-METHOD
• MOVE-MOUSE-TO-WINDOW-HEADER
• POP-UP-MENU
• CLICK-OVER-CLOSE-OPTION
• GOAL USE-L7-METHOD
• PRESS-L7-KEY /function key/

30
GOMS Example - Selection Rules

• Rule 1 USE-CLOSE-METHOD if hands are on mouse
• Rule 2 USE-L7-METHOD if hands are on keyboard

31
The GOMS family

• CardMoranNewell-GOMS
• Simplified version - Keystroke-Level Model
• Cognitive Complexity Theory
• Natural GOMS Language NGOMSL
• Cognitive Perceptual Motor-GOMS

32
The real GOMS family

• Card, Moran and Newell
• Peter Polson David Kieras
• Bonnie John Wayne Gray

33
The closure problem

• Another competence measure

34
Issues with GOMS family

• What limitations do you see?
• (see Web for more on GOMS)

35
Class Discussion GOMS

• Define each of the letters in the acronym GOMS
• What is the difference between an operator and a
  method?
• How do you derive task times, and what good are
  they, really?
• What are the assumptions of GOMS?
• When is GOMS appropriate?

36
Discussion Points GOMS

• Goal photocopying a piece of paper
• GO-TO-COPIER
• if (user is Bob)
• KEY-CARD-ACTIVATE-COPIER
• else
• COIN-ACTIVATE-COPIER
• PLACE-ORIGINAL-ON-GLASS
• MAKE-COPY
• Is this decomposition detailed enough?

37
Discussion Points GOMS

• Determine times for each operator, and for the
  task sequence, just add up the times to get total
  time sequence
• Assumes expert users behaving as rational
  problem-solvers why?
• Assumes you know a good sequence of tasks and can
  estimate times decently well
• GOMS power degrades when one of these
  assumptions does not hold

38
One More GOMS Question

• Case study with NYNEX telephone system
• Specialized GOMS analysis (equipped for parallel
  tasks) used to determine critical path, task
  timings
• Analysis concluded new system would be slower
  system was abandoned, saving millions of
  dollars
• Anything wrong with that conclusion?

39
Other models/variantsto know about

• know about means know they exist
• NGOMSL similar to GOMS, but expresses goals as
  noun-action pair
• more sophisticated, handles expert-novice better
• Cognitive Complexity Theory uses a hierarchical
  goal decomposition with production rules (if a,
  then b) for a generalized transition network
• better predictive power, size of production rule
  set a good measure of task complexity

40
Cognitive Complexity Theory

• Production rules
• Executable
• Competence
• match against an executable system model
• Performance
• measures of memory requirements

41
SummaryIssues in Cognitive Modeling

• Terminology
• Whats expert vs. novice?
• Granularity problems (see GOMS)
• Still no user, per se
• No notion of what the real users want
• Time-consuming and lengthy
• One user, one computer
• No social context