Chapter 8 Usability Specification Techniques Hix

1
Chapter 8 Usability Specification TechniquesHix
Hartson
2
Usability Specifications

• Quantitative Usability Goals

3
Usability Attributes

• Usability characteristics to be measured
• Initial Performance
• Long-term performance
• Learnability
• Retainability
• Advanced Feature Usage
• First Impression
• Long-term user satisfaction

4
How can attributes be measured?

• Objective Tasks (called benchmark tasks)
• Tasks must be representative of what users would
  perform
• Measure performance on benchmark tasks
• Tasks must be specific
• Do not tell user how to carry out tasks
• Should be simple, or small combinations of simple
  tasks
• Consider who your end users are!

5
Objective Tasks (cont.)

• Example
• For the attribute Initial Performance may measure
  how well they perform a specific function that is
  primary to the software. Time and error data can
  be collected.

6
Subjective Questionnaires

• Asks for opinions on use .
• QUIS is an existing validated questionnaire.
• Questionnaires produce objective data as well as
  subjective data.
• Example First Impression attribute, would want
  certain rankings on questionnaire.

7
Usability Specifications

• For the Task looking for the following
  information
• Current Level of Task Performance
• Worst Acceptable Level
• Planned Target Level
• Best Possible Level
• Observed Results

8
Types of measures

• Objective measures
• Time to complete a task
• Number or percentage of errors
• Percentage of task completed in a given time
• Ratio of successes to failures
• Time spent in errors and recovery
• Number of commands/actions performed
• Frequency of help and documentation use
• Number of repetitions of failed commands
• Number of available commands
• Number of time user expresses frustration or
  satisfaction

9
Set current levels based on

• Previous or existing system
• Similar competitive systems
• Performing computer tasks
• Performing manual tasks
• Market input
• From previous prototypes

10
Considerations when developing specifications

• Is each attribute practically measurable?
• Are the user classes specified clearly?
• Are the values for the levels reasonable?
• How well do the attributes capture usability for
  the design?

11
Chapter 10 Formative EvaluationHix Hartson
12
What is meant by formative evaluation?

• A formal evaluation plan during design process.
• To be begun as early as possible in design cycle.
  
• First evaluation to take place when 10 of
  project resources are expended.

13
Summative Evaluation

• A human factors engineers worst nightmare
• Evaluation only after completion of the design.

14
Types of Evaluation Data

• Objective Directly observed and measurable
• Subjective Opinions
• Quantitative Numerical data
• Qualitative lists of user problems,
  suggestions, etc.

15
Steps of Formative Evaluation

• Develop evaluation plan (or experiment)
• Selecting Participants
• Developing tasks and task orders
• Determining protocol and procedures
• Pilot testing
• Direct the evaluation

16
Direct the evaluation

• Data Generation
• Benchmark tasks
• User preference questionnaires
• Concurrent Verbal protocol
• Retrospective verbal protocol
• Critical incident taking
• Structured interviews

17
Direct the evaluation (cont.)

• Data Collection
• Real-time note taking
• Videotaping
• Audiotaping
• Internal instrumentation of the interface

18
Direct the evaluation (cont.)

• Analyzing the Data
• Compute averages for benchmark tasks
• Determining problems or user difficulty
• Determine effects on user performance
• Impact analysis
• Importance
• How important is this problem to the design
• Generate solutions
• Consider costs to fix problems
• Redesign, implement, retest

19
Formative Evaluation Pros Cons

• Pros
• 4 to 5 subjects find 80 of problems
• Sensitive to major problems
• Can be very through process
• Developers empathize with users
• Cons
• Time Consuming
• Expensive

20
Other Usability Testing Methods

• Heuristic
• Guidelines
• Computer Evaluation

21
Heuristic

• Usability engineer reviews and evaluates program
  with no standard procedure
• Pros
• Quickly identify problems
• Major problems are discovered
• Cons
• Must use more than one usability engineer
• Minor problems not discovered

22
Guidelines

• Evaluator looks at design to see if it meets
  guidelines
• Pros
• Finds general and recurring errors
• Easily applied
• Cons
• Major problems can be missed
• Guidelines are not exhaustive
• Not all programs are created equal
• Not all guidelines apply

23
Computer Evaluation

• Automated computer program evaluates software
• Pros
• Potential tool in the future
• Cons
• Expensive
• Currently finds only primitive problems
• Will designers lose creativity trying to design
  to meet tests?

24
Conclusions

• Heuristic can be cost effective
• Use more than one method
• Users determine success of software and companies