ISC 551 Chapter Four Notes

1
ISC 551 - Chapter Four Notes

• Expert Reviews, Usability Testing, Surveys, and
  Continuing Assessments

2
Expert reviews

• Heuristic evaluation determines conformance to a
  list of design goals
• Guidelines review checks for compliance with the
  guidelines document
• Consistency inspection examines consistency
  across a set of interfaces
• Cognitive walk-through simulates walking through
  (using) the interface to complete typical user
  tasks
• Formal inspection courtroom-style meeting to
  discuss/defend merits and weaknesses of the
  interface

3
Number of experts to use

• Research says experts are hard to find, so it is
  acceptable to have only one.
• Right?

4
Number of experts to use

• Research says experts are hard to find, so it is
  acceptable to have only one.
• Right?
• WRONG!
• Different experts find different problems, so
  using 3 to 5 experts can be productive

5
Experts should

• have expertise in either the particular
  application domain or the user-interface domain
• conduct a review on short notice and do it
  rapidly
• perform reviews early or late in the design phase
• be able to note possible problems for designer
  discussion, but leave the solution to the
  designers themselves

6
Usability testing

• User think aloud
• Videotaping
• Discount usability engineering
• Field tests

7
Competitive usability testing

• Who is competing with whom?
• Two or more prototypes for the same task
• How do you win?
• By performing better on usability tests than the
  other interface (on average)

8
Usability Testing - Scholarly vs. Practitioner
Approaches
9
Surveys

• Use of questionnaires to collect data
• Closed-ended questions
• Paper-and-pencil or computer-based
• No researcher present

10
Types of questions

• Likert scale
• strongly agree, agree, neutral, disagree,
  strongly disagree
• Bipolar adjective scale
• Frustrating 1 2 3 4 5 6 7 Satisfying

11
Advantages of Surveys

• Inexpensive
• Accommodate large samples
• Easy to administer, score, code
• Objective
• Generalizability
• Tested with statistics

12
Disadvantages of surveys

• Low response rate
• Requires careful development and pilot testing
  (questions must be correct)
• Reflects researcher biases
• No depth

13
Acceptance testing
Set goal
Measure performance
Performance acceptable?
Yes
No
Rework interface
Done
14
Acceptance test-writing

• Have a measurable goal, either
• mean (average) score, or
• percentage of subjects at a certain level
• Wording
• evaluated as true or false
• clearly
• One component/issue per test

15
What kind of information to collect during active
use?

• Data logging usage frequency data
• access to frequently-used features
• why not using other features?
• Customer satisfaction data?
• via bulletin boards, surveys, customer support
• long list of low-cost features

16
HCI scientific method (4.7)

• Issues
• theory-testing combined with practical problem
• variables, hypotheses
• sampling
• experimental design
• statistical analysis

17
References

• Crocker, L. and Algina, J. Introduction to
  Classical and Modern Test Theory, 1986, Harcourt
  Brace Jovanovich, Fort Worth, Texas
• Grover, V. A Tutorial on Survey Research From
  Constructs to Theory, 1997, URL
  http//theweb.badm.sc.edu/grover/grover.htm
• Newsted, P., Huff, S., and Munro, M. (Eds.), MIS
  Survey Research, URL http//www.acs.ucalgary.ca/
  newsted/surveys.html
• Shneiderman, Ben, Designing the User Interface
  Strategies for Effective Human-Computer
  Interaction, 1998, Addison Wesley Longman, Inc.,
  Reading, MA, 1998