Design Techniques and Evaluation

1
Design Techniques and Evaluation

• Jon Ivins

2
Design Techniques 1

• Many approaches
• Task analysis
• Data collection
• Defining the task
• Evaluation
• Designing the task
• Use Diagrams
• HTAs, flowcharts, etc.
• Problem is Does designers view match the
  users view of the task?

3
Design Techniques 2

• Techniques to specify dialogues
• STDs
• Simple but can get very messy with complicated
  dialogues
• Statecharts
• Easy to draw
• Can represent concurrency in GUIs c/f
  word-processing example p 73

4
Design Techniques 3

• Essentially PD techniques
• Scenarios e.g. acting out in the head
• Scripts more defined version of a scenario
• Metaphors e.g. cars
• Storyboarding as you did in MULT1003

5
Design Techniques 4

• There is no such thing as the best design
  technique
• Most suitable depends upon
• The designer
• The user
• The task

6
Usability

• Usability is how usable the system is
• Can be like nailing jelly to the wall!
• Must have defined usability criteria that are
  unambiguous and measurable
• Problem not everything is unambiguous and
  measurable in real-life!

7
Usability

• Peuple argues that there are two ways to evaluate
  usability
• Formative techniques
• Summative techniques
• Both have their place
• Both need to be used wisely
• Time, effort, interpretation, etc

8
Formative Techniques

• Usually performed at start to mid-point of
  project
• Can be informal
• Aim is to provide information to help designers
  improve interface / application
• Results tend to be qualitative
• Results are found quickly

9
Formative Techniques

• GOMS (See Peuple for a worked example)
• Goals
• What user is trying to do
• Operators
• Describes the activities the user and the system
  has to do e.g. keypress, mouse move, thinking,
  etc.
• Methods
• A way of achieving the required goal
• Selection rules
• How the best method is selected e.g. time to
  complete, etc.

10
Formative Techniques

• Walkthrough
• Allows user to test design
• Typically by trying out a prototype
• Helpful to clear up misconceptions by designer(s)
• Relies on the observation process to record
  actions / thought process of user
• Does DSDM remove the need for this type of
  evaluation? Or are walkthroughs implicit in DSDM?

11
Formative Techniques

• Heuristic evaluation
• Experts evaluate the design based upon heuristics
• Problems experience / ability of evaluators,
  balancing opinions.
• Dialogue error analysis
• Try to identify error the user might make
• Amend design or design a recovery stage for the
  user

12
Evaluation

• As in most things in life, evaluation needs to be
  planned.
• Develop measurable goals
• Select the techniques and tools to measure them
• Select the staff to carry out the evaluation
• Act on the results of the evaluation

13
Evaluation

• Goals
• Time to complete task
• Time to learn task
• How easy the task is to learn
• How easy is the task to remember
• Meeting benchmark criteria
• Examples given in Lecture 5

14
Evaluation

• Tools
• Formative tools
• Walkthroughs, heuristics, etc
• Summative
• Based on experimental techniques (normally)
• Usually evaluated by statistical techniques
  (discussed in next lecture)

15
Evaluation

• Try to use convergent validity to give confidence
  in findings
• Also need to consider
• Time available to evaluate
• Staff available to carry out the evaluation
• Cost
• Logistics
• Ethics

16
Summary

• We have considered the question Does
  designers view match the users view of the
  task?
• Considered many design techniques
• Task analysis, HTAs, STDs, storyboards, etc, etc
• Reprised Usability
• Introduced formative evaluation techniques