Title: CostJustifying Usability
1Cost-Justifying Usability
- Javad Sadeghi
- CPSC 681
- Nov 5, 2007
2Outline
- Motivation
- Background
- Discount usability
- Samples
- Methodology
- Worked example
3Motivation
Dennis Wixon, SIGCHI Bulletin, Volume 27, Number
2, April (1995), p94
4View from the other side of the table
- For each dollar a company invests in developing
the usability of a product, the company receives
10-100 in benefits and wins customer
satisfaction and continued business - For each dollar spent to fix a problem during
product design, 10 are spent to fix the same
problem in product development, and 100 or more
are spent to fix the same problem after product
release
Claire Marie Karat, A business case approach to
usability cost justification. In, R. Bias and D.
Mayhew, Eds. Cost-Justifying Usability, Academic
Press, NY, 1994.Â
5Background
- Cost-benefit of usability (Marilyn Mantei and
Toby Teorey) - Business case approach (Clare-Marie Karat )
- Cost-justifying usability (Bias and Mayhew)
- Discount usability (Jacob Nielsen)
- Web-based applications
Bias R. G., Mayhew D. J. Cost-Justifying
Usability An Update for the Internet Age, 2nd
Edition. Morgan Kaufmann Publishers, 2005
6Discount usability
- Le mieux est lennemi du bien
- Focuses on the good
- Reduces the costs
- Requires less experience and knowledge
- Three techniques
- Scenarios
- Simplified thinking aloud
- Heuristic evaluation
Nielsen J., Guerrilla HCI Using Discount
Usability Engineering to Penetrate the
Intimidation Barrier http//www.useit.com/papers/g
uerrilla_hci.html
7Scenarios
- Cuts down complexity by eliminating parts of full
system - Small and cheap to design and implement
- Paper mock-ups
Nielsen J., Guerrilla HCI Using Discount
Usability Engineering to Penetrate the
Intimidation Barrier http//www.useit.com/papers/g
uerrilla_hci.html
8Simplified thinking aloud
- Thinking aloud studies, videotape subjects,
perform detailed protocol analysis - Simplified version
- Take notes
- 35 test user per test
Nielsen J., Guerrilla HCI Using Discount
Usability Engineering to Penetrate the
Intimidation Barrier http//www.useit.com/papers/g
uerrilla_hci.html
9Heuristic evaluation
- Finding usability problems
- Evaluator inspect alone
- Current usability guidelines 1000 rules
- 10 basic usability principles
- Easy to learn
- Experts best
- Non-experts can find most problems
- Remaining, using simplified thinking aloud
Nielsen J., Guerrilla HCI Using Discount
Usability Engineering to Penetrate the
Intimidation Barrier http//www.useit.com/papers/g
uerrilla_hci.html
10Nielsens surveys on cost-justifying usability
- Usability costs
- 10 of the budget of each project
- Cost does not increase linearly with the budget
- Usability benefits
- 135 usability increase after redesign
- Return on investment (ROI)
- It is not linear with the costs
- The more users, the bigger ROI
Jakob Nielsen's Alertbox, http//www.useit.com/ale
rtbox/20030107.html
11Samples of cost-justifying usability
- Australian insurance company
- Usability-enhancing project, costing lt A 100,000
- Average of 7.8 errors per form
- More than one hour to repair the errors of each
form - Annually saving as a result gt A 530,000
- Redesigning application forms
- IBM employees login system
- Usability project, financing 20,700
- Making sign-on attempts faster
- Savings in work time during the first day of the
new system 41,700
Nielsen J., Usability Engineering. Academic
Press, 1993.
12Methodology
- Cost-justifying usability is
- A conservative approach
- Part of project planning
- Mayhew and Manteis framework
- A basic framework for cost-justifying usability
- Estimating costs
- Estimating benefits
- Justifying the assumptions made for benefits
Randolph G. Bias, Deborah J. Mayhew,
Cost-Justifying Usability. Academic Press, 1994.
13Estimating costs
- Steps
- Personnel hours and equipment costs
- Needs experience
- Additional one time setup
- Lab setup
- Usability tasks (e.g. user profile)
- Techniques (e.g. user questionnaire)
- Development of survey
- Pilot testing
- Distribution and collection
- Responding
- Coding and entering data
- Analyzing results
- Computer time
- Supplies and duplicating costs
Randolph G. Bias, Deborah J. Mayhew,
Cost-Justifying Usability. Academic Press, 1994.
14Estimating benefits
- Increased user productivity
- Decreased user errors
- Decreased training costs
- Savings gained from making changes earlier in
design - Decreased user support
- Relevant audience
- Internal development organization
- Vendor company
- Relevant categories of benefits
- Estimation
- Best measurement unit
- Assuming magnitudes
- Increased sales
- Decreased customer support
- Savings gained from making changes earlier in
design - Reduced cost of providing training
Randolph G. Bias, Deborah J. Mayhew,
Cost-Justifying Usability. Academic Press, 1994.
15Example
- Data entry system for 250 users
- Two to four primary screens
- Process 60 of them per day
- Users work 230 days a year
- Total cost per hour of 25
- The benefit of 1 second speedup in year
- 250 users 60 screens 230 days 1/3600 hours
25 23,958 / year
Randolph G. Bias, Deborah J. Mayhew,
Cost-Justifying Usability. Academic Press, 1994.
16Justifying the assumptions made for benefits
- Reasonable and likely minimum benefits
- Not precise, proven and specific
- Basing the predictions on known facts
- Published research
- Actual case histories
- Previous experience
Randolph G. Bias, Deborah J. Mayhew,
Cost-Justifying Usability. Academic Press, 1994.
17Cost-benefit analysis of heuristic evaluation a
case study
- Case study integrating system
- For internal telephone company
- 11 evaluators
- Find 44 usability problems
- 40 core problems
- 4 from other part of the interface
- Estimating the costs (evaluation time)
- Estimating the benefits (increased usability)
Nielsen J., Guerrilla HCI Using Discount
Usability Engineering to Penetrate the
Intimidation Barrier http//www.useit.com/papers/g
uerrilla_hci.html
18Estimates of the times spent
?
?
?
Nielsen J., Guerrilla HCI Using Discount
Usability Engineering to Penetrate the
Intimidation Barrier http//www.useit.com/papers/g
uerrilla_hci.html
19Time expenditure
- time(i) 47.8 5.2 i (Eq1)
- i number of evaluators
- Not exact for larger i
- Reduction reducing 2 observers to 1
- Revised time(i) 37.3 4.2 i (Eq2)
- Estimated hourly loaded cost 100
- Cost estimation 10,500
Nielsen J., Guerrilla HCI Using Discount
Usability Engineering to Penetrate the
Intimidation Barrier http//www.useit.com/papers/g
uerrilla_hci.html
20Benefit estimation
- Estimations of evaluators for improvements
Nielsen J., Guerrilla HCI Using Discount
Usability Engineering to Penetrate the
Intimidation Barrier http//www.useit.com/papers/g
uerrilla_hci.html
21Benefit estimation (cont.)
- Mean values
- 0.8 days -learning time reduction
- 18-expert speedup
- To be conservative
- 0.5 days - learning time reduction
- 10- expert speedup (3.3 of total work time)
- Assuming 2000 users (from 3000)
- User-days saved 1,000
- User-years saved 67 13,000 user days
- Total user days saved 14,000
Nielsen J., Guerrilla HCI Using Discount
Usability Engineering to Penetrate the
Intimidation Barrier http//www.useit.com/papers/g
uerrilla_hci.html
22Convert to monetary terms
- User-day cost of 100
- With actual saving of 50
- One-year saving 540,000
- Net present value (benefit) 500,000
- Benefit-cost ratio 48
Nielsen J., Guerrilla HCI Using Discount
Usability Engineering to Penetrate the
Intimidation Barrier http//www.useit.com/papers/g
uerrilla_hci.html
23Thanks!