Specifying Usability Requirements: The Need for Standards

1
Specifying Usability Requirements The Need for
Standards

• Mary Theofanos
• Brian Stanton
• Visualization and Usability Group

2
How do you specify usability requirements?
3
Despite usability horror stories

• Usability requirements are rarely identified by
  organizations purchasing or developing software

4
What is the state of usability in Government
procurements?

• Federal agencies have virtually no visibility of
  software product usability before we make
  procurement decisions
• We do not know how to include usability
  requirements in procurements.
• We do not know how to compare product usability
  or to plan for or measure usability costs

5
But Usability can be objectively defined and
measured

• This implies that we can
• Identify usability requirements
• Measure usability before we deploy or purchase a
  product

6
ISO Standards define 3 measures of Usability
Usability The extent to which a product can be
used by specified users to achieve specified
goals with effectiveness, efficiency and
satisfaction in a specified context of use

• Effectiveness -- a measure of user productivity,
  how well a user can perform his job accurately
  and completely. (i.e. completion rate, number of
  errors)
• Efficiency -- a measure of how quickly a user can
  perform work, the resources expended to
  accomplish the task. (i.e. time on task)
• Satisfaction--The degree to which users like the
  product a subjective response in terms of ease
  of use, frustration, and usefulness.

7
Example of an Operational Definition for
Usability for a travel system

• On their first try, within 15 minutes, 75 of
  Government travelers shall be able to correctly-
• Create a travel request form
• Select one departure flight and one return flight
• Designate one hotel
• Reserve one rental car
• Forward the travel request form for approval . .
• By their second try, within 15 minutes, 90 shall
  be able to complete the tasks correctly

8
In 1998, Industry approached NIST to develop a
method for factoring usability into procurement
decisions

• Goal Increase the visibility of software
  usability
• Reduce uncontrolled overhead costs of software
  usability problems, while improving user
  productivity and morale.
• Encourage software suppliers and consumer
  organizations to work together to understand user
  needs and tasks.
• Define and validate an industry-wide process for
  providing visibility of software usability to
  support product decision-making.

9
Participants span industry, government and
academia

• Government
• Brookhaven National Labs
• Census Bureau
• DISA
• DHHS
• GSA
• IRS
• Library of Congress
• OCLC
• OPM
• SSA
• USDA
• US Army Corps of Engineers
• US Air Force
• State of Georgia, DOT

• Universities
• Dalian Maritime Univ. China
• Rensselaer Polytechnic Inst.
• University of Indiana
• San Jose State
• SEI/CMU
• Shizuoka University
• Syracuse University
• UC Berkeley
• UCLA
• University of Maryland
• University of Michigan
• University of Copenhagen
• University College London
• University of Bologna

• Industry
• Boeing
• Microsoft
• Oracle
• Dell
• Fidelity Investments
• Motorola
• Apple
• General Electric
• Ford
• Honeywell
• Phillips
• Whirlpool
• SAP
• State Farm
• Xerox

10
The role of NIST has been to

• Act as a facilitator in bringing together
  industry usability professionals, academics, and
  government representatives.
• Maintain the documents and support the ISO
  standardization process
• Collect and analyze data to determine the value
  of incorporating usability into product
  decision-making and the impact of the adoption of
  IUSR products.

11
How do we incorporate usability requirements into
the procurement process?
12
The Common Industry Specification for Usability
Requirements
Developed by NIST and IUSR to provide a structure
for

• Defining usability requirements in sufficient
  detail to make an effective contribution to
  design and development
• Defining usability criteria that can be
  empirically validated subsequently if needed.

13
CISUR supports communication between usability
professionals

• Procurers can specify usability in a Request for
  Proposals or a contract for software
• Supplier organizations can determine if usability
  requirements specified by a customer are
  realistic for their product, and plan on how to
  ensure that a product meets these requirements.
• Suppliers can assess if the usability
  requirements specified for product development
  meet the needs of the customer organizations.

14
CISUR supports communication within and between
organizations

• Among members of the development team to specify
  requirements for use by the development team
• Between the customer and supplier of a custom
  product to define specific customer requirements
• Between a range of potential customers and a
  supplier of an off the shelf product, to define
  diverse requirements.

15
The CISUR identifies 3 components to specifying
requirements

• Context of Use description of intended users,
  their goals, equipment, and environment in which
  product will be used
• Performance and satisfaction criteria ways in
  which the usability of the product can be
  measured
• Test Methods how the product will be tested to
  determine whether the usability requirements have
  been met

16
The CISUR identifies 3 levels of specification
for the components
17
The CISUR was developed with the following
characteristics

• Independent of specific design process
• Facilitates iterative development of requirements
  
• Complements other user centered design standards
• Only applies to usability requirements
• Does not specify a format for specification
• As a complement to the CIF

18
Next Steps

• ISO Standardization
• Case Studies

19
Getting a copy of the CISUR

• http//www.nist.gov/iusr

20
Questions?

• Mary Theofanos
• NIST
• [phone]
• Maryt_at_nist.gov

Brian Stanton NIST [phone] brian.stanton_at_ni
st.gov

• www.nist.gov/iusr