User Studies

1
User Studies

• Defining the space for exploiting user knowledge
  in interaction with system components

2
Engineering Activity
Background Assumption
People produce error People
discover and solve them
Design Assumption
Routine work, rote thinking Development of
knowledge, understanding Flexibility
interchangeable jobs Flexibility skilled
people Standard operating environment Collaborat
ion and collaborative learning are necessary to
business take place in communities Social
interaction is nonproductive Communities are
funds of knowledge Automatation produce
reliability Skills through learning produce
reliability
Learning in not encouraged Learning is
supported
3
Engineering ws Work
Engineering view Work Perspective
Kyng, 1995
4
User vs Designer
Users Contribution
Developers Contribution
Kyng, 1995
5
Validity and Verifiability
Requirements need to be valid with respect to
the original problem statement And they need to
be verified with respect to the subsequent
design Examples of failure Invalid The
alphanumeric keyboard for the Air Traffic Control
system should provide 60 keys, laid out in three
separate groups of 5 x 4 keys so as to allow
for extra strengthening supports between the
groups. Unverifiable Decision making
assistance should be provided. Computer-generated
proposals will be generated by upstream ATC
expert systems and artificial intelligence, still
to be developed.
6
Defining Requirements
Building a model of the supported activities
(processes) Identifying the users Selecting
functional distributions/redundancies Setting
performance requirements
7
Processes Users

• Ethnography
• Interview
• Scenarios

8
Ethnography

• Ethnography is the work of describing a culture.
  The main objective of this activity is to
  understand another way of life from the native
  point of view. It is used to refer to both the
  work of studying a culture and the end product
  an ethnography'' or written text.
• Field work, then involves the disciplined study
  of what the world is like to people who have
  learned to see, hear, speak, think, and act in
  ways that are different. Rather than studying
  people ethnography means learning from people''
• Spradley, 1979.

9
Ethnography from Anthropology

• Its primary use is as a way of gaining insights
  into the life experiences of people whose daily
  lifestyle was different from those living in
  Western developed societies.
• The practice of ethnography is based on
• a commitment to studying activities in the
  natural'' setting in which they occur
• an interest in developing detailed descriptions
  of the lived experience
• a focus on what people actually do understanding
  the relation between activities and environment
• Ethnographic representations include textual
  accounts and summaries, videotape, storyboard and
  images as well as artifacts generated by the
  workprocess.

10
Ethnography Interactive Systems

• The socialization of Ethnography and IS began in
  the 1980s and was motivated by three emerging
  truths.
• First, there was a gradual awareness that the
  narrow focus on isolated individuals using
  computational artifacts was inadequate.
• Second, there was gradual agreement that human
  intelligence was socially constituted and
  achieved.
• Third, there was a growing interest in developing
  computer technologies that acknowledged and
  supported the cooperative nature of human activity

11
Method

• http//lucy.ukc.ac.uk/
• Fieldwork Online Stephen Lyon, a Ph.D. student
  in Anthropology at the University of Kent

12
Four basic roles

• Complete participant
• Participant observer
• Observer as participant
• Complete observer

13
Qualitative observation is ...

• Identifying people and situations
• Gaining access, consent, and trust
• Watching, listening, and experiencing
• Remembering and recording
• Sorting, coding, and analyzing
• Searching for patterns and uniqueness
• Finding the meaning in what is happening
• Hints

14
Querying Users via Interviews

• Excellent for pursuing specific issues
• vary questions to suit the context
• probe more deeply on interesting issues as they
  arise
• good for exploratory studies via open-ended
  questioning
• often leads to specific constructive suggestions
• Problems
• accounts are subjective
• evaluator can easily bias the interview
• prone to rationalization of events/thoughts by
  user
• users reconstruction may be wrong

15
How to Interview

• Plan a set of central questions
• mostly based on results of user observations
• Gets things started
• Focuses the interview
• ensures a base of consistency
• Try not to ask leading questions
• Start with individual discussions to discover
  different perspectives, and continue with group
  discussions
• the larger the group, the more the universality
  of comments can beascertained
• also encourages discussion between users

16
Scenario-Based Design

• Central to most scenario based design is a
  textual description or narrative of a use
  episode. This description is called a scenario.
• The scenario is described from the user point of
  view and may include social background, resource,
  constraints and background information.
• The scenario may describe a currently occurring
  use, or a potential use that is being designed
  and may include text, video, pictures, story
  boards, etc.
• The context might include details about the work
  place or social situation, and information about
  resource constraints. This provides some more
  help in understanding why users do what they do.
• In much current design work the users goals and
  context are often assumed implicitly, or may not
  be taken into account.

17
Scenarios

• The scenario then becomes the design object and
  may be augmented and rearranged as the design
  evolves.
• It can become a hypothetical interaction scenario
  for a new design and allow better understanding
  of the new design.
• It is also desirable to maintain a history of
  past scenarios as a way of capturing past design
  rationale.
• In one sense scenarios are not really new in
  design activity. It's extremely common in design
  to imagine "what if" situations, or to walk
  through a design in ones mind or in a group.
• Scenario based design is an effort to make some
  of what we do already more conscious and explicit.

18
Scenario-Based design by John M. Carroll.
The scenario perspective The function
views concrete descriptions
abstract descriptions focus on particular
instances focus on generic types work
driven technology driven
open-ended, fragmentary complete,
exhaustive informal, rough, colloquial
formal, rigorous envisioned outcomes
specified outcomes
19
Distribution and Performance

• Prototyping...

20
Activities

• Nested scenarios of growing complexity (cluster)
• Independent clusters