Assignment 2

1
Assignment 2

• Due Mar 6 (Tue)

2
Introduction

• Usability studies on Microsoft MSN Messenger
  Version 7
• Usability is a quality attribute that assesses
  how easy user interfaces are to use.

3
Assignment Overview

• You work for the Ace Consulting Company (), a
  consulting firm that specializes in evaluating
  interfaces.
• You and your team have been contracted to do a
  usability study on MSN Messenger version 7.
• You are to prepare a usability report for the
  vice-president in charge of that system's use and
  redevelopment.

4
Phase 1 Preparation

• Read the assignment description
• Try system out yourselves
• Prepare some example tasks
• Decide who you will employ as participants in
  your study (2 to 4 people)
• Assign jobs among group members The test
  administrator and the scribe

5
Phase 1 Preparation

• Write-up the pre-test questionnaire (10
  questions) that is used to determine the person's
  experiences and beliefs as specifically related
  to the task and system.
• steps for conducting a user observation (read
  online)
• Write up the post-test questionnaire

6
Phase 1 Preparation

• Select a core set of typical tasks - These tasks
  should be realistic ones that typical users would
  try to do with the system
• how to discover those typical tasks?
• 1) let people use their own real tasks
• 2) Ask a random sample of people who are using
  the system what they typically do with it, then
  generalize them.
• 3) Come up with some tasks by yourself

7
Phase 2 Conducting the Usability Study 

• Test process
• 1) Provide the introductions
• 2) Administer the pre-test questionnaire
• 3) Run the test
• 4) Administer the post-test questionnaire and
  conduct any necessary debriefing

8
Phase 2 Conducting the Usability Study 

• Throughout the exercise Active intervention and
  conceptual model formation.
• You are looking for places where the model is
  incorrect or undeveloped.
• Have them explain the meaning behind the
  different components of the introductory
  window/screen.
• Have them do their task.
• Ask them if they can now explain how some of the
  components actually work

9
Phase 2 Conducting the Usability Study 

• Run participants through the three main test
  cases
• Silent observer case (1 participant at least)
• Think aloud scenario (1 participant at least)
• Constructive interaction (2 participants at
  least)
• Note the experimenter should not help the
  participants with the task

10
Phase 2 Conducting the Usability Study 

• Post-test and debriefing

11
Phase 3 Writing up your results

• Section 1. Scenario
• Give a very brief reminder to the VP about what
  the system is, and then explain the role of your
  product evaluation team. Make sure you tell
  him/her the point of your work!

12
Phase 3 Writing up your results

• Section 2. Methodology
• Explain what you did.
• Some information to include the number of
  participants, which type of participants were
  assigned to each case
• pre-test and post-test questionnaires
• task descriptions
• which participant participated in which test
  scenario

13
Phase 3 Writing up your results

• Section 3 Observations
• Use selected raw and collapsed data,
  paraphrasing, comments, questionnaire and
  interview results to Summarize your observations
  in this section.
• observations should be separate from the
  interpretations

14
Phase 3 Writing up your results

• Section 4 Interpretation System strengths and
  weaknesses
• Identify common and important problems and
  strengths of the system.
• generalize all the minor 'symptoms' into a
  higher-level description of what the major
  problems are with this system

15
Phase 3 Writing up your results

• Section 5 Suggested improvements
• Describe the five most important changes that you
  would make to the design of the system, with
  explanations of why you think they are important

16
Phase 3 Writing up your results

• Section 6 Conclusion
• Summarize what you found and list your
  recommendations.

17
Phase 3 Writing up your results

• Appendix 1 Comparison of the different
  techniques
• Briefly summarize your experiences with each
  method, contrasting them for ease of use, the
  richness of the information obtained, their
  advantages, etc.
• recommend the methods you wish your group to use
  in the future

18
Phase 3 Writing up your results

• Appendix 2 Raw data
• All original observations/recordings

19
Next Week

• Presentation
• 1) One Task example
• 2) 3 Strength of the System
• 3) 3 Weakness of the System
• 4) Recommendations on how to improve the system
• Print out the above materials and hand it to me
  before your presentation