Software Usability Course notes for CSI 5122 University of Ottawa

1
Software UsabilityCourse notes for CSI 5122 -
University of Ottawa

• Section 2
• Conducting Research and Project requirements
• Timothy C. Lethbridge
• lttcl_at_site.uottawa.cagt
• http//www.site.uottawa.ca/tcl/csi5122

2
Main topics for these notes

• Places to search for background information
• Writing a report
• How you will be marked
• Presenting the results

3
General Research Sources - 1

• Databases that the University of Ottawa Library
  maintains a subscription to
• Must be using an on-campus computer
• Or connected by the VPN or proxy server
• See http//www.biblio.uottawa.ca for details on
  how to access the librarys digital resources
• Scopus
• http//www.scopus.com/scopus/search/form.url
• Very good general meta-search for scientific
  information
• Example searches to try
• Use colour user interface
• Cognitive walkthrough
• Usability engineering

4
General Research Sources - 2

• IEEE Xplore
• http//ieeexplore.ieee.org/Xplore/dynhome.jsp
• ACM Digital library and giude to literature
• http//portal.acm.org/portal.cfm
• Springerlink
• Access to Springer journals and Lecture Notes in
  Computer Science
• http//www.springerlink.com/home/main.mpx
• Google Scholar
• http//scholar.google.ca/
• Science Direct
• http//www.sciencedirect.com/
• HCIBib
• http//hcibib.org/

5
Key Journals

• ACM transactions on Computer-Human Interaction
• www.acm.org/tochi/
• Human-Computer Interaction
• http//hci-journal.com
• International Journal of Human Computer
  Interaction
• http//www.informaworld.com/openurl?genrejournal
  issn1044-7318
• International Journal of Human Computer Studies
• http//www.sciencedirect.com/science/journal/10715
  819
• Interacting With Computers
• www.elsevier.com/locate/intcom

6
Other periodicals

• ACM interactions
• Magazine style
• http//www.acm.org/interactions/
• ACM SIGCHI Bulletin
• Columns, reports, etc.
• http//bulletin.sigchi.org/

7
Key conferences

• Up to date list of some upcoming conferences
• http//degraaff.org/hci/conference.html
• CHI
• http//www.chi2009.org/
• HCI International
• http//www.hcii2009.org/
• Proceedings published by Springer
• Interact
• http//www.interact2009.com/

8
Key websites for usability

• The Nielsen- Norman Group Jakob Nielsen and Don
  Norman
• http//www.useit.com
• Some stuff is free, other material is not
• Don Normans Essays
• http//www.jnd.org/dn.pubs.html
• Usability First
• http//www.usabilityfirst.com/
• World Usability Day
• http//www.worldusabilityday.org/
• Usability Professionals Association
• http//www.upassoc.org/
• Resource list
• http//www.slostc.org/topics/usability/resources_l
  ist.html

9
A sample of books on usability

• See here for a good list of books
• http//www.hcibib.org/readings.html
• Some particularly noteworthy books
• A. Dix, J. Finlay, G. Abowd R. Beale.
  Human-Computer Interaction. 3rd Edition. Prentice
  Hall, 2004
• H. Sharp, Y. Rogers, J. Preece Interaction
  Design Beyond Human-Computer Interaction, John
  Wiley Sons, Inc, 2007
• D. J. Mayhew The Usability Engineering
  Lifecycle A Practitioner's Handbook for User
  Interface Design, Morgan Kaufmann, 1999
• B. Shneiderman. Designing the User Interface
  Strategies for Effective Human-Computer
  Interaction,Third Edition, 1997
• Donald A. Norman. The Psychology of Everyday
  Things., 1990
• Randolph G. Bias Deborah J. Mayhew (Eds.)
  Cost-Justifying Usability. Boston Academic
  Press, 1994
• Jakob Nielsen. Usability Engineering, Academic
  Press, 1993

10
Patent searches

• US patents
• http//patft.uspto.gov/
• Canadian patents
• http//patents.ic.gc.ca/cipo/cpd/en/search/basic.h
  tml
• Do a search on
• User interface

11
Examples of topics for one of your work items in
this course - 1

• A survey of usability metrics
• Discuss papers where different metrics were used
• Measuring individual differences
• There is a large body of research literature on
  this
• You could do a small experiment, or just report
  on some narrow topic from the literature
• Making software usable for the disabled
• Pick a type of disability and a class of
  application
• You can combine literature research with your own
  evaluation
• A comparison of software usability research and
  usability or ergonomics studies in other
  engineering disciplines

12
Work item topic ideas - 2

• International standards for usability
• See http//dx.doi.org/10.1006/ijhc.2001.0483
• Also see http//www.usabilitynet.org/tools/r_inter
  national.htm
• A study of how usability improvements have helped
  organizations financially.
• Research on the effects on usability of various
  design issues
• modality, choice of colours, menu design
  alternatives, etc.
• Reliability and validity of usability testing
• If you do two similar studies, will you get the
  same results?
• What are the threats to validity?

13
Work item topic ideas - 3

• Rapid (discount) approaches to usability
• Perhaps you could compare this to more expensive
  approaches
• Ethics in usability testing
• What different approaches are taken in different
  places
• What issues arise
• Effects of alternatives in experimental design
• Focus on what aspect of experiments
• Threats to validity
• Blocking
• Statistical analysis
• Compare some experiments in the literature

14
Work item topic ideas - 4

• Usability studies of a certain class of
  applications
• help systems
• web search engines
• word processors

15
No need to rush now but dont leave too late

• We will cover much material in the course
• So waiting for classes where I cover some
  material could help you
• But get working on your first project by the
  start of October
• Or else you will have too much to do at the end

16
Consider threats to validity - 1

• What could make your conclusions less valid?
• Conclusion validity
• Factors leading you to incorrectly believe some
  conclusion
• Seeing things that arent there due to bias
• Working with only one group of participants
• Participants learn from one step to the next
• Many other bias sources
• Certain statistical errors, like doing many
  T-tests
• Factors leading you to not reach a conclusion you
  should reach
• Not finding the needle in the haystack
• Not enough data
• Not enough participants
• Too much noise in the data
• Not asking the right questions
• Not using the right statistics

17
Consider threats to validity - 2

• Internal validity - are the relationships causal
• Could something other than what you think be
  causing the results you see
• Construct validity - did we measure what we
  wanted to measure?
• External validity - can we generalize the
  results?
• Are the results just true in this specific
  situation
• Good website on threats to validity
• http//www.socialresearchmethods.net/kb/introval.p
  hp

18
Writing a good research paper(for this course) -
1

• Step 1 Have something to say
• Learn the relevant background
• Search and read/skim the literature
• Follow the citations to read
• Papers that cite what you are reading
• Papers that what you are reading cites
• For studies you do yourself sketch out
• Method you followed
• Hypotheses
• Research questions answered
• Analysis of results
• Threats to validity
• Key conclusions

19
Writing a good research paper - 2

• Step 2 Develop a structure
• E.g.
• Abstract
• Introduction (write at end)
• Background
• Method
• Results and discussion
• Conclusions and future work (write at end)
• References

20
Writing a good research paper - 3

• Step 3 Fill in details
• When you get stuck, work on a different section
• Step 4 Review many times
• Does it tell a story?
• Are there details that could be left out?
• Are important elements missing?
• Is each sentence/paragraph well written?
• Have you used citations/references well?
• Ask somebody else to proofread

21
Write according to a publication format

• Here are some possibilities
• SIGCHI format
• http//www.sigchi.org/chipubform/
• ICSE format
• http//www.cs.uoregon.edu/events/icse2009/calls/fo
  rmat/
• ACM Journal formats
• http//www.acm.org/publications/submission

22
Some references on writing good papers

• http//people.csail.mit.edu/mernst/advice/write-te
  chnical-paper.html
• http//www.eg.bucknell.edu/cs475/F97-S98/handbook
  /research-paper.html
• http//www.sigplan.org/oopsla/oopsla96/how91.html
• http//www.cs.cmu.edu/jrs/sins.html

23
How I will evaluate research papers - 1

• 25 Writing quality
• Clarity, conciseness, ease of reading
• Document structure
• Introductions, body, conclusions
• If your first language is not English, you should
  consider getting somebody to proofread your
  report for you.
• 10 Quality of visual material
• Tables, lists, diagrams, figures, and appendices.
• If you are presenting data, focus on highlighting
  the most important aspects of the data.

24
How will I evaluate research papers - 2

• 25 Incorporation of background material
• Synthesis of ideas from various sources
• Journals and conferences (peer reviewed)
• A goal is to cite 10 of these
• Use online databases (discussed earlier)
• Books
• Good quality, reliable web-based material
• Should not be a majority

25
How will I evaluate research papers - 3

• Reference any relevant ideas discussed in the
  lectures.
• Do not repeat material I discussed in class
• Brief paraphrasing or quoting of my material is
  OK
• Properly formatted the bibliography at the end
• Give proper citations in the text for all the
  ideas you include that are not your own.
• Quotations must be clearly marked.
• At all costs, avoid accidental plagiarism

26
How will I evaluate research papers - 4

• 40 Depth of understanding, scientific maturity
  and/or originality
• You do not have to discover new principles,
• But a paper should be interesting for me to read,
  and make me think.
• Appropriateness of study design and execution
• If you are reporting on a study or experiment
• report what you did in detail,
• problems you encountered
• the rationale for choices
• Validity of conclusions
• You will lose marks if you did everything in a
  rush at the end.

27
Class presentation marking scheme - 1

• 25 Quality of overheads (or other A/V elements).
• Use large fonts (20 point or higher) and point
  form.
• Use diagrams, tables etc. if possible.
• Each slide should have about 7-15 lines of text,
  or a graphic.
• Avoid more than about 7 chunks in a list
• Use subheadings as on this page
• Each point should take no more than 2 lines.
• 25 Organization of material presented
• Did you provide enough background (but not to
  much)?
• Did you explain the experimental or study design
  (if appropriate)?
• Did you show interesting data?
• Did you show conclusions?

28
Class presentation marking scheme - 2

• 25 Delivery
• Pacing (1 slide every 1-5 minutes)
• Speaking clearly
• Avoiding reading what is written
• Leaving time for questions
• Handling of questions
• 25 Information content
• Amount the class would have learned from your
  presentation.
• You will not get good marks if you just repeat
  material from lectures