Silver Bullet or Fools Gold: Supporting Usability in Open Source Software Development

1
Silver Bullet or Fools GoldSupporting
Usability in Open Source Software Development

• Michael B. Twidale
• Graduate School of Library and Information
  ScienceUniversity of Illinois at
  Urbana-Champaign
• twidale_at_uiuc.edu
• www.uiuc.edu/twidale

2
Aims

• Introduce or emphasize certain themes
• Not trying to share a theory you can crank
  through to get stuff out of an SE problem
• Am trying to raise new ways to look at an SE
  problem
• Lightweight, Superficial overview
• Some provocative ideas to laugh at, argue over in
  the breaks

3
Background

• Computer Science at Lancaster University
• Computer Supported Cooperative Work
• Interface Design
• Computer Supported Collaborative Learning
• Interacting with Ethnographers
• Evaluation
• Understanding how people learn to use
  applications
• Designing new ways to visualize, share, discuss
  learn
• Often visualizing hidden knowledge
• Often designing new kinds of process visualization

4
What a Computer Scientist Learned at Library
School

• Other ways of looking at the problem
• Multidisciplinarity is good and normal,
• but still hard to do
• Service, Access, Empowerment
• Politics, Class, Gender, Image, Status
• New metaphors
• Ways of looking at oneself as a kind of computer
  scientist
• And ones research, discipline and approaches
• From CS airy-fairy wiffly-waffly
  interface-qualitative-evaluation-learning-applicat
  ions flake
• to LIS tech boy

5
Themes of the Talk

• Open Source Software Engineering (OSSE)
• Open Source Usability (OSSUI)
• Problems of both scarcity abundance
• Slice and Dice, Mix and Match
• Agility and Extremes
• Widening active participation
• Socio-technical robustness

6
Open Source Software Engineering

• Top Down, planned (seemingly)
• Requirements
• Specification
• Verification and Validation
• Testing
• Project management
• Configuration

7
The OSSE Mystery

• Bottom Up, anarchic (seemingly)
• Reqs An itch
• Spec Informalisms Scacchi
• Ver Testing Eyeballs
• Val Reputation, downloads, /. posts
• Mgmt Tom Sawyer meets Linus Torvalds,
• Be a nice guy, benevolent dictator, trusted
  lieutenants
• If you had studied SE and had never heard of OSS
  youd doubt it could it possibly work

8
Is OSS Brooks Silver Bullet?

• Is OSS a free lunch as well?
• Wheres the catch?
• How does it work at all?
• Does it scale over project type?
• Has it maxed out?
• What can we borrow?
• What can we improve?
• (too hard to answer all these here)

9
IS OSS SEs Platypus, Cyanobacteria?

• Weird exception, breaks the rules
• Mystery of how it possibly can survive
• Alien enough to be helpful for comparison
• Contrast with the proper way of doing things?
• Definition by difference
• Niche, boutique approach, exploding from 0.01 to
  10 of attention, then stopping?

10
Is OSS SEs Fruit Fly?

• Really easy to get lots of data (breeds fast,
  prolifically and can cope with lab study
  conditions)
• Really easy to look inside (accessible
  chromosomes)
• Really not too different from everything else
  when you look more closely

11
Sub-problem OSS Usability

• Is there a problem?
• Why might there be a problem?
• How is it being addressed?
• What can we do to help?
• Technically
• Managerially
• Socially

12
An OSS Usability Problem?

• Is OSS usability product or process different
  from Closed Source?
• Is OSS just a bit slower to catch on?
• Just like CSS used to be
• Maybe leave usability and concern for end users
  to the commercial world?
• Ignore product lines, or add wrappers, services
• That would keep M happy

13
(No Transcript)
14
Open Source Software Usability

• Not many OSS Usability Studies
• GNOME Smith et al., 2001, Athena Athena, 2001
  Greenstone Nichols et al., 2001
• Usability compared to CSS?
• Hard to measure
• Microsoft Excel v StarOffice Eklund et al.
  2002
• Firefox??
• Forked from Mozilla over usability
• Trades usability for functionality
• Big Battles
• Usability-centric design still rare in OSS

15
OSS Functionality v Usability?

• Scratching a functionality itch
• Global development infrastructure
• Very successful for infrastructure
• e.g. Apache, Linux
• Very successful for use by and for power users
• Works for
• Functionality
• Reliability
• Usability??!
• What about end users?

16
Why might there be a problem?

• If Grandma cant figure out which button to click
  on, she can always download the code, reprogram
  the interface to make it simpler to understand,
  upload the change to SourceForge and bingo!
• "They just don't like to do the boring stuff for
  the stupid people!" (Sterling, 2002)
• Developers are not typical end-users

17
Why might there be a problem?

• Usability experts do not get involved in OSS
• The incentives in OSS work better for improvement
  of functionality than usability
• OSS development is inclined to promote power over
  simplicity
• Open source projects lack the resources to
  undertake high quality usability work
• Commercial software establishes state of the art
  so that OSS can only play catch-up
• OSS has an even greater tendency towards software
  bloat than commercial software

18
Are OSS methods incompatible with usability?

• Design for usability really ought to take place
  in advance of any coding
• Usability problems are harder to specify and
  distribute than functionality problems
• Brooks law -gt modularity
• UI design -gt Within-application consistency

19
(No Transcript)
20
OSS Usability a problem?

• Lusers says it all /.
• The wrong kind of itch?
• With enough eyeballs all bugs are shallow
• The wrong kind of eyeballs for usability?
• Can we apply Usability to existing OSS?
• Can we apply OSS methods to other usability
  problems?

21
Some qualitative analysis

• Almost a virtual ethnography
• What gets talked about wrt interface design,
  usability, the user experience?
• How are usability bugs articulated and used?
• How are fixes and design discussions handled?
• Clarifying what is going on

22
Sources

• Bugzilla instances at
• Mozilla
• GNOME
• Mailing Lists
• Greesntone Digital Library software at
  greenstone.org
• Group Blogs
• planet.mozilla.org
• planet.gnome.org
• Newgroups

23
(No Transcript)
24
(No Transcript)
25
Bug Reporting
26
Feedback Samples

• From rajesh r
• Date 23 Jul 2003 095244 -0000
• Subject greenstone-users HTML page not
  processing
• Greenstone developers,
• I am facing a problem while putting my html pages
  to the library.
• The collection is creating properly and even the
  search is showing the results, but when i click
• the search result link it doesnt show the actual
  content of the html page.
• I also tried the pages after placing the files in
  the cgi-bin folder of apache. Still this is not
  working.
• Any help or advice is greatly appreciated.
• Thanks for your time and consideration.
• Raj.

27
Feedback Samples

• From Michael Dewsnip
• Date Tue, 29 Jul 2003 152830 1200
• Subject Re greenstone-users HTML page not
  processing
• Hi Rajesh,
• This is strange. A few questions
• Do you get an error message when trying to view
  the documents, or just a blank page?
• Are you using Windows or Unix? If Windows, are
  you using the local library, or do you use a
  separate web server?
• Have you made any changes to the format
  statements in the collection configuration file?
• What version of Greenstone are you using?
• If possible, the best thing would be if you could
  send me a link to your collection (so I can see
  the problem myself), or, if the collection is
  small enough, package the collection up and send
  it to me.
• This would make it much easier for me to track
  down the problem.
• Sorry I couldn't be of more help.
• Regards,
• Michael

28
Simplified Results I

• Problems of description
• Tell me exactly how and why you are confused
• Usability Bugs Breed
• Duplication due to description difficulty
• Level of analysis
• Metabugs
• App-wide Interface Consistency
• Talking about interfaces -
• Like dancing about architecture?
• Talking more like arguing!
• Hugely contested debates
• Deciding what if anything to do about it

29
Simplified Results II

• Marking invalid, since this is as designed and
  thus not a bug.
• This is not a bug. There's an extension called
  Tabbed Browsing Extensions that does exactly what
  you want.
• When is a bug not a bug?
• When its a request for enhancement
• INVALID
• WONTFIX
• Subjective v Objective bugs
• WORKSFORME

30
Bug Reports do a lot more than report bugs

• Bugzillas organise much of the OSS activity
• Complex networks of bugs
• Bug representation is used for
• Bugs
• Feature requests
• Project management
• Design
• Argumentation
• Etc.

31
ASCII Art
32
Why?

• Conventionally, early design should be informal
• Often paper-based
• ASCII seems very labour-intensive...
• But
• Good for multiple uses
• Newsgroups, chats, bugzilla,
• Can insert into textual discussions
• Easy to edit, Requires nothing more than text
  editor
• Habit

33
Where work happens...

• Not just in Bugzilla
• Newsgroups
• News sites, MozillaZine
• Comments
• Face-to-face
• Phone?
• Text chat
• IRC
• Blogs

34
Design by Blog

• Increasingly Bugzilla-like systems are being
  supplemented by Design-by-Blog
• planet.gnome.org
• Problems or UI designs are posted
• Accompanying informal design rationale
• Comments are made in other blogs
• Iterate
• Evolution blog contains several examples...

35
'Design by Blog' in Evolution
New Alarm Dialog Worked on creating a mock-up
for a re-design of the alarm notification dialog
in Evolution today, and after a few revisions,
Ben and I agreed on the image below. I'm working
on the code to get this implemented and into CVS,
so it should be popping up on your desktop soon.
103 comments 16 real comments
36
I emailed this to Rodney, but maybe other people
would be interested too... A couple of us out
here at the HCI design program at IU mocked up an
alternative A few rationales for our design
The button labels are more action-oriented,
instead of the generic "close". Snooze is only
one click now, instead of two. The "edit
appointment" button is connected visually to
the appointment information and the reminder
buttons are grouped with the snooze
information. Got rid of the word "Appointment"
at the top of the window, and the icon, as they
just clutter the window. Instead of providing a
long date string, we provide the pieces of
information people can act on the time until
they need to be at the meeting, and the time they
will be back at their desk. The most important
piece of information (how much time you have
until the appointment starts) is the focal point,
and it counts down, which conveys a sense of
immanency. Putting location, start time, and
end time together into a list isn't meaningful,
so we separate location information from
time information.
37
Evolution 2
New Alarm Dialog Again After my previous blog
entry about the Alarm dialog mock-up, a lot of
feedback was had, and I went off to implement a
version. What I currently have implemented is in
the screenshot below. If anyone knows a standard
localizable way to put the time/date information
into a single line, please, e-mail me. This
version has the times as currently displayed by
Evolution, as well as the appointment summary in
bold, instead of just an "Appointment" label, and
single-click snoozing.
87 comments 7 real comments
38
Evolution 3
New Alarm Dialog in CVS Finally, the new alarm
dialog is in CVS. A screenshot of what was
committed is below. The icon was moved to the
left side, and the secondary text/widgets were
re-aligned to match the HIG 2.0 Alert specs more
precisely. This cleans up the code quite a bit
and gets rid of the HTML widget that we were
using before.
27 comments 6 real comments
39
Mozilla Example
Prefwindow V For the past week I've been working
on a new Preferences window for Firefox - one
which will solve a number of the problems present
in the current one, such as the fact that it's a
modal sheet on OS X and GNOME. The current
Preferences window (aka "Prefwindow IV") dates
back to April of 2000, and so is among the oldest
XUL/JavaScript code in Firefox. I wrote it before
the advent of XBL, and so the new version
utilizes a XBL tagset to simplify the content of
the preferences panels. I'm developing this on
Windows, since that's where the good debuggers
and editors are, the modality/instant-apply-ness
of the window will be controlled by a hidden
pref. Much is left for me to do, but here's a
sneak peek of what's coming
Ben Goodgers blog 22 Dec 2004
40
Wikis Too
41
Implications

• Design happens in many places
• Not just in Bugzilla
• Easier to participate via Blog-comments
• But comment spam is a hindrance
• For researchers
• Harder to track design work lots of places
• Work is more fragile and harder to archive for
  future analysis
• BUT you do get to see lots of design process

42
What to do?

• Widening access
• More kinds of people
• More kinds of itches
• Different kinds of eyeballs
• Usability experts
• Interface designers
• Cognitive psychologies
• Just plain folks (JPFs)
• Designing tools to help

43
Supporting Analysis

• Easier reporting
• Support describing usability problems
• Also by end users
• The one click Huh?? datum, when aggregated, is
  valuable
• Handling a flood of error, bug, huh?
  reports
• Including buggy bug reports
• Offloading usability discussions from an
  overloaded bug reporting system
• Distributed user testing?

44
Development Environment Requirements

• All the existing OSS infrastructure Plus
• Support for
• graphical discussions
• interactivity discussions
• temporality discussions
• submitting rapid prototypes sketches
• comparing multiple candidate solutions
• low cost high speed evaluations
• distributed evaluations

45
What Next?

• Exploit the openness of OSS
• Empower all users to contribute to development
• Stop viewing users paternalistically
• Enable greater customer-developer interaction
• Blur end-user/developer
• Learn from, extend, Participatory Design
• Global usability infrastructure
• Not just fixing OSSs usability problems
• Work towards new, better ways of doing HCI, SE

46
New Extremes Xx

• Agile Methods
• XP Extreme Programming
• XE Extreme Evaluation
• XU Extreme Usability
• XPD Extreme Participatory Design
• Design Blogs as XXP
• Really fast, really tiny evaluations
• Micro user testing
• Faster easier ways to share, critique and improve
  design ideas, especially interface design ideas
• Open up the clubhouse

47
How to involve more people?

• Skill and interest cultures, communities of
  practice, may be trickier than national cultures
  to cross
• More people -gt more complexity
• More diversity -gt more complexity
• Pyramidal structure of OSS involvement
• Ideal Massively increase the base
• Will need complexity management tools

48
Widening participation

• OSS enables anyone with an internet connection to
  participate in a software development project
• Anyone with the right kind of skills
• Any hacker
• Transcends national cultures
• But strongly inside the hacker culture
• US, Finnish, Chinese, German, Indian hackers all
  working together
• Is that easier to manage than enabling a US
  hacker, US interface designer, US cognitive
  psychologist, US end user to work together?

49
Example of OSS-like wider participation

• Wikipedia
• Relatively easy to use
• Collaborative text creation rather than
  collaborative code creation

50
Internationalization Example

• Co-Design by non-techies
• Wikipedia inspired

51
(No Transcript)
52
(No Transcript)
53
(No Transcript)
54
(No Transcript)
55
End User Activities

• Workplace studies
• Studying boring stuff
• Uncovering the blindingly obvious
• Over The Shoulder Learning
• How did you do that?
• How do I?
• Brief, Informal Workplace Learning
• Also Collaborative Problem-Solving
• Also Innovation and Appropriation
• Design implications
• Affording, facilitating the above

56
Pragmatics of Workplace Help

• Coping, Satisficing and Workarounds
• Will that do?
• Not usually about finding the right or best
  way
• Dealing with hugely complex multi-application
  option-spaces
• Organizations tend to have many layers of
  technical infrastructure
• Users navigate within them, but they can get
  lost
• Sometimes looks like search or navigation
• Sometimes looks like debugging
• But without any decent diagnostic tools

57
The amazing power of Control-C---Control-
V

• Copy-Paste between multiple applications
• How regular people cobble computational elements
  together to do real work
• Bricolage
• Creating workflows
• (composing multiple apps)
• Dealing with exceptions
• Coping with change
• Using basic COTS software
• Design implications

58
Computing applications as office supplies

• Select, compose
• Mix and match
• How to design
• To afford appropriation?
• To enable end users to compose elements to do
  clever things
• To enable Just Plain Folks (JPFs) to do things
  that would have been done by computer scientists
• without having to become proper computer
  scientists

59
Precedents for widening access

• Early 1900s were about to run out of chaffeurs
• Solution let people be their own chauffeur
• Needs Easier to drive, more reliable automobiles
• 1940s were about to run out of telephonists
• Solution let people be their own telephonists
• Needs Direct dial, long distance, IDD
• 2000s about to run out of software engineers?
• So..

60
Designing for JPF Bricolage

• Facilitating creative appropriations and
  compositions
• Is it even possible?
• How?
• Dont assume slinging in a macro will cut it
• Beyond skinning,
• Not quite end user programming
• Every single human being is creative
• Florida

61
Conclusions

• Widening access is good
• Improving OSSUI is good
• Widening participation in the design process is
  good
• How can we design for it?
• How can we do it without forcing everyone to be a
  computer scientist?
• Can we design to afford JPFs bricolage?
• Process visibility is useful for SE analysis
• and maybe doing better SE?

62
To A Louse On seeing one on a Lady's bonnet at
ChurchRobert Burns 1785

• O wad some Pow'r the giftie gie usTo see oursels
  as others see usIt wad frae monie a blunder free
  usAn' foolish notionWhat airs in dress an' gait
  wad lea'e usAn' ev'n Devotion
• This translates as
• Oh, that God would give us the very smallest of
  giftsTo be able to see ourselves as others see
  usIt would save us from many mistakesand
  foolish thoughtsWe would change the way we look
  and gestureand to how and what we apply our time
  and attention

63
Religion, Mysticism Computing
64
Computers and Religion
65
Extra Slides
66
References

• Nichols, D.M. Twidale, M.B. (2005). Usability
  Processes in Open Source Projects. Submitted to
  Software Process - Improvement and Practice
  Journal Special Issue on Free/Open Source
  Software Processes.
• Twidale, M.B. Nichols, D.M. (2005). Exploring
  Usability Discussions in Open Source Development.
  Proceedings, Thirty-Eighth Annual Hawaii
  International Conference on System Sciences,
  HICSS-38, Track 7, p.198c.
• Nichols, D.M., McKay, D. Twidale, M.B. (2003).
  Participatory Usability Supporting Proactive
  Users. Proceedings, CHINZ'03, Dunedin, New
  Zealand.63-68.
• Nichols, D.M. Twidale, M.B. (2003). The
  Usability of Open Source Software. First Monday
  8(1).

67
Engineering as a metaphor

• Your program is like a bridge
• How can we build it so it wont fall down?
• How can we build it better, faster, cheaper?
• Learn from failures Petroski
• Quantify, Measure, evaluate, improve, experiment,
  compare Basili

68
Science v Engineering

• S Finding out universal truths
• Peer review
• Artisan Engineer in the machine shop at the
  service of the Gentleman Scientist in the lab?
• E Building Stuff, Fixing Stuff
• Constructing devices to measure whats out there
• Constructing devices to do something useful
  pragmatics of the possible

69
Who, what and how

• Should who you build for affect what you build?
• Should it affect how you build it?
• Who gets to decide?
• Power and politics
• Widening who gets to participate

70
Problems of scarcity and abundance

• Scarcity
• Traditional Computer Science
• Squeezing out the last cycle
• Optimization
• Spend peoples time to save computers time
• Abundance
• HW Cheap and getting cheaper
• Ubicomp
• Memory, drives, internet, bandwidth
• Choice overload
• Complexity of multi app multi device interactions
• Widening uses participation

71
Regular people have problems with many current
applications

• True of CSS as well as OSS
• What is it for?
• What does it do?
• How on earth do I get it to do that?
• Oh I give up!
• I have 7 remotes on my coffee table and Grandma
  cant switch on Jeopardy
• Even techies have some problems, but darent
  admit it

72
(No Transcript)
73
(No Transcript)
74
Reactions

• .. the volume of community reaction that
  thundered into my mailbox far surpassed what I
  had been expecting and the dominant theme, too,
  was a bit of a surprise No, the really
  interesting part was how many of the letters
  said, in effect, Gee. And all this time I
  thought it was just me Eric S. Raymond
• The problem isnt just that dear old A.T. cant
  use desktop Linux the problem is that even
  Linux geeks have trouble figuring it out.
  John Gruber on ESR

75
Even Techies suffer from poor usability

• But they can cope
• Less likely to give up
• Still may have problems with MS Word
• Debian, Ian Murdock
• Make Linux installation easier

76
Stick Shifts and Automatics

• Convenience v Control
• Hiding Complexity
• Increasing learnability, usability
• Losing a measure of control and power

77
Stick Shifts and Automatics

• Unix Stick Shift?
• Windows Automatic?
• MS Word v emacs , LaTex?
• Mindset of the über-geek
• Mindset of the auto-hacker

78
CS Design Aesthetic?

• Implicitly or explicitly taught
• Functionality is the only thing that matters
• Getting it to work at all is a big enough
  challenge
• Dont build a thing, build a thing builder
• Wherever possible, automate
• Choice is good, more choice is better
• Power is good, more power is better
• Raw power is best

79
CS Design Aesthetic II

• Build it because we can
• Adding a programming language makes all things
  better
• Its too hard for me to design for end users, so
  Ill build a reconfigurable system and the users
  can build their own apps
• Efficiency trumps learning effort
• Dont complain no-one likes a whiner
• If its too hard for you, maybe you dont belong

80
New Ways of Thinking

• How will a non-techie use this?
• What is the problem we are trying to solve?
• Can I make this design simpler?
• Try to simplify, not complexify
• An app that also contains its own programming
  language is not necessarily better
• Emacs is not the worlds best editor for everyone

81
The Status of Usability HCI in CS world
Hackerdom

• No money in it
• First to go in the budget/deadline crunch
• Usability is for someone else
• Cognitive Psychologists
• Graphic designers
• Usability is for Girlies - Real Men Code!
• Perceptions matter
• Even when they are incorrect

82
Potential Approaches to Improving OSS Usability

• Commercial approaches
• Sell a pretty wrapper round the raw power of OSS
• Apple OS, Debian, Red Hat, IBMs Strategy
• Technological approaches
• Automating usability analysis
• Academic involvement
• Us! authentic assignments, projects, Senior
  Theses, etc.

83
Potential Approaches to Improving OSS Usability 2

• Fragmenting usability analysis and design
• 1-shot user studies?
• Involving the experts
• Incentives to participate?
• Education and evangelism
• Selling usability within OSS community
• Involving the end users
• Bugzilla
• Creating a usability discussion infrastructure

84
Widening access

• Empowerment
• Like participatory design but even more radical
• Political implications
• Still need reliable software
• Clarity of what it does, opacity of how it does
  it
• Like cars put gas in, go places