Classical Open Source Software Process Model

1
Classical Open Source Software Process Model

• Alan Kelon Oliveira de Moraes
• alan_at_kelon.org
• Recife, November 27, 2006

2
(No Transcript)
3

• Open-source software is not a new idea but only
  recently have technical and market forces
  converged to draw it out of a niche role.
• Eric S. Raymond
• The Cathedral the Bazaar, 1999

4
open source companies had raised 900 million
from venture capitalists since 1999
Rivilin 2005
5
Summary

• What is open source and why should you care about
  it?
• Classical open source process model
• Remarks
• References

6
Definitions and Motivations

• What and why?

7
What is Open Source?

• Open Source software is the process of
  systematically harnessing open development and
  decentralized peer review to lower costs and
  improve software quality

Raymond 1999
8
Basic idea behind open source

• When programmers can read, redistribute, and
  modify the source code for a piece of software,
  the software evolves.
• People improve it, people adapt it, people fix
  bugs.

http//www.opensource.org
Open Source Initiative
9
Open source license definition

• Open source means more than source code
  availability
• The source must be available for redistribution
  without restriction and without charge
• The license must permit the creation of
  modifications and derivative works
• Must allow those derivatives to be redistributed
  under the same terms as the original work.

OSI Perens 1999 OReilly 1999
10
How is open source different?

• The term open source generally refers to software
  that differs from its commercially sponsored
  counterpart in three ways
• source codes ready availability to prepare
  derivative works
• collaborative production approach
• community governance of production

West 2005
11
It is still evolving...

• The term open source project does not have a
  clearly defined stable and shared meaning
  comparable to open source license
• The processes evaluated refer to classical open
  source projects

OMahony West 2005
12
Why should you care?

• Its important not to reduce the open-source
  debate to a question of NT versus Linux, or
  Microsoft versus the rest of the world
• Understand how the lessons of open source can be
  applied to software development across the board.
• The open-source process reflects a powerful
  global trend toward networked collaboration.
• Printing press spread of knowledge in
  Renaissance
• Open Source large scale cooperative development
  efforts today.

O'Reilly 1999
13
Open wallets for open source software

• In 1999 and 2000, according to VentureOne,
  venture capitalists invested 714 million in 71
  open-source companies
• Turbolinux raised 95 million
• Linuxcare raised at least 80 million
• Most of those projects collapsed

Rivilin 2005
14
Open wallets for open source software

• Venture capitalists are embracing open-source
  technology companies again
• 149 millions to 20 business in 2004
• At least three open-source start-ups raised 20
  million in April 2005
• The big difference is the increased adoption of
  open-source software by corporate users today
• Red Hat had 125 million in revenue in 2004 and
  now has a market capitalization around 2 billion.

Rivilin 2005
15
New business models

• JBoss's business model, built on selling support
  services, made sense Rivilin 2005
• Open Source has effect on commoditization, system
  architecture and network effect, and the
  development practices associated with software as
  a service O'Reilly 2005

16
Classical Open Source Process Model

• A brief survey

17
Hey, wait! What is a software process?

• A particular method of doing something, generally
  involving a number of steps or operations
  Webster dictionary
• The software process consists of the activities
  and associated information that are required to
  develop a software system Sommerville 1996
• A process defines specifically who does what,
  when, and how Fayad 1997
• Software process is a set of activities, methods,
  practices, and transformations which people use
  to develop and maintain software and the
  associated products Paulk et al. 1993

18
And what is a software process model?

• While a process is a vehicle for doing a job, a
  process description is a specification of how the
  job is to be done Osterweil 1987

19
Abstract generic process models

• Specification-based models
• Waterfall
• Incremental
• Evolutionary development models
• Prototyping
• Spiral

Sommerville 1996
20
The Cathedral and the Bazaar

• Development style of Linux
• Release early and often, and listen to your
  customers
• Delegate everything you can
• Be open to the point of promiscuity
• Present a plausible promise

Raymond 1999
21
Brooks vs. Linus

• Brooks Law Adding more programmers to a late
  project makes it later.
• Linus Law Given enough eyeballs, all bugs are
  shallow

Raymond 1999
22
A case study of open source software development
the Apache server

• Apache Server Process decision making-oriented
  process
• Emphasized decentralized workspaces and
  asynchronous communication
• Email lists exclusively to communicate with each
  other, and a minimal quorum voting system for
  resolving conflicts

Mockus et al. 2000
23
A case study of open source software development
the Apache server

• There is no explicit system-level design, or
  even detailed design
• There is no project plan, schedule or list of
  deliverables
• In order to keep track of the project status, an
  agenda file is stored in each product's
  repository, containing a list of high priority
  problems, open issues, and release plans.

Mockus et al. 2000
24
A case study of open source software development
the Apache server

• There is no single development process, but these
  are developer's common actions
• Discovering that a problem exists
• Determining whether a volunteer will work on it
• Identifying a solution
• Developing and testing the code within their
  local copy of the source
• Presenting the code changes to the group for
  review
• Committing the code and documentation to the
  repository.

Mockus et al. 2000
25
A case study of open source software development
the Apache server

• Quantitative analysis based on Developer mailing
  lists, CVS commit logs and Problem reporting
  database
• From may February 1995 to May 1999

Mockus et al. 2000
26
A case study of open source software development
the Apache server

• Almost 400 individuals contributing code
• The top 15 developers contributed more than 83
  of the MRs and deltas, 88 of added lines and 91
  of deleted lines.

Mockus et al. 2000
27
A case study of open source software development
the Apache server

• Around 3060 different people submitted 3975
  problem reports.
• The top 15 problem reporters (only 3 also in the
  core) submitted only 213 or 5 of Prs
• Almost 2600 developers submitted one report, 306
  submitted two, 85 submitted three
• The maximum number submitted by one person was 32.

Mockus et al. 2000
28
A case study of open source software development
the Apache server

• Time to resolve problem reports
• 50 of PRs are resolved within a day
• 75 within 42 days
• 90 within 140 days

Mockus et al. 2000
29
Descriptive Process Model for Open Source
Software Development

• Three basic views for software process models
  Humphrey 1989
• The state view
• The organizational view
• The control view

Johnson 2001
30
Descriptive Process Model for Open Source
Software Development

• State view
• Closed prototyping
• Iterative and incremental enhancement
• Concurrent development
• Large-scale peer review
• User-driven requirements

Johnson 2001
31
Descriptive Process Model forOpen Source
Software Development

• Organizational view
• Decentralized collaboration
• Trusted leadership
• Internal motivation
• Asynchronous communication

Johnson 2001
32
Descriptive Process Model for Open Source
Software Development

• Control view
• Informal planning
• Tiered participation
• Modular design
• Ubiquitous tool support
• Shared information space

Johnson 2001
33
An overview of the Software Engineering Process
in the Mozilla project

• The importance of tools
• Bugzilla
• Tinderbox
• Bonsai
• LXR

Reis and Fortes 2002
34
An overview of the Software Engineering Process
in the Mozilla project

• Great deal to develop and to document the Mozilla
  software process
• Rely heavily on communication tools such as
  mailing lists, irc, news, bugzilla
• Bug-driven development bugzilla
• Requirements
• A message thread is started on a public newsgroup
  regarding a change in functionality
• When a person has a specific idea for a change a
  bug will often be directly filed with no
  newsgroup discussion
• You can get a feature in if youre willing to
  write code for it -- Boris Zbarsky, Mozilla
  developer
• Code review and inspection bonsai
• Nightly build tinderbox

Reis and Fortes 2002
35
An overview of the Software Engineering Process
in the Mozilla project

• Effective version control
• A well-defined protocol for integrating source
  code changes
• A high degree of accountability for people who
  integrate code
• High modularity
• Good communication channels
• Documentation not always up to date

Reis and Fortes 2002
36
Understanding the Requirements for Developing
Open Source Software Systems

• Traditional SE
• Elicitation
• Analysis
• Specification and modeling
• Validation
• Communicating and managing

• Open Source
• Post-hoc assertion
• Reading, sense-making, accountability
• Continually emerging webs of discourse
• Condensing and hardening discourse
• Global access to discourse

Scacchi 2002
37
Free Software development lifecycle
Senyard and Michlmayr 2004
38
Bazaar style development lifecycle
Senyard and Michlmayr 2004
39
Quality assurance under the open source
development model

• User participation in open source projects is
  high
• up to 20 of the changes for almost 50 of the
  projects
• discovering 2040 of the faults in 20 of the
  projects.
• Change management processes and tools are at the
  cutting edge of large-scale collaborative
  software development
• Documentation is not a high priority for most
  projects TODO and install guide
• Little investment in utilizing testing techniques
  and tools, relying mainly in community testing

Zhao and Elbaum 2003
40
Open source software maintenance process framework
Koponen and Hotti 2005
41
Open source software maintenance process framework
Koponen and Hotti 2005
42
Concluding remarks

• Past, present and future

43
Past

• Hacker community activity
• Self-contained moviment

44
Present

• Open source gets momentum
• Developing F/OSS is different than software
  engineering Scacchi
• not better, not worse, but different and new
• more social, more accessible
• F/OSS systems dont need and probably wont
  benefit from classic software engineering
  Scacchi

45
Abstract generic process models

• Specification-based models
• Waterfall
• Incremental
• Evolutionary development models
• Prototyping
• Spiral
• Open Source

Sommerville 1996
46
So an open source process model is

• Evolutionary/maintenance-oriented development
  model
• Not just a technical development process
• Collaborative effort
• Based on community government
• User-driven requirements
• Modular architecture/design without UML stuff
• Heavily dependent on communication tools because
  it is distributed, asynchronous and discontinuous
• Indeed it is not a traditional software
  engineering process

47
Future

• Open source is still in its infancy today
• OSS projects need to be more professional
• Open Source Software Process
• Do not to move into Process Paralysis Yourdon
  1987

48
http//www.fabricadesol.com/in953
49
Neus et al. 2005
50
Neus et al. 2005
51
Something else?
52
References

• OMahony, S. and West, J. 2005. What makes a
  project open source? Migrating from organic to
  synthetic communities. Academy of Management
  conference, Technology and Innovation Management
  division, Honolulu, August 2005.
• O'Reilly, T. 1999. Lessons from open-source
  software development. Commun. ACM 42, 4 (Apr.
  1999), 32-37
• O'Reilly, T. 2005. The Open Source Paradigm
  Shift. In Feller, J., Fitzgerald, B., Hissam, S.
  A. and Lakhani. K. R. eds. Perspectives on Free
  and Open Source Software. The MIT Press, 461-482.
  Also available in http//tim.oreilly.com/articles/
  paradigmshift_0504.html since June 2004
• Open Source Initiative. http//www.opensource.org
• Perens, B. .1999. The Open Source Definition. In
  DiBona, C., Ockman, S., and Stone, M. eds. Open
  Sources Voices from the Open Source Revolution.
  O'Reilly and Associates, 171-188.
• Raymond, E. S. The Cathedral the Bazaar
  Musings on Linux and Open Source by an Accidental
  Revolutionary. O'Reilly and Associates, 1999.
  Also available in http//www.catb.org/esr/writing
  s/cathedral-bazaar/cathedral-bazaar/ since May
  1997
• Rivilin, G. 2005. Open Wallets for Open-Source
  Software. The New York Times, April 27, 2005.
• West, J., Jury, A., Sengupta, S., and Penberthy,
  S. 2005. Bookshelf. IEEE Softw. 22, 4 (Jul.
  2005), 114-118.

53
References

• Johnson, K. A. Descriptive Process Model for
  Open-Source Software Development, Master Thesis,
  Univ. Calgary, Alberta, 2001.
• Koponen, T. and Hotti, V. 2005. Open source
  software maintenance process framework, ACM
  SIGSOFT Software Engineering Notes, v.30 n.4,
  July 2005
• Mockus, A., Fielding, R. T., and Herbsleb, J.
  2000. A case study of open source software
  development the Apache server. In Proceedings of
  the 22nd international Conference on Software
  Engineering ( Limerick, Ireland, June 04 - 11,
  2000). ICSE '00. ACM Press, New York, NY,
  263-272.
• Reis, C. and Fortes, R. P. M. 2002. An overview
  of the Software Engineering Process in the
  Mozilla project. In Proceedings of Workshop on
  Open Source Software Development, Newcastle,
  155-175
• Scacchi, W. 2002. Understanding the Requirements
  for Developing Open Source Software Systems. In
  IEE Proceedings Software, volume 148, number 1,
  pp. 24-39.
• Senyard, A. and Michlmayr, M. 2004. How to Have a
  Successful Free Software Project. In Proceedings
  of the 11th Asia-Pacific Software Engineering
  Conference (Apsec'04) - Volume 00 (November 30 -
  December 03, 2004). APSEC. IEEE Computer Society,
  Washington, DC, 84-91
• Zhao, L. and Elbaum, S. 2003. Quality assurance
  under the open source development model. J. Syst.
  Softw. 66, 1 (Apr. 2003), 65-75.

54
References

• Neus, A., Scherf, P. and Subjects, R. 2005.
  Opening minds Cultural change with the
  introduction of open-source collaboration
  methods. IBM Journal of Research and Development,
  Volume 44, Number 2, Page 215.
• Sommerville, I. 1996. Software process models.
  ACM Comput. Surv. 28, 1 (Mar. 1996), 269-271.
• Humphrey, W. S. 1989 Managing the Software
  Process. Addison-Wesley Longman Publishing Co.,
  Inc.
• Yourdon E. A Game Plan for Technology Transfer,
  Tutorial Software Engineering Project
  Management. R.H. Thayer, ed. Computer Society
  Press, 1987
• Paulk, M.C. et al., Key Practices of the
  Capability Maturity Model, Version 1.1,
  technical report CMU/SEI-93-TR-025, Software
  Engineering Institute, Carnegie Mellon
  University, 1993.
• Fayad, M. E. 1997. Software development process
  a necessary evil. Commun. ACM 40, 9 (Sep. 1997),
  101-103.
• Osterweil, L. 1987. Software processes are
  software too. In Proceedings of the 9th
  international Conference on Software Engineering
  (Monterey, California, United States).
  International Conference on Software Engineering.
  IEEE Computer Society Press, Los Alamitos, CA,
  2-13.