On software maintenance and evolution

1
On software maintenance and evolution

• Michael W. Godfrey
• Software Architecture Group (SWAG)
• University of Waterloo

2
Overview

• Software aging
• Software maintenance
• Software evolution
• A case study of growth and evolution

3
Software Aging
4
Parnas on Software Aging

• Bit rot isnt the problem!
• Rather
• Our expectations change, new demands, new
  environments, emergent properties,
• e.g., MS-Word generates HTML, OLE, vmware,
  network drives, XML as a RDBMS,

5
Parnas on Software Aging

• BUT
• Large software systems are complex
• Difficult and expensive to change!
• Legacy systems are expensive assets!
• Ignorant surgery leads to brittle,
  hard-to-change systems
• Business pressures lead to JIT maintenance
• Accidental complexity accrues over time

6
Parnas on Software Aging

• Parnas advice
• Design for change!
• Isolate likely-to-change hotspots and virtualize
• Keep documentation up-to-date
• Perform design reviews
• Perform redesigns
• But we know this already, right?

7
Software maintenance
8
A thought experiment

• Ignore software for a moment
• What does the term maintenance connote?
• Keep it running, replace worn out bits, upgrades
  sometimes,
• Is this what software maintainers do?
• Mostly not!

9
Kinds of software maintenance

• What are the common categories of software
  maintenance?
• Corrective
• Adaptive
• Perfective
• Preventative
• Alas, there is much confusion about what these
  terms (ought to) mean!

10
Swansons original view ICSE 1976

• Corrective Fix program so that it meets its
  specs
• Processing failure
• wrong answer or program aborts
• Performance failure
• doesnt satisfy non-functional reqs, esp. real
  time performance
• Impl. failure
• programming standards ignored, impl.
  inconsistent/incomplete, reqs not implemented
  correctly
• Adaptive Fix to adjust to new underlying
  env./formats
• Change in data format
• Change in processing environment

11
Swansons original view ICSE 1976

• Perfective It works fine, but lets make it
  even better
• Processing inefficiency
• Even in performance is within specified params,
  may still want to make it faster
• Performance enhancement
• Make the output more readable
• (NOT related to response time)
• Maintainability
• Better use of comments, documentation
• (NOT refactoring/restructuring/redesigning)

12
Swansons original view ICSE 1976

• Corrective Fix program so that it meets its
  specs
• Adaptive Fix to adjust to new underlying
  env./formats
• Perfective It works fine, but lets make it
  even better
• BUT
• Where does preventative maintenance fit in?
• Where does make it faster fit in?

13
Swanson/Leintz CACM 1978

• Refined model, most commonly quoted
• Corrective
• fixing bugs
• Adaptive
• adapting to new environment
• but no outward semantic change, no new features
• Perfective
• any other kind of change intended to make the
  system better
• e.g, performance tuning, adding new features,
  preventative maintenance (redesign/refactoring)

14
IEEE Std on Software Engineering Terminology
1990

• Software maintenance
• The process of modifying a software system or
  component after delivery to correct faults,
  improve performance, or adapt to a changed
  environment
• Corrective maintenance
• Maintenance performed to correct faults in
  hardware or software.
• Adaptive maintenance
• Software maintenance performed to make a
  computer program usable in a changed
  environment.
• Perfective maintenance
• Software maintenance performed to improve the
  performance, maintainability, or other attributes
  of a computer program.
• Preventative maintenance
• Maintenance performed for the purpose of
  preventing problems before they occur.
• Q Adding new features?

15
IEEE Std on Software Maintenance Terminology
1998

• Software maintenance
• Modification of a software product after
  delivery to correct faults, improve performance
  or other attributes, or adapt to a modified
  environment
• Corrective maintenance
• Reactive modification of a software product
  performed after delivery to correct discovered
  faults.
• Adaptive maintenance
• Modification of a software product performed
  after delivery to keep a computer program usable
  in a changed or changing environment.
• Perfective maintenance
• Modification of a software product after
  delivery to improve performance or
  maintainability.
• Preventative maintenance is MIA
• Q Adding new features?
• Part of the environment?
• Not a maintenance activity per se?

16
The current view

• Theres no standard taxonomy but heres a
  reasonable view
• Corrective maintenance
• Fix bugs
• Adaptive maintenance
• Add support for new deployment environments
• May include adding new features
• Perfective maintenance
• Improve non-functional requirements (performance,
  security) apart from portability
• May include adding new features
• May include preventative maintenance
• Preventative maintenance
• Improve internal design w/o changing external
  semantics

17
Software maintenance vs. evolution
18
Software maintenance vs. evolution

• So far, weve treated initial delivery as the
  major milestone in a products lifespan.
• What about changes during initial development?
• Is it reasonable to make such a sharp distinction
  between pre- and post-initial delivery?
• Are there other important phases in a software
  systems?
• What are the forces at play as a system evolves?
• Is evolution a better term than maintenance?

19
Waterfall-like process model
Requirements engineering
Design and implementation
Testing
20
Staged model of maintenance and evolution
Bennet/Rajlich 2000

• The staged model recognizes that there are
  fundamentally different life phases of a software
  system!
• At each stage you are typically performing
  mini-waterfall development cycles differently!
• The problems, tensions, solutions, processes are
  very different!
• Need to understand what is appropriate when.
• Lots of research needs to be done here!
• What is an evolvable architecture?
• What are the likely effects of different kinds of
  change?
• etc

21
Staged model of maintenance and evolution
Bennet/Rajlich 2000

• Initial development
• First version developed, already possess the
  architecture and the fundamental system
  knowledge.
• Active evolution
• Takes place only when the initial development was
  successful, the goal is to adapt the application
  to the ever-changing user requirements.
• Servicing
• Once the architecture or the knowledge
  disappears, product is no longer evolvable (i.e.,
  enters the service stage)
• Phase out
• no more servicing.

22
Staged model of maintenance and evolution
Bennet/Rajlich 2000

• Initial development
• System architecture is devised
• Often create a runnable infrastructure as soon as
  possible, gradually fill in missing/fudged bits
  aka staged delivery
• Design for change!
• Consider strategically where you expect changes
  to occur and design your architecture accordingly
• Ideal Cost of making a future change is
  proportional to size/complexity of change, not
  size/complexity of system
• Change is the rule, not the exception.
• Developers are aware of this.
• QA artifacts developed
• Knowledge about the system is fresh, accurate
  experts can be found easily

23
Staged model of maintenance and evolution
Bennet/Rajlich 2000

• Active evolution
• Knowledge about system is good some personnel
  turnover since initial development.
• Simple changes and adaptation to new
  requirements/environments are mostly
  straightforward
• System architecture is flexible enough to allow
  for more wide-ranging changes to be planned and
  executed successfully
• Architecture can be redesigned if necessary, and
  still be coherent and reasonably faithful to the
  original system

24
Staged model of maintenance and evolution
Bennet/Rajlich 2000

• Servicing
• Simple changes and adaptation to new
  requirements/environments can still be done
• Architectural change is harder and risky!
• Knowledge about system is disappearing
• JIT maintenance starts to occur
• Effects of change (e.g., on other parts of
  system) become harder to predict
• Start to treat system more like a caged animal
  than an intellectual creation
• May perform preventative maintenance (program
  comprehension, reverse engineering) tasks to
  recover design, stave off death
• In practice, its very hard to swim upstream back
  to evolution stage
• Legacy wrapping, adapters, regression testing,
  impact analysis,

25
Staged model of maintenance and evolution
Bennet/Rajlich 2000

• Phase out
• System is getting too risky and expensive (,
  performance, time/effort) to keep.
• A new solution is sought
• An exit strategy must be devised
• Common approaches
• Create new adapter-style interface for
  clients/rest of system
• Perform data recovery and migration
• Deploy new solution to fit adapter interface,
  integrate with recovered data
• Shut down old system

26
Maintenance vs. evolution
27
Software maintenance vs. evolution

• Maintenance connotes
• Fixing, rather than intellectually enhancing
• Short-term, not long-term goals
• Many prefer the term evolution
• Fundamental change and adaptation
• Short- and long-term change
• Planned and unplanned phenomena my two
  cents

28
What is evolution?

• My answer
• Essential change in some identifiable thing (or
  family) over time.
• Warning Some very personal views follow.
• They are far from widely accepted.

29
Evolution Software vs. biology

• Biology has the hard scientific theory and
  empirical results, but evolution is a generally
  occurring phenomenon!
• Frogs, economies, ideas, music all evolve and
  software does too!
• Although the phenomenon of evolution is apparent
  widely, it is hard to be truly scientific in its
  study outside of biology.

30
Three fundamentals ideas of evolution

• Mechanisms that increase variation / act as an
  agent for essential change.
• Mechanisms that decrease variation / act as an
  inhibitor for essential change.
• Its all relative, man.
• Change occurs within an environment, which also
  evolves.
• That is, what succeeds today may not succeed
  tomorrow (but may succeed the day after).

31
Biological evolution in a nutshell

• Biological evolution is change in a gene pool
  of a population over time. talk.origins FAQ
• Genes/alleles are the essential encoding that is
  inherited by living creatures they comprise the
  genotype of the individual.
• A gene is the dedicated slot or memory location
• Alleles are the set of abstract values at those
  genes
• DNA are the bits that encode the allele messages
  (ACGT)
• The phenotype is the totality of what an
  individual becomes.
• The phenotype is determined by the genotype PLUS
  interaction with the environment.
• Non-genotypic change is not inheritable!
  Lamarck was wrong!

32
Biological evolution in a nutshell

• Consider
• Biston betularia, an English moth
• 2 dark in 1848, 95 dark in 1898 Manchester
• Hair colour, tattoos, height
• Morphological change ? evolution
• Evolution ? morphological change
• Sickle cell anemia
• Evolution is not progress!
• Biological evolution is a greedy algorithm
• Individuals are embedded in and alter their
  environments!
• They consume resource, alter the competitive
  balances, and modify their environment to suit
  themselves as they are able

33
Biological evolution in a nutshell

• Mechanisms that increase genetic variation
• Mutation (new values)
• Recombination (new combinations)
• Gene flow
• Genetic drift (chance increase in existing allele
  freq.)
• Mechanisms that decrease genetic variation
• Natural selection (!!)
• Sexual selection
• Genetic drift (chance decrease in existing allele
  freq.)

34
Biological vs. software evolution

• How hard should we push the analogy of biology
  into software?
• OK, genotype source code, but what is a
  software phenotype?
• What is a software individual/population?
• I wont try to answer this

35
Biological vs. software evolution

• One important fact
• Unlike biology, the prime agent for new
  values/traits of software is not chance, its
  marketing!
• Rate of change is much, much faster!
• Software evolution is (arguably) Lamarckian!

36
Biological vs. software evolution

• One important observation
• Not all of the effects of software change are
  planned or foreseen.
• Software reacts with its environment
  (development, deployment, political) and forms a
  complex feedback system that influences the
  future evolution of the software Lehman
• Software systems exhibit interesting emergent
  phenomena

37
SummaryBiological vs. software evolution

• If you take away nothing else from this, at least
  remember this
• We shape our tools, then our tools shape us.
• McLuhan
• We can plan and execute changes
• but we cannot understand in advance all of the
  effects of the changes, both on the environment
  and feeding back into the development of the
  software system.
• Understanding how software evolves requires
  studying both the planned and unplanned phenomena
  of software change.

38
Further reading

• Evolutionary biology
• The talk.origins archives
• http//www.talkorigins.org
• The Selfish Gene, Richard Dawkins
• Other evolution
• The Meme Machine, Susan Blackmore
• The Nature of Economies, Jane Jacobs