Care, Feeding and Growth of Software Systems: Some Challenges

1
Care, Feeding and Growth of Software Systems
Some Challenges

• Mathai Joseph
• Executive Director
• Tata Research Development Design Centre
• Pune, India
• Montevideo, 25 Nov 2004

2
Automating the Software Process

• Growth in any form of engineering has come
• from the automation of parts of its processes.
• Yet little of the software engineering process
• has been automated on an industrial scale.
• Requirements collection is still largely manual,
• Software design is manual,
• Coding is largely manual, and
• Test generation is manual, but text execution is
  automated.
• Today we have automated book-keeping of
• steps in the software process.

3
Challenges

• Major deficiencies
• Gaps between requirements expected properties
  of the delivered system
• Long development cycle manual coding speeds too
  slow
• Platform dependence of the solution, migration
  may require major effort including
  re-engineering
• Testing inadequate in relation to possible system
  use.
• Note also
• 30 of all software work is new development, but
  
• 70 is so-called maintenance.

4
Summary of Talk

• In this talk, I will describe some tools and
• techniques that have been developed for
• Addressing the major deficiencies of today, by
• Automating some steps in the software process.
• Metrics will be used to show both the degree of
• success and the many challenges ahead.

5
Software Services Solutions

• Software services and solutions companies have to
  manage complex portfolios of development and
  maintenance work.
• They may be the best judges of what will help
  them to
• Reduce one-time costs
• Cut the time-to-market
• Reduce recurring cost.
• They should also be the best people to building
  the tools and techniques aid their work.

6
Software Development

• In an idealized world, software development
  follows
• well-defined steps

Requirements analysis
Software specification
Software construction
Software testing
Software engineering methods are used to ensure
close adherence between actual and expected work.
7
Methods and Processes

• Software engineering methods fit into a quality
• process that checks
• The rigor of application of the method.
• The improvements that result.
• A quality process can be assessed to measure
• the maturity of an organization.
• e.g. as in the SEI Capability Maturity Model

8
Methods and Processes

• Software engineering methods fit into a quality
• process that checks
• The rigor of application of the method
• The improvements that result
• A quality process can be assessed to measure
• the maturity of an organization.
• Industrial software development needs
• methods and processes to be able to
• manage complex application development.

9
Use of Methods

• In practice, the use of any method requires
• adopting a discipline. This will be accepted if
• There is a satisfactory payoff
• The change improves a known problem area.
• The improvement can be measured and is
  acknowledged by the project team.
• Use of a method (however good) will not result
• in solutions to major structural programs in the
• development process.

10
Payoff

• Use of a software development method is seen to
  have
• value if it
• Provides useful data otherwise not available,
• Reduces the time or effort for completing a task,
• Adds measurable quality to the resulting
  software,
• Scales up effectively to project needs,
• Can be used reliably and repeatedly.
• Most methods meet some of these criteria no
  method
• meets all the criteria.

11
The Target

• A study of a large number of industrial software
• development projects in the past shows the
• following

Requirements analysis
System design
Coding
Testing
4.8
26.2
12.1
56.9
Actual effort
The time spent on requirements does not include
the time for analysis done at client sites.
Nevertheless, it is still low, as is the time for
design.
12
The Target

• Closer analysis shows that the time for coding
• probably includes time for re-work for analysis
  and
• design defects, as well as coding defects.

Requirements analysis
System design
Coding
Testing
4.8
26.2
12.1
56.9
Actual effort
of total defects corrected in phase
11
29
59
13
Example

• System One is a large, very complex financial
• services application. Software was developed
• using the MasterCraft software development
• environment
• Requirements analysis produced analysis models,
• System design used class models,
• Operations were defined in abstract imperative
  notation,
• Code was automatically generated from UML models
  and operation definitions.
• The method required all requirements analysis
• to precede system design.

14
System One

• Java
• XML, SwiftNet
• C
• Comms. middleware
• Mainframe Platform
• C
• DBMS

• High performance
• SWIFT message protocol
• Data interchange in XML
• Reliable queuing
• process architecture

Trade
Profiles
Master
15
Automated Development System One

• Requirements Analysis Team of 2030 people,
• 8 months (36 of total effort)
• Software Development Team of 60 people, 6
• months
• 180K lines of operation specification
• 1200 UML classes
• MasterCraft automatically generated code
• 5M lines of C for mainframe
• 1.5M lines of C for middle and front-end
• average of 250 lines produced, tested per
  programmer day
• Almost no rework of requirements during coding
• few coding defects.

16
Review of Achievements

• Compared to the normal development process,
• Coding was less important than analysis,
  modelling

• Total development time was vastly reduced
  because of automated code generation.

• Testing was complex but coding errors were few
  because of automated code generation

17
Adding Value
Requirements analysis
System design
Coding
Testing
4.8
26.2
12.1
56.9
Actual effort
I M P R O V E M E N T S
Use models/ specification
Automate where possible

• Automate
• code review
• test generation
• test execution

Check consistency, missing details
Action
Improve accuracy save later re-work
Improve re-use
Desired result
Platform independence
Improve coverage, save time
Program analysis
Formal technique
Model-checking
18
Current Focus

• Definition of requirements, use of model-checking
  
• Code generation for embedded applications.
• Machine-assisted code review.
• Automated test-case generation, automated test
  execution.

19
Practical Software Development

• Large-scale software development rarely
• proceeds along well-defined sequential phases!
• Requirements evolve during the development and
  after
• New requirements appear throughout the lifecycle.
• This has also been described as the spiral
• method of software development.

20
Software Development

21
Changes and Adaptations

• Each change in requirements means changes in the
  software specification and program code.
• New tests must be devised to take account of
  these changes.
• Different phases are interleaved throughout the
  lifecycle.
• An acceptance test is used to mark the end of the
  development period.

22
Software Development

23
Software Development

Development Testing
Maintenance
Acceptance
Acceptance is one point in the software
lifecycle.
24
Software Maintenance

• The post-acceptance period is typically called
• the maintenance period.
• Maintenance activities are
• Similar to those in the development period, but
• Emphasis on supporting and growing an operational
  software system.
• Typically, new code is added but no code is
• ever removed.

25
Software Maintenance

• Software maintenance consists broadly of four
• activities
• Remedial correcting errors 21
• Adaptive catering to changes in the operating
  environment 25
• Enhancing adding new features 50
• Improving making software more robust.
  4
• These activities may be interleaved.
• Note most maintenance work is not bug-fixing.
• http//fox.wikis.com/wc.dll?WikiSoftwareMainten
  anceSoftwareEng

26
Comparison

• TCS data
• Of all software development/maintenance work
• Development 47
• Maintenance 53
• In maintenance projects
• Remedial work 27.5
• Enhancement 58
• Conversion lt 0.5
• Other 14
• What is the role of testing in all this?

27
What is a Test?

• A test checks whether a property is exhibited by
  a program a test is specified as a sequence of
  inputs and expected outputs.

28
What is a Test?

• A test checks whether a property is exhibited by
  a program a test is specified as a sequence of
  inputs and expected outputs.
• A test is valid if the property is required by
  the specification S.

29
What is a Test?

• A test checks whether a property is exhibited by
  a program a test is specified as a sequence of
  inputs and expected outputs.
• A test is valid if the property is required by
  the specification S.
• Program P is said to pass test T if P exhibits
• the property required by T.
• We assume that a test is precise
• defines a property that must be satisfied by all
  executions of a program, and
• specifies all relevant inputs and outputs.

30
Test Coverage

• Program P has a bug iff there exists a valid test
  T such
• that P does not pass T.
• Complete test set T a set of valid tests such
  that every behaviour B that is not in S fails
  some test in T.
• If program P passes complete test set T then P
  satisfies S.
• Finding complete test sets is computationally
• infeasible.

31
Value space

• Input values can be divided into equivalence
  classes
• Output values can be divided into equivalence
  classes and related to sets of input values.
• Within each class, boundary values and some
  internal values can be chosen.
• Test coverage can be improved if I O is
• restricted to meaningful combinations.

32
Testing sequences

• Each element t of a test T of an output value
  against an
• input value must be executed in the context of a
• specific initial state.
• e.g. initial state si v1, v2, v3 Æ input
  i1vx
• Æ output Æ o1vy Æ final state si v6, v7,
  v8
• A test must thus have the initial state set up
  before it is
• executed.
• Can be done by defining a initializing test
  sequence.
• e.g. h t1, t2, t3, tn i

33
Automating Testing

• Generate valid test sequences
• From behavioural specifications
• UML use-cases,
• Message Sequence Charts
• State machines, etc.
• Generate initial states and inputs for each step
  in the sequence
• From structural specifications enhanced with test
  conditions and domain definitions
• Class diagrams, ER diagrams
• Run the tests
• Many tools available

34
Automating Testing

• Testify
• Gets test sequences and state specifications from
  models
• Supports manual specifications
• Generates initial state and inputs for each step
• Integrates into development frameworks
• Interfaces with test execution tools
• Suitable for testing by developers (as against
  intended users)
• Improved results obtained
• Size Tests Defects detected
• Small 100 40
• Large 523 116

35
Testing for Maintenance Activities

• Testing during maintenance must cover
• Adaptation
• Remedial changes
• Enhancement
• Platform changes

36
Remedial Changes

• An error detected in program P must be in
• behaviours that are not tested by some test set
• T.
• How do we generate an adequate test for each
• change made during testing?

37
Adaptive Maintenance Platforms

• Programs are deployed on platforms
• (middleware, OS, hardware, ) and system
• behaviour is platform dependent
• Platforms may affect
• Data representation
• System call (name, parameters)
• Protocol of interaction
• Problems
• How to model platforms and interaction between
  programs and platforms?
• How does a change in platform affect the program
  if we want to preserve overall behaviour?

38
Metering Testing

• The test phase can be used also to meter paths
• of usage of the system.
• During test set execution
• Keep statistics of major path usage (especially
  inter-module/inter-component)
• During bug-fixing, use this information to cut
  down on regression testing
• When tracing output equivalence classes to input
  classes, use path usage data.

39
Making Changes

• Navigating through a large program is difficult,
• but necessary before any changes can be
• made.
• Legacy programs often have little or no
• documentation.
• Possibilities
• Program analysis tool to generate a structure
  description from program code e.g. Revine
• Abstraction of specifications from the program
  e.g. Alur et al, 2004
• Problems large size, lack of traceability,
  language
• dialects, etc.

40
Problems to be Addressed

• Generate precise test for each change
• How to choose input values other than boundary
  values
• From output value equivalence class, generate
  input value equivalence class.
• Extract behaviour model from program.
• Maintain traceability between model and program.

41
Acknowledgments

• The work described here was done by many
  colleagues
• at TRDDC. The tools MasterCraft, Testify and
  Revine
• were developed at TRDDC and are now in commercial
  
• use in TCS.

42
Thank You