The Software Process

1
Implementation, Integration and Maintenance
CS 63 Software Engineering Topic 11 12
2
Implementation - Overview

• Choice of programming language
• Fourth generation languages
• Good programming practice
• Coding standards
• Module reuse
• Management aspects of module testing
• Challenges of the implementation phase

3
Implementation Phase

• Programming-in-the-many
• Choice of Programming Language
• Language is usually specified in contract
• But what if the contract specifies
• The product is to be implemented in the most
  suitable programming language
• What language should be chosen?

4
Choice of Programming Language (contd)

• Example
• QQQ Corporation has been writing COBOL programs
  for over 25 years
• Over 200 software staff, all with COBOL expertise
• What is most suitable programming language?
• Obviously COBOL

5
Choice of Programming Language (contd)

• What happens when new language (C, say) is
  introduced
• New hires
• Retrain existing professionals
• Future products in C
• Maintain existing COBOL products
• Two classes of programmers
• COBOL maintainers (despised)
• C developers (paid more)
• Need expensive software, and hardware to run it
• 100s of person-years of expertise with COBOL
  wasted

6
Choice of Programming Language (contd)

• Only possible conclusion
• COBOL is the most suitable programming language
  
• And yet, the most suitable language for the
  latest project may be C
• COBOL is suitable for only DP applications
• How to choose a programming language
• Cost-benefit analysis
• Compute costs, benefits of all relevant languages

7
Choice of Programming Language (contd)

• Which is the most appropriate object-oriented
  language?
• C is (unfortunately) C-like
• Java enforces the object-oriented paradigm
• Training in the object-oriented paradigm is
  essential before adopting any object-oriented
  language
• What about choosing a fourth generation language
  (4GL)?

8
Fourth Generation Languages

• First generation languages
• Machine languages
• Second generation languages
• Assemblers
• Third generation languages
• High-level languages (COBOL, FORTRAN, C)
• Fourth generation languages (4GLs)
• One 3GL statement is equivalent to 510 assembler
  statements
• Each 4GL statement intended to be equivalent to
  30 or even 50 assembler statements

9
Fourth Generation Languages (contd)

• It was hoped that 4GLs would
• Speed up application-building
• Applications easy, quick to change
• Reducing maintenance costs
• Simplify debugging
• Make languages user friendly
• Leading to end-user programming
• Achievable if 4GL is a user friendly, very
  high-level language

10
Productivity Increases with a 4GL?

• The picture is not uniformly rosy
• Problems with
• Poor management techniques
• Poor design methods
• James Martin suggests use of
• Prototyping
• Iterative design
• Computerized data management
• Computer-aided structuring
• Is he right? Does he (or anyone else) know?

11
Actual Experiences with 4GLs

• Playtex used ADF, obtained an 80 to 1
  productivity increase over COBOL
• However, Playtex then used COBOL for later
  applications
• 4GL productivity increases of 10 to 1 over COBOL
  have been reported
• However, there are plenty of reports of bad
  experiences

12
Actual Experiences with 4GLs (contd)

• Attitudes of 43 Organizations to 4GLs
• Use of 4GL reduced users frustrations
• Quicker response from DP department
• 4GLs slow and inefficient, on average
• Overall, 28 organizations using 4GL for over 3
  years felt that the benefits outweighed the costs

13
Fourth Generation Languages (contd)

• Market share
• No one 4GL dominates the software market
• There are literally hundreds of 4GLs
• Dozens with sizable user groups
• Reason
• No one 4GL has all the necessary features
• Conclusion
• Care has to be taken in selecting the appropriate
  4GL

14
Key Factors When Using a 4GL

• Large sums for training
• Management techniques for 4GL, not for COBOL
• Design methods must be appropriate, especially
  computer-aided design
• Interactive prototyping
• 4GLs and complex products
• Dangers of a 4GL
• Deceptive simplicity
• End-user programming

15
Good Programming Practice

• Use of consistent and meaningful variable
  names
• Meaningful to future maintenance programmer
• Consistent to aid maintenance programmer
• Issue of self-documenting code
• Exceedingly rare
• Key issue Can module be understood easily and
  unambiguously by
• SQA team
• Maintenance programmers
• All others who have to read the code

16
Good Programming Practice Example

• Module contains variables freqAverage,
  frequencyMaximum, minFr, frqncyTotl
• Maintenance programmer has to know if freq,
  frequency, fr, frqncy all refer to the same thing
• If so, use identical word, preferably frequency,
  perhaps freq or frqncy, not fr
• If not, use different word (e.g., rate) for
  different quantity
• Can use frequencyAverage, frequencyMyaximum,
  frequencyMinimum, frequencyTotal
• Can also use averageFrequency, maximumFrequency,
  minimumFrequency, totalFrequency
• All four names must come from the same set

17
Good Programming Practice Example (contd)

• Example
• Variable xCoordinateOfPositionOfRobotArm
• Abbreviated to xCoord
• Entire module deals with the movement of the
  robot arm
• But does the maintenance programmer know this?

18
Prologue Comments

• Mandatory at top of every single module
• Minimum information
• Module name
• Brief description of what the module does
• Programmers name
• Date module was coded
• Date module was approved, and by whom
• Module parameters
• Variable names, in alphabetical order, and uses
• Files accessed by this module
• Files updated by this module
• Module i/o
• Error handling capabilities
• Name of file of test data (for regression
  testing)
• List of modifications made, when, approved by
  whom
• Known faults, if any

19
Other Comments

• Suggestion
• Comments are essential whenever code is written
  in a non-obvious way, or makes use of some subtle
  aspect of the language
• Nonsense!
• Recode in a clearer way
• We must never promote/excuse poor programming
• However, comments can assist maintenance
  programmers
• Code layout for increased readability
• Use indentation
• Better, use a pretty-printer
• Use blank lines

20
Nested if Statements

• Example
• Map consists of two squares. Write code to
  determine whether a point on the Earths surface
  lies in map square 1 or map square 2, or is not
  on the map

21
Nested if Statements (contd)

• Solution 1. Badly formatted

22
Nested if Statements (contd)

• Solution 2. Well-formatted, badly constructed

23
Nested if Statements (contd)

• Solution 3. Acceptably nested

24
Nested if Statements (contd)

• Combination of if-if and if-else-if statements is
  usually difficult to read
• Simplify The if-if combination
• if ltcondition1gt
• if ltcondition2gt
• is frequently equivalent to the single condition
• if ltcondition1gt ltcondition2gt
• Rule of thumb
• if statements nested to a depth of greater than
  three should be avoided as poor programming
  practice

25
Programming Standards

• Can be both a blessing and a curse
• Modules of coincidental cohesion arise from rules
  like
• Every module will consist of between 35 and 50
  executable statements
• Better
• Programmers should consult their managers before
  constructing a module with fewer than 35 or more
  than 50 executable statements

26
Remarks on Programming Standards

• No standard can ever be universally applicable
• Standards imposed from above will be ignored
• Standard must be checkable by machine
• Examples of good programming standards
• Nesting of if statements should not exceed a
  depth of 3, except with prior approval from the
  team leader
• Modules should consist of between 35 and 50
  statements, except with prior approval from the
  team leader
• Use of gotos should be avoided. However, with
  prior approval from the team leader, a forward
  goto may be used for error handling

27
Remarks on Programming Standards (contd)

• Aim of standards is to make maintenance easier
• If it makes development difficult, then must be
  modified
• Overly restrictive standards are
  counterproductive
• Quality of software suffers

28
Module Reuse

• The most common form of reuse

29
Software Quality Control

• After preliminary testing by the programmer, the
  module is handed over to the SQA group
• Module Testing Management Implications.
• We need to know when to stop testing
• Costbenefit analysis
• Risk analysis
• Statistical techniques

30
Challenges of the Implementation Phase

• Module reuse needs to be built into the product
  from the very beginning
• Reuse must be a client requirement
• Software project management plan must incorporate
  reuse

31
Integration - Overview

• Implementation and integration
• Testing during the implementation and integration
  phase
• Integration testing of graphical user interfaces
• Product testing
• Acceptance testing
• Challenges of the implementation and integration
  phase

32
Implementation and Integration Phase

• Up to now Implementation followed by integration
• Poor approach 
• Better
• Combine implementation and integration
  methodically

33
Product with 13 Modules
34
Implementation, Then Integration

• Code and test each module separately
• Link all 13 modules together, test product as a
  whole

35
Drivers and Stubs

• To test module a, modules b, c, d must be stubs
• Empty module, or
• Prints message ("Procedure radarCalc called"), or
• Returns precooked values from preplanned test
  cases
• To test module h on its own requires a driver,
  which calls it
• Once, or
• Several times, or
• Many times, each time checking value returned
• Testing module d requires driver and two stubs

36
Implementation, Then Integration (contd)

• Problem 1
• Stubs and drivers must be written, then thrown
  away after module testing is complete
• Problem 2
• Lack of fault isolation
• A fault could lie in any of 13 modules or 13
  interfaces
• In a large product with, say, 103 modules and 108
  interfaces, there are 211 places where a fault
  might lie
• Solution to both problems
• Combine module and integration testing
• Implementation and integration phase

37
Top-down Implementation and Integration

• If module m1 calls module m2, then m1 is
  implemented and integrated before m2
• One possible top-down ordering is
• a, b, c, d, e, f, g, h, i, j, k, l, m
• Another possible top-down ordering is
• a
• a b, e, h
• a c, d, f, i
• a, d g, j, k, l, m

38
Top-down Implementation and Integration (contd)

• Advantage 1 Fault isolation
• Previously successful test case fails when mNew
  is added to what has been tested so far
• Advantage 2 Stubs not wasted
• Stub expanded into corresponding complete module
  at appropriate step

39
Top-down Implementation and Integration (contd)

• Advantage 3 Major design flaws show up early
• Logic modules include decision-making flow of
  control
• In the example, modules a, b, c, d, g, j
• Operational modules perform actual operations of
  module
• In the example, modules e, f, h, i, k, l, m
• Logic modules are developed before operational
  modules

40
Top-down Implementation and Integration (contd)

• Problem 1
• Reusable modules are not properly tested
• Lower level (operational) modules are not tested
  frequently
• The situation is aggravated if the product is
  well designed
• Defensive programming (fault shielding)
• Example
• if (x gt 0)
• y computeSquareRoot (x, errorFlag)
• Never tested with x lt 0
• Reuse implications

41
Bottom-up Implementation and Integration

• If module m1 calls module m2, then m2 is
  implemented and integrated before m1
• One possible bottom-up ordering is
• l, m, h, i, j, k, e, f, g, b, c, d, a
• Another possible bottom-up ordering is
• h, e, b
• i, f, c, d
• l, m, j, k, g d
• a b, c, d

42
Bottom-up Implementation and Integration (contd)

• Advantage 1
• Operational modules thoroughly tested
• Advantage 2
• Operational modules are tested with drivers, not
  by fault shielding, defensively programmed
  calling modules
• Advantage 3
• Fault isolation

43
Bottom-up Implementation and Integration (contd)

• Difficulty 1
• Major design faults are detected late
• Solution
• Combine top-down and bottom-up strategies making
  use of their strengths and minimizing their
  weaknesses

44
Sandwich Implementation and Integration

• Logic modules are implemented and integrated
  top-down
• Operational modules are implemented and
  integrated bottom-up
• Finally, the interfaces between the two groups
  are tested

45
Sandwich Implementation and Integration (contd)

• Advantage 1
• Major design faults are caught early
• Advantage 2
• Operational modules are thoroughly tested
• They may be reused with confidence
• Advantage 3
• There is fault isolation at all times

46
Summary
47
Object-Oriented Implementation and Integration

• Object-oriented implementation and integration
• Almost always sandwich implementation and
  integration
• Objects are integrated bottom-up
• Other modules are integrated top-down

48
Management Issues during Implem. and Int.

• Example
• Design document used by Team 1 (who coded module
  m1) shows m1 calls m2 passing 4 parameters
• Design document used by Team 2 (who coded module
  m2) states clearly that m2 has only 3 parameters
• Solution
• The implementation and integration. process must
  be run by SQA

49
Testing during Implem. and Integration Phase

• Product testing
• Performed to ensure that the product will not
  fail its acceptance test
• Failing the acceptance test is a disaster for
  management
• The client may conclude that the developers are
  incompetent
• Worse, the client may believe that the developers
  tried to cheat
• The SQA team must ensure that the product passes
  its acceptance test

50
Integration Testing of Graphical User Interfaces

• GUI test cases
• Mouse clicks
• Key presses
• And so on
• Cannot be stored in the usual way
• We need special CASE tools
• Examples
• QAPartner
• XRunner

51
Strategy for Product Testing

• The SQA team must try to approximate the
  acceptance test
• Black box test cases for the product as a whole
• Robustness of product as a whole
• Stress testing (under peak load)
• Volume testing (e.g., can it handle large input
  files?)

52
Strategy for Product Testing (contd)

• Check all constraints
• Timing constraints
• Storage constraints
• Security constraints
• Compatibility
• Review all documentation to be handed over to the
  client
• Verify the documentation against the product
• The product (software plus documentation) is now
  handed over to the client organization for
  acceptance testing

53
Acceptance Testing

• The client determines whether the product
  satisfies its specifications
• Performed by
• The client organization, or
• The SQA team in the presence of client
  representatives, or
• An independent SQA team hired by the client

54
Acceptance Testing (contd)

• Four major components of acceptance testing
• Correctness
• Robustness
• Performance
• Documentation
• (Precisely what was done by developer during
  product testing)

55
Challenges of the Implem. and Integration Phase

• Management issues are paramount here
• Appropriate CASE tools
• Test case planning
• Communicating changes to all personnel
• Deciding when to stop testing

56
Maintenance - Overview

• Why maintenance is necessary
• What is required of maintenance programmers
• Maintenance case study
• Management of maintenance
• Maintenance of object-oriented software
• Maintenance skills versus development skills
• Reverse engineering
• Testing during the maintenance phase
• Challenges of the maintenance phase

57
Maintenance Phase

• Maintenance
• Any change to any component of the product
  (including documentation) after it has passed the
  acceptance test
• Why is maintenance necessary?

58
Types of maintenance
59
Types of maintenance (contd)

• Corrective maintenance
• To correct residual faults
• Specification, design, implementation,
  documentation, or any other types of faults
• On average, 17.5 of maintenance
• Perfective maintenance
• Client requests changes to improve product
  effectiveness
• Add additional functionality
• Make product run faster
• Improve maintainability
• On average, 60.5 of maintenance

60
Types of maintenance (contd)

• Adaptive maintenance
• Responses to changes in environment in which
  product operates
• Product ported to new compiler, operating system,
  and/or hardware
• Change to tax code
• 9-digit ZIP codes
• On average, 18 of maintenance

61
Difficulty of Maintenance

• About 67 of the total cost of a product accrues
  during the maintenance phase
• Maintenance is a major income source
• Nevertheless, even today many organizations
  assign maintenance to
• Unsupervised beginners, and
• Less competent programmers

62
What is Required of Maintenance Programmers?

• Maintenance is one of the most difficult aspects
  of software production because
• Maintenance incorporates aspects of all other
  phases
• Suppose a fault report is handed to a maintenance
  programmer
• What tools does the maintenance programmer have
  to find the fault?
• The fault report filed by user
• The source code
• And often nothing else

63
What is Required of Maintenance Programmers?

• Maintenance programmer must have superb debugging
  skills
• The fault could lie anywhere within the product
• The original cause of the fault might lie in the
  by now non-existent specifications or design
  documents

64
Corrective Maintenance

• Suppose that the maintenance programmer has
  located the fault
• Problem
• How to fix it without introducing a regression
  fault
• How to minimize regression faults
• Consult the detailed documentation for product as
  a whole
• Consult the detailed documentation for each
  individual module
• What usually happens
• There is no documentation at all, or
• The documentation is incomplete, or
• The documentation is faulty

65
Corrective Maintenance (contd)

• The programmer must deduce from the source code
  itself all the information needed to avoid
  introducing a regression fault
• Now the programmer changes the source code
• Test that the modification works correctly
• Use specially constructed test cases
• Check for regression faults
• Use stored test data
• Add specially constructed test cases to stored
  test data for future regression testing
• Document all changes
• Major skills required for corrective maintenance
• Superb diagnostic skills
• Superb testing skills
• Superb documentation skills

66
Adaptive and Perfective Maintenance

• The maintenance programmer must go through the
  phases of
• Requirements
• Specifications
• Design
• Implementation and integration
• Using the existing product as a starting point
• When programs are developed
• Specifications are produced by specification
  experts
• Designs are produced by design experts
• Implementation is performed by implementation
  experts
• Integration is performed by integration experts
• Testing is performed by testing experts
• Documentation is produced by documentation experts

67
Adaptive and Perfective Maintenance (contd)

• But every maintenance programmer needs to be an
  expert in
• Specifications
• Design
• Implementation
• Integration
• Testing
• Documentation
• Conclusion
• No form of maintenance
• Is a task for an unsupervised beginner, or
• Should be done by a less skilled computer
  professional

68
The Rewards of Maintenance

• Maintenance is a thankless task in every way
• Maintainers deal with dissatisfied users
• If the user were happy,the product would not
  need maintenance
• The users problems are often caused by the
  individuals who developed the product, not the
  maintainer
• The code itself may be badly written
• Maintenance is despised by many software
  developers
• Unless good maintenance service is provided, the
  client will take future development business
  elsewhere
• Maintenance is the most important phase of
  software production, the most difficultand most
  thankless

69
The Rewards of Maintenance (contd)

• How can this situation be changed?
• Managers must assign maintenance to their best
  programmers, and
• Pay them accordingly

70
Management of Maintenance

• We need a mechanism for changing a product
• If the product appears to function incorrectly,
  the user files a fault report
• It must include enough information to enable the
  maintenance programmer to recreate the problem
• Ideally, every fault should be fixed immediately
• In practice, immediate preliminary investigation
• The maintenance programmer should first consult
  the fault report file
• All reported faults not yet fixed, and
• Suggestions for working around them

71
Management of Maintenance (contd)

• Previously reported fault
• Give information in fault report file to user
• New fault
• Maintenance programmer should try to find
• Cause of fault
• A way to fix it
• A way to work around problem
• New fault is now filed in the fault report file,
  together with supporting documentation
• Listings
• Designs
• Manuals

72
Management of Maintenance (contd)

• The file should also contain the clients
  requests for perfective and adaptive maintenance
• Contents of file must be prioritized by the
  client
• The next modification is the one with the highest
  priority
• Copies of fault reports must be circulated to all
  users
• Including estimate of when the fault can be
  fixed
• If the same failure occurs at another site, the
  user can determine
• If it is possible to work around the fault, and
• How long until it can be fixed

73
Management of Maintenance (contd)

• In an ideal world
• We fix every fault immediately
• Then we distribute the new version of product to
  all sites
• In the real world
• We distribute fault reports to all sites
• We do not have the staff for instant maintenance

74
Making Changes to the Product

• Corrective maintenance
• Assign a maintenance programmer to determine the
  fault and its cause, then repair it
• Test the fix, test the product as a whole
  (regression testing)
• Update the documentation to reflect the changes
  made
• Update the prologue comments to reflect
• What was changed,
• Why it was changed,
• By whom, and
• When

75
Making Changes to the Product (contd)

• Adaptive and perfective maintenance
• As with corrective maintenance, except there is
  no fault report
• There is a change in requirements instead

76
Making Changes to the Product (contd)

• What if the programmer has not tested the fix
  adequately?
• Before the product is distributed, it must be
  tested by the SQA group
• Maintenance is extremely hard
• Testing is difficult and time consuming
• Baselines and private copies

77
Ensuring Maintainability

• Maintenance is not a one-time effort
• We must plan for maintenance over the entire life
  cycle
• Design phaseuse information-hiding techniques
• Implementation phaseselect variable names
  meaningful to future maintenance programmers
• Documentation must be complete and correct, and
  reflect current version of every module

78
Ensuring Maintainability (contd)

• During the maintenance phase, maintainability
  must not be compromised
• Always be conscious of the inevitable further
  maintenance
• Principles leading to maintainability are equally
  applicable to the maintenance phase itself

79
The Problem of Repeated Maintenance

• Moving target problem
• Frustrating to development team
• Frequent changes have an adverse effect on the
  product
• Apparent solution
• Construct a rapid prototype
• Change it as frequently as the client wants
• Once the client is finally satisfied, the
  specifications are approved and product is
  constructed
• In practice
• The client can change the requirements next day
• If willing to pay the price, the client can
  change the requirements daily

80
Problem of Repeated Maintenance (contd)

• The problem exacerbated during the maintenance
  phase
• The more changes there are
• The more the product deviates from its original
  design
• The more difficult further changes become
• Documentation becomes even less reliable than
  usual
• Regression testing files are not up to date
• A total rewrite may be needed for further
  maintenance

81
Moving Target Problem (contd)

• Clearly a management problem
• In theory
• Developers must explain the problem at start of
  the project
• Specifications can be frozen until delivery
• Specifications can be frozen after perfective
  maintenance
• In practice
• It does not work that way
• A client with clout can order changes and they
  are implemented
• He who pays the piper calls the tune

82
Warning

• It is no use implementing changes slowly
• The relevant personnel are replaced
• Nothing can be done if the person calling for
  repeated change has sufficient clout

83
Maintenance of Object-Oriented Software

• The object-oriented paradigm promotes maintenance
• The product consists of independent units
• Encapsulation (conceptual independence)
• Information hiding (physical independence)
• Message-passing is the sole communication
• Reality is somewhat different!
• Three obstacles
• The complete inheritance hierarchy can be large
• The consequences of polymorphism and dynamic
  binding
• The consequences of inheritance

84
Maintenance versus Development Skills

• Skills needed for maintenance include
• Ability to determine cause of failure of large
  product
• Also needed during integration and product
  testing
• Ability to function effectively without adequate
  documentation
• Documentation rarely complete until delivery
• Skills in specification, design, implementation,
  testing
• All four activities carried out during
  development
• Skills needed for maintenance same as those for
  other phases

85
Maintenance versus Development Skills

• Key Point
• Maintenance programmers must not merely be
  skilled in broad variety of areas, they must be
  highly skilled in all those areas
• Specialization impossible for the maintenance
  programmer
• Maintenance is the same as development, only more
  so

86
Reverse Engineering

• When the only documentation is the code itself
• Start with the code
• Recreate the design
• Recreate the specifications (extremely hard)
• CASE tools help (flowcharters, other visual aids)
• Definitions
• Reengineering
• Reverse engineering, followed by forward
  engineering
• Lower to higher to lower levels of abstraction
• Restructuring
• Improving product without changing functionality
• Prettyprinting, structuring code, improving
  maintainability
• What if we have only the executable code?
• Treat as black box

87
Testing during the Maintenance Phase

• Maintainers view a product as loosely related
  modules
• They were not involved in development of the
  product
• Regression testing is essential
• Store test cases and outcomes, modify as needed

88
Challenges of the Maintenance Phase

• The development-then-maintenance model is
  unrealistic today
• The clients requirements frequently change
  before the product is delivered
• Faults often have to be fixed before the product
  is delivered
• Development from scratch is almost unknown today.
• Instead, products are built from reused
  components.
• Products are modified before delivery

89
Future of Software Engineering

• Must people have 20-20 hindsight
• Forecasting the future is much harder!
• Possible scenario 1
• Emphasis on formal techniques
• Formal object-oriented techniques
• Possible scenario 2
• Intensive reuse of hundreds of standard classes
• Possible scenario 3
• Totally different to anything currently envisioned