Process and V

1
Process and VVTraditional to Extreme

• EE599 Software VV
• Winter 2004
• Diane Kelly

2
References

• MicroSoft Secrets Cusumano, Selby, Simon
  Schuster Inc., 1995
• Extreme Programming Explained Kent Beck,
  Addison-Wesley, 2000
• The Mythical Man-Month, F.P.Brooks, Jr.,
  Addison-Wesley, 1995
• Managing the Development of Large Software
  Systems Concepts and Techniques, Winston Royce
  1970 WESCON technical papers Vol.14

3
Outline

• Where do VV activities fit into a software
  development process?
• Examples from Traditional to Extreme!
• Waterfall process model
• MicroSoft software development process
• Extreme Programming

4
Waterfall Model
System Requirements
Software Requirements
Analysis
Program Design
Coding
Testing
Operations
5
Premise of Waterfall Model (1)

• each step completed successfully provides strong
  foundation for next step
• forces developers to cope with data and control
  flows, errors, problems earlier in development
• each step should be performed by those skilled in
  those steps
• document everything software requirements,
  preliminary design, interface design, final
  design, test plan/test results, operating
  instructions
• To procure 5 million dollars of software, I
  would estimate a 1500 page specification is about
  right. (Royce)

6
Premise of Waterfall Model (2)

• Plan, control, and monitor testing
• extensive testing
• use test specialists
• visually inspect the code before testing
• plan the test process
• test every logic path
• use testing standards, procedures, tools
• Involve the customer
• include customers in reviews, preliminary design,
  final acceptance
• involvement should be formal, in-depth,
  continuing
• Royce suggests Do it twice.
• also suggestion of Fred Brooks (Mythical
  Man-Month)

7
Boehms Refined Waterfall Model
System Feasibility
Validation
Plans and Requirements
Validation
Product Design
Verification
Detailed Design
Verification
Code
Unit Test
Integration
Product Verification
Implementation
System test
Operations and Maintenance
Revalidation
8
Reasons for New Model

• test phase done very late in development
• each phase now culminated by a VV activity
• usual reasons for lack of success
• infeasibility and mismatches with user
  requirements
• verify that output of each phase is in step with
  input to that phase - keep in line with original
  requirements?
• Note that maintenance VV is vastly
  oversimplified!

9
Verification and Validation

• Verification
• Have we done the thing right?
• ISO 9000-3
• The process of evaluating the products of the
  given phase to ensure correctness and consistency
  with respect to the products and standards
  provided as input to that phase.
• Validation
• Have we done the right thing?
• ISO 9000-3
• The process of evaluating software to ensure
  compliance with specified requirements.

10
V Model
11
MicroSoft Incremental Build
12
The Microsoft Triangle

• Decision making focuses on three of the
  dimensions of quality in the end product
• Functions supported
• Timing to Market
• Implementation defects remaining
• rapid development, daily builds, testing buddies
• evolving requirements, work from vision statement
• heavy reliance on testing, multiple testing
  strategies
• well established plans and schedules

13
Approach

• fix project resources
• number of people
• amount of time
• set intended ship date
• define milestones backwards from target ship date
• no separate maintenance group
• fixes included in next product release
• three phases
• iterative approach
• more like a risk-driven incremental spiral life
  cycle
• planning phase, development phase, stabilization
  phase
• uses concept of buffering throughout

14
Development Phase

• 3 or 4 major milestone releases
• pretend these will be shipped
• developers write code
• testers write test suites
• user education staff write draft documentation
• developers fix errors continuously as testers
  find them
• 1/3 of total development phase is buffer time

• visual freeze
• major user interfaces frozen
• for benefit of user education group
• feature complete
• fully functional
• test scenarios can run
• still has bugs and performance issues
• code complete
• all features work as described in specs and users
  doc
• point where bug count should be steadily declining

15
Stabilization Phase

• 1/3 of total time in project
• extensive testing debugging
• no new features
• zero bug release
• no more high-severity bugs have been found
• all remaining bugs will be fixed in next release

16
Daily Build Process (1)

• Check Out
• private copies of source code
• from centralized master version
• multiple developers can check out same code at
  same time
• Implement Feature
• developer implements his/her own changes
• could take a few hours to several days
• Build Private Release
• developer builds own private copy of the product
  with newly implemented features
• Test Private Release
• developer tests own copy of product

17
Daily Build Process (2)

• Synch Code Changes
• compare private copies of source files back to
  master version
• other developers may have made changes in the
  mean time
• Merge Code Changes
• update private copies of source files to include
  any changes from other developers
• Build Private Release
• developer builds own copy of product with both
  own changes and changes of other developers
• Test Private Release
• test to make sure newly implemented code still
  works with others changes

18
Daily Build Process (3)

• Execute Quick Test
• done on private release
• highly automated
• makes sure new changes do not interfere with
  basic functionality of product
• does not test new feature directly
• e.g., quick test on EXCEL takes 30 minutes
• also called smoke test or regression test

• Check in
• developer officially checks his private copies
  into the master version
• has to synch and merge again in case of further
  changes
• if conflicts, may have to withdraw changes
• deadline for check in
• for EXCEL, 2PM

19
Daily Build Process (4)

• Generate Daily Build
• after check in deadline
• build master generates complete build of product
  from master version
• called daily build
• after successful build
• executes series of automated tests
• assures basic functionality works correctly
• makes daily build available to all project staff

20
Testing (1)

• quick tests on daily build
• developers create debug versions
• usually to view internal data structures
• improve memory usage and execution speed
• use assertions if then tests
• when making assumptions about data stored
  elsewhere in program
• instrumentation for user testing
• captures commands users select
• captures corresponding source code statements
  that get executed
• usability testing
• lab with people off the street
• user interfaces tested

21
Testing (2)

• weekly test releases
• to testing group for thorough testing
• use highly structured scripts
• guerrilla (gorilla?) testing
• try to break the product
• testing buddies
• work side by side with developers
• developer shares private release with his testing
  buddy
• do private release testing before code goes to
  daily build
• others
• performance testing
• internal usage
• beta testing

22
Final Notes

• little architectural documentation
• source code is one main document
• try to keep code as simple as possible
• compartimentalize code
• very little code comments
• uses Hungarian rules for variable naming
• half life of source code is as little as 18
  months
• Cusumano Selby recommend
• more use of metrics
• do use bug counts
• design and code reviews
• intensive code inspections
• detect and fix problems before incorporated into
  product
• refine product architecture
• educate junior developers

23
Extreme Programming (XP)Three Phases to Lifecycle

• Exploration
• find out what system will do
• write a story
• estimate a story
• split a story
• Commitment
• decide what subset of requirements to pursue this
  iteration
• sort by value (business)
• sort by risk (development)
• set velocity
• choose scope

• Steer
• guide development as reality molds plan
• iteration
• recovery
• new story
• reestimate
• iterations are 1 to 4 weeks

24
XP in a Nutshell

• pair programming
• constant testing
• evolving design
• immediate integration and testing

25
Pair Programming (1)

• all production code written with two programmers
  at one machine
• one partner types and thinks about best way to
  implement method
• other partner is thinking more strategically
• will this work
• is there another test case that might not work
• is there some way to simplify the system
• pairing is dynamic
• work with someone who has experience in the task
  ahead

26
Pair Programming (2)

• collective ownership
• anyone can change any code anywhere in the system
  at any time
• everyone takes responsibility for whole system
• coding standards
• all code written in accordance with standards
  that emphasize communication THROUGH the code
• have to have coding standards with so many
  changing and redeveloping the code
• must be adopted voluntarily by the team

27
Constant Testing (1)

• chance to think about what code is to do,
  independently of how it it implemented
• write every test you can think of that wont run
  immediately
• tests give confidence that things are working
• reduces time spent debugging
• use mostly unit and functional (customer) testing

28
Constant Testing (2)

• considering testing as a type of metric
• software features that cant be demonstrated by
  automated tests, dont exist
• types of testing
• programmers continually write unit tests
• customers write tests for features
• on-site customer
• include real, live customer on the team
• if need arises, then add
• regression testing, stress testing, monkey
  testing (nonsense input)

29
Evolving Design (1)

• What is simplest thing that will work?
• system designed as simply as possible
• complexity removed as soon as discovered
• Reduce emphasis on looking ahead to whats needed
  for tomorrow.
• Put something into production as early as
  possible.
• Have courage to throw code away and break tests.

30
Evolving Design (2)

• Refactoring to Simplify
• call timeout if code is getting too complex
• team gets together for a day and restructures
  whole system
• use CRC cards, sketches, etc.
• programmers restructure system without changing
  behaviour
• remove duplication, improve communication,
  simplify, add flexibility

31
Evolving Design (3)

• Design through Refactoring
• refactoring done at time programmer is adding new
  features
• based on test cases
• pick up first test case
• implement code to get test case running
• pick up next test case
• refactor code to get simplest implementation to
  get both test cases running
• repeat for rest of test cases

32
Continuous Integration

• integrate and build system many times a day
• every time a task is completed
• have a machine dedicated to integration

33
Limitations of XP

• maximum 10 programmers on a team
• wont work for extremely complex system
• wont work where there is long turn-around time
  for testing
• inhibits immediate feedback that XP is based on
• wont work if there is a long QA cycle
• wont work where there are major distractions
• needs dedicated, concentrated work
• performance testing at end?
• need cooperation in developers?
• limitations on size of system?

34
Focus on VV

• Royce
• plan, control, and monitor testing
• extensive testing
• use test specialists
• visually inspect the code before testing
• MicroSoft
• testing buddies
• multiple testing strategies
• no inspections
• XP
• write tests at the same time as the code
• tests give confidence
• no inspections