CS 425/625 Software Engineering Verification and Validation

1
CS 425/625 Software Engineering Verification
and Validation

• Based on Chapter 19 of the textbook SE-6 Ian
  Sommerville,
• Software Engineering, 6th Ed., Addison-Wesley,
  2000 and on the
• Ch19 PowerPoint presentation available at the
  books web-site
• www.comp.lancs.ac.uk/computing/resources/IanS/SE6/
  Slides/index.html
• November 10, 2003

2
Outline

• Introduction
• Testing and Debugging
• Verification and Validation Planning
• Software Inspections
• Program Inspections (forms of Software
  Inspections)
• Automatic Static Analysis

3
Introduction..

• Verification and validation (VV) the checking
  and analysis processes that ensure the software
  satisfies its specification and meets the needs
  of the clients who are paying for it
• Validation Are we building the right product?
• Verification Are we building the product
  right?
• Verification involves checking the software
  conforms with its specification while the more
  general process of validation ensures the
  software meets the needs of the clients
• VV is a whole life-cycle process, encompassing
  requirements reviews, design reviews, code
  inspections, and program testing

4
.Introduction.

• V V techniques
• Software inspections
• Software testing
• Software inspections
• Are static VV techniques as they do not require
  the software to be executed
• Consist of inspections, automated static
  analyses, and formal verifications of source code
  or system models
• Can only check the correspondence between the
  software and its specification
• Cannot demonstrate the system is operationally
  useful
• Cannot check non-functional requirements of the
  software

5
..Introduction...

• Software testing
• It is a dynamic VV technique as it needs an
  executable version of the software system
• The system is executed with test data and its
  operational behaviour is assessed
• Can reveal the presence of errors, not their
  absence
• A successful test is a test which discovers one
  or more errors
• The VV technique for checking non-functional
  requirements and for verifying system integration
• Should be used in conjunction with static
  techniques to provide full VV coverage

6
Introduction..

• Types of software testing
• Defect testing
• Intended to find inconsistencies between a
  program and its specification
• Tests designed to discover program faults and
  defects
• A successful defect test is one which reveals the
  presence of defects in a system
• Statistical testing
• Designed for softwares performance and
  reliability estimation
• By running tests that reflect actual user inputs
  and their frequency, an estimate of operational
  reliability can be made

7
.Introduction.

• Static and dynamic VV Fig 19.1, Somm00

8
..Introduction

• Verification and validation should establish
  confidence that the software is fit for purpose
• This does not mean the software is completely
  free of defects rather, it must be good enough
  for its intended use
• The required level of confidence depends on
• Softwares purpose the level of confidence
  depends on how critical the software is to an
  organisation
• User expectations users may have lower
  expectations of certain types of software
• Marketing environment getting a product to
  market early may have higher priority than
  finding its defects

9
Testing and debugging.

• Defect testing and debugging are distinct
  processes
• VV is concerned with establishing the existence
  of defects in a program
• Debugging is concerned with locating and
  repairing these errors
• Debugging involves formulating hypotheses about
  program behaviour then testing these hypotheses
  to find the errors

10
.Testing and debugging

• The debugging process Fig. 19.2, Somm00

11
V V Planning..

• The planning of VV should start early in the
  development process
• The plan should balance static verification and
  testing, specify testing standards and
  procedures, establish checklists for inspections,
  and define the software test plan
• Test planning breaks down VV into a number of
  stages, often organized according to the V-model
  (shown on next page)
• The focus is on setting standards and procedures
  for inspections and testing, not on describing
  product tests

12
.VV Planning.

• The V-model of development Fig. 19.3, Somm00

13
..VV Planning

• The structure of a software test plan
• The testing process
• Requirements traceability
• Tested items
• Testing schedule
• Test recording procedures
• Hardware and software requirements
• Constraints

14
Software Inspections.

• Involve people examining software code or models
  (representations) with the aim of discovering
  defects
• Do not require execution of a system thus can be
  used throughout the development process
• May be applied to any representation of the
  system requirements, design, test data, etc.
• Very effective technique for discovering errors

15
.Software Inspections

• In software inspections many different defects
  can be discovered in a single review of the
  source code or software model
• In testing, one defect may mask another hence
  several executions are required
• Software inspections reuse domain and programming
  knowledge so reviewers are likely to have seen
  the types of error that commonly occur
• Software inspections and software testing are
  complementary, not competing techniques (see also
  slides 4 and 5)

16
Program Inspections.

• Program inspections are a type of software
  inspections
• Consist of formal reviews conducted by teams and
  intended for program defect detection
• Defects may be logical errors, anomalies in the
  code that might indicate an erroneous condition
  (e.g., an un-initialized variable), or
  non-compliances with standards

17
.Program Inspections

• Inspection pre-conditions
• A precise specification must be available
• Team members must be familiar with the
  organisation standards
• Syntactically correct code must be available
• An error checklist should be prepared
• Management must accept that inspection will
  increase costs early in the software process
• Management must not use inspections for staff
  appraisal

18

..Program Inspections..
The inspection process Fig. 19.6, Somm00
19
Program Inspections.

• Program inspection procedure
• The inspection is planned by the moderator (or
  chairman)
• The system overview is presented to the
  inspection team by the program author (or owner)
• The code and associated documents are distributed
  to the inspectors (inspection team) in advance
  for individual preparation
• The inspection meeting takes place and errors are
  noted (the inspection also involves a reader and,
  possibly, a scribe)
• During re-work modifications are made by the
  author to repair discovered errors
• Re-inspection may or may not be required, based
  on moderators decision

20
.Program Inspections

• Inspection teams are made up of at least 4
  members
• Author of the code being inspected
• Inspector who finds errors, omissions and
  inconsistencies
• Reader who reads the code to the team
• Moderator who chairs the meeting and notes
  discovered errors
• Other roles are chief moderator and scribe

21
..Program Inspections..

• Inspection checklist
• A checklist of common errors should be used
  during the inspection
• This error checklist is programming language
  dependent
• The weaker the type-checking of the language, the
  larger the checklist
• Examples of possible errors initialisation,
  constant naming, loop termination, array bounds,
  etc.

22
Inspection checks (fault classes) Fig. 19.7,
Somm00
23
.Program Inspections

• Inspection rates
• 500 statements/hour during overview
• 125 source statement/hour during individual
  preparation
• 90-125 statements/hour can be inspected
• Inspection is therefore an expensive process
• Inspecting 500 lines costs about 40 man/hours
  effort 2800

24
Automated Static Analysis.

• Static analyzers are software tools for source
  code processing
• They parse the program text to discover erroneous
  conditions
• Very effective as an aid to inspections
  supplement but cannot replace inspections
• Particularly valuable for languages such as C
  that have weak typing (many errors can remain
  undetected by the compiler)
• Less cost-effective for languages such as Java
  that have strong type checking (many errors can
  be detected during compilation)

25
.Automated Static Analysis
Automatic static analysis checks Fig. 19.8,
Somm00
26
..Automated Static Analysis..

• Stages of static analysis
• Control flow analysis. Checks for loops with
  multiple exit or entry points, finds unreachable
  code, etc.
• Data use analysis. Detects un-initialised
  variables, variables written twice without an
  intervening assignment, variables which are
  declared but never used, etc.
• Interface analysis. Checks the consistency of
  routine and procedure declarations and their use

27
Automated Static Analysis.

• Stages of static analysis contd
• Information flow analysis. Identifies the
  dependencies of output variables. Does not
  detect anomalies itself but highlights
  information for code review (inspection)
• Path analysis. Identifies paths through the
  program and sets out the statements executed in
  that path. Again, potentially useful in the
  review process

28
LINT static analysis Fig 19.9, Somm00
138 more lint_ex.c include ltstdio.hgt printarray
(Anarray) int Anarray printf(d,Anarray)
main () int Anarray5 int i char c
printarray (Anarray, i, c) printarray
(Anarray) 139 cc lint_ex.c 140 lint
lint_ex.c lint_ex.c(10) warning c may be used
before set lint_ex.c(10) warning i may be used
before set printarray variable of args.
lint_ex.c(4) lint_ex.c(10) printarray, arg. 1
used inconsistently lint_ex.c(4)
lint_ex.c(10) printarray, arg. 1 used
inconsistently lint_ex.c(4) lint_ex.c(11) print
f returns value which is always ignored