Software%20Verification%20and%20Validation%20(V

1
Software Verification and Validation (VV)

• By Roger U. Fujii
• Presented by Donovan Faustino

2
Introduction

• The paper has these following topics.
• Software VV Concepts
• Planning for VV
• Life Cycle VV Tasks
• VV Techniques and Methods
• Measurements Applied to VV
• What is VV?

3
Definitions

• VV a system engineering discipline employing a
  rigorous methodology for evaluating and assessing
  the correctness and quality of software
  throughout the software life cycle.
• Verify a developers process is technically sound.

4
VV and QA

• VV and QA are not the same, but compliment each
  other.
• VV usually focuses on ensuring the requirements
  are being met, the overall project is focused on
  the correct objectives, and risk is being
  managed.
• QA is focused on the day to day aspects of a
  project and is used to determine if procedures
  are followed

5
VV Concepts

• Benefits of VV
• Early detection leads to a better solution rather
  than quick fixes
• Validating the solution is solving the right
  problem against software requirements
• Objective evidence of software and system
  compliance to quality standards
• Support process improvements with an objective
  feedback on the quality of development process
  and products

6
Field of Application

• Important to conduct VV in its interactions with
  the system of which it is part.
• VV examines the interaction with each of the key
  system components.
• The VV process addresses the following
  interactions with software environment,
  operators/users, hardware, and other software.

7
Planning for VV

• Scope of work
• Software Integrity Levels
• Development of the Software VV Plan (SVVP)
• Cost of VV

8
Planning for VV (cont)

• VV is more effective when initiated during the
  acquisition process and throughout the life cycle
  of the software.
• VV has importance levels or called Integrity
  Levels
• Example
• Medical device high level
• Personnel record-keeping system low level

9
Integrity Levels

• The level is a range of values that represent
  software complexity, criticality, risk, safety
  level, security level, desired performance,
  reliability, or other project-unique
  characteristics.
• Each level defines the minimum required VV
  tasks.
• ANSI/IEEE Std 1012 defines four levels. Level 4
  is assigned to high-assurance or critical systems

10
Development of the Software VV Plan (SVVP)

• How the VV process is to be accomplished
• Pg 36 gives a list. 13 key steps.

11
Cost of VV

• How much software VV is adequate?
• Rule of thumb for estimating VV efforts ranges
  10 - 33 depending on the software integrity
  level.
• 33 - life sustainig medical devices or nuclear
  weapons
• 20-25 - telecommunications or financial systems
• 10-18 systems desiring software quality but not
  high-integrity

12
Life Cycle VV Tasks

• Pg 37-39 explains what VV tasks go through the
  life cycle of the project
• Goes throughout the life cycle of VV starting
  from the Acquisition phase and all the way to the
  Maintenance phase.
• The tasks listed in the book is defined from the
  ANSI/IEEE Std. 1012.

13
Life Cycle VV Tasks

• Acquisition VV
• Supply VV
• Development VV (Concept, Requirements, Design,
  Implementation)
• Development VV (Test)
• Development VV (Installation and Checkout)
• Operation VV
• Maintenance VV

14
VV Techniques and Methods

• Audits, Reviews, and Inspection
• Analytic Techniques
• Dynamic Techniques

15
Audits, Reviews, and Inspection

• VV use these techniques to verify the software
  during its development process
• Peer Reviews
• Documentation inspections
• Requirements/design/code reading
• Test witnessing
• Installation audits

16
Analytic Techniques

• Static analysis of the software (i.e,
  requirements, design, or code) using graphical,
  mathematical formulas or diagrams.
• Effective in error detection at the software unit
  level

17
Analytic Techniques

• Control (data) flow diagramming
• Interface input/output/process diagramming
• Algorithm and equation analysis
• Database analysis
• Sizing and timing analysis
• Proof of correctness

18
Dynamic Techniques

• Involve the execution of the software.
• Effective at error detection when software units
  are combined at the integrated subsystem and
  system levels

19
Dynamic Techniques

• Simulation and modeling
• Hardware/software benchmark testing
• Hardware-in-the loop testing the system config.
  is heavily instrumented to simulate different
  test scenarios to be created.
• Scientific testing coding of the target
  requirements/design using a general-purpose
  computer and higher order language.

20
Measurement Applied to VV

• Uses various calculated measurements to determine
  when the analysis or testing is completed, where
  errors are mostly likely to occur in the
  software, and what development process or
  function is causing the largest number of errors.
• Based on these measurements, the software
  engineer can determine where to concentrate their
  efforts.

21
Measurement Methods

• Software Structural Metrics measures pinpoint
  program logic having greater logical or data
  complexity
• Statistics-Based Measurements examines program
  error rates, categorization of errors, and error
  discovery time periods
• Trend Analysis analyzing percent of errors with
  historical data
• Prediction-Based Measurement using reliability
  models to determine how much analysis and test
  effort to be done.

22
Conclusions

• The VV methodology and measurements are outlined
  in ANSI/IEEE Standard 1012.
• Provides the framework for achieving an effective
  VV effort
• VV is part of the software quality management
  process as defined in the IEEE SWEBOK.
• Complimentary to and supportive of the software
  quality assurance, reviews, and inspections.

23
Questions?