Software Verification and Validation V

1
Software Verification and Validation (V V)by
Roger U. Fujii and Dolores R. Wallace

• Presented by
• Sanjay Patel

2
Agenda

• Introduction
• Overview Of Software Verification and Validation
• Objectives of V V
• Responsibilities of V V
• Organizing a V V Effort
• Applying V V to a Software Life Cycle
• Effectiveness of V V
• Standards and Guidelines For Planning and
  Managing V V
• Summary
• Questions

3
Introduction

• Software V V defined as a systems engineering
  methodology to ensure that quality is built into
  the software during development.
• Analysis and test activities are performed by V
  V in order to evaluate and assess the software
  products and development processes during each
  SDLC phase.
• Software V V is complementary to and supportive
  of quality assurance, project management, systems
  engineering, and development.

4
Introduction (Contd)

• Relationship of V V to the development team

5
Overview of Software Verification and Validation

• Software Verification Process is a process for
  determining whether the software products of an
  activity fulfill the requirements or conditions
  imposed on them in the previous activities.
• Software Validation Process is a process for
  determining whether the requirements and the
  final, as-built system or software product
  fulfills its specific intended use.

6
Verification and Validation
7
Objectives of V V

• Software V V determines that the software
  performs its intended functions correctly.
• Ensure that the software performs no unintended
  functions.
• Measure and assess the quality and reliability of
  software.
• As a systems engineering discipline, software V
  V also assesses, analyzes, and tests the software
  on
• how it interfaces with systems elements
• Influences the performance, or reacts to stimuli
  from system elements

8
Responsibilities of V V Versus Other Groups
9
Organizing a V V Effort

• Whom does the V V reports?
• Four methods of organizing a V V effort are
  described
• Independent V V
• Embedded in the development system engineering
  group
• Embedded in the development quality assurance
  group
• Embedded in the user group.

10
Independent V V

• Independent of the development group and is
  called Independent VV or IVV
• The IV V group reports directly to the system
  program manager.
• The V V group establishes formal procedures for
  receiving software releases and documentation
  from the development team.
• V V tasks are oriented towards engineering
  analysis (algorithm analysis, control/data flow
  analysis), and comprehensive testing (such as
  simulation)
• Main objective is to determine whether the
  software satisfies critical system requirements.
• Sends all evaluation reports and discrepancy
  reports to both the program manager and
  development group.

11
Independent V V (Contd..)

• Advantages
• Detailed analysis and test of software
  requirements.
• Early detection of high risk software and system
  errors.
• Disadvantages
• Higher cost to the project
• Additional development interfaces

12
V V effort embedded in the Systems Engineering
Group

• Embedded in developments systems engineering
  group
• V V tasks are to review the groups engineering
  analyses (like algorithm development,
  sizing/timing), and testing (like test evaluation
  or review of the adequacy of the development test
  planning document)
• V Vs results are reviewed and monitored by
  systems engineering and quality assurance groups.
• Advantages
• Minimize cost impact to the project
• No system learning for the staff
• No additional development interfaces
• Disadvantages
• Loss of engineering analysis objectivity

13
V V effort embedded in the developments
quality assurance group

• V V group is part of quality assurance and
  maintains its relationship with systems
  engineering and other development groups in the
  same manner as quality assurance.
• Tasks include monitoring, auditing, reviewing
  content (for example, audit performance, audit
  support, test witnessing, walk-through support,
  documentation review)
• Advantages
• Low cost to the project
• Bringing V V analysis capabilities into
  reviews, audits, and inspections
• Disadvantages
• Loss of an independent software systems analysis
  and test capability

14
V V effort embedded in the user group

• Tasks consists of
• configuration management support of development
  products
• support of formal reviews
• user documentation evaluation
• test witnessing
• test evaluation of the development test planning
  documents
• user testing support
• Advantages
• Strong systems engineering and user perspective
  that can be brought to bear on the software
  product during development
• Disadvantages
• Loss of detailed analysis and test of incremental
  software products

15
Applying V V to a Software Life Cycle

• Minimum recommended V V tasks that are required
  by the ANSI/IEEE Standard for Software
  Verification and Validation Plans (SVVP) for each
  development phase.
• Considered effective and applicable to all types
  of software applications.
• A V V task specifies minimum input and output
  requirements for each V V task
• A V V task may not begin without specific
  inputs, and is not complete until specific
  outputs are completed.

16
Applying V V to a Software Life Cycle

• Minimum set of recommended V V tasks

17
Applying V V to a Software Life Cycle

• Management of V V
• Concept Definition Evaluation
• Requirements Analysis
• Design evaluation
• Implementation (Code) evaluation
• Testing
• Installation and Checkout activities
• Operations and Maintenance evaluation and test.

18
Applying V V to a Software Life Cycle
Management of V V

• Management tasks for V V span the entire life
  cycle.
• Tasks include
• Plan the V V process
• Coordinate and interpret performance and quality
  of the V V effort
• Report discrepancies to the user or development
  group
• Identify early problems and focus V V
  activities on them
• Provide a technical evaluation of the software
  performance and quality at each major software
  review
• Assess the full impact of proposed software
  changes
• Output of the V V activities consists of
• Software Verification and Validation Plan (SVVP)
• Tasks reports
• Phase summary reports
• Discrepancy reports
• Final report

19
Applying V V to a Software Life Cycle Concept
Definition Evaluation

• The main task is to evaluate the concept
  documentation to determine whether the defined
  concept satisfies user needs and project
  objectives (for example, statement of need,
  project initiation memo) in terms of system
  performance requirements, feasibility (for
  example, overestimation of hardware
  capabilities), completeness, and accuracy.
• The evaluation assesses the criticality of each
  software item defined in the concept

20
Applying V V to a Software Life Cycle
Requirements Analysis

• V V tasks verify the completeness of all the
  requirements
• Common tasks during requirement analysis include
  control flow analysis, data flow analysis,
  algorithm analysis, and simulation.
• V V test planning begins in the requirements
  phase and spans almost full range of life cycle
  phases and it consists of
• Component testing
• Integration testing
• System testing
• Acceptance testing

21
Applying V V to a Software Life Cycle Design
evaluation

• Most commonly used tasks are algorithm analysis,
  database analysis, timing/sizing analysis, and
  simulation.
• These V V tasks during the design phase assures
  that
• Requirements are not misrepresented or
  incompletely implemented.
• Unintended requirements are not designed into the
  solution
• Requirements are not left out of the design

22
Applying V V to a Software Life Cycle
Implementation (Code) evaluation

• Commonly used V V tasks are control flow
  analysis, database analysis, regression analysis,
  and sizing/timing analysis.
• Other tasks for code evaluation are
  walk-throughs, code inspection, and audits

23
Applying V V to a Software Life CycleTesting

• V V test technique is to develop test cases
  that violate software requirements.
• V V testing is effective at probing for errors
  and weaknesses that reveal hidden faults. It
  often finds errors for which some functional, or
  black box, test cases can produce the correct
  output despite internal errors.
• Optional tasks include regression analysis and
  test, simulation, and user document evaluation

24
Applying V V to a Software Life Cycle
Installation and Checkout activities

• V V validates that the software operates
  correctly with the operational hardware system
  and with other software, as specified in the
  interface specifications
• Optional tasks include regression analysis and
  tests, simulation, and test certification

25
Applying V V to a Software Life
CycleOperations and Maintenance evaluation and
test

• Any changes that are made during the operations
  and maintenance phase will result in repeating
  all life cycle phase V V activities in order to
  make sure that nothing is overlooked
• V V activities can be added or deleted
  depending on the type of changes made

26
Effectiveness Of V V

• Varies with project size, project complexity, and
  V V staff experience
• Some positive effects of V V on a software
  project include
• Better quality
• More rigorous development planning
• Better adherence by the development organization
  to programming language, developments standards
  and configuration management practices.
• Early error detection
• Better schedule compliance and progress
  monitoring
• Greater project management visibility into
  technical quality and progress
• Better criteria and results for decision making
  at formal reviews and audits.

27
Effectiveness Of V V

• Negative effects of V V on a software
  development project include
• Additional project cost of V V (10 30 )
• Additional interface involving the development
  team, user, and V V organization
• Lower development staff productivity

28
Standards and Guidelines for Planning and
Managing V V

• Planning V V with guidance from V V
  documents

29
Summary

• Software V V is a proven systems engineering
  discipline for generating correct and quality
  software
• Uncovers high risk errors early
• Evaluates the correctness of products against
  system and software requirements.
• Provides the user an incremental preview of
  system performance, with the chance to make early
  adjustments.
• Provides decision criteria for whether or not to
  proceed to the next development phase
• Software V V has become a powerful risk
  management tool by providing the detailed
  technical insight into the true performance of
  the software

30
Questions ?????