2003 NASA OSMA Software Assurance Symposium SARP Initiative 583

1
2003 NASA OSMA Software Assurance SymposiumSARP
Initiative 583

• The Use of a Virtual System Simulator
  Executable Specifications to Enhance SW
  Validation, Verification, and Safety Assurance

An Introductory BriefingBy Ted Bennett Paul
Wennberg
31 July 2003
2
Definitions

• Executable Specification (ES) Description of the
  dynamic behavior of a system or system element in
  an executable language, through the execution of
  which its behavior may be tested, validated
  verified. The ES used in this project are
  bounded with virtual interfaces analogous to the
  real parts they specify.
• Detailed Executable (DE) Virtual embedded
  control system element simulation running the
  unmodified object software developed for its
  real-world counterpart.
• Virtual System Integration Laboratory (VSIL)
  Virtual environment wherein embedded system
  element executable specifications and detailed
  executables may be interconnected and tested for
  verification validation purposes.

3
These Are the Problems

• Most embedded SW faults traceable to ambiguities
  errors in system rqmts. 1,2,3
• Poor comm. of rqmts. changes and poor comm.
  between teams during development also implicated
  as a major source of SW faults and significant
  schedule budget overruns 1, et al

• Conventional fault injection-based testing
  limited by cost and schedule constraints
• Present methods of collecting dynamic SW metrics
  are intrusive - typically requiring
  instrumentation of operating system or target
  software itself

4
Project Objectives

• Evaluate viability benefit of maintaining test
  consistency between VSIL using ES, VSIL using
  DE running executable SW
• Evaluate metric capabilities of VSIL
• New types of dynamic metrics,easier capture
  methods
• Reliability, accuracy benefit of noninvasive
  metric capture

5
Project Plan

1. Create simplified simulation of Shuttle Robotic
   Manipulator System (Robotic Arm)
2. VSIL simulation to comprise multiple ES and one
   computerized subsystem developed into DE
3. Write test suite to VV system design
4. Develop DE and control SW from target ES
5. Rerun all system tests with DE substituted for ES
6. Use VSIL to investigate metric objectives

6
ES in IcoSim VSIL

• Hierarchical
• Highly Bounded
• Firmly Anchored in Reality

7
ES/DE Test Consistency
DirectSubstitution

• Substitute DE for ES
• Run same test sequences
• When DE SW passes tests,it correctly
  implementsfunctionality verified in ES

8
Simulator Abstraction Charts
9
IcoSim Part Characteristics

• Highly modular
• Bounded
• Hierarchical
• Recursive
• Abstract or Detailed
• Simple or Complex
• Definable intrinsic failure modes

Hierarchical yes, modular yes but definitely not
a Venn diagram!
10
Potential Contributions

• Reduce interpretation induced SW faults due to
  ambiguities in system requirements
• Improve ability for dynamic, noninvasive test of
  system SW response to failure conditions
• Known behavioral characteristics failure modes
  of real part are intrinsic to virtual part and
  manifested under test control

11
Potential Contributions (Contd)

• Reduce SW faults caused by breakdown in
  communication of system Rqmts changes

• System design, ES, and DE changes verified in
  VSIL by Systems Engineering VSIL Development
  team
• Updates to VSIL and all tests maintained under
  configuration control and distributed as they
  occur to all team members

12
Potential Contributions (Contd)

• New capacity for empirical SW VV in cases where
  analysis was only viable means
• Realistic fault injection failure mode testing
• Complex digital signal processor designs
• Complete VSIL also provides useful tool for
• Post deployment command testing
• Post deployment SW change testing
• Anomaly/mishap analysis problem solving

13
Potential Contributions (Contd)

• Reduce project development costs time by doing
  bulk of integration testing in VSIL during SW
  development phase

14
Project Status

• Project started in April
• ES-based simulator operational
• System-level tests in development

Next Steps

• Create DE and write control SW
• Verify control SW passes system tests
• Collect dynamic metrics

15
Questions?
Please stop by Friday for a demo
16
References

• 1 Lutz, Robyn R. 1994. Analyzing Software
  Errors in Safety-Critical, Embedded Systems. Jet
  Propulsion Laboratory, California Institute of
  Technology, Pasadena, CA.
• 2 Ellis, A. 1995. Achieving Safety in Complex
  Control Systems. Proceedings of the
  Safety-Critical Systems Symposium. pp. 2-14.
  Brighton, England. Springr-Verlag. ISBN
  3-540-19922-5
• 3 Leveson N. G. 1995. Safeware - System,
  Safety and Computers. Addison Wesley 1995. ISBN
  0-201-11972-2