ContractBased Justification for COTS Component within SafetyCritical Applications

1
Contract-Based Justification for COTS Component
within Safety-Critical Applications

• Fan Ye, Tim Kelly
• Department of Computer Science
• University of York, York, UK
• fan.yetim.kelly_at_cs.york.ac.uk

2
Outline

• COTS and CBS
• COTS in Safety-Critical Systems
• Safety Case Contract
• Contract Establishment
• Contract Satisfaction
• How Safety Case Contracts can be Used
• Conclusion

3
COTS and CBS

• COTS Commercial-Of-The-Shelf
• Related terms MOTS, GOTS, NDI, SOUP OTS
• A COTS product is standard commercial software
  developed without any particular application in
  mind
• COTS-Based Systems (CBS) are composed of
  off-the-shelf parts integrated to achieve
  new/expanded system functionality (SEI)
• Rationale for COTS use better, cheaper and
  faster
• Software crisis inability to deliver a system
  of required quality, on time, within budget, if
  the system is delivered at all
• Increased pressures complexity, functionality,
  time-to-market Flexibility of evolving the
  system over the time
• In favour of reuse advances of the technology
  on CBSE (e.g., CORBA, DCOM, Java Beans) growing
  market of COTS

4
Use of COTS in SC Context

• Safety-critical systems development is guided by
  standards
• Achievement of safety by following suitable
  development processes
• Demonstration of the achievement by means of
  certification
• Safety case safety objectives, argument
  structure, evidence
• Challenges regarding to COTS use in SC context
• How to establish evaluation criteria
• Take application-specific safety consideration
  into account
• Ensure a selected COTS component doesnt
  compromise overall system safety
• Lack of evidence to support safety argument
• Limited visibility (limited access to development
  process description, design documentation, source
  code, fault histories)
• Safety assurance under change (e.g. COTS
  upgrades)
• Lack of systematic approach

5
Compositional Safety Case

• An attempt to establish a modular, compositional,
  approach to constructing safety cases that has a
  correspondence with the structure of the
  underlying system architecture
• The need for modular safety case construction
• Systems of Systems
• Integrated Modular Avionics
• OTS components designed for use in the Safety
  Critical Sector
• Safety Case Modules usefully composed if
• Objectives and Arguments complement each other
• Collective Evidence and Assumed Context
  Consistent
• E.g. Operational Usage Context must agree
• Where two or more modules can be composed, it is
  useful to record agreement
• Safety Case Contracts
• (Contracts already understood in S/W Engineering)

6
Safety Case Contract the concept 1/3

• Strength of Safety Arguments
• Both strong and weak safety arguments exist
• Arguments can be weak due to insufficient weight
  and poor relevance of evidence with respect to
  the objectives.
• Safety Assurance Level (SAL) the level of
  confidence that a safety argument element (goal
  or solution) meets its objective.
• Safety Case Contract
• To record the agreed safety relationship between
  the application and the COTS component
• Goal / claim, SAL, context

7
Safety Case Contract establish. 2/3

• Contract establishment

• STEP one identify application safety objectives
• How? Component Criticality Analysis

• STEP two fill in COTS claims
• Based on evidence provided by vendor (during COTS
  selection)

• STEP three refine both sides of the contract
• During COTS evaluation with real COTS at hand
• Application requirement complete? how about
  extra functions
• Vendor evidence credible?

STEP two
STEP one
three
STEP
8
Safety Case Contract satisfaction 3/3

• Contract satisfaction
• Matching COTS claims with application objectives
• Mismatch handling
• Goal mismatch
• SAL mismatch
• Context mismatch

Goal Mismatch
9
Contract Establishment app side 1/3

• Component Criticality Analysis
• System hazards analysis
• Identify hazards
• Determine severity catastrophic, critical,
  marginal, negligible
• COTS component failure modes identification
• HAZOP over preliminary system architecture
• FFA over COTS component functionality
• Fault tree analysis
• Establishing causal relationship
• Minimal cut set analysis identifies the level of
  protection
• Criticality determination

• Criticality Determining factors
• Hazard Severity Level
• Failure Probability
• Software Control Category
• Degree of Protection

10
Contract Establishment app side 2/3

• From criticality analysis results to contract
  terms
• Goal hazardous COTS component failure mode
• e.g. failure mode X shall not occur
• SAL criticality level for a failure mode
• The higher the criticality level, the higher the
  assurance level
• e.g. if failure mode X is of criticality level 3,
  then the goal failure mode X shall not occur
  should be assured to SAL 3
• Context assumption

11
Contract Establishment both-side 3/3

• Contract refinement with real COTS products at
  hand
• Are the application-side safety requirements
  complete? Unintentional interactions
• Unused COTS functionality
• Intentional communication mechanisms
  communication channel
• Unintentional communication mechanisms shared
  resources e.g. CPU time, memory.
• COTS-side safety claims establishment
• Validating evidence provided by vendor
• Examine how COTS component interact with rest of
  the system
• Confirm assumptions made about COTS
• Confirm functional and non-functional claims

12
CCA An Example 1/4

• An Example Eurofighter OSC System
• a database to hold the aircrafts system models
  and fault tree models
• facilitate enquiries about the risks of the
  failure of one or more functions

13
CCA An Example 2/4

• System Hazard OSC produces incorrect results
  indicating lower than actual risk associated with
  the failure of Function X (Hazard A)
• Hazard Severity Catastrophic
• COTS FMs e.g. fail to add would result in data
  source incomplete thus the system hazard

14
CCA An Example 3/4

• COTS DB failure modes

15
CCA An Example 4/4

• COTS FM Criticality e.g.
• Fail to add C4
• Fail to retrieve C4
• Fail to update C4

• What have been learned
• DB component highly critical

16
Contract Satisfaction 1/4

• Type of argument support
• Single support
• One child element satisfies the entire parent
  goal
• Linked support
• Several child elements interdependently satisfy
  the parent goal
• Convergent support
• Several child elements each separately satisfies
  the parent goal

17
Contract Satisfaction 2/4

• Direct match
• Application requirements can be readily supported
  by COTS claims with evidence available
• Single support

18
Contract Satisfaction 3/4

• Indirect match via Mitigation
• Context mismatch
• Goal and SAL mismatch
• Trade off between Goal and SAL
• Resolve goal mismatch through linked support
• Resolve SAL mismatch through convergent support

• Example of SAL mismatch handling
• Application requirement absolute non-occurrence
  of a COTS failure mode (SAL 4)
• COTS claim this can only be assured to SAL 2
  with evidence available
• Introduction of mitigation at application context
  to detect and handle COTS failure

19
Contract Satisfaction 4/4

• Mitigation strategies
• Acquire additional evidence
• Press for additional evidence from vendor
• 3rd party assessment
• Black-box testing functional, statistical,
  operational testing
• Application fault tolerance
• Fault tolerant architecture
• Design diversity / NVP (e.g. Controller
  Monitor)
• Component containment / COTS wrapping
• Mitigation according to component failure mode
  types

20
Use of Safety Case Contract

• Facilitate evaluation and selection of an
  appropriate COTS product
• Safety case construction
• Final COTS-based safety-critical application
  certification
• Safety maintenance in case of COTS upgrading
• The points of concern have already been
  identified
• Check if the safety case contract still upheld

21
Conclusion

• Use of COTS in safety-critical context desirable
• Challenges
• Compositional safety case construction
• Safety case contract
• Contract establishment
• Criticality analysis to establish evaluation
  criteria
• Contract satisfaction
• Direct match
• Mismatch handling
• Uses of the safety case contract