Certification Issues - PowerPoint PPT Presentation

1 / 32
About This Presentation
Title:

Certification Issues

Description:

Failure Prevents Continued Safe Flight and Landing. Level B ... Greenwich Meridian. Use of Service History for Certification. 9. Use of Service History ... – PowerPoint PPT presentation

Number of Views:42
Avg rating:3.0/5.0
Slides: 33
Provided by: georger8
Learn more at: http://www.sigada.org
Category:

less

Transcript and Presenter's Notes

Title: Certification Issues


1
(No Transcript)
2
Safety Critical SystemsandCertification
IssuesDO-178BAirborne Standard
3
SC190 / WG-52 Application Guidelines For
RTCADO-178b/ED-12b
RTCA
EUROCAE
SC-167
WG-12
AvionicsIndustry
CNS/ATM Community
SC-190 / WG-52
Cast Position Papers
CAST (Certification Authority Software Team)
SC-190 Products
4
DO-178B in a Nutshell
  • Highly Process Oriented
  • Requires Traceability
  • Requirements
  • High Level Design
  • Detailed Design
  • Source Code
  • Test Procedures
  • Test results
  • Test Test Test Test and ..

5
DO-178B Safety Levels
  • Level A
  • Catastrophic
  • Failure Prevents Continued Safe Flight and
    Landing
  • Level B
  • Hazardous/Severe-Major
  • Potential Fatal Injuries to a Small Number of
    Occupants
  • Level C
  • Major
  • Discomfort to Occupants or Possible Injuries
  • Level D
  • Minor
  • Increased Crew Workload

6
Development Team
  • What is COTS?
  • How can objectives of DO-178B be satisfied, using
    COTS?
  • There is much variation in applicants for COTS
    certification credit
  • Is DO-178B clear on the interpretations?

Certification Evidence Available as COTS
Product is COTS

Together these satisfy safety objectives
7
Testing without Source Code
Wrappers to Validate Parameters
Commercial O/S
Application
This cannot be trusted unless O/S is verified
8
Special Considerations
Use of Service History for Certification
Air Traffic Control system
  • Worked Correctly in US for years
  • Transferred to U.K.

Actual Plane 2
Plane 1
Displayed Plane 2
Greenwich Meridian
9
Use of Service History
Developed under a less stringent standard
(Military?)
Used for 4 years Problems tracked Quality Good!
Safety Critical System
Dead Code(Unintended Function)
Residual Error
Is this system safe for the next 4 years? At
Level A, B, C? We can bound inputs, but we
cannot check internal states without looking
inside
10
Black Box Testing
  • No single failure should prevent Continuous safe
    flight and landing.
  • Statistical testing cannot show absence of a
    single state that will cause a failure
  • Software has discontinuities
  • Software does not follow Gaus/Normal Distribution

There is no foundation for statistical reasoning
about software faults or safety
11
Verification Teamexamples of issues
  • What are low level requirements? How can they be
    tested
  • Data and Control flow coupling
  • Use of higher level test results for lower level
    requirements
  • What is the intent of structural coverage?
  • Traceability of source to object code for
    structural coverage
  • What is statement, decision, condition and MCDC
    coverage testing (Modified Condition/Decision
    Code)
  • Verification tool qualification
  • etc..

12
Coverage Analysisat Level B and C
  • Statement Coverage
  • Decision Coverage
  • Entry Points
  • Exit Points
  • All Decisions
  • All Outcomes

13
Fixing anomaliesexample for level B/Library
Filter
B 3 A Filter (B) X X A
Compiler
Object Code
Source level coverage required
Even for Filter
14
Boundary Level testingnot enough!
A B -- A and B are packed Boolean arrays
Run-time call to Bit_Block_Move (From, To,
Size) -- size in bits
  • Min, Mid, Max in combination gives 27 (useless)
    test cases
  • 5 Bits size
  • 32 Bit size
  • 67 Bit Size
  • From overlaps To
  • To overlaps From

Interesting test cases based on actual
code i.e. White Box Testing
15
Coverage at Level A
  • Coverage required at Machine Code level or
  • Show source to object code traceability
  • and test at source level or
  • Use different language/compilers
  • and use voting system
  • MCDC testing required
  • each condition must have effect on outcome
  • Tools which modify source for traceability
  • problem at level A
  • Mitigation method use 3 different compilers
  • (Now Look At Conditional Statements)

16
Conditions/Decisions
Conditions
if AB and C or Dlt3 then
Decision
Boolean Variable
Boolean Operators
17
What areConditions/Decisions
MCDC Coverage Requires all Branches AND all
Conditions Be Covered
18
More BooleanConditions
Cannot hide fromTesting Obligations
and are Boolean Operators!
19
Condition coverage
Coverage of Basic-Block may not capture
condition results
20
Avoiding MCDC Testing
Modified Condition/Decision Code
Use Adas short-circuit conditions if A0 and
then Blt 2 and then Cgt5 then Or in C write if
A 0 B lt 2 C lt 5
21
Why short-circuitconditions eliminateMCDC
if A0 and then Blt 2 and then Cgt5 then
if A0 then if Blt2 then if Cgt5 then
P end if end if end if
MCDC not required for this code
22
Testing strategymust evaluate conditions
if A0 and then Blt 2 and then Cgt5 then
if A0 then if Blt2 then if Cgt5 then
P end if end if end if
BUT !!!
Testing must show thateach then part has been
tested True and False
MCDC not required for this code
23
Inlining code
  • If decisions/conditions introduced
  • Decisions must be identified and verified (level
    B)
  • Conditions must be identified and verified (level
    A)
  • Verification may be done by analysis
  • Traced to derived requirements
  • ensure safety is not compromised
  • Code may be deactivated
  • As inlined code depends on local state it may be
    very hard to test the conditions in accordance
    with standards requirements
  • Intent - absence of unintended funtion
  • Dead code not allowed

24
Use of Tools
  • Tool Qualification is required if tool replaces a
    step of development process
  • Development tools
  • Examples - Compiler, Design to code generator
  • May introduce an error
  • In general - NOT qualified, not trusted
  • Verification tools
  • Examples - Coverage analyser
  • May conceal an error
  • May be qualified - Trusted for verification
    purposes
  • Additional verification process required if the
    tool is not trusted

25
CNS/ATM Process Integration
  • Information matrix
  • Regulators
  • Committees
  • Standards Bodies
  • Standard
  • Software Integrity Assurance Standards vs.
    Software Development Standards
  • Relationships between DO-178B
  • and IEC 61508

26
Ground BasedCommunity
  • Communications/ Navigation/Surveilance and Air
    Traffic Management - in the loop
  • Ground Based Systems affect airborne software
  • DO-178B addresses airborne only
  • Guidance being prepared to encompass needs of
    CNS/ATM community (SC-190 committee)
  • Standards tightening up

27
Certification Standardsare Improving
  • Holes in documents are being fixed
  • Understanding of Certification Requirements is
    spreading
  • Industry and Certification Authorities are
    collaborating on Guidance materials
  • It will get more difficult to shop around for a
    more lenient signature

28
( Legal)Safety Systems
Case Law Precedence Interpretations Standards Guid
elines
Laws Regulations Standards Guidelines
PROCESS
Traceability
Visibility
EVIDENCE / RECORD
Confidence / Safety
29
When Is Software Safe
30
When Is Software Safe
We Dont Know !!
31
What is our best guessabout the safety
  • When applicable processes have been followed
  • When we have verified the code from within
  • When this has been checked
  • and checked
  • and checked
  • and checked
  • and
    checked

  • and checked

32
The FAA/JAA Rulesare Strict
To Date no fatalities have been attributed to
Software Failure
Have we been lucky? Have we been safe?
Write a Comment
User Comments (0)
About PowerShow.com