Why Projects Fail

1
Why Projects Fail

• Lecture 2 what you can do about it
• Martyn Thomas
• Visiting Professor, OUCL

2
The Problemlast week, I said .

• Developing dependable software is difficult
  because of the size and complexity of real-life
  systems.
• The software industry is amateurish and immature.
  Most significant projects overrun dramatically
  (and unnecessarily) or totally fail.
• In next weeks lecture, I will explore why some
  failures have occurred, show you what a real-life
  zero-defect project looks like, and talk about
  what you need to know if you are to become a
  professional amongst all these amateurs.

3
Some failures

• Ariane V rocket
• Therac 25 radiotherapy
• Torpedo
• Buffer overflow security attacks
• Most high-profile project failures

4
Ariane V European Space Agency launch vehicle
5
Ariane V Explosion

• Initial launch exploded
• Failure traced to the inertial navigation system
  (INS).
• Overflow on conversion from 64-bit floating to
  16-bit integer exception not trapped
• primary and back-up INS both failed for the same
  reason, and stopped
• loss of INS led to auto-destruction.

6
Ariane V cause of failure

• INS software re-used from Ariane IV
• Ariane IV flight profile guaranteed this
  parameter could not overflow
• Ariane V specification was different, in a way
  that affected the requirements for the INS.
• Formal specification would highlight that the
  preconditions for re-using this module were not
  satisfied.

7
Radiotherapy
8
Therac 25

• (not the system on the previous slide)
• A system for treatment of tumours
• Mode 1 low energy electron beam treatment
• Mode 2 very high energy beam (25MeV) with a
  thick metal plate in front, for X-rays.
• Therac-20 had a mechanical switch to change beam,
  and an interlock to stop change to high energy
  without the plate.
• Therac 25 interlock was in software.

9
Therac-25 User Interface

• Set up treatment time
• Electron beam, type e
• X-ray beam, type x.
• System puts the plate in place before switching
  beam to X-rays.
• System Beam Ready, Operator types b to start
  treatment.
• Operator station in a different room from the
  patient, to protect staff from radiation

10
Therac Accident

• Ray Cox, oil worker, on the table for his regular
  e-beam treatment for a tumour on his shoulder.
• Operator goes to the other room
• types x, realises mistake, types ? edit, e,
  enter - all within 8 seconds. System says
  Malfunction
• cleared the error, got beam ready and hit b
• same error message, so tried again. Twice.
• Ray felt a painful jolt - not like previous
  treatments. Shouted in pain but no-one heard.
  Third time he got off the table and went to find
  the nurse.

11
Therac what went wrong?

• The operators actions exposed a race-condition
  in the (multi-tasking) code.
• The result was a full-power beam without the
  plate in place. 125-fold overdose!
• The particular sequence of actions had never
  occurred in testing.
• Made worse because audio intercom and video link
  both out of service. System error messages not
  informative (and usually meant treatment had not
  occurred).

12
Therac Failings

• Safety Case claimed 10-11 probability for
  computer selects wrong energy. No evidence for
  the claim.
• No low-complexity protection system (fuse and/or
  interlock).
• Poor software engineering.
• Poor investigation of reported accidents.
  Manufacturer did not consider possible software
  fault until several accidents

13
Nancy Levesons Torpedogaps in the specification

• How to stop a torpedo blowing up the launch ship?
• If it malfunctions or starts to come back
• sink it
• blow it up
• On live test, a torpedo failed whilst still in
  the torpedo tube

14
Buffer Overflows

• A very long string is passed to an interface,
  overwriting code or data.
• The corruption is designed to give control to the
  attacker.
• The problem is caused by the insecure design of
  the C programming language, in particular by a
  weak type structure and lack of bound checking.

15
Most project failures

• Most project failures happen because the
  requirements change, or are discovered to be
  wrong, late in the development or in service.
• Almost always, the information was available at
  the start of the project.
• Better methods for defining specifications and
  analysing them can avoid these problems.
• The agile approach of replacing specifications
  by Use Cases and Stories is dangerous lunacy.

16
Specifications

• Form must mirror the structure of the real world
  requirement. Principles of Program Design,
  Jackson, AP 1975
• Notation must support unambiguous statement of
  requirements and reasoning about properties. Only
  then can you understand the system, or the impact
  of changes.
• Mathematical foundations are essential - Alan
  Turing knew this in the 1940s.

17
Abstraction

• The two most important characteristics of a
  specification notation are (1) that it should
  permit problem-oriented abstractions to be
  expressed
• and (2) that it should have rigorous semantics
  so that specifications can be analysed for
  anomalies and to explore system properties.
• In this connection it might be worthwhile to
  point out that the purpose of abstracting is not
  to be vague, but to create a new semantic level
  in which one can be absolutely precise. Dijkstra
  1972

18
The Path to Security Assurance

• Randolph Johnson
• National Security Agency
• drjohns_at_orion.ncsc.mil

19
TOKENEER ID Station
20
OVERVIEW- Correct by Construction (C by C)
Process

• A software engineering process employing good
  practices and languages
• SPARK (Ada 95 subset with annotations)
• math based formalisms (Z) at early stages for
  verification of partial correctness.
• A supporting commercial toolset (Z/Eves,
  Examiner, Simplifier, Proof Checker) for
  specifying, designing, verifying/analyzing,
  developing safety or security critical software.

21
C by C S/W ENGINEERING PROCESS

• Seven Software Engineering Steps to High
  Assurance Security Software
• Requirements Analysis
• Security Analysis
• Specification
• Design
• Implementation
• System Test
• Assurance

22
7 Step Process (continued)

• Design (w/ (semi)formal documents)
• Refined functional spec (written in Z)
• INFORMED design document
• Details data store and flow, dependencies of
  modules (SPARK packages), etc.
• Links design statements to implementation
  modules-straightforward and tool supported
• Provides baseline orthogonal documents for
  developers and testers functional specs(Z),
  design docs (Z for behavior and environment
  dependencies), test specs

23
The Development Approach

• Requirements Analysis Step (REVEAL approach)
• Identify system boundaries
• Clarify dependencies on environment
• Security Analysis
• Develop Security Target Security Policy Model
  (CC) using Protection Profile
• Identify key properties to ensure security
• Validating functional spec with security
  properties
• Specification
• Define and document customer requirements in Z
  and English with customer feedback

24
Development Approach (continued)

• Implementation
• Coding in SPARK Ada with static analyzer
  (EXAMINER)
• prevents uninitialized variables, buffer
  overflows, incorrect info flows
• System test
• Incremental builds with increasing functionality
• Specification based testing done by SPRE
• 100 statement coverage tests
• 100 branch coverage tests

25
Process Summary
26
Assurance Process
27
Statistics of System
28
Experience with Strong Software Engineering

• Software development based on formal
  specification, strong static analysis, ISO 9001
  mature processes
• 0.1 - 1 faults / 1000 lines of software
• 1M lines 100 - 1000 faults on delivery instead
  of 6000 - 30000
• 100-fold improvement at no extra development
  cost!
• sources
• Pfleeger Hatton, IEEE Computer, pp33-42,
  February 1997
• Amey, http//www.sparkada.com/downloads/Mar2002Am
  ey.pdf

29
Formal methods can reduce risk, save money, and
deliver successful projects

• Z is more efficient at finding faults than the
  most efficient test phase
• http//www.sparkada.com/downloads/sholis.pdf
• compelling evidence that development methods
  that focus on bug prevention rather than bug
  detection can both raise quality and save time
  and money.
• http//www.sparkada.com/downloads/Mar2002Amey.pdf

30
But get the basics right first

• First ISO 9001 and/or CMM Level 3
• now you are under control
• ThenFormal methods for requirements
• now you really understand what you are trying to
  do
• Then ...Implementation using well-defined
  languages with formal annotations
• for example, SPARK

31
Two reasons for using Formal Methods

• Formal methods reduce development risks and save
  development budget
• FMs are cheaper
• Formal methods lead to far fewer delivered bugs
• delivered software is much higher quality
• the extra quality is free
• support costs are much reduced
• you can even guarantee the software

32
Summary

• We can build complex, dependable systems - but
  only if we always use strong software
  engineering.
• System and software engineering must be rigorous,
  disciplined, conservative and evolutionary,
  learning from what has worked dependably in the
  past. Formal methods are fundamental to strong
  software engineering.
• It will take numerate, motivated graduates to
  change things for the better you.