Software Failures

1
Software Failures

• Ron Gilmore, CMC
• Edmonton
• April 2006

2
Software Failures

• Santayana
• The software sector
• Observations
• Case Study Therac 25
• Lessons
• Engineering Comparisons
• Challenges

3
Santayana (1863 - 1952)

• Philosopher, essayist, poet, novelist
• The Life of Reason (1905)
• "Those who cannot remember the past are condemned
  to repeat it
• Lots of other great quotes
• Egypt, March 2006

4
Software Sector

• Young less than a century
• Amateurs
• Change, churn and failures
• Compare to roads, houses, bridges
• Professions evolving
• Standards evolving
• Best practices evolving
• Societal awareness evolving

5
Case Study Therac 25

• Radiation therapy machines
• Atomic Energy of Canada
• 1985 to 1987
• Six known incidents
• Massive radiation overdoses to patients
• Order of tens of thousands of rads
• At least five deaths!

6
Therac 25 Root Causes

• Institutional causes
• No independent code review
• Software not included in reliability design
• Documentation lean on error codes
• AECL did not initially believe complaints

7
Therac 25 Root Causes

• Design Issues
• No preventative hardware interlocks
• AECL re-used software from older models which had
  hardware interlocks
• No way for software to verify sensors were
  working
• Arithmetic overflow - safety checks bypassed
• Software written in assembly language

8
Therac 25 Lessons?

• Professions?
• Standards?
• Best practices?
• Societal awareness?

9
Engineering Comparisons

• More mature sector
• Certification, legislation, compliance
• Curriculum Tacoma Narrows Bridge
• Still London Pedestrian bridge
• Still Confusion re mandate, coverage
• Still budget schedule - oilsands

10
Challenges

• Education technical, business
• Sensitivity bad software can kill!
• Lots more examples
• Chinook helicopter
• Missile detection systems

11
Constructive Notions

• Awareness efforts
• Consequences
• Core competencies
• Systems classifications
• A Life threatening
• B Business threatening
• C Other