Impact of Computers on Society

1
Impact of Computers on Society

• 4. Can We Trust the Computer?
• Errors, Failures, Risk

2
Two Main Areas of Concern

• Reliability
• Accuracy
• Combined with
• Hardware
• Software

3
Some Failures

• ATT phone system shutdown, January 1990
• Subsequent smaller outages
• Caused by software flaws, hardware changes
• Airport baggage systems in Denver and Hong Kong
• CTB/McGraw-Hill educational testing, 1999
• FBI abandons criminal tracking DB, 2005
• IRS
• FAA Air Traffic Control
• Strategic Defense Initiative (aka Star Wars)
• Two major new systems at NSA do not work (2006)
• NSA nearly overloads local power grid (2007)

4
Have you experienced a failure?

• Your own computer?
• A business system?
• An educational system?
• A governmental system?

5
Program Correctness

• Ignore hardware for the moment
• Can you guarantee program correctness?
• Fairly easy to prove incorrectness
• Theoretically impossible to prove correctness
• A simple program with just 10 branches or
  decisions would require 10! tests thats
  3,628,800
• Exhaustive testing can only improve the odds

6
Three levels of failure

• Individual end user or consumer -- affects only
  that person
• System failure affects many, perhaps millions
• Critical failure may result in loss of life

7
Failures Databases

• Computer error is generally human error
• Law enforcement databases
• Not up to date
• Inaccuracies
• Sometimes presume guilt
• City of Frederick Maryland Police
• Sen. Edward Kennedy and others flagged by the air
  travelers No-Fly DB.
• Md. State Police spied on protestors and
  activists
• National Crime Information Center

8
Failures Systems

• Communications (ATT already mentioned)
• Business, financial, transportation
• FBI, FAA and others have abandoned systems
• System must be able to handle errors
• Question any sudden change in output
• James Stewart, nuclear engineer non-linear
  errors
• The problem of computational complexity
• A host of sources of errors
• MSD too cheap to buy a backup drive
• WordPerfect for DOSfloppy disc full

9
Failures Software Reuse

• The Object-Oriented programming paradigm
• Libraries of pre-coded reusable components
• Should increase reliability, but not always
• Designed for one environment, but used in another
• The Ariane 5 rocket overflow

10
Overflow/Underflow an example

• A byte has 8 bits, which equals 256
• Computers count from 0 to N-1
• What is the next number?
• 000 0000 0000
• 001 0000 0001
• 002 0000 0010
• 003 0000 0011
• And so forth
• 253 1111 1101
• 254 1111 1110
• 255 1111 1111
• 256 ??
• 256 1 0000 0000

11
Failures User Interfaces

• An entire course titled HCI
• Think of a device you hate
• Therac-25
• The max/min/close buttons in MS Win vs. Mac OS X
• Gauges in aircraft in WWII
• Inconvenient menu structures

12
Preventive Measures

• Exhaustive testing
• Remember that you can never prove there are no
  flaws
• Unforeseen problems and flaws may emerge later
• Over-engineering design for greater capacity
  than needed
• Systems redundancy the US space shuttle

13
Some protection against failure

• Legal recourse
• Very expensive for both parties
• Can put a firm out of business
• Cannot undo a wrong
• Warranties
• Shrink-wrap warranties very one-sided
• Sometimes difficult to get satisfaction

14
Additional protections

• Regulation
• Requires an agency to oversee
• Regulations are usually written after the fact
• Slows the introduction of new technology
  expensive and time-consuming
• Sometimes affected by political concerns
  influence of large donors

15
More Protection

• Professional licensing
• CS is not yet a true profession
• An accrediting organization (ACM, IEEE)
• A competency exam or apprenticeship
• Requirement for continuing education
• A recognized license
• An enforced code of ethics
• An adjudication process

16
Somebody Must Take Responsibility

• Responsibility shared between consumer/user, the
  vendor/integrator, and the company/developer
• RTM?
• Make realistic claims about the technology
• Provide a clear means of communication
• An area still in flux

17
Put Things in Perspective

• Its not all catastrophic some errors are minor
• You cant guarantee anything 100 of the time
• Some problems are old ones in a new guise
• What is an acceptable error rate?
• Consider how much damage was done?
• A document that didnt print correctly
• A massive failure involving thousands of
  individuals and businesses
• Injury or loss of life

18
How Dependent Are We?

• VERY!
• Can you think of something you used today that
  did not involve a computer at some stage of its
  design, manufacture, or use?

19
Computer Models

• A mathematical model of a real-world process
• Link Trainers in WWII
• Modern aircraft trainers
• Population growth
• Vehicle accident simulation
• Aircraft design the Boeing 777
• Climate models
• Chaotic systems
• The butterfly effect a small change in input
  results in a large difference in output

20
Limits on Models

• Computational complexity
• Unknown factors
• Use of approximation
• Incomplete data
• Incorrect data

21
Four Considerations when Creating a Computer Model

• How well is the process (the thing to be
  simulated) understood?
• How good are the data?
• What simplifications have been made?
• If possible, how do the results compare with
  observed reality?

22
Dont exaggerate

• People have a tendency to exaggerate
• People have a tendency to play down negative
  aspects
• If it sounds too good to be true