CSC2108: Automated Verification or Everything you Wanted to Know about Model-Checking

1
CSC2108 Automated Verification orEverything you
Wanted to Know about Model-Checking

• Instructor Marsha Chechik
• Contact chechik_at_cs.toronto.edu
• Bahen 3248, 416-978-3820
• TA Shiva Nejati
• shiva_at_cs.toronto.edu
• Web page http//www.cs.toronto.edu/chechik/cours
  es07/csc2108

2
Software Engineering and Formal Methods
3
Motivation

• To err is human but to really foul things up
  requires a computer
• Farmers Almanac for
  1978
• Capsules of
  Wisdom
• 60 of all major industrial disasters from 1921
  to 1989 occurred after 1975
• Nancy Leveson
• Safeware Computers and Technology

4
Example auto-pilot

• Problem
• Design a part in auto-pilot that avoids
  collision with other planes.
• Solution
• When distance is 1km, give warning to other
  plane and notify pilot. When distance is 300m,
  and no changes in the course of other plane were
  noticed, go up to avoid collision

5
Problem with solution

• Both planes have the same software. Both go up...

6
This happens in real software!

• Some famous bugs
• NASA Space Rover, Intel floating point processor,
  etc.
• Hard to predict all behaviours!
• US aircraft went to southern hemisphere and
  flipped when crossing the equator
• Air traffic controller US to Britain.
• It never dealt with problem of 0 degrees
  longitude.
• Result software folded Britain along Greenwich
  Meridian
• Software written for US F-16
• accidents when reused in Israeli aircraft flown
  over the Dear Sea
• (altitude lt sea level)
• Year 2000 problem

7
Yet more such examples

• NASA Space Shuttle software (in use since 1980)
• 16 severity-level 1 software errors
• 8 remained in code that was used in flights
• none encountered during flights
• total size - only 400,000 words

8
So, why not test?

• Testing only shows presence of bugs not their
  absence!

9
Formal Methods

• Mathematically-based techniques for describing
  properties of systems
• Provide framework for
• Specifying systems (and thus notion of
  correctness)
• Developing systems
• Verifying correctness
• Of implementation w.r.t. the specification
• Equivalence of different implementations
• Reasoning is based on logic
• Amenable to machine analysis and manipulation
• In principle, can verify everything that is true
  in the system!
• Given enough time, skill and patience

10
Why arent FMs used more?

• Formal methods can revolutionize development!

Formal methods are difficult, expensive, not
widely useful and for safety-critical systems
only
11
and one more problem

• Need to know what to build (specification) before
  you start building

water-fall model

• Unrealistic!
• May need to discover what to build iteratively
• Software changes all the time

12
Formal Methods Light

• Partial application of formal methods
• only parts of systems are specified
• Emphasis on analysis of some properties
• security, fairness, deadlock freedom, rather than
  complete verification
• Debugging rather than assurance
• Automation
• Most successful lightweight technique
• Model-Checking

13
Model-Checking
14
Overview of Automated Verification
Correct?
15
CTL Model-Checking

• CTL Branching-time propositional temporal logic
• Model - a tree of computation paths
• Example
• Kripke Structure
• Tree of computation

S1
S2
S3
16
Models Kripke Structures

• Conventional state machines
• M ltS, A, s0, I , Rgt
• S is a (finite) set of states
• A is a (finite) set of propositional
• variables
• s0 is a unique initial state (s0 ? S)
• I S ? 2A is a labeling function that maps each
  state to the set of propositional variables that
  hold in it
• R ? S ? S is a (total) transition relation

17
Propositional Variables

• Fixed set of atomic propositions p, q, r
• Atomic descriptions of a system
• Printer is busy
• There are currently no requested jobs for the
  printer
• Conveyer belt is stopped
• How to choose them?
• Should not involve time!

18
CTL Computation Tree Logic

• propositional temporal logic.
• allows explicit quantification over possible
  futures
• Syntax
• True (?) and False (?) are CTL formulae
• propositional variables are CTL formulae
• if? and ? are CTL formulae, then so are? ? , ?
  ? ? ,? ? ?
• EX ? --- ? holds in some next states
• EF ? --- along some path, ? is true in a
  future state
• E? U ? --- along some path, ? holds until ?
  holds
• EG ? --- along some path, ? holds in every
  state
• Universal quantification AX ? , AF ? , A? U
  ? , AG ?

19
Examples
20
Examples, Contd
?
?
?
?
AF (all future)
EF (exists future)
?
?
?
?
?
?
?
?
?
AU (all until)
EU (exists until)
21
CTL (Contd)

• Examples
• Properties that hold
• (EX p)(s0)
• (Ap U q)(s0)
• ( EX AF p )(s0)
• Properties that fail
• (A? p U q)(s0)

22
Some Statements To Express

• It is possible to get to a state where started
  holds, but ready does not hold
• EF (started ? ?ready)
• When a request occurs, it will eventually be
  acknowledged
• AG (request ? AF acknowledge)

23
CTL Model-Checking

• Receive
• Kripke structure K
• Temporal logic formula ?
• Assumptions
• Finite number of processes
• Each having a finite number of finite-valued
  variables
• Finite length of a CTL formula
• Algorithm
• Label states of K with subformulas of that ?
  are satisfied there and working outwards towards
  ?.
• Output states labeled with ?
• Example EX AG (p ? Ep U q)

24
CTL Model-Checking (Contd)

• EX ?
• Label any state with EX ? if any of its
  successors are labeled with ?

• AF ?
• If any state s is labeled with ?, label it with
  AF?
• Repeat
• label any state with AF?
• if all of its successors
• are labeled with AF?
• until there is no change

25
Counterexamples

• Explain
• Why the property fails to hold

• to disprove that ? holds on all elements of S,
  produce a single element s ? S s.t. ?? holds on
  s.
• counterexamples restricted to universally-quantifi
  ed formulas
• counterexamples are paths (trees) from initial
  state illustrating the failure of property

• AG p

26
Symbolic model-checking

• Idea of model-checking
• recursively go through the structure of the CTL
  property
• associating each subformula with a set of states
  where each subproperty is true
• Symbolic model-checking
• effective cure for state explosion problem
• use symbolic representation for sets of states
• use symbolic representation for transition
  relation
• use binary decision diagrams (BDDs) to encode
  these
• Example
• x?y in classical logic

27
Questions

• Where does the Kripke structure come from?
• Computing it from several concurrent processes
• Models of concurrency maximum parallelism,
  interleaving
• What happens if Kripke structure is infinite?
• Abstraction!
• How to represent large Kripke structures?
• Abstraction!
• Symbolic encoding of transition relation
• SAT-based encoding (BMC)
• Symmetry (not part of the course)
• Compositional verification
• What other methods are there?

28
2108 Automated Verification

• A course on model-checking technology
• Theory
• Different temporal logics (CTL, LTL, mu-calculus)
• Foundations
• Automata theory (automata on infinite strings,
  tree automata, alternating tree automata),
  lattice theory, fixpoint computations
• Introduced as needed!
• Different model-checking approaches
• Explicit-state, Symbolic, Bounded (BMC)
• Game-theoretic
• Automata-theoretic
• Tools
• SMV, SPIN, Concurrency workbench

29
Automated Verification (Contd)

• Making this work
• Decision diagrams, SAT solvers
• Compositional verification
• Partial order reduction
• Abstraction and refinement
• Some applications
• Debugging
• Requirements monitoring
• Test case generation
• Extensions
• Query-checking
• Model-checking over partial and inconsistent
  systems
• Etc.

30
2108 Pragmatics

• Time Wednesdays, 2-5 p.m.
• Readings
• Book
• Clarke, Grumberg, Peled. Model-Checking. MIT
  Press, 1999
• Many research papers
• Course work
• 2-3 small modeling exercises
• Regular (small) paper-and-pencil homeworks
• Research project
• Possibly presentation of a topic related to
  model-checking
• Prerequisites
• Discrete mathematics, general inclination towards
  theory
• Prior experience with concurrent systems a plus

31
Pros and Cons of Model-Checking

• Often cannot express full requirements
• Instead check several smaller properties
• Few systems can be checked directly
• Must generally abstract
• Work better for certain types of problems
• Very useful for control-centered concurrent
  systems
• Avionics software
• Hardware
• Communication protocols
• Not very good at data-centered systems
• User interfaces, databases

32
Pros and Cons, Contd

• Largely automatic and fast
• Better use for debugging
• rather than assurance
• Testing vs model-checking
• Usually, find more problems by
• exploring all behaviours of a downscaled
  system
• than by
• testing some behaviours of the full system

33
Readings

• List of readings on the web site
• Chapter 1 of the textbook
• Several papers on formal methods in general.
  Links from the reading list on the course web
  page
• For next week
• Chapter 3 of the textbook