Extended Static Checking for Java or Light-weight formal methods: from objects to components

1
Extended Static Checking for JavaorLight-weight
formal methodsfrom objects to components
K. Rustan M. LeinoMicrosoft Research, Redmond, WA

• Joint work with Cormac Flanagan, Mark
  Lillibridge, Greg Nelson, James B. Saxe, Raymie
  Stata
• Compaq SRC

6 Nov 2002FMCO 2002, Leiden, The Netherlands
2
Objects and components

• Some technical problems
• Meanings of specifications
• Ingredients of formalisms
• How to build tools
• Other problems
• Cost-effectiveness
• Getting programmers to use the above

3
Formal methods
coverage
programverification
extended static checking
decidability ceiling
light-weightformal methods
typechecking
effort
Note Illustration not to scale
4
Users view
ESC/Java
Warning messages
Annotated Java program
public class Bag private /_at_non_null/ int
a private int n //_at_ invariant 0 lt n n lt
a.length public Bag(/_at_non_null/ int
initialElements) n initialElements.length
a new intn System.arraycopy(initialElement
s, 0, a, 0, n) public void add(int x)
if (n a.length) int b new
int2(a.length1) System.arraycopy(a, 0, b,
0, n) a b an x n public
int extractMin() int m Integer.MAX_VALUE
int mindex 0 for (int i 0 i lt n i)
if (ai lt m) mindex i m
ai if (0 lt n) n-- amindex
an return m // The program text
continues down here, but if youre // reading
this, you probably arent paying attention to //
the talk.
Bag.java18 Array index possibly too
large Bag.java45 Precondition possibly violated
5
ESC/Java distinguishing features

• Annotation language captures design decisions
• Powered by automatic theorem prover
• Not decidable
• Not sound or complete
• Performs modular checking

6
Method-modular checking

• Check that each method satisfies its
  specification, assuming that all called routines
  satisfy theirs
• Reason about implementation when it is written,
  not when it is used or extended

7
Demo
8
Design tradeoffs

• Missed errors
• Spurious warnings
• Annotation overhead
• Performance

9
Tool architecture
Annotated Java program
Translator
Verification condition
Valid
Automatic theorem prover
Resource exhausted
Counterexample context
Post processor
Warning messages
10
Tool architecture, detail
Annotated Java program
Sugared command
Translator
Primitive command
Passive command
Verification condition
Automatic theorem prover
Counterexample context
Post processor
Warning messages
11
Tool architecture, detail
12
Annotation language
Annotated Java program

• Simple
• non_null
• Method annotations
• requires E
• modifies w
• ensures P
• exsures (T x) Q
• Object invariants
• invariant E

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
13
Annotation language
Annotated Java program

• Simple
• non_null
• Method annotations
• requires E
• modifies w
• ensures P
• exsures (T x) Q
• Object invariants
• invariant E

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
14
Annotation language
Annotated Java program

• Specification expressions
• side-effect free Java expressions
• no , no method calls
• \result, \old(E)
• ensures \result \old(x)
• gt
• (\forall T x P), (\exists T x P)
• (\forall int j 0 lt j j lt n gt aj gt 0)
• \typeof(E), \type(T), lt
• requires \typeof(x) \typeof(this)

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
15
Annotation language
Annotated Java program

• Specification expressions
• side-effect free Java expressions
• no , no method calls
• \result, \old(E)
• ensures \result \old(x)
• gt
• (\forall T x P), (\exists T x P)
• (\forall int j 0 lt j j lt n gt aj gt 0)
• \typeof(E), \type(T), lt
• requires \typeof(x) \typeof(this)

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
16
Annotation language
Annotated Java program

• Concurrency
• monitored_by lock
• /_at_ monitored_by this / long x
• \locksetlock
• requires \locksetthis
• lock0 lt lock1
• \max(\lockset)
• requires \max(\lockset) lt this

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
17
Annotation language
Annotated Java program

• Concurrency
• monitored_by lock
• /_at_ monitored_by this / long x
• \locksetlock
• requires \locksetthis
• lock0 lt lock1
• \max(\lockset)
• requires \max(\lockset) lt this

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
18
Annotation language
Annotated Java program

• Ghost variables
• ghost public T x
• ghost public int objectState
• ghost public \TYPE elementType
• set x E
• set objectState Open
• set elementType \type(T)

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
19
Annotation language
Annotated Java program

• Ghost variables
• ghost public T x
• ghost public int objectState
• ghost public \TYPE elementType
• set x E
• set objectState Open
• set elementType \type(T)

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
20
Annotation language
Annotated Java program

• Miscellaneous
• assert E
• assume E
• assume x gt 0 // because x yy
• nowarn
• x aj //_at_ nowarn
• axiom E
• axiom (\forall int x x gtgt 2 gt 0)

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
21
Sugared commands
Annotated Java program

• S,T assert E
• assume E
• x E
• raise
• S T
• S ! T
• S T
• loop inv E S ? T end
• call x t.m(E)

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
22
Sugared commands
Annotated Java program

• x t.f.g
• assert t ! null
• tmp select(f, t)
• assert tmp ! null
• x select(g, tmp)
• if (x lt 0) x -x /_at_ assert x gt 0 /
• ( assume x lt 0 x -x
• assume !(x lt 0)
• )
• assert x gt 0

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
23
Sugared commands
Annotated Java program

• x t.f.g
• assert lblneg(Null_at_58.9, t ! null)
• tmp select(f, t))
• assert lblneg(Null_at_58.11, tmp ! null)
• x select(g, tmp)
• if (x lt 0) x -x /_at_ assert x gt 0 /
• ( assume x lt 0 assume lblpos(Then2807,
  true) x -x
• assume !(x lt 0) assume lblpos(Else2807,
  true)
• )
• assert x gt 0

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
24
Primitive commands
Annotated Java program

• S,T assert E
• assume E
• x E
• raise
• S T
• S ! T
• S T
• loop inv E S ? T end
• call x t.m(E)

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
25
Primitive commands
Annotated Java program

• //_at_ requires Pre modifies w ensures
  Postvoid m(U u)
• call x t.m(E)
• var u in
• u E
• assert Pre
• var w0 in
• w0 w
• havoc w
• assume Post
• end
• end

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
26
Passive commands
Annotated Java program

• S,T assert E
• assume E
• x E

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
27
Passive commands
Annotated Java program

• if (x lt 0) x -x /_at_ assert x gt 0 /
• ( assume x0 lt 0 assume x1 -x0 assume x2
  x1
• assume !(x0 lt 0) assume x2 x0
• )
• assert x2 gt 0

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
28
Weakest preconditions
Annotated Java program

• wp(assert E, Q) E ? Q
• wp(assume E, Q) E ? Q
• wp(ST, Q) wp(S, wp(T,Q))
• wp(S T, Q) wp(S, Q) ? wp(T, Q)
• wp(S, Q) wp(S, true) ? wlp(S, Q)
• wlp(S, Q) wlp(S, false) ? Q

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
29
Verification condition
Annotated Java program

• Universal background predicate
• (FORALL (t) (lt t t))
• Type-specific background predicate
• (lt T_T T_java.lang.Object)
• Verification condition BPUniv ? BPT ?
  VCmethod

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
30
Verification condition
(BG_PUSH (AND (lt T_T T_java.lang.Object)
(EQ T_T (asChild T_T T_java.lang.Object))
(DISTINCT arrayType T_boolean T_char T_byte
T_short T_int T_long
T_float T_double T_.TYPE
T_T T_java.lang.Object))) (EXPLIES (LBLNEG
vc.T.abs.2.2 (IMPLIES (AND (EQ
elems_at_pre elems) (EQ elems (asElems
elems)) (lt (eClosedTime elems) alloc) (EQ
LS (asLockSet LS)) (EQ alloc_at_pre alloc))
(NOT (AND (EQ _at_true (is x2.21
T_int)) (OR (AND (OR
(AND (lt x2.21 0) (LBLPOS
trace.Then0,3.15 (EQ _at_true _at_true))
(EQ x3.17 (- 0 x2.21)) (EQ
x2.21lt1gt x3.17)) (AND (NOT
(lt x2.21 0)) (LBLPOS
trace.Else1,3.4 (EQ _at_true _at_true))
(EQ x2.21lt1gt x2.21))) (NOT (LBLNEG
Assert_at_4.8 (gt x2.21lt1gt 0)))) (AND
(OR (AND (lt x2.21 0)
(LBLPOS trace.Then0,3.15 (EQ _at_true
_at_true)) (EQ x3.17 (- 0 x2.21))
(EQ x2.21lt1gt x3.17)) (AND
(NOT (lt x2.21 0)) (LBLPOS
trace.Else1,3.4 (EQ _at_true _at_true))
(EQ x2.21lt1gt x2.21))) (LBLNEG
Assert_at_4.8 (gt x2.21lt1gt 0)) (NOT
(LBLNEG Exception_at_5.2 (EQ ecReturn
ecReturn))))))))) (AND (DISTINCT
ecReturn)))
Annotated Java program

• class T
• static int abs(int x)
• if (x lt 0) x -x
• //_at_ assert x gt 0

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
31
Theorem prover Simplify
Annotated Java program

• Nelson-Oppen cooperating decision procedures
• conguence closure
• linear arithmetic
• partial orders
• quantifiers
• Key features
• automatic no user interaction
• refutation based searches for counterexamples
• heuristics tuned for program checking
• labels
• time limit

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
32
Counterexamples and warnings
Annotated Java program

• Counterexample labels (IndexTooBig_at_26.5
  vc.Bag.add.20.2 trace.Then0,21.23)
  context (AND (NEQ tmp1!a23.23
  null) (NEQ this null) (EQ alloc_at_pre
  alloc) (EQ tmp4!n26.6 0)
  (lt alloc (vAllocTime tmp3!a26.4)) )
• Bag add(int) ...--------------------------------
  ----------------------------------------Bag.java
  26 Warning Array index possibly too large
  (IndexTooBig) an x Execution
  trace information Executed then branch in
  "Bag.java", line 21, col 23.---------------------
  --------------------------------------------------
  -

Sugared command
Primitive command
Translator
Passive command
Verification condition
Automatictheorem prover
Counterexample context
Post processor
Warning messages
33
ESC/Java research platform

• Annotation inference
• Daikon Ernst, Houdini Flanagan Leino
• Other checking tools
• Calvin Qadeer et al., Stale-value concurrency
  checker Burrows Leino
• Formal verification without driving theorem
  prover
• Simplify theorem prover
• SLAM Ball, Rajamani, et al., oolong Leino et
  al.,
• Teaching
• Kansas State University Dwyer Hatcliff

34
Family of languages and tools

• ESC/Java
• Java Modeling Language (JML) Leavens et al.
• LOOP Jacobs et al.
• JML JUnit Cheon Leavens

35
Further researchChecking components

• Component-modular checking
• Fewer specifications
• More inference
• Specifying and checking side effects
• Modifies clauses (frame problem), alias
  confinement

36
Conclusions

• ESC/Java tool and platform for research
• JML family of specification languages and tools
• Ready for use by disciplined programming teams
  and in teaching
• Future challenges
• component-modular checking
• more specification methodologies
• Download ESC/Java research.compaq.com/SRC/escSo
  urces now available, too!