Runtime Verification of Software Systems

1
Run-time Verification of Software Systems

• Oleg Sokolsky
• Department of Computer and Information Science
• University of Pennsylvania
• January 21, 2003

2
Acknowledgements

• Collaboration
• Professors
• Insup Lee
• Sampath Kannan
• Students
• Mahesh Viswanathan
• Moonjoo Kim
• Usa Sammapun
• Support
• ONR, NSF, ARO

3
Outline

• Introduction
• Why I work in formal methods?
• Overview
• What is run-time verification, and why do we
  care?
• Main body
• A framework for run-time verification and its
  implementation
• Conclusions
• Beyond run-time verification

4
What are formal methods?
Behavioral properties, e.g. temporal logic
Models provide a higher-level view of the system
Check before you build
Correct by construction
Debug and fix errors
5
Advantages of formal methods

• Modeling as means of reuse and lifecycle support
• Analyze before you build
• Early detection of errors
• Correct by construction
• Implementation by refinement preserves properties
  of the model
• Analysis provides counterexamples
• Debug and fix errors easier

6
Do formal methods deliver?
Modeling is tedious and hard to get right
State explosion
Seldom goes all the way
Cryptic and large
7
Formal methods drawbacks

• Garbage in, garbage out
• Large detailed models are almost as hard to craft
  as code, and have less tool support
• Analysis is as good as the model
• Only small models can be exhaustively analyzed
• Rather than proving correctness, formal methods
  provide sophisticated debugging support
• Only simple models can be refined into code
• If implementation is not fully automatic, what
  have we learned from analyzing the model?

8
Is it hopeless? Of course not!

• What do we need?
• Better modeling languages
• More domain-specific, more declarative, better
  tool support
• Model-driven implementation techniques
• Model-driven code generation rather than
  refinement
• Model-driven test generation
• Novel applications of formal methods
• Run-time verification

9
Outline

• Introduction
• Why I work in formal methods?
• Overview
• What is run-time verification, and why do we
  care?
• Main body
• A framework for run-time verification and its
  implementation
• Conclusions
• Beyond run-time verification

10
Motivation

• Problem
• No matter what we do, the system can misbehave at
  run time
• We check models, not implementations
• We cannot always check the model exhaustively
• Solution
• Validate system executions at run-time

11
Advantages and disadvantages

• Check implementation, not model
• Avoid state explosion
• -
• Only one execution is checked
• Assess coverage metrics?
• And what if it breaks?
• Provide feedback for recovery?

12
What is run-time verification?
Check the same properties
Simpler checking algorithm
Extract only relevant information
Low-level execution data vs. high-level
requirements
13
Monitoring behavioral properties

• Formulas in a temporal logic
• Always evaluated over a finite execution trace
• Safety properties
• something bad does not happen
• Raise alarm when the bad happens
• Liveness properties
• Requires non-traditional interpretation
• Ultimately, properties determine what
  observations are relevant

14
Checking a property of a trace

• Satisfaction relationt
• Simple algorithm, linear in the trace length
• At each step, trace becomes longert
• Furthermore, traces are too big to store
• Need a different approach

15
Incremental checking of a trace

• In fact, we do not need to check the whole trace
  over and over again
• Keep a checker state
• values of all subformulas
• Upon each observation, update checker state

16
Requirements vs. observations

• System requirements are high-level and
  independent of an implementation
• Run-time observations are low-level and
  implementation-specific
• Software variable assignments, function calls,
  exceptions, etc.
• Network send, receive, route packets, update
  routing tables, etc.
• Need an abstraction layer to match the two

17
Trace extraction

• Too much information is just too much!
• Trace is a sequence of observations
• A temporal projection of execution
• Observation is a projection of system state
• Keep only relevant state components
• Too little information is a problem, too
• Did you miss anything important?
• Can you see everything you need?
• Software state is observable
• Networks/hardware may have unobservable state
• Can you see well enough?

18
Outline

• Introduction
• Why I work in formal methods?
• Overview
• What is run-time verification, and why do we
  care?
• Main body
• A framework for run-time verification and its
  implementation
• Conclusions
• Beyond run-time verification

19
MaC Monitoring and Checking

• Designed at UPenn
• Components
• Architecture for run-time verification
• Languages for monitoring properties and trace
  abstraction
• Steering in response to alarms
• Prototype implementation for Java programs
• Implementation of checking algorithms
• Automatic instrumentation

20
The MaC framework
Program
Requirement Spec
Input
Static Phase
Run-time Phase
21
MaC languages
PEDL

• Abstract state
• events
• conditions
• auxiliary variables

• Run-time state
• control locations
• object state
• local variables

SADL
MEDL

• PEDL Primitive Event Definition Language
• abstraction
• MEDL Meta Event Definition Language
• abstract transformation
• SADL Steering Action Definition Language
• feedback

22
Properties events and conditions

• Natural distinction for monitoring properties
  instantaneous vs. durational
• Instantaneity depends on time granularity
• Motivation for the distinction
• Cannot monitor every time instance
• If you saw something in an observation, is it
  still there while you are not looking?
• Yes it is a condition
• No it is an event

23
Example hundred years war

• The war is a condition
• Battles are events
• Battle durations notwithstanding
• Events change the state of conditions
• end(War)FallOfBordeaux
• FinalDefeat FallOfParis,FallOfBordeaux)

24
Property specification
Program
Requirement Spec
Input
Static Phase
low-level observations
high-level events
Run-time Phase
steering
25
Meta Event Definition Language

• Express requirements using the events and
  conditions, gathered from an execution
• define events and conditions using incoming
  primitive events and checker state variables
• Describe the safety requirements
• properties (conditions that must always be true)
• alarms (events that must never be raised)
• Independent of the monitored system

26
MEDL events and conditions

• Grammar
• Obvious tautologies
• c start(c),end(c))
• start(e1,e2))e1
• end(e1,e2))e2

27
Semantics of events and conditions

• 2-sorted past temporal logic
• Interpreted over timed traces
• Semantic function defines
• The value of a condition at a given time instance
• When an event occurs
• Two-valued or three-valued interpretation

28
MEDL auxiliary variables

• Checker state, in addition to values of events
  and conditions, contains auxiliary variables
• Used to define additional events and
  conditionsevent 5th_e e when (e_count5)
• Updates triggered by eventse -gt e_count 1
  
• Traces are interpreted over both system state
  variables and checker state variables

29
Trace extraction and abstraction
Program
Requirement Spec
Input
Static Phase
low-level observations
high-level events
Run-time Phase
steering
30
Primitive Event Definition Language

• Abstracts low-level run-time observations into
  high-level events
• each primitive event or condition used in MEDL
  must have a PEDL definition
• Semantically, a subset of MEDL
• Single-state definitions
• By necessity, PEDL is system dependent

31
PEDL for Java

• Monitored objects
• Class variables and local variables of methods
• Updates of monitored objects define primitive
  events
• Expressions over monitored objects define
  primitive conditions
• Monitored methods
• Calls and returns define primitive events
• inside a call to a monitored method is a
  primitive condition

32
Instrumentation Process

• Variable names are available inside classfile
• Bytecode instructions to be instrumented
• local variables ltTgtstore, ltTgtstore_ltngt and iinc
• global variables putfield and putstatic
• exec points beginning and return points of the
  method
• Instrumentation algorithm
• inserts code for monitoring
• updates target addresses of jump instructions
• updates the size of operand stack

33
Filter

• Probes inserted into the target system
• Thread for handling update buffers
• Updates reports are synchronized for correct
  event ordering.

34
Event recognition
Program
Requirement Spec
Input
Static Phase
low-level observations
high-level events
Run-time Phase
steering
35
Event recognition

• Event Recognizer evaluates a PEDL script on each
  message from the filter
• store the last seen value of each monitored
  variable
• Specified by PEDLcondition foo x y 5
• x y foo
• undefined
• 1 undefined
• 2 true send event to checker
• 4 true
• 2 false send event to checker

36
MEDL evaluation
Program
Requirement Spec
Input
Static Phase
low-level observations
high-level events
Run-time Phase
steering
37
Property checking

• A MEDL specification can be seen as an automaton
  with auxiliary store running on a stream of
  events provided by the event recognizer

aux. variables
38
Efficient evaluation of MEDL

• Checker state
• Value of each condition and auxiliary variable
• Timestamp of most recent occurrence of each event
• Representation forest of dependencies between
  events and conditions
• Lazy bottom-up evaluation
• Based on statically assigned height

39
Steering
Program
Requirement Spec
Input
Static Phase
low-level observations
high-level events
Run-time Phase
steering
40
Steering

• Steering is a way to provide feedback to the
  system that a violation has occurred
• Requires exact knowledge of the system
  functionality
• System should be ready to receive feedback
• User specifies when it is safe to steer and what
  is the appropriate action
• Not a recovery mechanism, but a vehicle to invoke
  recovery mechanism

41
Steering process
steering condition satisfied
action invocation received
violation
action executed
system
event received
action invoked
detection
checker
42
Steering Action Definition Language

• Identifies objects used in steering
• Defines steering actions
• Specifies steering conditions
• locations in the code where the actions can be
  executed
• steering action change2SC call
  (IPdm).setSC()
• before read DecisionModulevolts
• Invocation in response to events (in
  MEDL)SEviolation -gt invoke change2SC

43
Implementation of steering

• Injector class
• Action bodies as methods
• Listener thread
• accepts action invocations and sets invocation
  flags
• Calls to action bodies at fixed locations
• defined in the steering script, added by
  instrumentation

Injector class
Checker
execution
invoke
Invocation flags
test
steering conditioni satisfied
0
i
n
Action bodies
0
steering conditioni satisfied
i
call
n
44
MaC Prototype with steering
Program (Java byte code)
Monitoring Script (PEDL)
Steering Script (SADL)
Requirements (MEDL)
SADL Compiler
MEDL Compiler
PEDL Compiler
Instrumentation Information
Filter Generator (JTREK)
Compiled MEDL
Compiled PEDL
Instrumented
Code
Event Recognizer
Checker
45
Case studies I

• Particle formations based on artificial physics
• Balance attraction and repulsion
• Susceptible to disturbances
• Monitoring
• Check the progress of pattern formation
• Restart pattern formation if disturbance is
  observed
• Suspend and then restore repulsion
• Statistically improves quality of formations

46
Case studies II

• MaC-based Simplex architecture
• Controller for an inverted pendulum with
  hot-swapping of experimental controllers
• Enhanced performance, uncertain stability
• Use checker to compute safety envelope
• Use steering to switch between controllers

47
Inverted Pendulum in MaC
Experimental Controller
Experimental Controller
Experimental Controller
Safety Controller
angle, track
monitor
Device Drivers
Decision Module
Switching logic
J N I
volts
steer
48
Outline

• Introduction
• Why I work in formal methods?
• Overview
• What is run-time verification, and why do we
  care?
• Main body
• A framework for run-time verification and its
  implementation
• Conclusions
• Beyond run-time verification

49
Summary

• Run-time verification is a recent (since 1999)
  direction in formal methods
• Bridge the gap between model and implementation
• Avoid state explosion in checking algorithms
• Annual workshop on run-time verification
• 4th workshop is a satellite to TACAS 04

50
Summary

• MaC is a framework for run-time verification
• Architecture
• Monitoring languages
• Checking algorithms
• Applications for checking and steering
• Java programs
• Network simulations
• QoS requirements (on-going)

51
Related work

• Other run-time verification approaches
• Java Path Explorer, Time Rover
• Incremental checking algorithms, collection of
  statistics,
• Model-based diagnostics
• State estimation
• Event definition languages
• Monitoring tools
• Automatic instrumentation
• Post-mortem analysis

52
Future directions

• Integration of static and run-time analysis
• Suppose we did partial model checking?
• Synchronous checking
• Faster detection of problems
• More efficient extraction of data
• Distributed MaC
• Collection from distributed platforms
• Decentralized checking
• Monitoring coverage estimation
• How much confidence have we gained?

53
Other research directions

• Modeling approaches
• Hierarchical hybrid systems
• Uniform resource modeling
• Code generation from hybrid models
• Feasibility constraints on models
• Model-based test generation
• Use model checkers to generate the right
  counterexamples
• Modeling of genetic pathways by hybrid systems