Statistical Analysis of Probabilistic Systems

1
Statistical Analysis of Probabilistic Systems

• Koushik Sen
• University of Illinois at
• Urbana-Champaign, USA

Joint work with Gul Agha, Nirman Kumar, José
Meseguer and Mahesh Viswanathan
2
Motivation

• Network protocols
• Delays in communication
• Failures in communication
• Malicious participants
• Large and Geographically Distributed Systems
• Interact with unpredictable and hostile
  environment
• Failure of a component
• Probabilistic in Nature

3
Probabilistic Model

• Stochastic Modeling
• Associate probability with different
  uncertainties
• Probability distribution for time (delay)
• Probabilities for failures
• Formal Probabilistic Models
• Discrete-Time Markov Chains (DTMC)
• Continuous-Time Markov Chains (CTMC)
• Markov Decision Processes (MDP)
• Generalized Semi-Markov Processes (GSMP)
• Probabilistic Rewrite Theories (PRwTh) FMOODS03

4
PMaude

• Specifications in Probabilistic Rewrite Theories
• Not restricted to finitary probabilistic rewrite
  theories.
• Execute Specifications to get sample traces
• Non-determinism is resolved by Maude in fair way.
• Probabilistic choices are made by sampling.
• Library for Sampling

5
Clock Example

• sort Clock .
• op clock Nat Float -gt Clock ctor .
• op broken Nat Float -gt Clock ctor .
• op sample Float -gt Bool .
• crl tick clock(N,C) gt if B then clock(N
  1,C - (C / R0))
• else broken(N, C - (C / R0)) fi
• if B sample(C) .
• eq sample(C) if C gt R0 then true
• else sampleBerWithP(C / R0)
• fi .

6
Sample Runs

• Maudegt rew clock(0,1000.0) .
• result Clock broken(227, 7.5188020199780931e2)
• Maudegt rew clock(0,1000.0) .
• result Clock broken(237, 7.4253439021252518e2)
• Maudegt rew clock(0,1000.0) .
• result Clock broken(195, 7.8258459583034676e2)
• Maudegt rew clock(0,1000.0) .
• result Clock broken(206, 7.7189105943066818e2)
• Maudegt rew clock(0,1000.0) .
• result Clock broken(200, 7.7770565471541227e2)
• Maudegt rew clock(0,1000.0) .
• result Clock broken(214, 7.6420583477641014e2)
• Maudegt

7
Sampling Library

• op sampleBerWithP Float -gt Bool .
• op sampleExpWithMean Float -gt Float .
• op sampleExpWithRate Float -gt Float .
• op sampleNormal Float Float -gt Float .
• op sampleT Float -gt Float .
• op sampleChi Float -gt Float .
• op sampleF Float Float -gt Float .

8
Dynamic Analysis

• Simulate system or Observe the system execution
• Collect traces
• Analysis Techniques
• Statistical Analysis of Performance and
  Reliability properties (VeStA) CAV04 QEST04
  submitted
• Construct Predicate Abstraction using Machine
  Learning
• Model-check to find counter-example FSE04
  submitted
• Construct Reachable Region as a recognizable
  language
• Regular Model Checking, Widening, Acceleration,
  Lever
• Runtime Verification (Eagle) VMCAI04,PADTAD04
  Space Rover at NASA Ames
• Predictive Analysis (JMPaX) FSE03, TACAS04,
  PADTAD04
• Distributive Analysis (DiAna) ICSE04 WODA04

9
Probabilistic Analysis

• Specify performance and reliability properties in
  some probabilistic logic
• Probabilistic Computation Tree Logic (PCTL)
• Continuous Stochastic Logic (CSL)
• Model-Check against the property
• Numerical (PRISM, ETMCC)
• Accurate, Computationally intensive
• Statistical (ProVer, SMART, VeStA )
• Approximate, Scalable

10
Continuous Stochastic Logic (CSL)

• Computation Tree Logic (CTL) extended with
  continuous time and probabilities
• Plt 0.5(lt10 full)
• Probability that queue becomes full in 10 units
  of time is less than 0.5
• Pgt0.98( retransmit Ult200 receive)
• Probability that a message is received
  successfully within 200 time units without any
  need for retransmission is greater than 0.98

11
Statistical Approach - Blackbox
12
VeStA

• Get sample traces with labels
• L(s0) a0-gt L(s1) a1-gt -an-1-gt L(sn)
• Learn model
• Edge-labeled Continuous Time Markov Chain
• New Machine Learning
• Statistically verify learned model
• Hypothesis Testing

13
Simple Approach

• Traces with complete state information
• s0 a0-gt s1 a1-gt -an-1-gt sn
• s0 a0-gt s1 a1-gt -am-1-gt sm
• Easy to infer probabilistic model
• Black-box systems
• Traces with no complete state information
• L(s0) a0-gt L(s1) a1-gt -an-1-gt L(sn)
• Infer model through learning

14
Machine Learning Algorithm
Learn bisimulation relation over States
15
Details of ML algorithm

• Define bisimulation of states
• s s
• L(s) L(s)
• 8 a 9 a ?(a) ?(a) and ?(s,a) ?(s,a)
• Vice-versa
• Cannot test bisimulation exactly
• Approximate bisimulation test
• Statistical tests
• Hoeffdingd bound to test Bernoulli Distribution
• F-distribution to test Exponential Distribution
• In the limit the algorithm learns the exact model.

16
Verification Idea

• To verify a formula P p? set up two statistical
  hypothesis tests
• Test 1
• H0 ( of true)/( of samples) p - ?1 say
  YES and calculate p-value
• H1 ( of true)/( of samples) gt p - ?1
• Test 2
• H0 ( of true)/( of samples) p ?2 say
  NO and calculate p-value
• H1 ( of true)/( of samples) lt p ?2
• If p - ?1 lt ( of true)/( of samples) lt p ?2
• say I dont know

17
On Nested Formulas

• Evaluate Satisfaction of nested Probabilistic
  formulas
• Yes
• No
• I dont know
• Resolve in adversial fashion
• Yes when verifying a formula of the form P p?
• No when verifying a formula of the form P p?

18
Evaluation

• Tandem Queuing Network
• Cyclic Polling System
• Grid World Example
• Answers matched the numerical model-checker
• P-value (?) of the order 10-8
• Very high confidence in our result
• 4-times faster than ProVer and PRISM
• Disadvantage Space requirement is high
• Required to store all samples before
  model-checking

19
Discussion

• No need for Model
• Black-box Checking
• Can we do better if we have the model?
• Use PMaude specifications
• Use Machine Learning to verify Liveness properties