Model Checking for Probabilistic Timed Systems

1
Model Checking for Probabilistic Timed Systems

• Jeremy Sproston
• Università di Torino
• VOSS Dagstuhl seminar
• 9th December 2002

2
The problem

• Model checking probabilistic timed systems
• In probabilistic systems
• Probabilistic choice between alternatives
• Example electronic coin flipping in randomized
  algorithms
• In timed systems
• Timing parameters are critical for the correct
  functioning of the system
• Example the system must meet a certain deadline
• In probabilistic timed systems
• Coexistence of probabilistic choice and timing

3
The focus

• Probabilistic versions/extensions of timed
  automata (Alur and Dill 1994)
• Timed automaton
• finite-state graph clocks clock
  constraints
• Clocks are an appropriate device for modelling
  time-dependent behaviour
• A clock is a real-valued variable which increases
  at the same rate as real time
• Clocks can be reset when system transitions
  occur
• Therefore, clocks can measure the exact amount
  of time elapsed since a particular transition

4
Timed automata

• Finite-state graph
• clocks
• clock constraints (examples x?3, x-ygt5)
• Example light switch

5
Timed CTL

• CTL a request will always follow a response
• ??(request -gt (?? response))
• TCTL timed CTL
• Alur, Courcoubetis and Dill (1993)
• Henzinger et al. (1994)
• A request will always follow a response within 5
  milliseconds
• ??(request -gt (??? 5 response))
• Use ?T for the satisfaction relation of TCTL

6
Timed automata semantics

• Problem underlying semantic model is
• infinite-state (node space) x R(number of
  clocks)
• infinitely branching for example
• Model checking classically assumes a finite state
  space

7
Model checking for timed automata

• Reduce to a finite state space clock equivalence
• Partitioning bounded by the maximal constant used
  in the timed automaton or the TCTL formula
• Clock equivalent states satisfy the same clock
  constraints now and in the future

y
2
1
x
1
2
8
Model checking for timed automata

• Region equivalent states have the same
• node
• clock equivalence class
• Construct finite-state region graph (transition
  system)
• States region equivalence classes
• Transitions

Time transitions
Discrete transitions E.g. crossing an edge with
x0
9
Model checking for timed automata

• Let
• TA be a timed automaton,
• ?T be a TCTL formula,
• RG(TA, ?T) be the region graph of TA, ?T
• TA ?T ?T if and only if RG(TA, ?T) ? ?
• where ? and ? are untimed versions of ?T and
  ?T
• Key result of Alur, Courcoubetis and Dill (1993)

10
Real-time probabilistic processes

• Alur, Courcoubetis and Dill (1991ICALP,
  1991Real-Time)
• Similar to Generalized Semi-Markov Processes
  (Whitt (1980), Glynn (1989))
• A fully probabilistic model

11
Real-time probabilistic processes

• Finite-state graph
• clocks
• clock scheduling function
• probabilistic branching over edges
• probabilistic clock resetting
• Example light switch

yUniform(1,30) x3
12
Timed CTL revisited

• Interpreting branching-time logic over fully
  probabilistic systems
• s ? ?? means the probability that the
  computations starting in s satisfy ? is gt 0
• s ? ?? means the probability that the
  computations starting in s satisfy ? is 1
• Alur, Courcoubetis and Dill (1991ICALP)
  interpret TCTL (branching-time) over real-time
  probabilistic processes

13
Timed CTL revisited

• For example
• ??(request -gt (??? 5 response))
• With probability 1, a request is followed by a
  response within 5 milliseconds
• Use R-TCTL to denote the logic, and ?R for its
  satisfaction relation

14
Real-time probabilistic processes semantics

• Real-time probabilistic processes use clocks, so
  are infinite-state Markov processes
• Clocks are set to negative values drawn from
  continuous probability distributions
• When at least one clock reaches 0, a transition
  is triggered

15
Model checking for real-time probabilistic
processes

• Again, reduce to a finite state space using (a
  version of) clock equivalence
• The set of clocks to reach 0 first is the same
  for all clock equivalent states

y
-1
-2
-3
x
-1
-2
-3
16
Model checking for real-time probabilistic
processes

• Construct finite-state region graph (transition
  system)
• States region equivalence classes
• Transitions

Discrete transitions E.g. crossing an
edge triggered by y reset y within (1,2)
Time transitions
17
Model checking for real-time probabilistic
processes

• Let
• RTPP be a real-time probabilistic process
• ?R be a R-TCTL formula,
• RG(RTPP, ?R) be the region graph of RTPP, ?R
• RTPP ?R ?R if and only if RG(RTPP, ?R) ? ?
• where ? and ? are untimed versions of ?R and
  ?R
• Key result of Alur, Courcoubetis and Dill
  (1991ICALP)

18
Probabilistic timed automata

• Introduced by Jensen (1995), Kwiatkowska et al.
  (2002)
• Finite-state graph clocks clock constraints
• probabilistic branching over edges
• Example light switch

x0
x0
0.01
0.99
0.99
on
off
x?3
0.01
x?2
19
Probabilistic timed CTL

• PCTL (Probabilistic CTL) Hansson and Jonsson
  (1994), Bianco and de Alfaro (1995)
• The system will fail with probability lt 0.01
• Plt0.01? failure
• PTCTL (timed PCTL) Kwiatkowska et al. (2002)
• The system will fail within 5 hours with
  probability lt 0.01
• Plt0.01?? 5 failure
• Use ?P to denote the satisfaction relation of
  PTCTL

20
Model checking probabilistic timed automata

• Probabilistic timed automaton semantics
• Infinite-state, infinite-branching Markov
  decision process
• Again, reduce to a finite state space using clock
  equivalence

y
2
1
x
1
2
21
Model checking probabilistic timed automata

• Construct finite-state region graph (Markov
  decision process)
• States region equivalence classes
• Transitions
• Time transitions are as standard
• Discrete transitions for example

0.99
x0
0.99
0.01
0.01
on
on
fail
fail
ylt3
xlt7
22
Model checking probabilistic timed automata

• Construct finite-state region graph (Markov
  decision process)
• States region equivalence classes
• Transitions
• Time transitions are as standard
• Discrete transitions for example

on
0.99
y0
x0
0.99
0.01
0.01
on
on
fail
fail
ylt3
xlt7
23
Model checking probabilistic timed automata

• Let
• PTA be a probabilistic timed automaton,
• ?P be a PTCTL formula,
• RG(PTA, ?P) be the region graph of PTA, ?P
• PTA ?P ?P if and only if RG(PTA, ?P) ? ?
• where ? and ? are untimed versions of ?P and
  ?p
• Key result of Kwiatkowska et al. (2002)

24
Continuous probabilistic timed automata

• Introduced by Kwiatkowska et al. (2000)
• Finite-state graph clocks clock constraints
• probabilistic branching over edges
• probabilistic clock resetting
• Example light switch

yUniform(0,29) x0
y30
0.01
y30
0.99
x?2
off1
on
off2
y
x,y
x?3 ? y?30
y?30
y?30
0.99
0.01
y30
25
Model checking continuous probabilistic timed
automata

• Continuous probabilistic timed automata semantics
• Infinite-state, infinitely branching
  probabilistic-nondeterministic system with
  continuous probability distributions
• Again, reduce to a finite state space using clock
  equivalence

26
Model checking continuous probabilistic timed
automata

• Problems with clock equivalence an example by
  Alur
• Clock x is reset within (0,1) in node A clock y
  is arbitrary
• Some time elapses in node A
• Then we move to node B clock y is reset within
  (0,1)
• 3 cases (1) xlty, (2) xy, (3) xgty
• Probability of (2) is 0, but we do not know the
  probabilities of (1) and (3) (clock equivalence
  abstracts from the duration of the time
  transition in node A)

x1
A
B
x
y
xlt1
y1
27
Model checking continuous probabilistic timed
automata

• A partial solution change the granularity of the
  time scale
• For example, from granularity of 1 to granularity
  of 0.5
• Say we know that x ? (0,0.5)
• Say that y is then set within (0.5,1)
• We know that ygtx

1
1
0.5
1
1
0.5
28
Model checking continuous probabilistic timed
automata

• Given a time granularity, construct a
  finite-state region graph (Markov decision
  process)
• States region equivalence classes
• Transitions
• Time transitions are standard
• Handling of probabilistic branching over edges is
  straightforward
• But how do we deal with resetting clocks
  according to continuous probability
  distributions?

29
Model checking continuous probabilistic timed
automata

• Representing continuously distributed clock
  resets in the region graph
• Integrating over time-unit intervals gives the
  probability of a clock being set within an
  interval
• E.g. with a time granularity of 1, we integrate
  over intervals such as (0,1), (1,2),
• E.g. with a time granularity of 0.5, we integrate
  over intervals such as (0,0.5), (0.5, 1),
• But the relationship between the ordering on the
  fractional parts of the newly set clocks and the
  clocks which keep their old values is not
  obtainable
• The probabilistic choice regarding this
  relationship is replaced with a nondeterministic
  choice

30
Model checking continuous probabilistic timed
automata

• Let
• CPTA be a probabilistic timed automaton,
• ?P be a PTCTL formula,
• n?1 be the chosen time granularity,
• RG(CPTA, ?P, n) be the region graph of CPTA, ?P,
  n
• CPTA ?P ?P if RG(CPTA, ?P, n) ? ?
• where ? and ? are untimed versions of ?P and
  ?p
• Key result of Kwiatkowska et al. (2000)

31
Model checking continuous probabilistic timed
automata

• Replacing probabilistic choice with
  nondeterministic choice introduces the
  possibility of an error in the computed
  probabilities
• But we know that the maximum probability that
  CPTA satisfies a path formula is bounded from
  above by the maximum probability that the
  RG(CPTA, ?P, n) satisfies the path formula
  (similar with minimum)
• For example
• CPTA ?P Plt0.01? failure
• if
• RG(CPTA, ?P, n) ? Plt0.01? failure

32
Conclusions model checking timed automata

• Achieved success in the form of the development
  of tools such as UPPAAL (Uppsala/Aalborg) and
  KRONOS (Grenoble)
• Use of zone-based algorithms
• Manipulate sets of clock equivalence classes

33
Conclusions model checking real-time
probabilistic processes

• Activity died off after Alur, Courcoubetis and
  Dills 1991 papers
• Interest renewed by the development of process
  algebras with generally distributed delays
  (Bravetti et al., DArgenio et al)
• Model checking of Semi-Markov Chains
  Infante-Lopez et al. (2001)

34
Conclusions model checking probabilistic timed
automata

• Model checking using PRISM (Kwiatkowska, Norman
  and Parker (2002)) and
• Region graphs
• Discrete-time semantics (given restrictions on
  clock constraints to x?c and x?c)
• Based on discrete-time semantics for timed
  automata developed by Henzinger et al. (1992),
  Asarin et al. (1998), Bozga et al. (1999)
• Case studies FireWire (Kwiatkowska et al.
  (2002FAC)), IEEE802.11 (Kwiatkowska et al.
  (2002PAPM-PROBMIV))

35
Conclusions model checking probabilistic timed
automata

• Zone-based algorithms for probabilistic timed
  automata
• Must carefully distinguish zones which have
  different probabilities
• Kwiatkowska et al. (2001CONCUR, 2002TCS)
• Case study FireWire
• Kwiatkowska et al. (2002FAC), Daws et al. (2002)

36
Conclusions model checking continuous
probabilistic timed automata

• Increasing the time granularity blows up the
  state space
• Exists a need to concentrate on restricted
  subclasses