Guest Lectures On Reliability Modeling

1
CSE565 Software Verification, Validation, and
Testing
Guest Lectures On Reliability Modeling
2
Terminology
3
Failure Probability and Reliability Function
Let T be a random variable for the time interval
from time 0 to the time of the first system
failure. The Cumulative Distribution Function
F(t) ProbT t. The Probability Density
Function f(t) dF/dt. F(t) is called failure
probability in the interval 0, t. Reliability
function R(t) of a system is the probability that
the system has survived in the time interval 0,
t, given that it is operational at time 0. R(t)
1 F(t) 1 ProbT t ProbT gt t.
4
Reliability Function and Failure Rate
The failure rate, z(t), is defined as It is the
rate at which the system changes from working
state to failed state at time t. z(t) Dt is
the probability that the system is in working
state at time t, but in failed state at time t
Dt.
5
Reliability Function for Computer Hardware
?
6
Availability Function and Repair Rate
Availability A(t) of a system is the probability
that the system is working at time t. The repair
rate m(t) is the rate at which the system changes
from failed state to working state at time
t. m(t) Dt is the probability that the system
is in failed state at time t, but in working
state at time t Dt.
7
Availability Function and Repair Rate
Availability is the probability that a system
stays in the sate "working". A(t Dt) A(t) -
A(t)z(t)Dt (1 A(t))m(t)Dt
8
Software Reliability Models Overview
Reliability Models
Design phase
Testing debugging phase
Operation phase
Validation phase
(reliability predication)
(reliability estimation)
(reliability assessment)
(reliability validation
9
Time-Between-Failures (Time Domain) Models
Also called Time Domain Models Software
reliability is usually defined as the
probability R(t)  Probno of failures within
time period 0, t where t is the exposure
period whose time unit is the calendar or CPU
time. R(t) is assumed to follow certain
probability distribution, for example, R(t)
e-z(t), where z(t) is the failure rate. The
main concern of these reliability growth models
is to estimate the value of the failure rate
function z(T).
10
Shooman Model Shooman 1973
Error model where Er number of errors remains
at time T E0 number of errors at T 0 in the
program under test I0 number of
instructions Ec number of errors corrected in
the time interval 0, T E0, I0, and Ec are
decided by testing
11
JM Model Jelinski and Moranda 72

• Assumption of the model
• There are N independent software faults in the
  program at the beginning of testing
• Each fault is equally likely to cause a failure
  during testing
• A detected fault is removed with certainty in a
  negligible time and no new faults are introduced
  during testing and debugging process
• The software failure rate at time t, after the
  ith fault is removed, is proportional to the
  current faults and is given by

where, f is a constant
12
Axiomatic Models

• Software reliability is postulated to obey
  certain universal laws.
• One of the well-known models Software Science
  Model
• The no. of bugs
• where
• K constant
• V volume of program
• E0 number of errors at T 0 in the program
  under test
• Axiomatic models a special issue IEEE Trans.
  Soft. Eng. 1979.

13
Fault Seeding Models
How do we estimated the number of fishes in a
lake?
1) Insert a certain number of faults into the
program under test 2) Test the program How
many detected faults are inserted? How many
detected faults are original? How many original
faults are detected? 3) Calculate the total
number of original faults according statistics.
14
Input Domain Models
Examples MacWilliams73, BrownLipow75, Nelson78.
Software reliability is defined as the
probabilityR(N) Probno of failures over N
application runs where N is the exposure period
whose time unit is the number of application
runs. Assuming that input cases are selected
independently, then R(N) can be expressed
by R(N) (R(1))N RN where, R ? R(1) is the
reliability per application run. Now the
question is to estimate R.
15
Estimate R
R, the reliability per test run, can be defined
by the ratio of the number of test runs in which
failures are observed and the total number of
test runs when infinite number of different input
cases are applied for test runs R 1 F 1
Because of test time limit only a subset of
the entire input domain can be applied to test
the program in practice. Thus the reliability per
test run, R, is usually estimated by
16
MacWilliams 73 and BrownLipow 75
MacWilliams 73 the s input cases are selected
randomly from the input domain. BrownLipow 75
the input domain is partitioned into m classes.
If si input cases are selected from class Ci and
fi failures are observed, the reliability can be
calculated by where P(Ci) is a probability
function reflecting the input profile in terms of
classes.
17
Modeling Complex Systems (Software and Hardware)

• A large system can be decomposed into smaller
  components.
• Evaluate the reliability of the components
• Evaluate the reliability of the system based on
  known component reliabilities
• Combinatorial Models
• Markov Models

18
Combinatorial Models
Combinatorial modeling is a failure-to-exhaust
approach, in which the system divided into
non-overlapping modules. Combinatorial modeling
is based on the following assumptions 1 Module
failures are independent 2 Failed modules always
yield incorrect results 3 The system fails, if it
doesnt have the minimal set of functioning
modules required. 4 Once the system or a module
fails, the subsequent activities cannot bring the
system back to a functional state.
19
Series Models
20
Parallel Models
21
Example 1 Parallel and Series Systems
Which is more reliable?
22
Which is more reliable?
Let Ra Rb Rc Rd R, then
23
TMR (Triple Modular Redundancy) Systems
RTMR Rv(R3 3R2(1-R)) Rv(2R3 - 3R2)
24
MTTF of a TMR System
For a simple system
MTTFTMR lt MTTF
Pitfall TMR system is less reliable than a
simple system.
25
MTTF of a TMR System
MTTF is computed from 0, ?, it has little
meaning after certain time, especially for highly
dependable systems
26
NMR (N-Modular Redundancy) Systems
27
Success Diagram for Nonseries/Nonparallel Models
Rsys Rm Prob(system works m works) (1
Rm) Prob(system works m fails)
28
Success Diagram Example
B works
B fails
Rsys RB Prob(system works B works) (1
RB) Prob(system works B fails) Rsys R6
3R5 R4 2R3
29
Markov Models
Markov models are more generic than combinatorial
models. They can handle repairs and much more
complex situations . Assumption Any
component may in one the two states working or
failed Probability of state transition depends
only on the current state. ß Failure rates and
repair rates are constants. ß Transition
probability is proportional to the time that the
component stays at a state. ß Exponential
distribution of the reliability/availability
30
Steps of Applying Markov Models
A system consists of multiple components ß Constr
uct state transition diagram
(1)
31
Step 1 Construct state transition diagram
Example 1 Simplex system with repair
32
Step 1 Construct state transition diagram
Example 2 TMR system with repair
33
Step 1 Construct state transition diagram
Example 3 A ring system with different node and
link failure rates a and b
34
Step 2 Construct differential equations
A(t) p0(t)
The question is how to obtain the probability of
each state.
p0 (t Dt) (1 l  Dt)   p0 (t) m  Dt 
p1 (t) p1 (t Dt) l  Dt p0 (t) (1 m 
Dt)   p1 (t)
Solve the differential equations to obtain (p0
(t), p1 (t)).
35
Step 2 Construct differential equations
l
0
1
m
36
Step 2 Construct differential equations
37
Step 2 Construct differential equations
In general, assume a STD has n states and is
fully connected. Any state has n incoming and n
outgoing transitions
aij ? 0 is the transition rate from state i to
j. For i, j 1, 2, ..., n, and i ? j.
38
Step 2 Construct differential equations
The probability in state j at t Dt the
probability in state j at t incoming prob
outgoing prob
Math manipulation Divide Dt on both sides, let
Dt ? ?
39
Step 2 Construct differential equations
40
Example 1
R(t) p1(t) p2(t)
41
Example 2
R(t) p1(t) p2(t) p3(t)
42
SUMMARY

• Basic concepts of reliability and reliability
  modeling
• Hardware reliability models
• Software reliability models
• System reliability models consisting of multiple
  components
• Combinatorial models
• Markov models