Models for Software Reliability

1
Models for Software Reliability

• N. El Kadri
• SEG3202

2
Definitions

• 1991 IEEE standard the probability of
  failure-free software operation for a specified
  period of time in a specified environment
• The quality of the product improves over time,
  and we talk about reliability growth
• Software-reliability growth problem estimating
  and predicting the reliability of a program as
  faults are identified and attempts made to fix
  them.
• Fault density - The number of faults per thousand
  lines of executable source code.

3
Using Reliability Models
4
Taxonomy of Software Reliability Models
5
Time-dependent Reliability Growth Models
6
Time-dependent Reliability Growth Models
7
Discussion

• One problem with models is an overwhelming
  number of models has been proposed to address the
  issue of software reliability assessment.
• We must be aware that no single model can be
  recommended universally to users under any
  circumstances.
• The best models may vary from time to time and
  differ form application to application.

8
Time-Independent Reliability Models Fault
Injection

• Estimates the number of faults N in the system
  in the case where we know the outcomes of the
  already detected faults.
• Insert into a software module a certain number A
  of faults
• Proceed with its testing
• Count the number of failures due to injected
  faults (f) Count the number of failures due to
  inherent faults (i )
• Estimate the number of remaining faults

9
Time-Independent Reliability Models Fault
Injection

• Example
• Insert into a software module A25 faults
• Proceed with its testing (results 32 failures)
• Count the number of failures due to injected
  faults (f17)
• Count the number of failures due to inherent
  faults (i15)
• Estimate the number of remaining faults

10
Fault Injection Confidence Levels about the
Number of Faults

• If not all artificial faults have been found
  (fltA), then
• If all artificial faults have been found (fA),
  then

11
Fault Injection Example (cont.)

• Confidence levels about the number of faults
• Let us set E to 10.
• f lt A gt

12
Fault Injection Example 2

• Assume all artificial faults have been found
  (fA)
• Given E10, certain confidence level ? and iltE,
  
• how to determine the number of faults to be
  injected?

13
Time-Independent Reliability Models Input Domain

• Software reliability depends on how software
  operates on a certain input domain.
• This point of view relates to selecting software
  test cases over the input domain according to how
  software is used
• Software usage information includes the
  environment information where software is used,
  as well as the information on the actual
  frequency of usage of different operations,
  functions, or features that the system offers.
• The usage information is quantified through
  operational profiles

14
Input Domain Equivalence Classes
Software Module
15
Operational Profiles

• Such a strategy is called Statistical Testing and
  it has at least two benefits
• Testing concentrates on the parts of the system
  that are most likely to be used and hence should
  result in a system that the user finds more
  reliable.
• Using the techniques which we have presented
  earlier, we have confidence that reliability
  predictions based on the test results will give
  us an accurate prediction of reliability as seen
  by the user.

16
Operational Profiles

• Methodology to develop operational profile
• 1. Determine customer profile or usage context
  profile.
• 2. Determine user profile.
• 3. Determine system modes and their profile.
• 4. Determine the functional (requirements)
  profile.
• 5. Determine the operational (implementation)
  profile.

17
Operational profile

• A set of relative frequencies (or probabilities)
  of occurrence of disjoint software operations
  during its operational use
• A software-based system may have one or more
  operational profiles.
• Operational profiles are used to select test
  cases and direct development, testing and
  maintenance efforts towards the most frequently
  used or most risky components.

18
Operational profile

• Construction of an operational profile is
  preceded by definition of a customer/user
  profile, a system mode profile, and a functional
  profile.
• Profiles are constructed by creating detailed
  hierarchical lists of customers, users, modes,
  functions and operations that the software needs
  to provide under each set of conditions.
• For each item estimate the probability of
  its occurrence and thus provide a quantitative
  description of the profile.
• If usage is available as a rate (e.g.,
  transactions per hour) it needs to be converted
  into probability.  

19
User Profile - Example
20
System Mode
21
System Modes Example

• We can identify five system modes
• Normal Traffic load
• High Traffic load
• Start/Restart
• Administration
• Troubleshooting
• The first three are only relevant to the
  subscriber user type, the fourth to the operator
  user type and the fifth to the customer care
  system user type.

22
Functional Profile
23
Operational Profile
24
Operational Profile
25
Input Domain Equivalence Classes
Software Module
26
Input Domain Equivalence Classes Operational
Profiles

• Each Path through the Operational Profile
  hierarchy defines an equivalent class
• Path E1 subscriber, normal traffic, MO SMS,
  originating MO SMS
• Each path Ej has its associated probability Pj
  stating that the inputs will come from it under
  normal operation of the system
• Assuming the inputs are independent from each
  other, the probability of a path is a product of
  the corresponding operational probabilities,
  i.e. for E1 the associate probability is
• P1 0.90.60.050.95 0.35

27
Input Domain Model Reliability Estimation

1. Assumption c equivalent classes Ei
2. With each class comes its operational profile
3. Let Pi be the probability stating that the
   inputs will come from Ei under normal operation
   of the system
4. nj is the number of test cases sampled from the
   jth input domain Ei, where fj out of them
   resulted in software failures
5. The estimated reliability is computed as

28
System reliability

• Knowing the reliability of individual components
  Ri, one can easily compute the reliability of
  some architectures as follows
• (a) Series configuration (b) Parallel
  configuration

29
Classes of Reliability Models and their main
Features
30
Reliability Measures
31
Availability
32
ANSI/IEEE 982.1-1988

• Includes guidance for the following
• Applying product and project measures throughout
  the software life cycle
• Optimizing the development of reliable software
  with respect to constraints
• Maximizing the reliability in its actual use
  environment
• Developing the means to manage reliability in the
  same manner that cost and schedule are managed

33
More Information