Lecture 8 Risks

1
Lecture 8 Risks Metrics Risk Exposure
Prob(failure) x Cost of Failure
CS 540 Quantitative Software Engineering

2
Software Risk

• Risk is the possibility that an undesirable event
  in the life of a software project can happen
• Requires uncertainty and loss
• Project Risk cost, schedule, completion
• Technical Risk feasibility, viability, etc.
• Business Risk contractual compliance, payment,
• Legal Risk damages, liability

3
Risk event is undesired a possible project
impacting event
Activity
Activity
4
What makes a project risky ?
5
Software Engineering Institute (SEI) Risk Analysis

• Incomplete requirements
• Tight schedule
• Understaffed
• Staff morale
• Design problems
• Feature set too large
• Technology immature
• Late planned deliveries of OS/3PS
• SOX compliance procedures

6
Factors that increase risk

• Excessive Schedule Pressure (65 of projects)
• Management Malpractice
• Inaccurate and Inadequate Metrics
• Poor cost Estimates
• Silver Bullet Syndrome
• Creeping Featurism
• Quality shortfalls
• Size growth

7
Risk dos and donts

• Dont overestimate the risks
• Dont do too much contingency planning
• Dont underestimate the risks, this leads to
  panic management later
• Dont look for scapegoats
• Deal only with the top 10 risks at a time

8
Boehms Risk Management Taxonomy
9
Software Intensive Systems-of-Systems Risks-
Boehm CrossTalk, May 2004

• 1. Acquisition management and staffing
• 2. Requirements/architecture feasibility
• 3. Achievable software schedules
• 4. Supplier integration
• 5. Adaptation to rapid change
• 6. Quality factor achievability and tradeoffs
• 7. Product integration and electronic upgrade
• 8. Software COTS and reuse feasibility
• 9. External interoperability
• 10. Technology readiness

10
Architecture Risk Taxonomy
.                                              
                                                  
                                                  
     
11
Risk Taxonomy(adapted from CMU/SEI-93-TR-6)
12
Ten most software serious risks (Capers Jones
view 1992)

• Schedule pressure
• Malpractice
• Creeping requirements
• Low quality
• Low productivity
• Silver Bullet syndrome
• Inaccurate metrics
• Inadequate measurement
• Inaccurate estimating
• Canceled projects

13
Common telecommunications software risks (Capers
Jones view 1992)

• Excessive paperwork
• Lack of reusable components
• Poor Management tools
• Aging software
• Security and viral protection
• Corporate hardware focus
• Low status of software staff
• Inadequate specialization

14
Risk Determination from

• Historical Data
• Project and Customers experience with Wide-band
  Delphi.
• Business plan assumptions
• Expert opinions

15
Typical Risks
16
Risk Histogram
17
Omission vs. Commission
18
Risk Analysis

• Let Prob (E) be the probability of a favorable
  outcome that can be any one of m events. The
  risk of E happening is
• Prob (E) m/n,
• Where m is the number of favorable events
• n is the total number of possible events.
• Let RE be the risk exposure.
• RE Prob (E not happening) x Loss
• 1-Prob (E) x Loss
• Recall that the Win-Win Spiral Model
  makes a risk exposure evaluation for every
  cycle.

19
Uniform Loss

• Uniform Loss
• loss/day x (days late),
• For a project employing 100 people,
• Loss cost of retaining 100 people per day
  daily project overhead daily opportunity cost
  of savings or profits from this and other
  projects days late

20
Taguchi Loss

• Loss (budget)Max incurred cost-budget2, where
• budget (cost per day) (estimated days)
• Max Maximum acceptable loss/ Maximum over
  budget
• For a project employing 100 people,
• Loss Loss Budget) daily opportunity cost of
  savings or profits from this and other projects
  days late

21
Schedule Risks

• Tight Schedules
• Estimation Errors
• Feature Creep

22
Gamma Function model for schedule slipsIEEE
Software 9/97 pp 740-745
0.0035
Prob. of delay
0.0005
23
Estimation Accuracy International Software
Benchmarking Standards Group 2000 project study

• 2000 projects completed within past decade
• 25 met both schedule and effort estimates
• 11 better than expected
• 40 met only one estimate
• 35 met neither estimate

24
Estimation Process

• 40 use only decomposition
• 15 use Function Points
• 30 use both
• 20 use estimation models
• 20 had dictated schedules

25
Ten deadly estimation sins

• Confusing targets with estimates.
• Saying yes when you really mean no.
• Committing to estimates too early.
• Underestimating
• Estimating in the impossible zone
• Overestimating savings from new tools or methods
• Using only one estimation technique
• Not using estimation software
• Not including risk impacts in estimates
• Providing off-the-cuff estimates

26
Specification for Development Plan

• Project
• Feature List
• Development Process
• Size Estimates
• Staff Estimates
• Schedule Estimates
• Organization
• Gantt Chart

27
Architecture Reviews

• These reviews have been around for at least 11
  years/ 500 reviews at ATT and Lucent (SARB,
  Systems Architecture review Board)
• Stresses architecture and evaluating it early in
  the project
• Architecture in this context is viewed as a
  solution to a problem for a client and should
  cover a broad range from cost to client needs
• After you establish an architecture
• Decide what and when to test
• Establish success criteria
• Make decisions based on your findings

28
Arch Review Payoff

• Average review pays back 12 times its cost
• Reduced development effort and interval - finds
  defects early and identifies risks
• Higher product quality and lower cost
• Company wide learning - annual report
• Yes, projects were canceled after reviews and the
  attitude of the project team was often surprising

29
Contingency Planning

• Contingency is the resources needed to reduce
  the probability of a risk to 0.2
• Risk Reduction leverage (RE Lev)
• RE Lev RE (nominal) RE (at contingency)
  (Risk reduction Cost)

30
Risk Exposure Sweet Spot
31
Boehm
32
Risk Heuristics

• Unrealistic schedule is a common project risk.
• Schedule slip rates remain the same throughout a
  project unless steps are taken to change the
  causes of slippage
• Contingency funding should bring the risk
  probability to 0.2
• Periodically and quantitatively evaluate risks-
  project meeting are the place to consider the
  top-ten.
• Minimize risks by doing the hard part first.
• Human Interfaces and customer acceptance are
  major risks to most projects.

33
Software Risk Analysis

• Identify
• Communicate
• Analyze
• Plan
• Track
• Control

34
SEI risk assumptions

• Risks are often known but rarely discussed.
• Management attitudes must be non-judgmental and
  supportive so that controversial views can be
  heard
• Project success is not based solely on risk
  management

35
Analyze Risk

• Probability of risk, USAF Handbook categories are
  very low, low, medium, high and very high
• Impact of risk, USAF Handbook categories are
  negligible, marginal, critical and catastrophic
• Risks are rarely independent
• A matrix is used to determine overall risk for
  different categories (e.g., effort, performance,
  schedule, cost, support)

36
Sample Impact/Probability Matrix(used to
calculate overall risk)
37
Plan for the Risks

• What can you do
• Mitigate with contingency plans and triggers
• Avoid the risk by changing something
• Accept the risks and the consequences if it
  occurs
• Study the risk further so that you can decide on
  one of the above
• Specify risk importance
• Define data needed to track risk status
• Define who is responsible for Risk Management and
  what is the cost

38
Risk Tracking and Control

• Use Action Item system
• Track status of risks and actions taken to
  address them
• Define Risk Exposure
• RE Prob(risk) x Cost(risk)
• Review ten risks with highest RE at every
  project meeting

39
Software Risk Summary

• Risk is the possibility that an undesirable event
  in the life of a software project can happen
• Requires uncertainty and loss
• Project Risk cost, schedule, completion
• Technical Risk feasibility, viability, etc.
• Business Risk contractual compliance, payment,
• Legal Risk damages, liability

40
Software Risk Mitigation Summary

• Project Risk cost, schedule, completion
• Estimation, simplifications, staffing changes
• Technical Risk feasibility, viability, etc.
• Prototyping, experimentation, invention, multiple
  path
• Business Risk contractual compliance, payment,
• Contractual mitigation time and materials,
• Legal Risk damages, liability
• Limitations of liability, termination for
  convenience