Lecture

1
SE325/425 Principles and Practices of Software
Engineering Lecture 2 Risk Management Winter
2005DePaul University Jane HuangSchool of
Computer Science, Telecommunications, and
Information Systems
2
Elements of Risk Analysis
What are the risks involved in getting to work?
Reduce the occurrence and/or impact of the risk.
Identify new risks as they occur report on all
known risks.
3
Proactive Risk Strategies

• Begins long before technical work is initiated.
• Potential risks are identified.
• Probability and impact of risks are assessed.
  Software risks always involve the two
  characteristics of uncertainty and loss.
• Uncertainty The level of certainty about
  whether the event may or may not happen.
• Loss What is the impact of the event if it does
  occur?
• Risks prioritized by importance.
• Software team establishes a plan to manage the
  risk.

Dont hide from project risk. Face it head on!
4
Three Types of Software Risk

• Project RisksThreaten the project plan. Ie if
  the risks materialize, then it is likely that the
  project schedule will slip and costs will
  increase.

• Budgetary
• Schedule
• Personnel
• Resources

• Customers
• Requirements problems
• Project complexity and size.

• Technical RisksThreaten the quality and
  timeliness of the software to be produced.

• Design
• Implementation
• Interfacing

• Verification
• Maintenance

5
Three Types of Software Risk (cont)

• Business RisksThreaten the viability of the
  product to be built.

• Building a great product that no-one wants
  anymore. (Market risk)
• Building a product that no longer fits into the
  overall business strategy for the company
  (Strategic risk).
• Building a product that the sales force doesnt
  understand how to sell.
• Losing the support of senior management due to a
  change in focus or a change in people.
  (Management risk).
• Losing budgetary or personnel commitment (Budget
  risk)

6
Another way to Categorize Risk

• Known RisksRisks that can be uncovered after
  careful evaluation of the project plan, business
  and technical environment, and other reliable
  sources of information (ie unrealistic delivery
  dates, lack of user input etc)
• Predictable RisksRisks that can be extrapolated
  from past projects. (Staff turnover, poor
  communication with the customer)
• Unpredictable RisksJoker risks that are hard
  to predict.

7
Risk Identification
Aaaghh!

• A systematic attempt to specify threats to the
  project plan.
• For each risk category there are two types of
  risk
• Generic risks are a potential threat to every
  software project.
• Product specific risks can only be identified by
  those with a clear understanding of the product,
  its environment, its underlying technology.

If you dont actively attack the risks, they
will attack you! Tom Gilb
8
Risk Identification (cont.)
Controls
Project ResourcesProject RequirementsRisk
Management Plan
Outputs
Identify Risk
Risk StatementRisk context
Inputs
UncertaintyKnowledgeConcernsIssues
Mechanisms
Risk Checklists (See SEI risk taxonomy)Risk
AssessmentRisk Management FormRisk Database
9
Risk Item Checklist
A checklist is useful for supporting risk
identification of known and predictable risks in
the following subcategories

• Product size
• Business Impact - imposed by management or the
  marketplace.
• Customer characteristics
• Process definition
• Development environment ie availability and
  quality of tools.
• Technology to be built complexity, newness
  etc
• Staff size and experience

10
Product Size Risks

• Project risk is directly proportional to product
  size.
• Measure the following sizes against previous
  projects. If those projects were successful
  results are similar, then risk is probably low.
  If a large negative deviation is observed then
  risk is HIGH.

• Estimated size of the product in LOC or FP?
• Degree of confidence in estimated size estimate?
• Estimated size of product in number of programs,
  files, transactions?
• Percentage deviation in size of product from
  average for previous products?
• Number of users of the product?
• Anticipated volatility of the requirements?
• Amount of reused software?

11
Business Impact Risks

• The following items help identify generic risks
  associated with business impact
• Effect of product on company revenue.
• Visibility to senior management.
• Reasonableness of delivery deadline
• Number of customers who will use the product
  consistency of their needs.
• Number of other products that it will interact
  with.
• Sophistication of end users.
• Governmental constraints.
• Costs associated with late delivery or a
  defective product?

12
Customers

• There are many different types of customer to
  deal with
• Customers have different needsDo they know what
  they need or not? Are they willing to sweat the
  details?
• Customers have different personalitiesSome
  enjoy the process and some hate it. Some will
  accept a shabby product just to get the process
  over with while others will insist on a high
  quality product.
• Customers have varied associations with their
  suppliesPersonal relationships vs. impersonal
  phone contacts.
• Customers may be contradictoryThey want
  everything yesterday for free. A good
  requirements engineer knows how to bring the best
  out of the customer.

13
Customer Related Risks

• The following items help identify generic risks
  associated with the customer
• Have you worked with the customer in the past?
• Does the customer have a solid idea of what is
  required?
• Is the customer willing to commit significant
  time to the requirements gathering process?
• Is the customer willing to establish rapid
  communication links with the developer?
• Is the customer willing to participate in
  reviews?
• Is the customer technically sophisticated in the
  product area?
• Does the customer understand the software
  process?
• Risks should be investigated if the answer to any
  of these questions is YES.

14
Process Risks

• An ill defined software process and/or an ad hoc
  approach to analysis, design, and testing can
  introduce risk.
• The following are sample questions that should be
  asked to identify process risk
• Do you have a consistent repeatable process that
  is actually used?
• Do you train all developers in the process?
• Are formal technical reviews part of this
  process?
• Do you have a mechanism for managing change? (ie
  formal rfc system configuration management).
• Do you have specific methods that you use for
  each phase of the process?
• Is the process supported by tools?
• Do you manage the process through use of metrics?

15
Technology Risks

• Pushing the limits of technology is challenging
  exciting, yet very risky.
• Questions to identify risk include
• Is the technology to be built new to your
  organization?
• Do the requirements require the creation of new
  algorithms?
• Does the software interface with new or unproven
  hardware or unproven vendor products?
• Do the requirements require the creation of
  components that are unlike anything your
  organization has previously built?
• Do requirements demand the use of new analysis,
  design, or testing methods?
• Do requirements put excessive performance
  constraints on the product?

16
Development Risks

• The software engineering environment supports the
  project team, the process, and the product.
• If the environment is flawed, it can be a source
  of significant risk
• Is a software project management tool available?
• Are tools for analysis and design available?
• Are compilers and code generators available and
  suitable for the product to be built?
• Are testing tools available and suitable?
• Are the software tools integrated with each
  other?
• Are team members trained in the use of the tools?
• Are tool mentors available?

17
Staff Size and Experience Risks

• CEOs have frequently observed that people make
  the most significant difference to the success of
  the organization.
• Are the best people available?
• Do the people have the right combinations of
  skills?
• Are enough people available?
• Are staff committed for the duration of the
  product?
• Are some people working on multiple projects?
• Have staff received necessary training?

18
Risk Components and Drivers

• The US Air Force requires project managers to
  identify
• risk drivers that affect software risk
  components.
• Performance risk the degree of uncertainty that
  the product will meet its requirements and be fit
  for its intended use.
• Cost risk the degree of uncertainty that the
  project budget will be maintained.
• Support risk the degree of uncertainty that the
  software will be easy to correct, adapt, and
  enhance.
• Schedule risk the degree of uncertainty that
  the project schedule will be maintained and that
  the product will be delivered on time.

19
How risk averse are you?

• Risk seeking people
• I like action, and I act impulsively at times.
• I seek excitement for the thrill of the
  experience.
• I am resourceful and prefer not to plan or
  prepare.
• I am more self oriented than service oriented.
• I like to anticipate another persons position.

• Risk averse people
• I like being dependable and Im usually punctual.
• I am not likely to take chances.
• I am responsible and prefer to work efficiently.
• I am more service oriented than self oriented.
• I value institutions and observe traditions.

• Risk neutral people
• I trust my intuition, and I am comfortable with
  unknown.
• I think about the future and have long-range
  objectives.
• I am naturally curious and often ask, Why?
• I enjoy generating new ideas.
• I work best when I am inspired.

20
Risk Management

• Lets take a look at one of Indys escapades in
  the opening sequence of Raiders of the lost arc.
• Did his approach to risk management prove
  successful?
• As we will see this is an example of reactive
  rather than proactive risk management.

Dont worry, Ill think of something!!
21
Post-Implementation Review

• Project Manager Dr. Indiana Jones
• Project Objective To retrieve priceless gold
  statue for the museum
• Did the project meet objectives? No. Indy got the
  statue but lost in in the last phase to evil
  forces
• Project Costs High. One guide dead, a severely
  bruised Indy and a priceless archeological find
  the cave which contains photo-electric cells for
  example destroyed
• Project Benefits None - Indy did not save
  statue for society and research
• Overall success From all perspectives the Cave
  Project was dismal failure. Not only did the
  project fail to meet its objectives but one
  person died and the project manager was placed at
  risk throughout the project

22
Post-Implementation Review

• In addition, the project manager Dr. Indiana
  Jones didn't undertake planning or risk
  assessment and went into the Cave project
  completely un-prepared.
• Finally, Dr. Jones has exhibited no learning from
  the project and has continued throughout the
  entire three releases of the Indiana Jones
  Integrated Project (Release1 Raiders of the Lost
  Ark Release 2 The Temple of Doom Release 3
  The Last Crusade) to walk into other Cave
  Projects without any planning or formal risk
  assessment
• Conclusion Dr. Indiana Jones should be demoted
  from Project Manager to Trainee ASAP.

23
Risk Projection

• Each risk is rated according to likelihood and
  probability.
• The project manager performs the following risk
  projection activities
• Establish a scale that reflects the perceived
  likelihood of the risk.
• Delineate the consequences of the risk.
• Estimate the impact of the risk on the project.
• Note the overall accuracy of the risk projection.

24
1
2
1
2
1
2
1
2
Potential consequence of 1. of undetected
software errors or faults, 2. if desired outcome
is not achieved.
25
Activity 1Risk Analysis

• Form small groups of 4-6 people.(Try to make
  different groups from last time)
• Analyze the sample development risks from the NEC
  Corp project.
• Derive risk exposure class for each risk.
• Identify the top ten risks and identify a
  plausible risk reduction strategy.
• For the same risks identify a plausible
  contingency plan. (ie what you will do if the
  risk actually occurs.)

25 minutes
26
Security Risks

• An important area of risk management in software
  engineering involves security.
• Need to identify, mitigate, and/or develop
  contingency plans to deal with threats,
  vulnerabilities, and attacks.
• Threat Any potential occurrence, malicious or
  otherwise, that can have an undesirable effect on
  assets and resources associated with a computer
  system.
• Vulnerability A characteristic of a computer
  system that makes it possible for a threat to
  occur.
• Attack An action taken by a malicious intruder
  that involves the exploitation of certain
  vulnerabilities in order to cause an existing
  threat to occur.

Fundamentals of Computer Security Technology,
Edward Amoroso, Prentice Hall, 1994.
27
Types of Threats

• Disclosure ThreatThe dissemination of
  information to an individual for whom that
  information should not be seen. (ie a leak)
• Integrity ThreatUnauthorized change to
  information stored ona computer system or in
  transit between computer systems.
• Denial of Service ThreatAccess to some computer
  system resource is intentionally blocked as a
  result of malicious action taken by another user.
• IF denial of service occurs for a critical
  component such as activation of an alarm system,
  it impacts the security of the system.

Fundamentals of Computer Security Technology,
Edward Amoroso, Prentice Hall, 1994.
28
System Security Engineering Process
Security Engineering allows you to determine the
optimal security approach for a particular system
based on an identification of all relevant
factors.
Fundamentals of Computer Security Technology,
Edward Amoroso, Prentice Hall, 1994.
29
Threat Identification

• Brain storming
• Ongoing construction of a list of threats as they
  are noticed during development.
• Threat tree decomposition of an abstract threat
  into very specific ones.

Non-systematic approach to threat identification.
May easily miss important threats.
A more rigorous and systematic approach to
identifying threats.
Threat
Subthreat
Fundamentals of Computer Security Technology,
Edward Amoroso, Prentice Hall, 1994.
30
Example Hospital Computer System
HospitalComputer System
Non Patient Medical History
Patient Medical History
Lifethreatening
Non-lifethreatening
Disclosure
Integrity
Denial of Service
Disclosure
Integrity
Denial of Service
Patients medical information is corrupted.
Confidential patient information is disclosed
Confidential patient information is not available
Fundamentals of Computer Security Technology,
Edward Amoroso, Prentice Hall, 1994.
31
Using threat trees for Calculations

• Values can be associated with each node in the
  threat tree.
• Probability of occurrence
• Potential damage if threat occurs
• Level of effort required to enact the threat
• Threats can be DISJUNCTIVE or CONJUNCTIVE

Threat
Threat
Sub-threat
Sub-threat
Sub-threat
Sub-threat
OR
AND
DISJUNCTIVEThe threat occurs if either of the
subthreats materialize.
CONJUNCTIVEThe threat occurs if ALL of the
subthreats materialize.
32
Effort in Threat
Integrity threatEffort MODERATE
OR
Integrity sub -threatEffort MODERATE
Integrity sub -threatEffort HIGH

• In this example the attacker would likely choose
  the moderate effort option, therefore the
  integrity threat on the higher node is moderate.
• Attacker MAY attempt the high effort option if it
  involved less chance of being caught etc.

Fundamentals of Computer Security Technology,
Edward Amoroso, Prentice Hall, 1994.
33
Supporting System Security Engineering
Threat
OR
Subthreat ACriticality 4Effort 2Risk 2
Subthreat BCriticality 5Effort 1Risk 5
Criticality 5Effort 1Risk 1

• Analyze each of the risks to determine optimal
  placement of security protections.
• Following implementation of security protections,
  Subthreat Bs risk is reduced (shown in green)

Fundamentals of Computer Security Technology,
Edward Amoroso, Prentice Hall, 1994.
34
Software Engineering Risk ModelSERIM

• Applicable for small and medium sized
  organizations.
• Primarily qualitative with quantitative analysis.
• Applicable to full life-cycle including
  maintenance phase.
• Maintainable during project development.
• Compatible with any development model, such as
  spiral, waterfall, joint application design,
  recent application design, incremental, or
  prototyping.
• SERIM uses risk questions to derive numerical
  probabilities for a set of risk factors and
  analyses them using a simple spreadsheet.

35
Case Study Aircraft Computer System
TemperatureSensor (ts)
Connection to electromechanical servo to
control engine (e)
MainController(mc)
PositionSensor (ps)
Connection to electromechanical servo to
control cooling system (cs)
FlightRecorder (fr)

• Specify system architecture and prioritize the
  need for each component to be protected.
• Top priority mc
• High e, all interconnections, ps
• Moderate ts, cs
• Flight recorder low

36
Identify threats, vulnerabilities, attacks.
Threats to system components
Denial of ServiceG10, E2, R5
Disclosure
Integrity
fr
ms
ps
ts
e
int
cs
Dev
OpG5, E5, R1
Dev
OpG10, E2, R5
Fundamentals of Computer Security Technology,
Edward Amoroso, Prentice Hall, 1994.
37
SERIM Terminology

• Risk Elements Technical, Cost, Scheduling
  problems.
• Risk Factors - more specific subcategories of the
  three general risk elements (for example
  organizational risk). A risk factor can be
  related to one or more risk elements.
• SERIM identifies ten primary risk factors of
  organization, estimation, monitoring,
  development, tools, risk culture, usability,
  correctness, reliability, and personnel.
• Each software risk factor can have an influence
  on each risk element. That influence can be
  categorized as low, medium, or high.

38
SERIM Terminology
The impact of each risk factor on each risk
element.
The effect of risk factors on each risk
categories.
39
Risk questions

• One approach to assessing risk is to use risk
  questions to measure risk factors.
• Answers to the questions can be recorded in a
  yes-or-no format (0 or 1) or as a numerical range
  of possible responses.
• Response ranges can use any numerical value from
  0 through 1. For example, a response range could
  be defined as none 0, little 0.2, some 0.5,
  most 0.8, and all 1.0.

The relationship of risk questions to risk
factors and risk elements.
40
Organizational questions
O1. Are you using or do you plan to use
experienced software managers? O2. Has your
company been producing software similar to this
in the past? O3. Is there a documented
organizational structure either in place or
planned? O4. Is the organizational structure
stable? O5. What is the confidence level of your
management team? O6. Does good communication
exist between different organizations supporting
the development of the software project? O7. Are
software configuration management functions being
performed? O8. Are software quality functions
being performed?
Devise a numerical range of responses For
example in question O1 1 Mgrs with little or no
SE experience. 0.5 Mix of experienced and
non-experienced managers. 0 Only experienced
managers.
Scores
41
Calculating P(An) Example Organizational success
2. Calculate Imp / Impact Total.
3. Calculate of Impact X success.
1. Sum the impacts.
42
Quantitative Risk Analysis

• The probability of a successful project P(A) is
  derived using the following equation where E
  the number of risk elements.
• P(A) ? P(An)/(E)
• This equation assumes that all risk elements are
  equal in weight. P(A) is the probability of a
  successful project.
• If different risk elements have different impacts
  (weights), then P(A) w1P(A1) w2P(A2)
  w3P(A3), where w1 is a positive number and
  w1 w2 w3 1.

E
n1
43
Example

• 5 risk elements at the following probabilities
  for successA1 0.9, A2 0.75, A3 0.85, A4
  0.6, A5 0.95
• Risk elements weighted as
• A1 0.2, A2 0.4, A3 0.1, A4 0.1, A5 0.2
• Probability for successP(A) (0.90.2)
  (0.750.4) (0.850.1) (0.60.1)
  (0.950.2)
• P(A) 0.815
• Question Should we proceed??????

44
Decision Making Time

• Can you mitigate any of the risks?ie reduce the
  possibility of the risk or mitigate its impact IF
  it does occur.
• How much risk are you able to tolerate?
• What is at stake? ie is the risk worthwhile?

45
RMMM Plan

• Introduction
• Scope and purpose of document
• Overview of major risks
• Responsibilities
• Management
• Technical staff
• Project Risk Table
• Description of all risks above cut-off
• Factors influencing probability and impact
• Risk mitigation, monitoring, management
• Risk n
• Mitigation
• General strategy
• Specific steps to mitigate the risk
• Monitoring
• Factors to be monitored
• Monitoring approach
• Management
• Contingency Plan