A Structured Approach to Incident Response

1
A Structured Approach to Incident Response

• Morphing Incident Response into Incident
  Management

Peter Stephenson CISSP, CISM, FICAF, PhD Director
Information Assurance CeRNS The Center for
Regional and National Security Eastern Michigan
University
2
Introduction, Overview, Scope of the
Problem, and New Approaches
3
Elements of Incident Response (?)

• Shultz Shumway
• Preparation
• Detection
• Containment
• Eradication
• Recovery
• Follow-up
• vanWyk Forno
• Detection (Identification)
• Damage Assessment (Coordination)
• Damage Control (Mitigation)
• Damage Reversal (Investigation)
• Lessons Learned (Education)

4
Do You Get the Idea that There is Some
Confusion Here????
5
Forget What You've Been Told About Incident
Response!
6
Incident Response is REACTIVE... Incident
Management is PROACTIVE.
7
Heres How Its Really Done A Holistic Approach

• Preparation
• Incident response is an element of risk
  management
• Response
• Identify
• Contain
• Isolate for later analysis
• Recover
• Formal post mortem

8
The Incident Occurs

• The team already is in place and has been trained
• The plan already is in place and has been
  rehearsed and approved
• Resources already are in place
• Threats, vulnerabilities and impacts are known
• A triage approach has been established in advance

9
Managing the Incident

• Identify
• Identify that there is an incident
• Interdict
• Contain
• Respond in accordance with the plan
• Business needs come first, but
• Isolate for later analysis wherever possible
• Use a Command and Control architecture
• Recover
• Formal post mortem

10
Command and Control Architecture
11
Command Control Requirements

• Team structure
• Includes non-team members such as system
  administrators
• Central command
• Analysis, monitoring and management tools
• Link analyzers
• Forensics
• IDS monitors
• Network management
• All analysis is done by central command
• Issues suggestions to field teams
• Dispatches and makes team assignments
• Makes decisions
• Field teams
• Perform pre-designated tasks
• Commands eyes, ears and hands

12
Over the Next Two Days We Will

• Today
• Explore the role of incident management in the
  risk management process
• Introduce the incident environment
• Case study of an incident post mortem
• Explore types of incidents
• Build incident management into a risk management
  plan
• Learn how to train an incident management team
  and work with outside resources
• Tomorrow
• Match incident types to management techniques
• Explore some incident management tools
• Command and Control Architecture
• Manage a mock incident
• Map incident management to regulatory
  requirements

13
Risk Management

• Elements
• Threats
• Vulnerabilities
• Impacts
• Countermeasures
• Process
• Analysis - ongoing
• Response to analysis ongoing
• Incident management plan and practice
• Focus on business continuity
• Before, during and after the incident

14
Risk vs. Vulnerabilities

• Definition of risk
• The likelihood that a threat agent will
  successfully exploit a vulnerability to create an
  unwanted or adverse impact. (Jones)
• Risk consists of the agent causing the threat,
  the exploitable vulnerability, the impact of a
  successful attack and mitigating factors
• Vulnerabilities
• Weaknesses that may be exploited to cause an
  unwanted result

15
(No Transcript)
16
The Stages of the Incident Management Process

• Risk analysis
• Develop a triage plan for business continuity
• Risk and incident management plan
• Feed back risks that could enable an incident
  into the plan for proactive remediation
• Training, tools and relationships
• Everyone who could be touched by an incident is
  on the team
• Manage the incident

17
The Stages of an Incident

• Interdiction
• Containment
• Recovery
• Analysis

18
What We Mean By

• Interdiction
• Stopping or interrupting the incident
• Containment
• Isolating damage and preventing it from spreading
• Recovery
• Returning the business to the pre-incident state
• Analysis
• Post-incident root cause analysis (post mortem)

19
Security Policy Domains

• A domain whose objects are all governed by the
  same security policy. There are several types of
  security policy domain, including access control
  policy domains. Corba
• The scope over which a security policy is
  enforced. There may be subdomains for different
  aspects of this policy. NIST
• an environment or context that is defined by a
  security policy, a security model, or
  architecture, and includes a set of system
  resources and a set of entities that have the
  right to access the resources. -"Requirements
  for the Multidimensional Management and
  Enforcement (MSME) System."

20
Example of Security Policy Domains
21
Policy Domains - Details

• Concerned with both the scope of the domain and
  the interconnections between it and other
  security policy domains
• Represent communications channels
• May be authorized or covert
• Impact data flows
• May contain multiple components or a single high
  sensitivity/criticality component
• Policy may be explicit (corporate policy,
  procedure or guideline)
• Policy may be implicit (configuration of devices
  governed by a policy)
• Instantiation of the policy

22
An Incident Post Mortem Case Study
23
Late in January of 2003 a worm variously known as
SQLSlammer and Sapphire was released into the
wild. It flourished on the Internet faster than
any other worm or virus in the Internets
history. Internet storm centers around the world
watched in amazement as the worm literally
took over the Internet in a matter of
hours. Shortly after the release of the worm a
large financial services company was infected.
The worm spread throughout the organizations
27,000 user network in the United States in under
ten minutes, completely flooding the network. It
was noticed, although it did no damage, in the
companys international subsidiaries. This
session describes the post-incident root cause
analysis process and the analysis of the
infection of the victims enterprise.
24
Victims Environment

• Enterprise architecture
• Switches with routed core
• Limited safeguards
• High performance high availability
• Firewalled perimeter, few, if any, internal
  firewalls
• Perimeter network, wireless LAN, extranets
• 500 SQL servers, 1,000 other MSDE (Microsoft
  Database Embedded) systems
• Flat security policy domains
• No patched SQL Servers
• Internal political considerations
• Viewed the threat as trivial and did not patch
  servers
• Lack of communication between IT and security
• Naïve view of system safeguards in general

25
The Team

• Technical, investigation and business process
  specialists
• Lead investigator and forensic specialist
• Two network/platform specialists
• One code specialist
• Three business process analysts
• Quality control/evidence manager
• The team had 5 days to analyze the event on a
  27,000 user network spanning two countries

26
Areas of Investigation

• Technical data analysis
• Business process analysis
• Regulatory compliance and loss analysis

27
Evidence Categories

• Logs from monitoring devices
• Logs from hosts and servers
• Interviews with involved personnel
• Interviews with business and technical managers
• Device configuration files
• Network maps
• Event observation timelines
• Notes of relevant meetings
• Response team notes and observations

28
Impediments

• Logs missing or incomplete
• Interviews uncovered conflicting statements
• Critical devices such as edge routers had been
  purged following the incident
• Intrusion detection logs could not be extracted
  from their original digital format
• Project manager stopped the investigation when he
  saw we were getting close to a solution
• Fundamental business processes that would enable
  the investigation were not in place

29
Hypotheses

• Lack of firewalls between the internal network
  and the VPN, wireless network and extranets
• SQL Servers not patched
• Lack of policy enforcement
• Naïve view of layered security requirements
• Flat network security architecture

30
A Model of an SQLSlammer Infection
31
Conclusions

• The worm entered the network through the
  unprotected wireless LAN
• There were possible countermeasures that could
  have prevented the success of the incident
• Blocking covert channels into the network
• If even one countermeasure failed, the attack
  would succeed
• Nothing to be gained by patching the SQL servers
• Because there were three covert channels without
  countermeasures, the attack had to succeed
• Proof The worm was blocked at the only entrance
  to the foreign subsidiary and, although it was
  seen by the intrusion detection system, it never
  entered the foreign enterprise and no infection
  resulted

32
Understanding the Incident Management Environment
33
Incident Types

• Penetration
• Fraud
• Denial of service
• Virus/worm infection

34
Penetration

• Purposes
• Data theft
• Extortion
• Joy riding
• Web defacement
• Incident management preparation
• Preventative and detective controls
• Managing penetration incidents
• Interdiction terminate connection if on-line,
  notify law enforcement as appropriate, launch
  internal investigation
• Containment Locate root kits, compromised
  accounts, etc. and correct, correct vulnerability
  that allowed initial penetration
• Recovery analyze damage and respond
• Analysis generally part of the containment and
  recovery stages, formal incident post mortem at
  completion of incident

35
Fraud

• Incident management preparation
• Preventative and detective controls
• Managing fraud incidents
• Interdiction terminate connection if on-line,
  notify law enforcement as appropriate, launch
  internal investigation
• Containment Locate root kits, compromised
  accounts, etc. and correct, correct vulnerability
  that allowed initial penetration
• Recovery analyze damage and respond
• Analysis generally part of the containment and
  recovery stages, formal incident post mortem at
  completion of incident

36
Denial of Service

• Incident management preparation
• Harden and test perimeter
• Relationship with ISP
• Managing denial of service incidents
• Interdiction terminate connection via ISP
  backbone routers
• Containment if necessary, terminate Internet
  connection until attack subsides, isolate
  vulnerable/critical assets temporarily
• Recovery analyze damage and respond
• Analysis formal incident post mortem at
  completion of incident

37
Virus and Worm Infection

• Incident management preparation
• Harden and apply defense in depth
• Use security policy domains
• Relationship with ISP
• Managing virus and worm incidents
• Interdiction terminate connection via ISP
  backbone routers to stop incoming worm or virus
  attack
• Containment isolate infected security policy
  domains temporarily
• Recovery analyze damage and respond, dont
  re-open infected domains until all domains have
  been cleared
• Analysis formal incident post mortem at
  completion of incident

38
IMPORTANT CONCEPT One Size Does Not Fit All -
- Each Type of Incident Requires a
Specific,Planned Response Unique to Your
Organization
39
Building Incident Management Into Your Risk
Management Plan

• Incidents pose risks and must be planned for in
  advance
• Resources needed
• Management procedures
• Reducing the risks that could permit an incident
• Threat management
• Preventative and detective controls
• Understand how the elements of risk affect
  incident management

40
Elements of Risk

• Malicious threat factors
• Capability
• Motivation
• Access
• Catalysts
• Inhibitors
• Amplifiers
• Vulnerabilities
• Very difficult to manage
• Impacts
• Cannot quantify the unquantifiable
• Complex impacts impact impacts
• Mitigating factors
• Safeguards or countermeasures that reduce,
  redirect or eliminate impact

41
Threat Components and Relationships
42
30 General Threats CC-PKB

• Administrative errors of commission
• Administrative errors of omission
• Hostile administrator modification of user or
  system data
• Administrator violates user privacy policy
• A critical system component fails
• Software containing security-related flaws
• Failure of a distributed system component
• Hacker undetected system access
• Hacker attempts resource denial of service
• Hacker eavesdrops on user data communications
• Cryptanalysis for theft of information
• Hacker masquerading as a legitimate user or as
  system process
• Message content modification
• Exploitation of vulnerabilities in the physical
  environment of the system

• Social engineering
• Malicious code exploitation
• Unexpected disruption of system or component
  power
• Recipient denies receiving information
• Sender denies sending information
• A participant denies performing a transaction
• Legitimate system services are spoofed
• Hostile user acts cause confidentiality breaches
• User abuses authorization to collect data
• User errors cause confidentiality breaches
• User error makes data inaccessible
• User errors cause integrity breaches
• User errors undermine the system's security
  features
• User's misuse causes denial of service
• User abuses authorization to modify data
• User abuses authorization to send data

43
Vulnerabilities

• More difficult to manage than risks
• New vulnerabilities constantly emerging
• Zero-day exploits
• Vulnerability groups are easier to manage than
  individual vulnerabilities
• 30 50 new vulnerabilities reported per week
• Symantec reported an average of 47 per day!
• Patch management, done correctly, is virtually
  impossible on large enterprises
• Vulnerability groups are easy to define
• 53 Basic families from the CVE

44
Vulnerability Groups

• Penetration and escalation of privileges
• Denial of service
• Unusual access
• Usually application-based
• Flooding
• Probe

A Common Intrusion Specification Language
http//home.comcast.net/prstephenson/other_paper
s/CISL-Original.PDF
45
Determining Impacts

• Is there a credible threat and threat agent?
• Are there vulnerabilities for the threat agent to
  exploit?
• Would exploiting the vulnerability cause harm to
  the organization?
• If the answers to all three are yes, manage the
  risk if not, ignore it.
• BUT - - do NOT be naïve about your answers!
• Recall the Slammer case study

46
Applying Countermeasures or Safeguards

• Many small safeguards may be better than one
  large one
• Defense in depth
• Apply where you can spread the cost wherever
  possible
• Avoid causing a bottleneck with a single large
  safeguard
• Use administrative controls where possible
  because they cost the least
• ALWAYS plan for the unplanned-for incident

47
Managing Threats During the Incident

• First stage objective
• Applying threat inhibitors and removing access to
  interdict the application of the threat against
  vulnerabilities
• May be technical
• Filtering at the perimeter
• Identifying threat agents and removing catalysts
• Second stage objective
• Contain damage and prevent spread
• Water-tight bulkheads already should be in place
• Security policy domains

48
Managing Vulnerabilities During the Incident

• First stage objective
• Take primary vector vulnerable systems off line
• Second stage objective
• Isolate vulnerable systems
• Third stage objective
• Remove vulnerabilities, obtaining evidence
  wherever possible
• Fourth stage objective
• Analyze evidence to understand how the
  vulnerability responded to the threat

49
Managing Impacts During the Incident

• First stage objective
• Reduce through interdiction and redirection
• Second stage objective
• Limit spread
• Security policy domains and water-tight bulkheads
• Third stage objective
• Ensure that recovery is complete
• Fourth stage objective
• Analyze and quantify
• Lessons learned MUST be applied

50
The Incident Management Team

• EVERYONE is on the team!
• Core team manages and plans for the incident
• Everyone who may be touched by an incident has a
  role to play and must be prepared to play his or
  her role
• Plan for and implement a command and control
  architecture
• Train and rehearse key players in the use of the
  architecture
• Lesson learned from the Slammer case study there
  were early warning signs for two hours prior to
  meltdown that would have helped avoid serious
  damage, but the network ops center operators
  missed them

51
The Command and Control Architecture

• Layered response
• First responders
• Incident Command Center
• Field responders
• Centralized incident management
• Each layer has specific responsibilities
• Each layer requires specific training
• Each layer requires specific tools

52
First Responders

• The first individuals likely to detect the
  incident
• System administrators
• Network control center operators
• Network security team
• Training
• Signs of an incident
• Local policies, procedures and incident
  management roles
• Including their roles
• Incident management contact list
• Evidence management requirements and techniques
• Tasks
• As defined in the Incident Management Plan
• Stage 1 know what, when and how to interdict
• Stage 2 know security policy domains and their
  containment procedures
• Stage 3 be prepared to follow recovery
  instructions
• Stage 4 participate in post mortems

53
Incident Command Center

• Key team members who will manage the incident
• Center of all control and all command decisions
• Performs all analysis during the incident
• Collects all data
• Training
• All elements of incident management including
  local policies and procedures
• Tasks
• As defined in the Plan
• Stage 1 collect data from first responders,
  determine interdiction process, direct
  interdiction procedures, involve outside
  resources as necessary
• Stage 2 determine appropriate containment
  perimeters, direct containment procedures
• Stage 3 collect evidentiary materials such as
  logs, forensic images, etc., determine recovery
  triage, direct and coordinate recovery
  procedures, involve outside resources as
  necessary
• Stage 4 conduct incident post mortems

54
Field Responders

• All individuals who may be affected by the
  incident and who may be called upon to assist in
  its management
• Training
• Elements of incident management that impact them
  directly
• Tasks
• As defined in the Plan
• Stage 1 none
• Stage 2 as directed by Command Center
• Stage 3 As directed by Command center
• Stage 4 participate in post mortems as
  requested

55
Tools of Incident Management

• Evidence collection and management
• Intrusion detection systems
• EnCase Enterprise
• Sniffers
• Log management systems
• Command and control
• Communications tools
• Cell phones, BlackBerries, alternative email
  systems, etc.
• Evidence management interface
• SceneSearch Corporate version
• (http//www.barons-inc.com/software.htm)

56
We're Done for Today- - See You Tomorrow!!
57
TEAM EXERCISE Building an Incident Management
Plan You Will Have 2 Hours After Which Teams
Will Present Their Plans
58
Questions Your Plan Should Address

• What kind of incident does your plan address?
• How do you know its an incident?
• Has it passed, is it in progress, or do expect it
  to recur?
• What is the early warning mechanism?
• How does your plan deploy
• Command Center
• First Responders
• Field Responders
• External (to the response team) resources
• How does your plan
• Interdict?
• Contain?
• Recover?
• Analyze?

59
Matching Incidents to Incident Management
Techniques

• Penetration
• Identify attacker
• Permit or deny continued penetration attempts
  from this attacker? Honeypots? Special logging?
• Identify attackers actions
• Fraud
• Nearly the same as penetration
• Manage internal fraud per corporate policy
• Denial of service
• Brute force response to stop the impacts speed
  and ISP relationships important
• Virus/worm infection
• Focus upon containment first, then interdiction
  speed important

60
Some Incident Management Tools

• EnCase Enterprise
• ProDiscover
• SceneSearch
• Ethereal
• NetForensics

61
EnCase Enterprise Open Ports
62
EnCase Enterprise Running Processes
63
EnCase Enterprise Open Files
64
EnCase Enterprise NIC Settings
65
EnCase Enterprise IP Filtering Settings
66
SceneSearch Evidence Management
67
SceneSearch Evidence Report
68
Ethereal Sniffer Capture
69
NetForensics External Penetration
70
TEAM EXERCISE Mock Incident You Will Have 1
Hour After Which Teams Will Present Their
Responses.
71
Mock Incident Test Your Plan

• Use your Incident Management Plan
• Set up command and control
• Describe how you would manage the incident from
  first indications up to, but not including, post
  mortem
• Take notes on all teams our next exercise is
  the post mortem
• You may not use any resources that you actually
  do not have

72
Scenarios

• Group 1
• You get an email demand addressed to the CEO by
  name at his private email address and addressing
  him by his first name. It is a demand for a
  consulting contract or the 150,000 credit card
  numbers they stole from you will begin to become
  public. This happens at 430PM Friday before a
  long weekend.
• Group 2
• Denial of service attack against your main site
  that starts rotating among sites before you can
  stop it at any single site it just stops and
  moves.
• Group 3
• Your SysAdmin quits and leaves a Trojan that lets
  him get back in. He goes to work for your main
  competitor and starts downloading customer info
  from you.
• Group 4
• A hacker finds a flaw in your on-line banking
  system and penetrates to the back-end system
  where he alters his account, creates multiple
  bogus accounts and starts moving money off shore
  using bogus, but convincing, letters of credit

73
CLASS EXERCISE Post-Incident Root
Cause Analysis (Post Mortem) You Have 1 Hour to
Work in Your Groups After Which We will Conduct
the Post Mortem In Groups and As a Class
74
Incident Post Mortem

• Identify root cause
• Identify steps taken at each stage
• Critique processes and procedures
• What countermeasures were missing? Which were
  effective?
• What management tools were missing? Which were
  effective?
• Identify lessons learned
• Critique the Plan and its execution

75
Mapping Post Mortem To Regulatory Requirements

• Final reports should include a set of mappings to
  applicable regulatory requirements
• Begin by mapping to ISO17799
• Then map directly to your applicable requirements
• Mapping to your internal policies is a good idea.

76
Mapping Example
77
CLASS EXERCISE Requlatory Mapping of Post
Mortem You Have 1/2 Hour to Work in Your
Groups After Which We Will Discuss Your
Mappings In Groups and As a Class
78
Regulatory Mappings

• Map your post mortem results to the ISO 17799
  items on the next slide
• Be prepared to discuss why your findings violate
  a mapping
• Be prepared to discuss a recommended
  countermeasure

79
ISO 17799 Groupings

• Information Security policy
• Organizational Security
• Infrastructure
• Third party access
• Outsourcing
• Asset Classification
• Accountability for assets
• Information classification
• Personnel Security
• In job definition
• User training
• Responding to incidents
• Physical and Environmental Security
• Secure areas
• Equipment security
• General controls
• Communications Operations Management
• Operational procedures and responsibilities
• System planning and acceptance

• Access Control
• Business requirements
• User access management
• User responsibilities
• Network access control
• Operating system access control
• Application access control
• Monitoring system access and use
• Mobile telecommuting and teleworking
• Systems Development Maintenance
• Security requirements of systems
• Security in application systems
• Cryptographic controls
• Security of system files
• Security in development and support
• Business Continuity Management
• Aspects of BCM
• Compliance
• Compliance with legal requirements

80
That's All, Folks... It's Been A Pleasure and I
Hope This Was Useful to You!!
Email me at peter.stephenson_at_emich.edu My web
site is http//people.emich.edu The CeRNS web
site is http//cerns.emich.edu Join the CeRNS
Mail List at https//list.emich.edu/mailman/listi
nfo/cerns