Event Correlation Models for Detection of Attacks

1
Event Correlation Models for Detection of Attacks

• Annarita Giani, Alvaro Cárdenas, Shankar Sastry
• University of California, Berkeley

2
The Need to Correlate Events

• Large amount of sensors for network monitoring
• Intrusion Detection Systems
• Network traces
• File Integrity Checkers
• Large amount of Alerts
• Overloaded operators
• Hard to make sense of alarms
• Need a principled way of combining alerts
• Reduce false alarms
• Discover multistage attacks

3
We Introduce New Analytical Models for Event
Correlation

• We introduce new analytical models
• Process Query Systems (PQS) for the detection of
  cognitive attacks
• Flow Attribution and Aggregation
• Alert correlation in intrusion detection systems
  via Neyman-Pearson theory
• The value of analytical models in computer
  security is based on our previous work
• SP 06 We introduced a new analytical framework
  to explain and compare previous heuristic work
  for the evaluaiton of IDSs
• Infocom 07 We showed how a detection algorithm
  with a formal adversary model outperforms (for
  all adversaries) previously proposed detection
  schemes based on heuristics

4
Outline

• Motivation and Terminology
• Process Query System (PQS) Approach
• Flow Attribution and Aggregation
• IDS Alert Correlation
• Conclusion and Acknowledgments

5
Process Query System
Observable events coming from sensors
Hypothesis
Models
PQS ENGINE
Tracking Algorithms
6
Framework for Process Detection
FORWARD PROBLEM
INVERSE PROBLEM
An Environment
Indictors and Warnings
6
129.170.46.3 is at high risk 129.170.46.33 is a
stepping stone ......
that are used for control
that detect complex attacks and anticipate the
next steps
5
consists of
1
Hypotheses
Multiple Processes
Track 1
l1 router failure
Track 2
Track 3
l2 worm
l3 scan
Hypothesis 1
Hypothesis 2
that produce
2
that are seen as
4
that PQS resolves into
Unlabelled Sensor Reports
Events
.
.
Time
Time
3
Real World
Process Detection (PQS)
7
PQS in Computer Security
5
1
2
8
7
Internet
12
BRIDGE
Worm Exfiltration Phishing
DMZ
PQS ENGINE
WWW Mail
observations
WS
WinXP LINUX
8
Complex Phishing Attack Steps
as usual browses the web and
Stepping stone
Web page, Madame X
. visits a web page. inserts username and
password. (the same used to access his machine)
1
100.20.3.127
accesses user machine using username and password
2
5
165.17.8.126
records username and password
attacks the victim
uploads some code
4
3
downloads some data
Attacker
6
51.251.22.183
9
Most General Phishing Attack Model
ATTEMPT or UPLOAD
ATTEMPT
DOWNLOAD
ATTEMPT or UPLOAD
1
2
3
4
RECON
ATTEMPT
DOWNLOAD
RECON
Stricter models reduce false positives, but less
strict models can detect unknown attack sequences
10
Covert Channel in Interpacket Delays
Noisy Channel
SENDER
RECEIVER
INTERNET
0 1 0 0 0 1 0 1 0
0 1 0 0 0 1 0 1 0
We shall not spend a large expense of
timeBefore we reckon with your several
loves,And make us even with you. My thanes and
kinsmen,
We shall not spend a large expense of
timeBefore we reckon with your several
loves,And make us even with you. My thanes and
kinsmen,
11
Binary Asymmetric Channel Capacity
Capacity Highest amount of information per
symbol that can be transmitted with arbitrarily
small error probability.
Error Probability
Bit/symbols
24 hops.
Sent
Received
12
Statistical Detection
delays
sample mean
of packets with delay
max of packets with the same delay
MODEL
Delay tenth of a sec
Level of confidence
Threshold used in the PQS experiments
bits
13
Outline

• Motivation and Terminology
• Process Query System (PQS) Approach
• Flow Attribution and Aggregation
• IDS Alert Correlation
• Conclusion and Acknowledgments

14
Flow Analysis Data Reduction
Flow Aggregation
Fewer events to be analyzed
EVENTS
Hundreds per hour
Flow Attribution
FLOWS
Thousands per hour
Current Analysis
PACKETS
Hundreds of thousands per hour
How data move
BYTES
Billions per hour
15
Flow Attribution and Aggregation
FLOW ATTRIBUTION
FLOW AGGREGATION
Recognizing that different flows (components),
apparently totally unrelated, nevertheless belong
to the same broader action (event). Views flows
as components of broader activities. The goal
is to correlate flows based on certain criteria.
The final goal is to attribute flows to people.
Intermediate steps are a required part of the
attribution process. Uses logs that can explain
a flow as legitimate or malicious. The goal is
to explain flows.
16
Outline

• Motivation and Terminology
• Process Query System (PQS) Approach
• Flow Attribution and Aggregation
• IDS Alert Correlation
• Conclusion and Acknowledgments

17
Alert Correlation in IDSs

• Each IDSi has an estimated operating performance
  of
• Probability of Detection
• PDi
• Probability of False Alarm
• PFAi
• Comparing the likelihood ratio
  to a threshold t we introduce a fusion center
  with PD and PFA optimal in a Neyman-Pearson sense

18
Example Combination of two IDSs

• Let
• 0 denote no alarm
• 1 denote an alarm
• Examples of the likelihood ratio of two IDSs
• Neyman-Pearson theory tells us one IDS is always
  suboptimal

19
Outline

• Motivation and Terminology
• Process Query System (PQS) Approach
• Flow Attribution and Aggregation
• IDS Alert Correlation
• Conclusion and Acknowledgments

20
Contribution

• Identification of a new generation of threats
• Process Query System (PQS) based approaches to
  detect complex attacks and covert channels
• Flow aggregation definition, analysis and
  application
• Covert Channel Detection statistical theory of
  undetectable covert communication

21
Proposed Work

• Information theoretic approach to the design and
  analysis of undetectable covert channels.
• Development of a graph theoretic approach to the
  analysis of flow attribution and aggregation for
  computer security.
• Definition of metrics to capture closeness
  between activities and group them into patterns
• Integration of aggregated flows into a tracking
  system (PQS or Markov Chain Monte Carlo Data
  Association for Multiple -Target Tracking
  developed at UC Berkeley)
• Identification of new application domains