INTRUSION DETECTION ALARM CORRELATION: A SURVEY

1

• INTRUSION DETECTION ALARM CORRELATION A SURVEY

Urko Zurutuza Ortega San Sebastian, December
1-3, 2004
2
Overview
Introduction

• Introduction
• IDMEF
• Alarm preprocessing
• Alarm Analysis
• Alarm Correlation
• Conclusions and Further Work
• Questions

IDMEF
Alarm preprocessing
Alarm analysis
Alarm correlation
Conclusions Further Work
Intrusion Detection Alarm Correlation A Survey
IADAT International Conference on
Telecommunications and Computer Networks 2004
3
Background
Introduction

• Computer security Research group in MGEP-MU
• Intrusion Detection 17 years of research
• Intrusion Any set of actions that attempt to
  compromise the CIA (Confidentiality, Integrity,
  and Availability) of a resource

• Security models
• Audit and evaluation mechanisms
• Intrusion detection

4
Background
Introduction

• Intrusion Detection
• The process of monitoring the events occurring in
  a computer system or network and analyzing them
  for signs of intrusions.
• Types
• Many arecomplementary

5
Background
Introduction

• Intrusion Detectors
• .

6
Background
Introduction

• Problem
• Unmanageable amount of alarms
• gt10,000 alarms per day sensor!
• gt90 of alarms are false positives!
• Solution
• ALARM CORRELATION
• The reciprocal relationship between two or more
  objects or series of objects

7
Background
Introduction

• Alarm correlation process
• The necessary steps to be taken in order to
  discover the connection between different series
  of security events

8
Surveyed authors
Introduction

• Frédéric Cuppens et al., MIRADOR project
• Ecole Nationale Supérieure des Télécommunications
  de Bretagne, France.
• Peng Ning et al., Intrusion Alarm Correlation
  project
• North Carolina State University, USA.
• Klauss Julisch, CLARAty project
• IBM Research Laboratory , Zürich, Switzerland
• Some others Manganaris et al., Debar Wespi,
  Tim Bass, Clifton Gengo, Valdés Skinner, Dain
  Cunningham

9
INTRUSION DETECTION MESSAGE EXCHANGE FORMAT

• Different IDS vendors,different alarm formats
• A future standard for IDS alarm output(now
  Internet draft)
• Model implementedas a DTD, to describeXML
  documents

IDMEF
10
Alarm Preprocessing - Cuppens

• Cuppens et al.
• Alarms compliant with the IDMEF
• The XML documents translated to Logical
  Predicates
• The facts are converted into relationships of a
  relational data base eschema

Alarm preprocessing
11
Alarm Preprocessing - Ning

• Ning et al.
• Logical predicates, to model alarms as
• Prerequisites
• Consecuences
• of an attack
• Hyper-alert, a triplet of (fact, prerequisite,
  consecuence)
• Fact set of names of the attributes, and their
  values
• Prerequisite a logic conbination of predicates
• Consecuence a logic conbination of predicates
• Example. Buffer Overflow against Sadmind remote
  administration tool
• SadmindBufferOverflow (VictimIP, VictimPort,
• ExistHost (VictimIP) ? VulnerableSadmind
  (VictimIP),
• GainRootAccess(VictimIP))
• Representation implemented by a DBMS

Alarm preprocessing
12
Alarm Preprocessing - Julisch

• Julisch
• Alarms tuples over a Cartesian product, in a
  multi-dimensional space.
• Dimensions alarm attributes (source/dest IP,
  sorce/dest port, type of alarm, timestamp,)
• Alarm log a set of alarms
• Taxonomy a generalization hierarchy, represented
  as trees for every given attribute

Alarm preprocessing
13
Alarm analysis

• Group simple alarms in higher level ones fusion
• Using data fusion techniques of military
  applications (Bass, 2000)
• Using data mining (associacion rules) to
  discover frequent alarm sets and thus improve
  anomaly detection (Manganaris et al., 2000)
• Using data mining to filter alarms (looking for
  frequent alarm sequences) (Clifton and Gengo,
  2000)
• Using probabilistic similarity measures (Valdés
  and Skinner, 2001)
• Using explicit rules looking for alarms
  containing identical attributes (duplicates)
  (Debar and Wespi, 2001)

Alarm analysis
14
Alarm analysis

• Cuppens et al.
• Using an expert system where the similarity
  measures are specified by expert rules
• Rules for similarity relationship between
  attributes
• classification similarity
• temporal similarity
• source similarity
• target similarity
• After measuring similarity, alert instances are
  assigned to global alerts (or clusters)
• Redundancies are avoided so a specific event
  emerges just once,even if several alerts have
  detected it

Alarm analysis
15
Alarm analysis

• Ning et al.
• Fusion of alerts is allowed during and after
  correlation (once the attack scenario is created)
• Hyper-alert a single alert or several linked
  ones
• Different utilities for alarm analysis
• aggregation/disaggregation of alerts
• focused analysis
• analysis of clustering
• frequency analysis
• link analysis
• association analysis

Alarm analysis
16
Alarm analysis

• Julisch
• Similarity between alarms based on defined
  taxonomies
• The closest their attributes are within certain
  taxonomy, the more similar two alarms will be
• Attribute-Oriented Induction data mining
  heuristic algorithm is implemented
• Generalised alarms are obtained
• This allows discovering which the root causes of
  having those alarms are
• Removing the root causes, future load of alarms
  are reduced over a 90

Alarm analysis
17
Alarm correlation

• Group high level alarms into attack scenarios
• Relaxing probabilistic similarity measures of the
  attack types (Valdés and Skinner, 2000)
• Two different alarms reported with the same
  source and destination addresses, and close in
  time
• Using data mining (decision trees) for the
  estimation of the probability of a new alert to
  belong to a certain scenario (Dain and
  Cunningham, 2001)
• Using the explicit rules algorithm. It performs
  the aggregation relationship where alerts are
  added depending in common characteristics and
  forming situations (Debar and Wespi, 2001)

Alarm correlation
18
Alarm correlation

• Cuppens et al.
• Use their LAMBDA languaje (based on LP), to
  specificate elementary attacks
• Specify logic links between the post-condition
  of an attack and the pre-condition of another
  attack
• Automatically derive correlation rules from the
  specification of elementary attacks
• The system receives the fusion alerts and tries
  to correlate them one by one using correlation
  rules
• When a complete or a partial intrusion scenario
  is detected, a scenario alert is generated

Alarm correlation
19
Alarm correlation

• Ning et al.
• Using predicates to represent prerequisites and
  consequences of attacks
• Check if an hyper-alert contributes to the
  prerequisite of a later one
• The result is a graph representing the attack
  scenario

Alarm correlation
20
Conclusions Further Work

• Correlation improves the accuracy of IDS
• Decreases the number of alarms to handle
• Decreases the number of false positives
• Helps on anomaly detection
• Improves the knowledge of attacks
• Different terms for correlation, common criteria
  needed!
• Different ways to issue the problem
• Correlation between IDS alarms
• Correlation taking into account other information
  (firewall logs, network topology, vulnerability
  assessment tools, )

Conclusions Further Work
21
Conclusions Further Work

• Similarity measures between attributes cannot
  discover the real reasons of why the alerts are
  correlated
• Previously specified scenarios are limited to
  discover only known scenarios and a great hand
  work is needed
• Great amount of configuration parameters
• Correlation must be in real time effective
  response

Conclusions Further Work
22
Questions
Thank You
?
Questions