Security event processing with SEC Simple Event Correlator

1
Security event processing with SEC (Simple Event
Correlator)

• Risto Vaarandi
• Estonian Defence Forces

2
Outline

• Event correlation what it is and why do we need
  it?
• Simple Event Correlator (SEC) motivation,
  features and performance.
• Sample event correlation scenarios.
• SEC application experience and further reading.

3
Event correlation what it is?

• Commonly used formal definition (Jakobson and
  Weissman, 1995) event correlation is an
  interpretation procedure where a new meaning is
  assigned to a set of events that occur within a
  predefined time interval.
• A simple example if a network link down event
  is not followed by network link up within 5
  seconds, emit an output event network link
  broken otherwise create an internal event
  network link short outage. If more than 10
  network link short outage events have been
  observed within 3 hours, emit an output event
  network link quality degradation.
• Please note that event correlation is a real-time
  (or near-real-time) event processing technique!

4
Event correlation why do we need it?

• Event correlation produces events that are more
  meaningful to a human.
• Event correlation can significantly reduce the
  number of events that are presented to the human.
• These considerations are important, because in
  todays IT systems millions of events can occur
  in every hour no hope for a human to follow the
  events and react to them in a timely manner!

5
Example - centralized event management
infrastructure
Central log server
Application
event receiver
event correlator
Notifications to the monitoring console
Server
events
Router
network
Event logs
Applications, servers, and network devices send
their events to the central log server. Events
are stored and correlated on the central log
server, and output alerts from the event
correlator are sent to the central monitoring
console.
6
Simple Event Correlator (SEC) motivation

• Existing event correlation products are mostly
  heavyweight and expensive solutions
• many products can only be deployed as a
  client-server system with a requirement for a
  dedicated high-end event correlation server,
• some products have been designed for specific
  tasks only (like network fault management),
• several products depend on other complex and
  expensive software packages (e.g., Oracle),
• there are products that cant be deployed without
  the assistance from vendor consultants!
• there exist only a few free and open-source event
  monitoring tools that have some support for event
  correlation.
• SEC implements a completely different event
  correlation paradigm lightweight and platform
  independent event correlation with an open source
  tool

7
SEC design and features

• Employs rule-based approach for event correlation
  events are correlated according to rules
  written by a human analyst.
• Lightweight design the size of SEC is lt 300KB
  and lt 9000 lines, runs as a single process.
• Modest resource requirements e.g., with 10,000
  event correlation operations and contexts active,
  lt 20MB memory is consumed.
• Platform independence SEC is written in Perl,
  thus runs on almost any UNIX flavor and Windows
  (with CygWin Perl).
• No dependencies on other software (Perl only).
• Reads input events from one or more textual event
  logs (plugins can be written for binary formats).
• Input events are matched with regular
  expressions, Perl subroutines, string patterns,
  and truth values.
• Produces output by executing custom command lines
  (e.g., sends an SNMP trap or an e-mail), writing
  to files and FIFOs, etc.
• Licensing GNU GPLv2 (free and open-source
  software).

8
Snort IDS alert consolidation I

• The following ruleset processes Snort IDS
  syslog messages
• Apr 4 101055 snorthost.mydomain auth.alert
  snort18800
• 1252814 SMTP PCT Client_Hello overflow
  attempt Classification
• Attempted Administrator Privilege Gain
  Priority 1
• TCP 192.168.5.4328813 -gt 192.168.250.4425
• If a Snort priority 1 alert has been seen for a
  certain source IP address, create
• a context for this IP, in order to consolidate
  all events for the same source IP
• also, send a short note to the local admin that
  an attack has begun
• typeSingle
• ptypeRegExp
• patternsnort\\d\ \\d\.\Priority
  1\ \S (\d\.)?\d -gt \d\.?\d
• context!ATTACK_FROM_1
• continueTakeNext
• descPriority 1 attack started from 1
• actioncreate ATTACK_FROM_1 \
• pipe 's' mail -s 'Snort priority 1
  attack from 1 (alert)' root

9
Snort IDS alert consolidation II

• If a Snort priority 1 alert is observed for a
  certain source IP address, then
• 1) store it to the context for that IP (the
  context has been created by the previous rule)
• 2) extend the context lifetime for 5 minutes.
• If the context lifetime ends (in other words,
  there have been no more events from
• the given source IP), send all events from the
  context event store to the local admin
• as a single mail.
• typeSingle
• ptypeRegExp
• patternsnort\\d\ \\d\.\Priority
  1\ \S (\d\.)?\d -gt \d\.?\d
• contextATTACK_FROM_1
• descPriority 1 incident from 1
• actionadd ATTACK_FROM_1 0 \
• set ATTACK_FROM_1 300 ( report
  ATTACK_FROM_1 \
• mail -s 'Snort priority 1 attack from 1
  (report)' root )

10
User account probe detection I

• Process sshd PAM authentication messages from
  Solaris system log, in order to detect repeated
• probing of different user accounts
• Apr 3 142019 myhost sshd25888 ID 800047
  auth.error error
• PAM Authentication failed for risto from
  myhost2
• Apr 3 142023 myhost sshd25888 ID 800047
  auth.info Accepted
• keyboard-interactive/pam for risto from
  192.168.27.69 port 9729 ssh2
• If a user fails to login with SSH and this is
  not followed by a successful login within 30
  seconds,
• generate an event LOGIN_FAILED_FOR_username
• typePairWithWindow
• ptypeRegExp
• patternsshd\\d\ \ID \d auth\.error\
  error PAM Authentication failed for (\S) from
  \S
• descPAM authentication failed for 1
• actionevent LOGIN_FAILED_FOR_1
• ptype2RegExp
• pattern2sshd\\d\ \ID \d auth\.info\
  Accepted keyboard-interactive/pam for (1)\
• from \S port \d ssh2
• desc2PAM authentication successful for 1

11
User account probe detection II

• Count LOGIN_FAILED_FOR_username events from the
  previous rule if 10 distinct usernames
• have been seen within 10 minutes, alert the
  local admin then switch to silent mode for 1
  hour
• typeSingleWithThreshold
• ptypeRegExp
• patternLOGIN_FAILED_FOR_(\S)
• context!USER_1_ALREADY_COUNTED !COUNTING_OFF
• continueTakeNext
• descTen login failures for distinct users have
  been observed
• actionpipe 's' mail -s 'PAM alert' root create
  COUNTING_OFF 3600
• window600
• thresh10
• Set up a context for the username after it has
  been counted, in order to prevent repeated
  counting
• typeSingle
• ptypeRegExp
• patternLOGIN_FAILED_FOR_(\S)
• context!USER_1_ALREADY_COUNTED !COUNTING_OFF

12
User account probe detection III
PairWithWindow
Feb 18 121023 PAM authentication failed for
risto
Feb 18 121033 Accepted keyboard-interactive/pam
for risto
Feb 18 121511 PAM authentication failed for
john
Feb 18 121556 ntpd ..
Feb 18 121649 PAM authentication failed for
john
doesnt produce a match anymore at 121719
logonly PAM authentication successful for risto
(at 121033)
create USER_john_ALREADY_COUNTED
Single
event LOGIN_FAILED_FOR_john
LOGIN_FAILED_FOR_john
SingleWithThreshold
10 LOGIN_FAILED events for distinct users within
10m
neither USER_john_ALREADY_COUNTED nor
COUNTING_OFF context exists at 121541, so start
the counting operation with the window of 10
minutes and set event counter to 1
pipe send mail alert create COUNTING_OFF (for
getting 1 mail per 1 hour)
13
SEC performance

• SECs event processing speed depends on a number
  of factors the size and nature of the rule
  base, the nature of input data, and the
  underlying hardware.
• Experiment1 Linux workstation with a 3.0GHz P4
  CPU, one rule with a regular expression pattern,
  1 of input lines were matching and written to a
  file by the rule event processing speed was
  17,000 events per second.
• Experiment2 Linux server with a 3.2GHz P4 CPU,
  100 rules with complex regular expression
  patterns and non-trivial output actions (at least
  2-3 actions executed per second), banking card
  events with a complex structure as input event
  processing speed was 3,000 events per second.

14
SEC experience

• Has been developed by Risto Vaarandi for 8
  years.
• Employed by many institutions large
  North-American telecom companies, government and
  financial institutions, universities, medical and
  car industry, etc.
• Has been used for security management, network
  management, system monitoring, etc.
• Has been employed for correlating events from
  many sources Snort IDS, HP OpenView,
  CiscoWorks, BMC Patrol, Nagios,
  syslogd/syslog-ng, various routers and firewalls,
  modems and other network equipment, etc.
• Has been packaged for major Linux and BSD UNIX
  distributions RedHat, Fedora, Debian, Ubuntu,
  Gentoo, OpenBSD, FreeBSD.
• SEC mailing list has 300 members.

15
SEC references

• SEC home page http//simple-evcorr.sourceforge.ne
  t.
• John P. Rouillard, University of Massatchusetts
  at Boston, Real-time Logfile Analysis Using the
  Simple Event Correlator (SEC), USENIX 2004
  System Administration Conference (see also
  http//www.cs.umb.edu/rouilj/sec/).
• Jeff Becklehimer and Cathy Willis, Cray Inc. and
  Josh Lothian, Don Maxwell, and David Vasil, Oak
  Ridge National Laboratory (ORNL), Real Time
  Health Monitoring of the Cray XT Series Using the
  Simple Event Correlator (SEC), 2007 Cray User
  Group Conference (see also http//www.nccs.gov/wp-
  content/uploads/2007/08/becklehimer_paper1.pdf).
• James Turnbull, Hardening Linux (Chapter 5),
  Apress 2005 (see also http//www.apress.com/book/v
  iew/1590594444).
• Risto Vaarandi, Simple Event Correlator for
  real-time security log monitoring, Hakin9
  Magazine 1/2006 (6) (see also http//en.hakin9.org
  /attachments/pdf/hakin9_05_2006_10_EN_str28-39.pdf
  ).