Intrusion Detection Issues

1
Intrusion Detection Issues

• UCD Computer Security Lab
• Jeff Rowe

2
The UCD Computer Security Lab

• Co-Directors Karl Levitt, Matt Bishop
• 5 Contributing Professors - Felix Wu, Prem
  Devanbu, Michael Gertz, Biswanath Mukherjee, Raju
  Pandey
• 3 Staff Researchers Jeff Rowe, Poornima
  Balasubramanyan, Rick Crawford
• 20 Graduate Students

3
UCD Seclab Accomplishments

• First network based IDS
• First specification based IDS
• First large-scale distributed IDS
• Current Research Areas
• Intrusion Detection
• Automated Response
• Vulnerablities
• Privacy
• Policy

4
Goals of Computer Network Security (CIA)

• Confidentiality
• protect information from unauthorized disclosure
• Integrity
• protect against unauthorized modification of
  information or references to that information
  (e.g., labels)
• Access
• Ensure access to information and system does not
  fall below threshold

5
Generalized Attack Scenario

• Move from outside system to inside system
• Move from not enough privilege to enough
  privilege
• Carry out deeds

6
Complex Attack Scenario
7
Intrusion-Detections Role in Network Security

• Design a securable system
• Harden system where appropriate
• firewalls
• encrypted links
• Audit the system
• static checks
• COPS, SATAN, Tripwire
• risk analysis
• Monitor the system

8
What Data to Monitor

• Audit trails generated by OS
• Accounting records
• Error messages
• syslog
• Network traffic
• directly
• router or RMON mibs
• Application-generated information
• Databases

9
Reasons to Monitor

• Gives a better understanding of the network
  environment
• Indicates policy violations
• Determines what has been compromised
• Provides accountability
• Deters someone in the first place
• Detects activity in a timely fashion
• Provides evidence to build a case

10
Reasons not to Monitor

• Its better to prevent than detect
• Many alarms may be false
• Oftentimes it is unclear what to do even when a
  real attack is detected
• Potential invasion of privacy
• Voluminous amounts of data to store and analyze
• Obligation to track down the sources of the attack

11
Example Audit Record
12
Example ls
13
ls, cont.
14
ls, cont 2.
15
Intrusion-Detection Definition
Intrusion-detection is the automated analysis of
data representing activity on a system for the
purpose of identifying activity which violates
the the confidentiality, integrity, or
accessibility of the system.
16
Goals of an Intrusion-Detection System

• Detect intrusions in a timely fashion
• Present summary analysis
• Detect a wide variety of intrusion types
• NOT be annoying

17
Insider vs. Outsider
Outsider
Insider

• Needs access to system
• Works quickly to avoid detection
• Installs trapdoors for further access
• Works in unfamiliar environment

• Has access to system
• Works leisurely
• Future access guaranteed
• Works in familiar environment
• Knows what is important

18
Intrusion-Detection Architecture
19
Intrusion-Detection Architecture, cont.
20
Distinguish Between Algorithms and Systems

• Most early systems have faded
• Systems success owes only a small part to the
  technology
• market for technology
• timing (byte code, p-code, revenue bomb)
• marketing
• usability

21
Classes of Algorithms/Approaches

• Signature-based
• NetRanger
• STAT/USTAT
• Nearly all commercial systems NFR, ISS
  RealSecure, Snort,
• Anomoly-based
• NIDES ( Theresa Lunt SRI )
• Emerald ( Phi Porras - SRI)
• JAM ( Sal Stolfo )
• Specification-based
• Calvin Kos Execution Monitor, SHIM

22
Example Code Red Worm
"GET /default.ida?NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
NNNNNNNNNNNNNNNNNNNNN NNNNNNNNNNNNNNNNNNNNNNNNNNNN
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN NNNNNN
NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
NNNNNNNNNNNNNNN NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNu90
90u6858ucbd3u7801u9090u6858
Univ. of Moscow Home Page
Hacked by Chinese
Jeff Rowes Home Page
Nokia.com Home Page
Hacked by Chinese
Hacked by Chinese
23
Signature-based Detection

• Detects known classes of bad behavior
• Can encode policy violations
• Uses a large signature database
• Pros
• Low false alarm rate
• An alarm for scripted attacks
• Cons
• May miss unknown attacks
• Trivial variants may be missed
• No clear distinction between serious and minor
  attack signatures
• First signature based network IDS developed at
  UCD by Todd Heberlein

24
CodeRed Signature

• The string GET /default.ida?NNNNNNNNNNNNNN
  present in network requests
• Network connections initiated by web server to
  port 80 on external machines.
• Large number of failed connections to port 80.
• Trivial variations trick signature based systems

25
Anomaly Detection

• Most early IDS efforts focused on anomaly
  detection
• Statistically-based
• Detects deviations from expected (or normal)
  behavior
• Needs profile of normal behavior (needs to have
  a training phase)
• Pros
• Can uncover unknown misuse patterns
• Normal profiles can me generated automatically
  through machine learning
• Cons
• Anomalous does not mean misuse
• Can generate a lot of false alarms

26
Anomalous CodeRed Behavior

• IIS process creates a new home page file
• IIS process creates an executable file (Trojan
  Horse) in an unusual directory
• IIS process opens a socket and connects to
  external web servers.
• Not all anomalies are attacks

27
NIDES Analysis

• Analysis is subject-based
• user (rowe)
• login session (rowe logged in at ttyxx)
• application
• entire system
• router
• Subject represented by a feature vector
• ltQ1, Q2, ..., Qngt
• Qi represents single measure (CPU usage)

28
Single Variable Intrusion-Detection

• Qi, when analyzed?
• session close
• continuously
• Analysis at 10am vs. 5pm?
• total count
• recent activity
• Similar to 30 day moving average in stocks

29
Two Variable Intrusion Detection

• Aggregating two S scores
• Euclidean approach
• Score sqrt(S12 S22)
• IS or T2 calculation
• IS lt S1,S2 gtC-1 ltS1,S2gtt S12 S22
• T2 (S1 S2)/n

30
Specification-based Analysis

• Developed by Calvin Ko, UC Davis
• Motivation
• programs, especially utilities, are more well
  behaved than humans
• many attacks involve exploitation of privileged
  program
• Easier to check then to verify programs
• Middle ground between Anomaly detection and
  Signature detection.

31
Specification Language

• Notation
• Parallel environment grammars
• Specifying trace policies

32
Example Trace Policies

• Rdist
• Fingerd
• Binmail

33
Web Server Specification

• Web server listens on port 80 and replies to
  request.
• Web server reads from document tree
• Web server executes from cgi-bin directory
• Web server writes to log files
• Catches CodeRed and variations.

34
Running your own IDS

• Snort is an open source network IDS
• Mainly a single packet analyzer
• Simple stateful pre-processors are included
• Supports plugins You can quickly prototype your
  own IDS algorithms
• MySQL, PHP GUI, Response add-ons
• Runs on Linux, FreeBSD, Windows, etc.
• www.snort.org

35
Whats Next?

• IDS report correlation
• Automatic response to IDS alerts
• Forensics when IDS fails
• Privacy and IDS monitoring

36
IDS Report Correlation

• Put together the big picture.
• Portscan, buffer overflow, anomalous activity all
  in the same time frame.
• Reduce false alarms
• Inter-domain cooperation
• Report collection and correlation methods that
  scale to large networks

37
GrIDS

• Correlates data including IDS reports to detect
  distributed coordinated attacks
• Report aggregation and monitoring scheme scalable
  to large networks
• Developed at UCD by Stuart Staniford
• First large scale IDS

38
GrIDS Graph-building Engine

• Graphs abstract network activity
• Configurable rulesets determine when graphs
  combine
• Host-based data is included as node attributes
• Attack assessment is made using global aggregated
  graph attributes.

Network based reports
Host based reports
Data Sources
spurr
kanab
kanab
Ruleset Correlation
R
Aggregated graph
olympus
spurr
kanab
sierra
erebus
tallac
R
R
39
GrIDS Hierarchy

• Graph building engines arranged in a hierarchy
• Unresolved graph fragments propagate up

Audits
Widgets Inc.
A
Human Resources
Financial
B
Audits
Payroll
Office of CFO
Personnel
Training
Office of CFO
H
Payroll
I
J
Audits
E
C
x,y,...
F
G
D
Payroll
Office of CFO
x,y,...
x,y,...
40
GrIDS Experiment Scalability

• GrIDS was installed and monitoring CS Dept
  network with gt100 hosts
• Test GrIDS scaleability using real network traffic

41
Results

• All data sources reside at the bottom of the
  hierarchy
• Rulesets accept everything with destination port
  lt 1023 ! 80
• Report volume at each engine is measured in
  proportion to the average input report volume.

1.3
1.6
0.79
0.80
1.0
0.79
1.0
1.0
Data source input traffic volume is defined to be
one.
42
IDS Report Correlation

• How do reports from attack signatures fit
  together.
• Reports of single attacks that go nowhere can be
  assigned lower priority.
• Is there a model of attacks that is extensible
  and useful for real correlation

43
The Attack Model

• Concepts
• Abstract attack situations
• Typically subtasks in a scenario attack
• Policy statements
• Many to one relation between concepts
• Generate alerts in an IDS
• Capabilities
• Conditions that must exist for a concept to hold
• Oftentimes assumed to exist outside the model
  (e.g. SYN-flood)
• Both required and provided by Concepts

44
The Attack Model
Concept
45
Automated Responses

• Time is of the essence. Even when attacks are
  detected it may be too late.
• Detection is trivial for Denial-of-Service
  attacks.
• Need a way to stop some attacks in the shortest
  possible time period
• Response strategies are not always obvious, even
  for human administrators.

46
Intrusion Detection and Isolation Protocol (IDIP)

• First Stage Quick, local IDIP generated
  response.
• Second Stage IDIP Discovery Coordinator (DC)
  removes redundant responses.
• Third Stage IDIP DC recognizes any global attack
  patterns and issues an optimized response.

47
An Example IDIP Enabled Network Enclave

• Hosts are grouped into neighborhoods of hosts
• Neighborhoods boundaries are protected locally
  by filtering routers/firewalls
• A centralized Discovery Coordinator directs
  global activity

The Internet
Discovery Coordinator
IDIP Enabled Routers/Firewalls
Subnets
Hosts
48
SYN-flood Attack from the Internet

• Attacker is issuing a SYN-flood to port 25 of
  the central e-mail hub
• The IP-header is forged to produce random, fake
  source addresses
• Result E-mail communications channel is
  effectively blocked by the attacker

The Internet
Target Mail Hub
Discovery Coordinator
IDIP Enabled Routers/Firewalls
Source
Subnets
Hosts
49
Immediate IDIP blocking

• IDS detecting the attack requests blocking from
  its neighbors
• IDIP enabled devices that have seen the attack
  block traffic from ... port to
  mail-hub.ucdavis.edu port 25.
• Result Attack is thwarted at the cost of
  blocking e-mail communications between IDIP
  neighborhoods (n.b. Still better than unhindered
  SYN-flood).

!
!
The Internet
!
Target Mail Hub
Discovery Coordinator
IDIP Enabled Routers/Firewalls
Subnets
!
Hosts
SYN-flood detected
50
IDIP Discovery Coordinators Role

• IDS detecting the attack will send reports to
  the IDIP DC.
• DC correlates the multiple sightings, realizing
  its one attack.
• DC selects the relevant response to a randomized
  SYN-flood attack ( block ... port to
  mail-hub.ucdavis.edu port 25).
• DC finds the location with minimal blocking
  costs and requests response
• Result Attack is prevented at the cost of
  blocking e-mail communications from arbitrary
  Internet hosts.

The Internet
Target Mail Hub
Attack!
Attack!
Attack!
Discovery Coordinator
Optimized Response
51
IDIP Cost Model Overview

• Responding without regard to self-inflicted costs
  is an open invitation for automated mass
  denial-of-service.
• Cost of our own responses must be weighed against
  the cost of potential attack damage.
• Cost model receives a list of effective
  responses.
• Selects from the list a response with minimal
  impact on normal system operation.
• Sends the selection to IDIP enabled responders.

52
Assigning Costs (Example)
Mission Policy for Email Sending mail to
internal machines Vin Sending mail to
external machines Vext
From mission policy, critical servers and service
interdependencies lt...gt lt,,,gt 0 lthost
cgt lt,,,gt port 25 Vext lt128.120..gt lthost
cgt port 25 Vin
Response request Filter ltgt to lthost c port 25gt
Attack Cost 4vin vext Cost to block at r1
lt not an option gt Cost to block at
r2 2vin vext Cost to block at r3 vext
Table of connectivity values for sending mail
between hosts
r1
a
r3
r2
internet
b
d
c
(Mail hub)
Topology
53
Prevention Forensics

• Protecting against future attack instances
  requires determination of the cause.
• Forensics need only proceed till a response
  blocking future attacks is determined, not to the
  ultimate root cause.
• The HACQIT Forensic Agent will automatically
  detect the suspicious events in the application
  and network traffic logs.
• Reports from the Forensic Agent are used by the
  Response Agent to block future instances.

54
Forensic Methods

• Simple rules
• The last log entry in a crashed application is
  the suspect transaction
• Recent anomalous application transactions are
  suspect (DEMIDS)
• Buffer replay
• An offline mirror of the application is
  maintained
• Suspect transactions recorded by the application
  are replayed against the mirror to find the
  one(s) that reproduce the behavior.
• Replay with subtle modifications zeros in on the
  true cause. (e.g. reordering of parameters,
  randomized time intervals, subtraction of
  obviously benign transactions)
• System Interaction rules
• Specify system dependencies between audit logs,
  processes and the file system

55
Principles of Forensics

• Strength of evidence depends upon investigation
  goals
• e.g. weak evidence may be enough for a civil suit
• Strong evidence is required for criminal cases
• Strength requirements for computer forensics
• Evidence requirements depend upon response
  options
• Response goals are set by the mission model
• Weak forensic evidence may be enough to trigger
  one type of response where strong evidence is
  required for another.

56
Automated Forensic Methodology
Abstract Forensic Leads

• Login Record 322 12/11/2001
• Code Compiled 327 12/11/2001
• Network Connection 402 12/11/2001
• Inconsistent File System 411 12/11/2001
• IDS Alarm 412 12/11/2001
• Login Record 412 12/11/2001
• Suspicious Transaction in Application Log 413
  12/11/2001

Forensic Evidence Analysis
Effect 1
Cause A
Cause B
Effect 2
Cause C
Cause D
Effect 3
Cause F
Cause E
Effect 4
Effect 4
57
Strength of Evidence Selects Response
Forensic Leads
Attack Buffer Overflow Response Block
Transaction

• Login record 322 12/11/2001
• Code compiled 327 12/11/2001
• Network connection 402 12/11/2001
• Suspicious Request (75 Buffer Overflow) 413
  12/11/2001
• IDS alarm 412 12/11/2001
• Network connection 412 12/11/2001
• Network connection 414 12/11/2001
• Suspicious Request (90 Race Condition) 407
  12/11/2001
• Recently created critical system file 414
  12/11/2001
• Unusual locale for critical system file 414
  12/11/2001
• Network connection 416 12/11/2001

Attack Race Condition Response
Non-Interference Policy
Attack Remote Access Response Block traffic
Attack Invalid CGI Inputs Response Validate CGI
Parameters
Strength of Evidence

• Required strength of evidence depends upon the
  response options
• If the response Block Traffic is acceptable,
  then weak evidence of unauthorized remote access
  is sufficient
• If a more focused response is required then
  evidence will need to be stronger to discriminate
  the type of attack

58
Privacy Issues

• Can raw data be sanitized to remove traces of
  individuals or organizations?
• What is the tradeoff between privacy and the
  ability of an IDS to detect attacks.
• How can data be shared across trust boundaries in
  a way that maintains privacy.