Fast Portscan Detection Using Sequential Hypothesis Testing

1
Fast Portscan Detection Using Sequential
Hypothesis Testing

• Authors Jaeyeon Jung, Vern Paxson, Arthur W.
  Berger, and Hari Balakrishnan
• Publication IEEE Symposium on Security and
  Privacy 2004
• Presenter Ryan Cunningham

2
A quick note

• All images and equations taken directly from the
  publication

3
Port scanning

• Network reconnaissance technique
• Usually a prelude to an attack
• Difficult to detect
• Traffic difficult to distinguish from regular
  traffic
• Stealth scans can occur very slowly
• Some scans are legitimate
• Search engine spiders
• SSH, peer-to-peer applications, etc.

4
Previous detection techniques

• Limit distinct connection attempts from one IP
• Network Security Monitor
• Snort
• Also detects malformed packets
• Limit failed connection attempts from one IP
• Bro
• Sensitive to service on specific port
• Robertson et al. showed threshold very important

5
Previous detection techniques

• Probabilistic model
• Developed by Leckie et al.
• Assesses typical traffic a machine receives
• Also assesses the traffic a remote machine is
  likely to send
• Combines these probabilities
• If the result is too much, an alert is sounded
• Generates too many false positives

6
Previous detection techniques

• SPICE
• Similar to probabilistic model
• Used to detect low traffic stealth scans
• Too computationally intensive for real world

7
Data set

• Traffic from two sites
• LBL
• 6,000 hosts
• Sparse address space 4.4
• ICSI
• 200 hosts
• Dense address space 42

8
Data set

• Anonymized TCP logs from Bro
• Recorded for one 24 hour period
• Bro NIDS flags for comparison and validation

9
Data set

• Unsuccessful Login attempt analysis

10
Data set

• Ratio of successful login attempts to
  unsuccessful login attempt analysis

11
Observations

• Scans usually come from one host
• Scans make lots of failed connection attempts and
  few successful connection attempts
• Scans should ideally be detected quickly
• False positive rate should be configurable

12
Sequential Hypothesis Testing

• Proposed by Wald in the 1940s
• Method of doing repeated hypothesis testing as
  sequential data is gathered
• Deciding between two hypotheses
• Each time a data point arrives, decide
• Accept H0 (in our case, benign traffic)
• Accept H1 (in our case, port scan traffic)
• Wait for more data (next connection attempt)

13
Sequential Hypothesis Testing

• We specify parameters a and b
• a gt false positive rate
• b lt detection accuracy
• We must estimate parameters q0 and q1
• q0 probability a benign connection attempt is
  successful
• q1 probability a scanner connection attempt is
  successful

14
Sequential Hypothesis Testing

• For each test, we compute the likelihood ratio
• Where

15
Sequential Hypothesis Testing

• Compare likelihood ratio to

• If
• L lt h0 then this is benign traffic
• L gt h1 then this is scan traffic
• Otherwise, wait for another connection

16
Sequential Hypothesis Testing

• We can estimate the expected number of
  connections required to decide with

• Derivation is long and messy

17
Sequential Hypothesis Testing
18
Algorithm
19
Results

• Efficiency true positive / total reported
  positive
• Effectiveness true positive / total actually
  positive

20
Results

• Comparison with Snort and Bro
• N bar average number of local hosts scanned
  before decision is made

21
Contributions

• Extremely fast port scan detection algorithm
• High accuracy
• Low false positive rate
• Sound statistical foundation
• Soundly evaluate the weaknesses of their approach
• Good use of appendixes
• Cure for insomnia

22
Weaknesses

• Buffer of activity
• Attacker can spoof multiple IP addresses
• How is filled buffer dealt with?
• Flush buffer
• Attacker can use this to hide scan activity
• Maintain larger buffer
• Attacker can keep going until system crashes
• Distributed port scans undetectable
• Botnets are increasing in popularity

23
Weaknesses

• Test assumes independent connection attempts
• As suggested in paper, an attacker could exploit
  knowledge of the system to connect to some
  systems while doing surveillance on others
• No real time testing conducted, only simulation
• Reasoning is a little circular
• Poor use of language

24
Improvements

• Implement and test in real time
• Perform suggested improvements in paper
• Differentiate between different services
• Differentiate between rejected and unanswered
  connection attempts
• Use a honeypot to see if complete three way hand
  shake is completed (to detect spoofed IPs)
• Should have kept some of the data away as a sort
  of test data set