Network Intrusion Detection and Mitigation

1
Network Intrusion Detection and Mitigation

• Yan Chen
• Northwestern Lab for Internet and Security
  Technology (LIST)
• Department of Computer Science
• Northwestern University
• http//list.cs.northwestern.edu

2
Our Theme

• Internet is becoming a new infrastructure for
  service delivery
• World wide web,
• VoIP
• Email
• Interactive TV?
• Major challenges for Internet-scale services
• Scalability 600M users, 35M Web sites, 2.1Tb/s
• Security viruses, worms, Trojan horses, etc.
• Mobility ubiquitous devices in phones, shoes,
  etc.
• Agility dynamic systems/network,
  congestions/failures
• Ossification extremely hard to deploy new
  technology in the core

3
Battling Hackers is a Growth Industry!
--Wall Street Journal (11/10/2004)

• The past decade has seen an explosion in the
  concern for the security of information
• Internet attacks are increasing in frequency,
  severity and sophistication
• Denial of service (DoS) attacks
• Cost 1.2 billion in 2000
• Thousands of attacks per week in 2001
• Yahoo, Amazon, eBay, Microsoft, White House,
  etc., attacked

4
Battling Hackers is a Growth Industry (contd)

• Virus and worms faster and powerful
• Melissa, Nimda, Code Red, Code Red II, Slammer
• Cause over 28 billion in economic losses in
  2003, growing to over 75 billion in economic
  losses by 2007.
• Code Red (2001) 13 hours infected gt360K machines
  - 2.4 billion loss
• Slammer (2003) 10 minutes infected gt 75K
  machines - 1 billion loss
• Spywares are ubiquitous
• 80 of Internet computers have spywares installed

5
The Spread of Sapphire/Slammer Worms
6
How can it affect cell phones?

• Cabir worm can infect a cell phone
• Infect phones running Symbian OS
• Started in Philippines at the end of 2004,
  surfaced in Asia, Latin America, Europe, and
  recently in US
• Posing as a security management utility
• Once infected, propagate itself to other phones
  via Bluetooth wireless connections
• Symbian officials said security was a high
  priority of the latest software, Symbian OS
  Version 9.
• With ubiquitous Internet connections, more severe
  viruses/worms for mobile devices will happen soon
  

7
The Current Internet Connectivity and Processing
8
Current Intrusion Detection Systems (IDS)

• Mostly host-based and not scalable to high-speed
  networks
• Slammer worm infected 75,000 machines in lt10 mins
• Host-based schemes inefficient and user dependent
• Have to install IDS on all user machines !
• Mostly signature-based
• Cannot recognize unknown anomalies/intrusions
• New viruses/worms, polymorphism
• Statistical detection
• Hard to adapt to traffic pattern changes
• Unscalable for flow-level detection
• IDS vulnerable to DoS attacks
• Overall traffic based inaccurate, high false
  positives

9
Current Intrusion Detection Systems (II)

• Cannot differentiate malicious events with
  unintentional anomalies
• Anomalies can be caused by network element faults
• E.g., router misconfiguration, signal
  interference of wireless network, etc.
• Isolated or centralized systems
• Insufficient info for causes, patterns and
  prevalence of global-scale attacks

10
Global Router-based Anomaly/Intrusion Detection
(GRAID) Systems

• Online traffic recording and analysis for
  high-speed networks
• Leverage sketches for data streaming computation
• Online adaptive flow-level anomaly/intrusion
  detection and mitigation
• Leverage statistical learning theory (SLT)
  adaptively learn the traffic pattern changes
• E.g., busy vs. idle wireless networks, with
  different level of interferences, etc.
• Unsupervised learning without knowing ground truth

11
GRAID Systems (II)

• Integrated approach for false positive reduction
• Signature-based detection
• Network element fault diagnostics
• Traffic signature matching of emerging
  applications
• Hardware speedup for real-time detection
• Collaborated with Gokhan Memik (ECE of NU)
• Try various hardware platforms FPGAs, network
  processors
• Scalable anomaly/intrusion alarm fusion with
  distributed hash tables (DHT)
• Automatically distribute alerts with similar
  symptoms to the same fusion center for analysis

12
GRAID Detection Sensor

• Attached to a router or access point as a black
  box
• Edge network detection is particularly powerful

Monitor each port separately
Monitor aggregated traffic from all ports
Original configuration
13
GRAID Sensor Architecture
Remote aggregated sketch records
Sent out for aggregation
Reversible k-ary sketch monitoring
Part I Sketch-based monitoring detection
Normal flows
Sketch based statistical anomaly detection (SSAD)
Local sketch records
Streaming packet data
Keys of suspicious flows
Filtering
Keys of normal flows
Statistical detection
Signature-based detection
Per-flow monitoring
Network fault detection
Part II Per-flow monitoring detection
Suspicious flows
Traffic profile checking
Intrusion or anomaly alarms to fusion centers
Modules on the critical path
Modules on the non-critical path
Data path
Control path
14
Scalable Traffic Monitoring and Analysis -
Challenge

• Potentially tens of millions of time series !
• Need to work at very low aggregation level (e.g.,
  IP level)
• Each access point (AP) can have 200 Mbps a
  collection of 10-100 APs can easily go up to 2-20
  Gbps
• The Moores Law on traffic growth ?
• Per-flow analysis is too slow or too expensive
• Want to work in near real time

15
Sketch-based Change Detection(ACM SIGCOMM IMC
2003, 2004)

• Input stream (key, update)

• Summarize input stream using sketches

• Build forecast models on top of sketches

• Report flows with large forecast errors

16
Evaluation of Reversible K-ary Sketch

• Evaluated with tier-1 ISP trace and NU traces
• Scalable
• Can handle tens of millions of time series
• Accurate
• Provable probabilistic accuracy guarantees
• Even more accurate on real Internet traces
• Efficient
• For the worst case traffic, all 40 byte packets
• 16 Gbps on a single FPGA board
• 526 Mbps on a Pentium-IV 2.4GHz PC
• Only less than 3MB memory used
• Patent filed

17
GRAID Sensor Architecture
Remote aggregated sketch records
Sent out for aggregation
Reversible k-ary sketch monitoring
Part I Sketch-based monitoring detection
Normal flows
Sketch based statistical anomaly detection (SSAD)
Local sketch records
Streaming packet data
Keys of suspicious flows
Filtering
Keys of normal flows
Statistical detection
Signature-based detection
Per-flow monitoring
Network fault detection
Part II Per-flow monitoring detection
Suspicious flows
Traffic profile checking
Intrusion or anomaly alarms to fusion centers
Modules on the critical path
Modules on the non-critical path
Data path
Control path
18
Current IDS Insufficient for Wireless Networks

• Most existing IDS signature-based
• Especially for wireless networks
• Detect denial-of-service attacks caused by the
  WEP authentication vulnerability, e.g., Airespace
  
• Current statistical IDS has manually set
  parameters
• Cannot adapt to the traffic pattern changes
• However, wireless networks often have transient
  connections
• Hard to differentiate collisions, interference,
  and attacks

19
Statistical Anomaly/Intrusion Detection and
Mitigation for Wireless Networks

• Use statistics from MIB of AP to understand the
  current wireless network status
• Metrics considered capacity, transmission fail
  count, multiple retry count, duplicate count,
  received fragment count, etc.
• Infer the wireless network status congested ?
  Interfered ?
• Automatically adapt to different learned profiles
  on observing status changes
• Applicable to both WLAN and celluar network
  infrastructure protection

20
Intrusion Detection and Mitigation
Attacks detected Mitigation
Denial of Service (DoS), e.g., TCP SYN flooding SYN defender, SYN proxy, or SYN cookie for victim
Port Scan and worms Ingress filtering with attacker IP
Vertical port scan Quarantine the victim machine
Horizontal port scan Monitor traffic with the same port for compromised machine
Spywares Warn the end users being spied
21
GRAID Sensor Architecture
Remote aggregated sketch records
Sent out for aggregation
Reversible k-ary sketch monitoring
Part I Sketch-based monitoring detection
Normal flows
Sketch based statistical anomaly detection (SSAD)
Local sketch records
Streaming packet data
Keys of suspicious flows
Filtering
Keys of normal flows
SIGCOMM04
Statistical detection
Signature-based detection
Per-flow monitoring
Network fault detection
Part II Per-flow monitoring detection
Suspicious flows
Traffic profile checking
Intrusion or anomaly alarms to fusion centers
Modules on the critical path
Modules on the non-critical path
Data path
Control path
22
Research methodology

• Combination of theory, synthetic/real trace
  driven simulation, and real-world implementation
  and deployment

23
Potential Collaborative Research Areas with
Motorola

• Wireless virus/worm detection
• Spyware detection
• Both by operators at infrastructure level (e.g.,
  access point)
• Intrusion detection and mitigation for cellular
  network infrastructure
• Automatic attack responding and survival for
  Motorola infrastructure products

24
Thank You!

• More Questions?

25
Backup Slides
26
RF Management and Monitoring (e.g., Airespace)

• Rogue Access Point/Ad-Hoc networks
• RF Interference
• Fake Access Point
• AP Impersonation
• Spoofed Deauthenticate Frame
• Honeypot AP

27
Network Diagnosis and Fault Location

• Infrastructure ossification led to thrust of
  overlay applications
• Traceroute gives hop-by-hop round-trip latency
• Asymmetric routing
• Cant get hop-by-hop loss rate !
• Network tomography
• Infer the properties of links from end-to-end
  measurements
• Limited measurements -gt under-constrained system,
  unidentifiable links
• Existing work uses various constraints and
  assumptions
• Tree-like topology
• The number of lossy links is small

28
Our Approach Virtual Links

• Minimal link sequences (path segments) whose loss
  rates uniquely identified
• Locate the faults to certain link(s)
• The first lower-bound on the network tomography
  granularity
• Use algebraic scheme to find virtual links
• Leverage our work on overlay network monitoring
  (ACM SIGCOMM IMC 2003, ACM SIGCOMM 2004)

29
GRAID Sensor Architecture
Remote aggregated sketch records
Sent out for aggregation
Reversible k-ary sketch monitoring
Part I Sketch-based monitoring detection
Normal flows
Sketch based statistical anomaly detection (SSAD)
Local sketch records
Streaming packet data
Keys of suspicious flows
Filtering
Keys of normal flows
Statistical detection
Signature-based detection
Per-flow monitoring
Network fault detection
Part II Per-flow monitoring detection
Suspicious flows
Traffic profile checking
Intrusion or anomaly alarms to fusion centers
Modules on the critical path
Modules on the non-critical path
Data path
Control path
30
Intrusion/anomaly Alarm Fusion

• Individual IDS has bad accuracy due to limited
  view
• Crucial to collect information from multiple
  vantage points distributed IDS (DIDS)
• Each IDS generate local symptom report, send to
  sensor fusion center (SFC)
• Help understand the prevalence, cause and
  patterns of global-scale attacks
• Existing DIDS
• Centralized fusion
• Distributed fusion with unscalable communication

31
GRAID Sensor Interconnection

• Though Cyber Disease DHT (distributed hash table)
  for alarm fusion
• Scalability
• Load balancing
• Fault-tolerance
• Intrusion correlation

32
Basic Operations of CDDHT

• put (disease_key, symptom report)
• Send report to SFC
• attack_info get (disease_key)
• Query about certain attacks from SFC
• Each operation only O(n) hops
• n is the total number of nodes in CDDHT

33
CDDHT Disease Key Design
Intrusion ID Characterization Field(s) Characterization Field(s) Characterization Field(s)
DoS Attack 0 Victim IP (subnet) Victim IP (subnet) Victim IP (subnet)
Scans 1 0 (for vertical block scan) Source IP address Destination IP (for vertical scan)
Scans 1 0 (for vertical block scan) Source IP address 0 (for block scan)
Scans 1 1 (for horizontal coordinated scan) Scan port number Source IP (for horizontal scan)
Scans 1 1 (for horizontal coordinated scan) Scan port number 0 (for coordinated scan)
Viruses/Worms 2 0 (for known virus/worm) 0 (for known virus/worm) Worm ID
Viruses/Worms 2 1 (for unknown virus/worm) 1 (for unknown virus/worm) Destination port number
34
Other Challenges of CDDHT

• Load balancing
• Supporting complicated queries
• E.g., aggregate queries
• Attack resilience
• OK to have some IDS sensors compromised
• What about SFCs?

35
Conclusion for GRAID Systems

• Online traffic recording and analysis on
  high-speed networks
• Online statistical anomaly detection
• Integrated approach for false positive reduction
• Signature-based detection
• Network element fault diagnostics
• Traffic signature matching of emerging
  applications
• Hardware speedup for real-time detection
• Scalable anomaly/intrusion alarm fusion with
  distributed hash tables (DHT)