The Internet Motion Sensor: A Distributed Blackhole Monitoring System

1
The Internet Motion SensorA Distributed
Blackhole Monitoring System

• Michael Bailey, Evan Cooke, Farnam Jahanian,
• Jose Nazario, David Watson
• 
  Presenter Anup Goyal

2
What is a Blackhole?
3
Telescope/ Blackhole/ DarkNet/ Sink

• Monitoring of unused Address space is very useful
• CAIDA-Network Telescope
• Internet Motion Sensor blackhole
• Team Cymru DarkNets
• IUCC/IDC Internet Telescope
• Isink
• Investigating DDOS
• Tracking Worms
• Characterizing emerging Internet Thread

4
Motivation

• Passive Monitoring has been around for a while,
  why build another one?
• Need broader coverage
• Need to be able to differentiate threats
• Our Solutions
• Distributed Monitoring Infrastructure
• Increas coverage of threat activity
• Lightweight Active Responder
• Capture payloads to differentiate threats
• Payload signature and caching
• Improve performance and scalability

5
Why do you need to be distributed?
6
Local Preference across sensors
7
Distributed Monitoring Infrastructure
8
Differentiate Threats?
9
Why not Build Service Modules?
10
(No Transcript)
11
Lightweight Active Responder

• Goal obtain enough fidelity to differentiate
  threats with the minimum resource cost
• TCP needs to establish a connection before data
  is sent
• Use Lightweight SYN-ACK active Responder

12
Blaster Worm Live Host
13
Blaster Worm Passive Monitor
14
Blaster Worm - IMS
15
Active Responder Limitations

• Application-level responder may be required to
  differentiate threats (e.g. Agobot)
• Threats like Agobot can be differentiated using
  scanning behaviour
• Sensors can be fingerprinted and avoided
• IMS focused on globally scoped threats (threat
  model does not include targated manual attacks)
• Many sensors of different sizes in many networks
  near live hosts makes avoidance very hard
• Rotate active responders within address block

16
Payload Signature and Caching
17
IMS Deployment

• Initial /8 deployment in 2001. Currently 60
  address blocks at 18 networks on 3 continent
• Tier 1 ISPs, Large Enterprise, Broadband,
  Academic, National ISPs, Regional ISPs

18
Deployment Statistics

• 17,094,016 IPs monitored
• 27 /8 block with an IMS sensor
• 1.25 of routed IPv4 space
• 21 of all routable /8 blocks have at least one
  sensor
• IMS Has provided insight into
• Internet Worms
• Reconnaissance/Scanning
• DDos

19
Internet Worms
20
Scanning
21
SCO DDoS
22
Summary Outreach

• Summary
• IMS provides a lightweight but effective platform
  for tracking, and characterizing Internet threats
• Successful deployment on 60 address blocks at 18
  organization
• Outreach
• IMS currently used daily by operator community to
  investigate new threats
• We provide reports and forensics to NSP-SEC
• We are now focusing academic networks
• Data traces will be available through DHS PREDICT
  project

23
Scalability, Fidelity, and Containment in
thePotemkin Virtual Honeyfarm

• Michael Vrable, Justin Ma, Jay Chen, David Moore,
  Erik Vandekieft, Alex C. Snoeren, Geoffrey M.
  Voelker,Stefan Savage
• Presenter Anup Goyal

24
Background

• Large-scale host exploitation a serious problem
• Worms, viruses, bots, spyware
• Supports an emerging economic criminal enterprise
• SPAM, DDoS, phishing, piracy, ID theft
• Quality and sophistication of malware increasing
  rapidly

25
Motivation

• Intelligence about new threats is critical for
  defenders
• Principal tool is the network honeypot
• Monitored system deployed for the purpose of
  being attacked
• Honeyfarm Collection of honeypots
• Provide early warning, accurate inference of
  global activity,cover wide range of software
• Design issues
• Scalability How many honeypots can be deployed
• Fidelity How accurately systems are emulated
• Containment How well innocent third parties are
  protected
• Challenge tension between scalability and
  fidelity

26
Honeyfarm Scalability/Fidelity Tradeoff
27
Approach

• Dedicated honeypot systems are overkill
• Can provide the illusion of dedicated systems via
  aggressive resource multiplexing at network and
  host levels

28
Network-level Multiplexing

• Most addresses dont receive traffic most of the
  time
• Apply late binding of IP addresses to honeypots
• Most traffic that is received causes no
  interesting effects
• - Allocate honeypots only long enough to identify
  interesting behavior
• Recycle honeypots as soon as possible
• How many honeypots are required?
• - For a given request rate, depends upon
  recycling rate

29
Effectiveness of Network-Level Multiplexing
30
Host-Level Multiplexing

• CPU utilization in each honeypot quite low
  (milliseconds to process traffic)
• Use VMM to multiplex honeypots on a single
  physical machine
• Few memory pages actually modified when handling
  network data
• Share unmodified pages among honeypots within a
  machine
• How many virtual machines can we support?
• - Limited by unique memory required per VM

31
Effectiveness of Host-Level Multiplexing

• Further 2-3 order of magnitude improvement

32
The Potemkin Honeyfarm Architecture

• Two components
• Gateway
• VMM
• Basic operation
• Packet received by gateway
• Dispatched to honeyfarm server
• VM instantiated
• Adopts IP address

33
Potemkin VMM Requirements

• VMs created on demand
• VM creation must be fast enough to maintain
  illusion
• Many VMs created
• - Must be resource-efficient

34
Potemkin VMM Overview

• Modified version of Xen 3.0 (pre-release)
• Flash cloning
• Fork copies from a reference honeypot VM
• Reduces VM creation timeno need to boot
• Applications all ready to run
• Delta virtualization
• Copy-on-write sharing (between VMs)
• Reduces per-VM stateonly stores unique data
• Further reduces VM creation time

35
Flash Cloning Performance

• Time required to clone a 128 MB honeypot
• Control tools overhead 124 ms
• Low-level clone 11 ms
• Device setup 149 ms
• Other management overhead 79 ms
• Networking setup overhead 158 ms
• Total 521 ms
• 0.5 s already imperceptible to external observers
  unless looking for delay, but we can do even
  better

36
Delta Virtualization Performance

• Deployed using 128 MB Linux honeypots
• Using servers with 2 GB RAM, have memory
  available to support 1000 VMs per physical host
• Currently tested with 100 VMs per host
• Hits artificial resource limit in Xen, but this
  can be fixed

37
Containment Policies

• Must also care about traffic going out
• We deliberately run unpatched, insecure software
  in honeypots
• Containment Should not permit attacks on third
  parties
• As with scalability, there is a tension between
  containment and fidelity
• Various containment policies we support
• Allow no traffic out
• Allow traffic over established connections
• Allow traffic back to original host

38
Containment Implementation in Gateway

• Containment policies implemented in network
  gateway
• Tracks mappings between IP addresses, honeypots,
• connections
• Modular implementation in Click
• Gateway adds insignificant overhead ( 1 ms)

39
Traffic Reflection

• Example gateway policy
• Redirect traffic back to
• Honeyfarm
• Packets sent out to third parties. . .
• . . .may be redirected back into honeyfarm
• Reuses honeypot creation functionality

40
Challenges

• Honeypot detection
• If malware detects it is in a honeypot, may act
  differently
• How easy it is to detect virtualization?
• VMware detection code used in the wild
• Open arms race between honeypot detection and
  camouflage
• Resource exhaustion
• Under high load, difficult to maintain accurate
  illusion
• Large-scale outbreak
• Honeypot denial-of-service
• Challenge is intelligently shedding load

41
Summary

• Can achieve both high fidelity and scalability
• Sufficient to provide the illusion of scale
• Potemkin prototype 65k addresses -gt 10 physical
  hosts
• Largest high-fidelity honeypot that we are aware
  of
• Provides important tool for study of and defenses
  against malware