Vortex: Enabling Cooperative Selective Wormholes

1
Vortex Enabling Cooperative Selective Wormholes

• John R. Lange
• Peter A. Dinda and Fabian Bustamante
• Prescience Lab
• Department of Electrical Engineering and Computer
  Science
• Northwestern University
• http//presciencelab.org

2
Cooperative Selective Wormholes (CSW)

• Mechanism to enable widely distributed NIDS
  traffic aggregation
• Possible for any researcher
• Requires no hardware, router configuration, large
  unused address blocks
• Operates as a volunteer Initiative
• Software client for commodity PCs
• Utilize unused resources on existing PCs
• Internet diverse enough to make CSWs effective?
• Are there enough unused resources?
• Analysis of Northwestern University Network
• Design requirements for an effective CSW
• Vortex Experimental implementation

3
CSW Vision
4
Outline

• Motivation
• Defining Cooperative Selective Wormholes (CSW)
• Feasibility
• Configuration Coverage
• Case study at Northwestern University
• Vortex
• Initial CSW implementation
• CSW Invisibility Requirements
• Invisibility to Volunteers
• Invisibility to Attackers
• Conclusion and Future Work

5
Motivation

• Intrusion Detection Systems are necessary to
  detect and analyze security threats
• Attack vectors, worms, etc
• Need to analyze large amounts of live traffic to
  be effective
• Current aggregation techniques exist but are
  relatively expensive
• Barrier of entry to unconnected/unfunded
  researchers
• Current aggregation techniques sample at low
  frequency
• Limited distribution due to cost and design

6
Existing Approaches

• Network telescopes
• Aggregate traffic bound to large areas of dark
  address space
• Many addresses but localized coverage
• Large configuration changes to large routers
• Hardware wormholes
• Hardware device sitting in a data/network center
• Assigned local IP address
• Tunnels traffic back to NIDS
• Larger distribution
• Large deployment cost (HW hosting fees)
• Weaver, N., Paxson, V., and Staniford, S.
  Wormholes and a honeyfarm Automatically
  detecting novel worms. In DIMACS Large Scale
  Attacks Workshop (2003)

7
Goal

• Provide large scale distributed traffic
  aggregation for any security researcher
• No deployment cost
• Large scale traffic aggregation
• Large sensor distribution
• Compatible with any generic backend IDS system
• Aimed primarily at VM based honeynets/honeypots
• Software client compatible with many standard
  OSes that operates at the network packet level
• Cooperative Selective Wormholes

8
Wormhole

• Remote Network Presence
• Separation of aggregation infrastructure from
  backend analysis engine
• Proposed by Weaver, Paxson, and Staniford
• Colocated hardware device
• Weaver, N., Paxson, V., and Staniford, S.
  Wormholes and a honeyfarm Automatically
  detecting novel worms. In DIMACS Large Scale
  Attacks Workshop (2003)
• Tunnel traffic from a remote sensor to a backend
  NIDS
• Allows distributable aggregation sensors
• CSWs are implemented as software

9
Cooperative

• Uses volunteer resources
• SETI_at_Home, Folding_at_Home
• Very successful at recruiting volunteers
• Hundreds of thousands of clients
• Donate unused network ports and Bandwidth
• Not CPU cycles
• Targeted at commodity PCs
• Available for wide range of OS environments
• Potential for much larger scale aggregation

10
Selective

• Aggregates traffic ignored by local client
• Completely transparent
• Invisible to local environment
• Invisible to remote hosts
• Allows layering of network configurations
• Linux server can share IP address with a Windows
  workstation
• Non intersecting open port configuration

11
Coverage

• Statistical relevance of aggregated traffic
• Internet as a sample space
• Sample from the entire space
• Sample from a meaningful distribution
• Random distribution ideal
• Horizontal Coverage
• Sensor Distribution over IP address space
• Geographical and numerical location
• Dependent on network location of sensor
• Issue for all traffic aggregation techniques
• Vertical Coverage
• Sensor Distribution over port space
• Dependent on configuration diversity
• Currently unknown for the internet
• Unique to CSW

12
Vertical Coverage

• CSW depends on configuration diversity
• E.g. 100 of hosts run a web server
• Cannot listen for web server traffic through CSW
• Ideals
• Small set of common non-intersecting
  configurations
• Random port configurations
• Would like to deploy sensor configurations in the
  same density as true configurations
• E.g. 15 of the volunteer hosts run a web server
• 15 of the hosts run a web server CSW

13
Northwestern Case Study

• Attempt to measure configuration diversity
• Network Port Scan
• Northwestern University network
• 1000 random active IP addresses
• TCP ports 1-1024
• Sorted Vortex capable hosts by hand
• Removed printers, network switches, etc
• Scan Results
• 401 Vortex capable hosts
• Probably higher removed 253 hosts reporting no
  open ports
• Possibly fully firewalled hosts
• 123 unique port signatures
• Port signature port configuration
• Analysis
• Identify presence of prevalent configurations
• Determine degree of separation of signatures
• Determine availability of common machine
  configurations

14
Configuration Diversity
Hosts per Signature
60
135, 139, 445

50
139, 445


40
80, 443
No. of Hosts
30
20
10
0
1
2
3
4
5
6
7
8 9
Number of Open Ports

• Identify prevalent port signatures
• Top 3 Web server and 2 windows configurations
  (34 of host set)
• 20 of hosts have unique signature

15
Signature Separation

• Determine the degree of intersection present in
  the signatures
• E.g.
• 20 unique signatures that all contain port 80
• Local and CSW signatures must not intersect
• A given client is available if its signature does
  not intersect a CSW signature
• E.g.
• A web server is an available host for a mail
  server CSW
• Determine lower limit of availability

16
Entire Signatures
Percentage of non-intersecting configurations
100
80
60
Percentage of Available Hosts
40
20
0
0
2
4
6
8
10
12
14
16
18
20
Number of Open Ports

• Smallest signatures have the highest
  availability
• Most common signatures have small number (2-3)
  of ports
• Availability does not decrease to zero
• Always have some available host

17
Port Combinations

• Separation based on signature subsets
• Attempting to extract common port sets
• NIDS might be interested in subsets of machine
  ports
• Example (scanning for web server attacks)
• Port signature 22, 80, 443
• Combinations (2 ports) 22, 80, 22, 443, 80,
  443
• Considered combinations of 1 to 5 ports
• Sorted by popularity
• Scatter plot of the availability

18
Port Combinations (1 port)
100
In Use
Available
80
60
Host Percentage
40
20
0
10
20
30
40
50
60
70
80
0
Port Combinations

• 50 availability in the worst case

19
Port Combinations (2-5 ports)
2 Port Combination
3 Port Combination
100
100
80
80

• Take Away
• 20 availability worst case

60
60
Host Percentage
Host Percentage
40
40
20
20
0
0
0
0
600
800
1000
2000
3000
4000
200
400
4 Port Combination
5 Port Combination
100
100
80
80
60
60
Host Percentage
Host Percentage
40
40
20
20
0
0
0
10000
10000
20000
30000
40000
0
5000
15000
20
Common Configurations

• Machine configurations likely of interest to
  NIDS operators
• Standard OS and Software configurations

21
Common Configuration Availability
80
70
60
50
Available Host Percentage
40
30
20
10
0
LIN
WIN
LHS
LMS
SOL
SWS
EXG
DNS
WFS
SMB
LWS
WDC

• Cross platform configurations are worst (Linux
  Samba and Windows Exchange)

22
Feasibility Summary

• Substantial heterogeneity of port configurations
• CSWs are feasible
• Still dependent on the demographic of volunteers
• Dont know effect of NATs or hardware firewalls
• UPNP, firewall extension

23
Vortex

• Experimental CSW implementation
• Provides CSWs for any generic backend IDS
• Based on experience with network virtualization
  and high performance grid computing (Virtuoso)
• VTL (HPDC 2007)
• Transparent network service framework
• Transparent to host environment (pcap libnet)
• Packet manipulation and network device interface
• VNET (HPDC 2005)
• Overlay network for virtual machines
• Layer 2 tunnel
• And many others
• http//virtuoso.cs.northwestern.edu

J. Lange and P. Dinda, Transparent Network
Services via a Virtual Traffic Layer for Virtual
Machines, Proceedings of the 16th IEEE
International Symposium on High Performance
Distributed Computing, (HPDC 2007) A. Sundararaj,
A. Gupta, and P. Dinda, Increasing Application
Performance in Virtual Environments through
Run-time Inference and Adaptation, Proceedings of
the 14th IEEE International Symposium on High
Performance Distributed Computing (HPDC 2005)
24
Vortex Architecture
VM Based Honeypot
Commodity PC
IDS Analysis Backend
Windows/UNIX
VM
VNET Proxy
Vortex
Apps
Physical Honeypot
Operating System
VTL
Firewall
PCAP
libnet
NIC
VNET Overlay
Backend Network
25
Cloaking

• Vortex ensures that packets are handled correctly
  by all parties
• Network equipment handles packets normally
• Volunteer host does not see any CSW traffic
• No listening ports are opened on the client
• Backend system receives legitimate traffic
• Remote hosts are not disrupted
• Two fundamental issues
• Addressing
• Packet Suppression

26
Addresses

• Only if the backend system is running a honeypot
• CSW traffic to the honeypot must be accepted
• IP and MAC addresses must match incoming packets
• IP addresses configurable on the host
• MAC addresses must be rewritten in packets
• ARP traffic forwarded to and from the backend

27
Packet Suppression

• TCP stacks respond with RSTs to closed ports
• Vortex intercepts traffic to closed ports
• Packets must be dropped before reaching TCP stack
  or RSTs must be suppressed
• Client Firewalls now prevalent
• Windows firewall and iptables
• Allow dropping of packets to blocked/closed ports
• Windows Default
• Iptables 1 line rule change
• VTL captures packets before reaching firewall

28
Vortex Architecture
VM Based Honeypot
Commodity PC
IDS Analysis Backend
Windows/UNIX
VM
VNET Proxy
Vortex
Apps
Physical Honeypot
Operating System
VTL
Firewall
PCAP
libnet
NIC
VNET Overlay
Backend Network
29
Invisibility to Volunteers

• CSWs rely on volunteer machines
• Cannot cause problems
• i.e. must operate transparently to host
  environment
• Cannot add vulnerabilities
• Potential issues
• Port collisions
• Performance degradation
• Privacy risks
• Security risks

30
Port Collisions

• CSWs must avoid interfering with valid host
  traffic
• Cannot operate on open ports
• Solution
• Must disable wormholes on ports that are opened
• Requires monitoring of client port usage
• Current (polling)
• Windows win32 API
• Linux /proc
• Future (event notification)
• Windows NDIS driver hook
• Unix Library interposition

31
Performance Degradation

• CSWs must not monopolize hosts resources
• Primarily bandwidth
• Common issue with volunteer systems
• Solution
• Bandwidth rate limiter for CSW traffic
• User configurable

32
Bandwidth Tests

• Determine impact of CSWs on available bandwidth
• ttcp benchmarks to host and through CSW to
  backend VM
• Two volunteer setups
• Volunteer on same LAN as backend
• Volunteer connected through DSL
• 4 common tests
• Raw host bandwidth
• Raw Vortex bandwidth
• Bandwidth available to host with Vortex saturated
• Bandwidth available to Vortex with host saturated
• 2 DSL only tests
• Bandwidth limited (Linux iproute2 netem)
• Bandwidth available to host when Vortex is rate
  limited
• Bandwidth used by Vortex when rate limited

33
Bandwidth Measurements (LAN)
Host Bandwidth
Vortex Bandwidth
12,000
12,000
10,000
10,000
8,000
8,000
Bandwidth (KB/s)
6,000
6,000
4,000
4,000
2,000
2,000
0
0
Raw
Vortex Flooded
Raw
Host Flooded
34
Bandwidth Measurements (DSL)
Host Bandwidth
Vortex Bandwidth
350
350
300
300
250
250
200
200
Bandwidth (KB/s)
150
150
100
100
50
50
0
0
Raw
Host Flooded
Raw
Vortex Flooded
Vortex Limited
Limited 10kB/s
35
Privacy Risks

• CSWs must not capture volunteers private traffic
• Only aggregate traffic on unused ports
• Cannot protect against user unknowingly but
  willfully communicating through wormhole
• CSWs should minimize amount of configuration
  information stored
• List of open ports, etc

36
Security Exposure

• CSWs should not increase security risks for
  volunteers
• Volunteer vulnerabilities
• Vortex operates at layer 2 and does minimal
  processing
• Ethernet layer headers
• Volunteer liability
• Compromised honeypot backend transmitting
  through Volunteer
• Policy decision
• Block all outgoing traffic from backend
• Insert traffic at location other than volunteer
  machine

37
Invisibility to Attackers

• Packet format fingerprinting
• Backend honeypot and volunteer using different
  TCP stacks
• Rewrite packets to match host if really necessary
• Latency difference
• CSWs inherently increase the latency of the
  connection
• No obvious solution that doesnt impact host
  performance
• Delaying volunteers traffic to match CSW latency
  not viable
• However if latency variance is large, CSW
  latency could be masked

38
Latency Tests

• SYN/SYN-ACK sequence latency
• Avoids application layer
• Simultaneously measured latency of
• Host (left bar)
• CSW (right bar)
• Attackers
• Planetlab nodes
• CSW Volunteer nodes
• DSL, Cable, LAN connections
• 100 trials
• Calculated mean and variance

Syn-SynAck Latency (DSL)
120
80
Latency (ms)
40
0
MIT
UCLA
Toronto
OSU
UT
Syn-SynAck Latency (LAN)
100
80
60
Latency (ms)
40
20
0
MIT
UCSD
Toronto
OSU
UTEP
39
Future Work

• Boinc volunteer project?
• http//boinc.berkeley.edu/
• Integration with existing IDS?

40
Conclusion

• Cooperative Selective Wormholes can provide large
  scale traffic aggregation for anyone
• Possible to run invisible wormholes on client
  machines
• Port configuration diversity enough for CSWs to
  be feasible

41

• Prescience Lab
• http//plab.cs.northwestern.edu
• Vortex
• http//vortex.cs.northwestern.edu (Coming Soon!?)
• Virtuoso
• http//virtuoso.cs.northwestern.edu
• John Lange
• http//www.artifex.org/jarusl

42
Latency Tests

• Determine degree of CSW masking due to latency
  variance
• If latency variance is large, CSW latency could
  be masked
• Latency measurements
• SYN/SYN-ACK sequence latency
• Avoids application layer
• Latency of host as well as CSW traffic
• Attackers
• Planetlab nodes
• Volunteers
• DSL, Cable, LAN connections