ESVT: A Toolkit Facilitating Use of DETER

1
ESVT A Toolkit Facilitating Use of DETER
Lunquan Li, Jiwu Jing, Peng Liu, TJ, Jisheng,
George Kesidis, David Miller Penn State
University September 28, 2005 Newport Beach, CA
2
Motivation

• Specific testbeds need specific tools
• EMIST tools are DETER specific
• Tools are a vehicle to make the evaluation
  methods developed by EMIST available to
  experimenters
• EMIST tools make DETER experiments easier
• EMIST tools save the experimenters time and
  energy

DETER
Experimenter
EMIST tools
General purpose tools
3
EMIST Tool Effort

• PSU ESVT toolkit
• UCD NTGC network traffic generation and control
  tool
• ICSI/PSU worm scale-down equations
• UCD emulated worm attack generation tool
• PSU KMSim Slammer-like attack generator
• SRI/UCD worm simulation tools
• UCD XML worm specification tool
• UCD BGP routing data viz tool
• PSU NTD traffic data mining tool
• Purdue scriptable event system
• Purdue sys info logging tool
• SPARTA/McAfee DDOS trace analysis and viz scripts
• Purdue data analysis and viz scripts

4
ESVT Status

• ESVT 1.0 -- May 2004
• Windows platform
• C
• User manual
• Sample DETER experiment package
• ESVT 2.0 -- May 2005
• 34,494 lines of C code
• ESVT made open source in July 2005

Downloads
ESVT 1.0 Executable 70 times ESVT 2.0
Executable 26 times ESVT 2.0 Source code 12
times
Download ? http//emist.ist.psu.edu
5
EMIST Tool Design Space
Pre-Execution
Post-Execution
Execution
-- Draw topology -- Import topology -- Configure
a node -- Setup virtualization -- Generate TCL
scripts -- Setup meters -- Upload programs --
Setup trace logger -- Configure bandwidth,
latency, etc. -- Specify attacks -- etc.
-- Attack injectors -- Background traffic
generators -- Replay trace data -- Trace
logger -- Event logger -- Meters -- Virtual
nodes -- Internet interface simulator --
Event coordination -- Conf. tracking -- Pause,
reconfigure, resume -- etc.
-- Trace analysis (scripts) -- Visualization --
Traffic data mining -- Data aggregation --
Animation, replay -- Database integration --
User-defined views -- TCPDUMP2Netflow -- Analysis
workflow learning -- etc.
6
ESVT Overview
-- May 2004 Version 1.0 -- May 2005 Version
2.0
Pre-Execution
Post-Execution
Execution
-- Draw topology -- Import topology -- Configure
a node -- Setup virtualization -- Generate TCL
scripts -- Configure bandwidth, latency,
etc. -- Specify attacks
-- Attack packet injectors (KMSim) -- Trace
logger -- Virtual nodes -- Internet interface
simulator
-- Visualization -- Traffic data mining -- Data
aggregation -- Animation, replay -- Database
integration -- User-defined views --
TCPDUMP2Netflow
To be integrated.
7

• EMIST topology specification in TCL
• - Virtual sub-network nodes
• - Internet interface
• - Normal vulnerable nodes
• - Bandwidth, latency, addresses, OS
• Other auxiliary TCL scripts

Step 1. Setup the experiment using
ESVT
Step 2. Setup the DETER environment
- Worm program - Traffic generator program -
Internet interface program - Virtual node
program - Normal node program - Vulnerable node
program - TCPDUMP setup - EMULAB GUI can be used
here
Step 3. Run the experiment on DETER

• Worm propagation snapshots
• Worm propagation animation
• Link traffic bar chart (dynamic)
• Worm replay

Step 4. Visualize the results using
ESVT
8
Year 3 Themes of ESVT

• BGP ESVT
• Integration
• Integrate ESVT into the broader SEW (Security
  Experimenters Workbench) concept
• Integrate NTD and other trace audit tools into
  ESVT
• Support PREDIT
• Use ESVT to help experimenters understand the
  characteristics of various DHS data sets

9
ESVT Screenshots
Demo this afternoon
10
The topology of the worm experiment done by Nick
Weaver et al. in 2004.
11
Internet Interface
Switch
Host
Enterprise topology 925 hosts, 70 switches, 7
routers
12
A topology imported from GT-ITM format.
13
Node configuration in a zoomed-in topology.
14
set lan70 ns make-lan "n(969) n(978) " 100Mb
0ms --Total Switch 3, Computer 58,
Susceptible ones 1. set link969 ns duplex-link
n(979) n(977) 100Mb 0ms DropTail Running
programs section tb-set-node-startcmd n(902)
"/proj/worm/e1k/scripts/run_virtual n-902-lan3
160" tb-set-node-startcmd n(903)
"/proj/worm/e1k/scripts/run_virtual n-903-lan4
160" tb-set-node-startcmd n(936)
"/proj/worm/e1k/scripts/run_virtual n-936-lan37
160 .. tb-set-node-startcmd n(943)
"/proj/worm/e1k/scripts/run_virtual n-943-lan44
160" tb-set-node-startcmd n(945)
"/proj/worm/e1k/scripts/run_tcp 945
160" tb-set-node-startcmd n(946)
"/proj/worm/e1k/scripts/run_virtual n-946-lan47
160" tb-set-node-startcmd n(969)
"/proj/worm/e1k/scripts/run_virtual n-969-lan70
160" tb-set-node-startcmd n(972)
"/proj/worm/e1k/scripts/run_tcp 972
160" tb-set-node-startcmd n(973)
"/proj/worm/e1k/scripts/run_tcp 973
160" tb-set-node-startcmd n(974)
"/proj/worm/e1k/scripts/run_tcp 974
160 tb-set-node-startcmd n(978)
"/proj/worm/e1k/scripts/run_tcp 978
160" tb-set-node-startcmd n(979)
"/proj/worm/e1k/scripts/run_internet 979 160" ns
rtproto Static ns run
network address/prefix 10.1.1.1/16 node
virtual node map file n- TYPE(B/I/V/R) S/N
(GUI node index) (Last segment of
IP) n-902 V N 29 254 n-902 V N 27
253 n-902 V N 32 252 n-902 V N 36
251 n-902 V N 38 250 n-902 V N 40
249 n-902 V N 43 248
A TCL script generated by ESVT support
virtualization set up trace loggers set up
the Internet interface etc.
15
-- Use a SQL query to instrument a network-wide
traffic view. -- MySQL database integration.--
Support both TCPDUMP and NetFlow formats.
16
Data sources for link visualization are defined
by a SQL query
17
User-defined link visualization options to
define views
18
Sample visualization output. Click on any plot
will zoom-in and show further details.
19
Animation the network event replay toolbar with
a pop-up link traffic chart.
20
BGP ESVT the first shot.
21
Questions?
22
PSU KMSim Slammer-like Attack Generator

• KMSim is a simulation code, consisting of coupled
  Kermack-McKendrick epidemic equations, to model
  the spread of a bandwidth-limited, randomly
  scanning Internet worm
• Benefit a family of worms can be flexibly
  simulated by tuning few parameters

23
PSU NTD Traffic Data Mining Tool

• This tool can detect the significant clusters,
  i.e., clusters whose traffic is greater than a
  threshold (either in terms of packet number or
  bytes)
• Cluster definition source IP, destination IP,
  source port, destination port or protocol
• NTD is an efficient implementation of that
  described by Estan et al. in SIGCOMM 03
• NTD is offline
• A tool for efficient mining of the
  multidimensional traffic cluster hierarchy for
  digesting, visualization, and modeling

24
EMIST Tool Effort

• ICSI/PSU worm scale-down equations
• PSU ESVT toolkit
• PSU KMSim Slammer-like attack generator
• PSU NTD traffic data mining tool
• Purdue scriptable event system
• Purdue sys info logging tool
• Purdue data analysis and viz scripts
• SPARTA/McAfee DDOS trace analysis and viz scripts
• SRI/UCD worm simulation tools
• UCD emulated worm attack generation tool
• UCD NTGC network traffic generation and control
  tool
• UCD XML worm specification tool
• UCD BGP routing data viz tool
• Officially released

25
Purdue Scriptable Event System

• During a DETER experiment, many events may happen
• time events, cmd events, etc.
• Although local event response can be
  pre-programmed on a single test machine,
  synchronized event response among a set of test
  machines cannot be pre-programmed
• This tool allows runtime coordinated event
  response via a coordinator-participant model
• Each test machine can run a participant stub that
  communicates with the coordinator to report
  events and receive response instructions
• The global event response plan can be flexibly
  scripted by the experimenter

26
Purdue Sys Info Logging Tool

• This tool logs system level statistics associated
  with a certain network interface

timestamp, bytes_per_sec, pack_per_sec,
bytes_per_sec_up,pack_per_sec_up, memtotal,
memused, uptime, idletime, established TCP
connections, half open TCP connections,TCPSlowSt
artRetrans count,TCPAbortOnTimeout count,errs
on the device drivers, drops on the device
drivers
27
UCD Emulated Worm Attack Generation

• All nodes host a worm generation daemon.
• Nodes wait for worm attack instructions.
• Propagation behavior of worm is varied by varying
  the instructions.
• An XML specification of worm propagation serves
  as the instructions.

28
UCD Network Traffic Generation and Control (NTGC)