Evaluation Methods for Internet Security Technology EMIST

1
Evaluation Methods for Internet Security
Technology (EMIST)
NSF Cyber Trust PI Meeting and DETER
workshop Newport Beach, CA, Sept. 2005
2
EMIST TEAM

• PSU G. Kesidis(PI), P. Liu, P. McDaniel, D.
  Miller
• UCD K. Levitt (PI), F. Wu, J. Rowe, C.-N. Chua
• ICSI V. Paxson (PI), N. Weaver
• Purdue S. Fahmy (PI), N. Shroff, E. Spafford
• SPARTA D. Sterne (PI), S. Schwab, R. Ostrenga,
  R. Thomas, S. Murphy, R. Mundy
• SRI P. Porras, L. Breismeister
• overall PI, expt lead/co-lead, EMIST ESVT
  lead
• PMs Joe Evans (NSF) and Douglas Maughan (DHS)
• Sister project DETER cyber security testbed

3
Outline

• Team.
• Goals.
• Publications.
• Tools released.
• Talks for DETER workshop Wed 09/28/05.
• Y3 activities.

4
EMIST goals

• Develop scientifically rigorous testing
  frameworks and methodologies for defenses against
  attacks on network infrastructure scale-down
  with fidelity.
• Develop experiments to yield deeper understanding
  of how previous attacks have, and future attacks
  will, affect the Internet and its users.
• Develop prototypical experiments (benchmarks) and
  associated databases of
• topologies and topology generators
• attack and background traffic traces and
  generators
• defenses
• special-purpose devices (meters, virtual nodes,
  etc.)
• metrics for scale-down fidelity, performance,
  overhead, etc.

5
EMIST goals (cont)

• Consult in the build-out of the DETER testbed and
  demonstrate its usefulness to vendors,
  researchers and customers of defense technology.
• Allow for open, convenient, rigorous, unbiased
  and secure testing of cyber defenses on DETER in
  order to expedite their commercial deployment.
• Quickly and publicly disseminate our results.

6
2004 EMIST publications

•  
• N. Weaver, I. Hamadeh, G. Kesidis and V. Paxson,
  Preliminary results using scale-down to explore
  worm dynamics, in Proc.  ACM WORM, Washington,
  DC, Oct. 29, 2004.
• P. Porras, L. Biesemeister, K. Levitt, J. Rowe,
  K. Skinner, A. Ting, A hybrid quarantine
  defense, in Proc. ACM WORM, Washington, DC, Oct.
  29, 2004.
• S.T. Teoh, K. Zhang, S.-M. Tseng, K.-L. Ma and S.
  F. Wu, Combining visual and automated data
  mining for near-real-time anomaly detection and
  analysis in BGP, in Proc. ACM VizSEC/CMSEC-04,
  Washington, DC, Oct. 29, 2004. 

7
2005 EMIST publications

• A. Kumar, N. Weaver and V. Paxson, "Exploiting
  Underlying Structure for Detailed Reconstruction
  of an Internet-scale Event", in Proc. ACM IMC
  2005.
• R. Pang, M. Allman, M. Bennett, J. Lee, V.
  Paxson, B. Tierney, "A First Look at Modern
  Enterprise Traffic ", in Proc. ACM IMC 2005.
• S. Schwab, B. Wilson, R. Thomas, Methodologies
  and Metrics for the Testing and Analysis of
  Distributed Denial of Service Attacks and
  Defenses, MILCOM, Atlantic City, NJ, Oct. 2005.
• L. Li, S. Jiwasurat, P. Liu, G. Kesidis,
  Emulation of Single Packet UDP Scanning Worms in
  Large Enterprises, In Proc. 19  International
  Teletraffic Congress (ITC-19), Beijing, Aug.
  2005.
• Q. Gu, P. Liu, C.-H. Chu, Hacking Techniques in
  Wired Networks, In The Handbook of Information
  Security, Hossein Bidgoli et al. (eds.), John
  Wiley Sons.
• S. Sellke, N. B. Shroff, and S. Bagchi, "Modeling
  and AutomatedContainment of Worms", In
  Proceedings of the International Conference in
  Dependable Systems and Networks (DSN), June 2005.
• R. Chertov, S. Fahmy, and N. B. Shroff,
  "Emulation versusSimulation A Case Study of
  TCP-Targeted Denial of Service Attacks",Purdue
  University Technical Report, September 2005.
• L. Briesemeister and P. Porras. Microscopic
  simulation of agroup defense strategy. In
  Proceedings of Workshop on Principles of
  Advanced and Distributed Simulation (PADS), pages
  254-261, June 2005.
• C. H. Tseng, T. Song, P. Balasubramanyam, C. Ko,
  and K. Levitt, "A Specification-based Intrusion
  Detection Model for OLSR, in Proc. RAID, Sept.
  2005.

8
2005 EMIST publications

• K. Zhang, S. Teoh, S. Tseng, R. Limprasittipom,
  C. Chuah, K. Ma, andS.F. Wu. PERFORMING BGP
  EXPERIMENTS ON A SEMI-RELISTIC INTERNET
  TESTBEDENVIRONMENT. in the 2nd International
  Workshop on Security inDistributed Systems
  (SDCS), conjunction with ICDCS, 2005.
• W. Huang, J. Cong, C. Wu, F. Zhao, and S.F. Wu.
  DESIGN, IMPLEMENTATION,AND EVALUATION OF
  FRITRACE. in 20th IFIP International
  InformationSecurity Conference, May, 2005,
  Chiba, Japan, Kluwer AcademicPublishers.
• G. Hong, F. Wong, S.F. Wu, B. Lilja, T.Y.
  Jansson, H. Johnson, and A.Nilsson.
  TCPTRANSFORM PROPERTY-ORIENTED TCP TRAFFIC
  TRANSFORMATION.in GI/IEEE SIG SIDAR Conference
  on Detection of Intrusions and Malware
  Vulnerability Assessment (DIMVA), Vienna,
  Austria, July, 2005, LNCS,Springer.
• J. Crandall, S.F. Wu, and F. Chong. EXPERIENCES
  USING MINOS AS A TOOLFOR CAPTURING AND ANALYZING
  NOVEL WORMS FOR UNKNOWN VULNERABILITIES.
  inGI/IEEE SIG SIDAR Conference on Detection of
  Intrusions and Malware Vulnerability Assessment
  (DIMVA), Vienna, Austria, July, 2005,
  LNCS,Springer.
• G.H. Hong and S.F. Wu. ON INTERACTIVE INTERNET
  TRAFFIC REPLAY. in the8th Symposium on Recent
  Advanced Intrusion Detection (RAID),
  Seattle,September, 2005, LNCS, Springer.
• J. Crandall, Z. Su, S.F. Wu, and F. Chong. ON
  DERIVING UNKNOWNVULNERABILITIES FROM ZERO-DAY
  POLYMORPHIC METAMORPHIC WORM EXPLOITS.To
  appear in 12th ACM Conference on Computer
  Communication Security(CCS2005), Alexandria,
  November 7-11, 2005.

9
EMIST tools

• EMIST Experiment Specification and Visualization
  Tool (ESVT) 2.0 released in May 05 with
• more advanced traffic viz features including link
  data and SQL interface, and
• ability to import output from a scale-free
  topology generator (with associated plotting
  tool).
• Offline netflow audit tool released in May 05.
• Online Scriptable Event System (SES) and, data
  analysis measurement tools.
• XML worm configuration and worm modeling.
• TCPOpera traffic generator and ELISHA viz tool.
• BGP topology capture tool.
• Experimental technical reports.

10
ICSI worm demo source models for testing
net-based detectors

• We are developing layer 4 (TCP/UDP) source
  models.
• Process of representing normal systems
• Derived from traces of a medium-scale enterprise
  (10K hosts)
• Store traffic information in database
• Classify host types application sessions based
  on measurements
• Create background traffic by sampling hosts and
  sessions
• Near-term goal is to mimic the Layer 4 behavior
  of normal hosts
• Testing against Approximate TRW worm containment
• Overlay worm traffic by adding worm-functionality
  to models
• Longer term goals
• investigate abstract source models
• apply to other containment technology

11
UC Davis / SRI worm demo collaborative
host-based defense

• Hosts that are not protected by network defenses
  can protect themselves from worm attack by
  collaborating with collections of other hosts to
  exchange alerts.
• A preliminary end-host collaborative worm defense
  exchanging failed connection reports will be
  demonstrated
• with respect to its ability to protect against
  worm spread
• in the presence of realistic background traffic.
• A 2000 virtual node experiment that uses our two
  tools
• the NTGC traffic generator and
• the UCD Worm Emulator

12
SPARTA DDoS demo

• FloodWatch defense deployed on both PCs
  andCloudShield appliances, as well as Juniper
  routers.
• A range of data collection and EVST visualization
  tools will be explored.
• The theme is examination of the experimental
  methodology, in particular
• the degree to which accurate detection and
  response characteristics can be calculated versus
• the limited fidelity of generated background
  traffic.

13
Purdue Method and Tools for High-Fidelity
Emulation of DoS Attacks

• Simulation versus emulation of DoS attack
  experiments are compared.
• As a case study, we considered low-rate
  TCP-targeted DoS attacks.
• Specific measurement-fidelity issues of the DETER
  testbed were resolved.
• We found that software routers such as Click
  provide a flexible experimental platform, but
  require detailed understanding of the underlying
  network device drivers to ensure they are
  correctly used.
• We also found that an analytical model and ns-2
  simulations closely match with typical values of
  attack pulse lengths and router buffer sizes.

14
UCD Requirements and Toolsfor Routing
Experiments

• Tools Requirements and Design (with SPARTA)
• ER (Entity Relationship) Information
  Visualization
• Experiments
• Interaction of BGP/OSPF/P2P
• Cross-layer routing dynamics/interactions
• Per-Update OASC Experiment
• Analysis of address ownership
• DDoS/Routing Interaction (with Purdue)
• DDoS impacts on BGP

15
PSU BGP demoLarge-Scale eBGP Simulator (LSEB)

• Our goal is large Internet-scale (global) routing
  attack modeling and measurement.
• Methodology
• intial AS topologies drawn from PREDICT
  Routeviews
• 20k java threads running across DETER hosts
• simulate all BGP message level interactions
• maintain route tables for all reachable prefixes
• Future work 
• realistic AS forwarding delay models 
• modeling iBGP 
• scale-down of experiments with more
  complex/realistic BGP speakers
• defense deployment and evaluation on DETER

16
PSU ESVT demo

• ESVT rendering of UDP/TCP worm emulation in an
  enterprise  
• We have emulated SQL slammer on a 1000 node
  enterprise network and compared the realism
  achieved by VM (jail), real LANs, and virtual
  nodes.  
• We are currently emulating TCP Blaster worm
  considering issues including the fidelity of our
  Blaster modeling technique, and the impact of
  background traffic.  
• Note that no defense is involved, just a local
  block of dark addresses used for detection.

17
Y3 Activities

• Release of reusable code developed for on-going
  attack/defense experiments, in particular
• ESVT 3.0 with integrated trace audit tool,
  spectral analysis, etc.
• Synthesize background traffic analogous to trace
  data in DETER experiments on same topology.
• BGP ESVT.
• Continued outreach, in particular BGP ESVT
  components to the ops community.
• Collaborate with DETER on, e.g., experimental
  workbench (SEW), RIB output collection.

18
Y3 Activities (cont)

• For each attack experiment, a summary document
  that described in particular
• Experimental methodologies.
• Metrics for experimental realism in defense
  evaluation.
• Benchmark attack experiments for specific classes
  of defenses.
• Experimental Tech Reports
• Experiment archiving and repeatability issues.
• Critical assessments of all items in deterlabs
  experimenters tools web pages.
• Summer 2006 attack/defense demonstration
  experiments.