Hamsa: Fast Signature Generation for Zero-day Polymorphic Worms with Provable Attack Resilience

1
Hamsa Fast Signature Generation for Zero-day
Polymorphic Wormswith Provable Attack Resilience

• Zhichun Li, Manan Sanghi, Yan Chen, Ming-Yang Kao
  and Brian Chavez

Lab for Internet Security Technology
(LIST)Northwestern University
2
The Spread of Sapphire/Slammer Worms
3
Desired Requirements for Polymorphic Worm
Signature Generation

• Network-based signature generation
• Worms spread in exponential speed, to detect them
  in their early stage is very crucial However
• At their early stage there are limited worm
  samples.
• The high speed network router may see more worm
  samples But
• Need to keep up with the network speed !
• Only can use network level information

4
Desired Requirements for Polymorphic Worm
Signature Generation

• Noise tolerant
• Most network flow classifiers suffer false
  positives.
• Even host based approaches can be injected with
  noise.
• Attack resilience
• Attackers always try to evade the detection
  systems
• Efficient signature matching for high-speed links

No existing work satisfies these requirements !
5
Outline

• Motivation
• Hamsa Design
• Model-based Signature Generation
• Evaluation
• Related Work
• Conclusion

6
Choice of Signatures

• Two classes of signatures
• Content based
• Token a substring with reasonable coverage to
  the suspicious traffic
• Signatures conjunction of tokens
• Behavior based
• Our choice content based
• Fast signature matching. ASIC based approach can
  archive 6 8Gb/s
• Generic, independent of any protocol or server

7
Unique Invariants of Worms

• Protocol Frame
• The code path to the vulnerability part, usually
  infrequently used
• Code-Red II .ida? or .idq?
• Control Data leading to control flow hijacking
• Hard coded value to overwrite a jump target or a
  function call
• Worm Executable Payload
• CLET polymorphic engine 0\x8b, \xff\xff\xff
  and t\x07\xeb
• Possible to have worms with no such invariants,
  but very hard

8
Hamsa Architecture
9
Hamsa Design

• Key idea model the uniqueness of worm invariants
• Greedy algorithm for finding token conjunction
  signatures
• Highly accurate while much faster
• Both analytically and experimentally
• Compared with the latest work, polygraph
• Suffix array based token extraction
• Provable attack resilience guarantee
• Noise tolerant

10
Hamsa Signature Generator

• Core part Model-based Greedy Signature
  Generation
• Iterative approach for multiple worms

11
Outline

• Motivation
• Hamsa Design
• Model-based Signature Generation
• Evaluation
• Related Work
• Conclusion

12
Problem Formulation
Signature Generator
Signature
false positive bound r
With noise
NP-Hard!
13
Model Uniqueness of Invariants
U(1)upper bound of FP(t1)
U(2)upper bound of FP(t1,t2)
The total number of tokens bounded by k
14
Signature Generation Algorithm
token extraction
t1
u(1)15
tokens
Suspicious pool
Order by coverage
15
Signature Generation Algorithm
Signature
t1
t2
u(2)7.5
Order by joint coverage with t1
16
Algorithm Analysis

• Runtime analysis O(T(MN))
• Provable Attack Resilience Guarantee
• Analytically bound the worst attackers can do!
• Example K5, u(1)0.2, u(2)0.08, u(3)0.04,
  u(4)0.02, u(5)0.01 and r0.01
• The better the flow classifier, the lower are the
  false negatives

Noise ratio FP upper bound FN upper bound
5 1 1.84
10 1 3.89
20 1 8.75
17
Attack Resilience Assumptions

• Two Common assumptions for any sig generation sys
• Two Unique assumptions for token-based schemes
• Attacks to the flow classifier
• Our approach does not depend on perfect flow
  classifiers
• With 99 noise, no approach can work!
• High noise injection makes the worm propagate
  less efficiently.
• Enhance flow classifiers

18
Improvements to the Basic Approach

• Generalizing Signature Generation
• use scoring function to evaluate the goodness of
  signature
• Iteratively use single worm detector to detect
  multiple worms
• At the first iteration, the algorithm find the
  signature for the most popular worms in the
  suspicious pool.
• All other worms and normal traffic treat as
  noise.

19
Outline

• Motivation
• Hamsa Design
• Model-based Signature Generation
• Evaluation
• Related Work
• Conclusion

20
Experiment Methodology

• Experiential setup
• Suspicious pool
• Three pseudo polymorphic worms based on real
  exploits (Code-Red II, Apache-Knacker and
  ATPhttpd),
• Two polymorphic engines from Internet (CLET and
  TAPiON).
• Normal pool 2 hour departmental http trace
  (326MB)
• Signature evaluation
• False negative 5000 generated worm samples per
  worm
• False positive
• 4-day departmental http trace (12.6 GB)
• 3.7GB web crawling including .mp3, .rm, .ppt,
  .pdf, .swf etc.
• /usr/bin of Linux Fedora Core 4

21
Results on Signature Quality
Worms TrainingFN TrainingFP EvaluationFN EvaluationFN EvaluationFP Binaryevaluation FP
Worms Signature Signature Signature Signature Signature Signature
Code-Red II 0 0 0 0 0 0
Code-Red II '.ida?' 1, 'u780' 1, ' HTTP/1.0\r\n' 1, 'GET /' 1, 'u' 2 '.ida?' 1, 'u780' 1, ' HTTP/1.0\r\n' 1, 'GET /' 1, 'u' 2 '.ida?' 1, 'u780' 1, ' HTTP/1.0\r\n' 1, 'GET /' 1, 'u' 2 '.ida?' 1, 'u780' 1, ' HTTP/1.0\r\n' 1, 'GET /' 1, 'u' 2 '.ida?' 1, 'u780' 1, ' HTTP/1.0\r\n' 1, 'GET /' 1, 'u' 2 '.ida?' 1, 'u780' 1, ' HTTP/1.0\r\n' 1, 'GET /' 1, 'u' 2
CLET 0 0.109 0 0.06236 0.06236 0.268
CLET '0\x8b' 1, '\xff\xff\xff' 1,'t\x07\xeb' 1 '0\x8b' 1, '\xff\xff\xff' 1,'t\x07\xeb' 1 '0\x8b' 1, '\xff\xff\xff' 1,'t\x07\xeb' 1 '0\x8b' 1, '\xff\xff\xff' 1,'t\x07\xeb' 1 '0\x8b' 1, '\xff\xff\xff' 1,'t\x07\xeb' 1 '0\x8b' 1, '\xff\xff\xff' 1,'t\x07\xeb' 1

• Single worm with noise
• Suspicious pool size 100 and 200 samples
• Noise ratio 0, 10, 30, 50, 70
• Noise samples randomly picked from the normal
  pool
• Always get above signatures and accuracy.
• Multiple worms with noises give similar results

22
Speed Results

• Implementation with C/Python
• 500 samples with 20 noise, 100MB normal traffic
  pool, 15 seconds on an XEON 2.8Ghz, 112MB memory
  consumption
• Speed comparison with Polygraph
• Asymptotic runtime O(T) vs. O(M2), when M
  increase, T wont increase as fast as M!
• Experimental 64 to 361 times faster (polygraph
  vs. ours, both in python)

23
Outline

• Motivation
• Hamsa Design
• Model-based Signature Generation
• Evaluation
• Related Work
• Conclusion

24
Related works
Hamsa Polygraph CFG PADS Nemean COVERS Malware Detection
Network or host based Network Network Network Host Host Host Host
Content or behavior based Contentbased Contentbased Behaviorbased Contentbased Contentbased Behavior based Behaviorbased
Noise tolerance Yes Yes (slow) Yes No No Yes Yes
Multi worms in one protocol Yes Yes (slow) Yes No Yes Yes Yes
On-line sig matching Fast Fast Slow Fast Fast Fast Slow
Generality Generalpurpose Generalpurpose Generalpurpose Generalpurpose Protocolspecific Serverspecific Generalpurpose
Provable atk resilience Yes No No No No No No
Information exploited egp egp p egp e eg p
25
Conclusion

• Network based signature generation and matching
  are important and challenging
• Hamsa automated signature generation
• Fast
• Noise tolerant
• Provable attack resilience
• Capable of detecting multiple worms in a single
  application protocol
• Proposed a model to describe the worm invariants

26
Questions ?