Title: Hamsa: Fast Signature Generation for Zero-day Polymorphic Worms with Provable Attack Resilience
1Hamsa Fast Signature Generation for Zero-day
Polymorphic Wormswith Provable Attack Resilience
- Zhichun Li, Manan Sanghi, Yan Chen, Ming-Yang Kao
and Brian Chavez
Lab for Internet Security Technology
(LIST)Northwestern University
2The Spread of Sapphire/Slammer Worms
3Desired Requirements for Polymorphic Worm
Signature Generation
- Network-based signature generation
- Worms spread in exponential speed, to detect them
in their early stage is very crucial However - At their early stage there are limited worm
samples. - The high speed network router may see more worm
samples But - Need to keep up with the network speed !
- Only can use network level information
4Desired Requirements for Polymorphic Worm
Signature Generation
- Noise tolerant
- Most network flow classifiers suffer false
positives. - Even host based approaches can be injected with
noise. - Attack resilience
- Attackers always try to evade the detection
systems - Efficient signature matching for high-speed links
No existing work satisfies these requirements !
5Outline
- Motivation
- Hamsa Design
- Model-based Signature Generation
- Evaluation
- Related Work
- Conclusion
6Choice of Signatures
- Two classes of signatures
- Content based
- Token a substring with reasonable coverage to
the suspicious traffic - Signatures conjunction of tokens
- Behavior based
- Our choice content based
- Fast signature matching. ASIC based approach can
archive 6 8Gb/s - Generic, independent of any protocol or server
7Unique Invariants of Worms
- Protocol Frame
- The code path to the vulnerability part, usually
infrequently used - Code-Red II .ida? or .idq?
- Control Data leading to control flow hijacking
- Hard coded value to overwrite a jump target or a
function call - Worm Executable Payload
- CLET polymorphic engine 0\x8b, \xff\xff\xff
and t\x07\xeb - Possible to have worms with no such invariants,
but very hard
8Hamsa Architecture
9Hamsa Design
- Key idea model the uniqueness of worm invariants
- Greedy algorithm for finding token conjunction
signatures - Highly accurate while much faster
- Both analytically and experimentally
- Compared with the latest work, polygraph
- Suffix array based token extraction
- Provable attack resilience guarantee
- Noise tolerant
10Hamsa Signature Generator
- Core part Model-based Greedy Signature
Generation - Iterative approach for multiple worms
11Outline
- Motivation
- Hamsa Design
- Model-based Signature Generation
- Evaluation
- Related Work
- Conclusion
12Problem Formulation
Signature Generator
Signature
false positive bound r
With noise
NP-Hard!
13Model Uniqueness of Invariants
U(1)upper bound of FP(t1)
U(2)upper bound of FP(t1,t2)
The total number of tokens bounded by k
14Signature Generation Algorithm
token extraction
t1
u(1)15
tokens
Suspicious pool
Order by coverage
15Signature Generation Algorithm
Signature
t1
t2
u(2)7.5
Order by joint coverage with t1
16Algorithm Analysis
- Runtime analysis O(T(MN))
- Provable Attack Resilience Guarantee
- Analytically bound the worst attackers can do!
- Example K5, u(1)0.2, u(2)0.08, u(3)0.04,
u(4)0.02, u(5)0.01 and r0.01 - The better the flow classifier, the lower are the
false negatives
Noise ratio FP upper bound FN upper bound
5 1 1.84
10 1 3.89
20 1 8.75
17Attack Resilience Assumptions
- Two Common assumptions for any sig generation sys
- Two Unique assumptions for token-based schemes
- Attacks to the flow classifier
- Our approach does not depend on perfect flow
classifiers - With 99 noise, no approach can work!
- High noise injection makes the worm propagate
less efficiently. - Enhance flow classifiers
18Improvements to the Basic Approach
- Generalizing Signature Generation
- use scoring function to evaluate the goodness of
signature - Iteratively use single worm detector to detect
multiple worms - At the first iteration, the algorithm find the
signature for the most popular worms in the
suspicious pool. - All other worms and normal traffic treat as
noise.
19Outline
- Motivation
- Hamsa Design
- Model-based Signature Generation
- Evaluation
- Related Work
- Conclusion
20Experiment Methodology
- Experiential setup
- Suspicious pool
- Three pseudo polymorphic worms based on real
exploits (Code-Red II, Apache-Knacker and
ATPhttpd), - Two polymorphic engines from Internet (CLET and
TAPiON). - Normal pool 2 hour departmental http trace
(326MB) - Signature evaluation
- False negative 5000 generated worm samples per
worm - False positive
- 4-day departmental http trace (12.6 GB)
- 3.7GB web crawling including .mp3, .rm, .ppt,
.pdf, .swf etc. - /usr/bin of Linux Fedora Core 4
21Results on Signature Quality
Worms TrainingFN TrainingFP EvaluationFN EvaluationFN EvaluationFP Binaryevaluation FP
Worms Signature Signature Signature Signature Signature Signature
Code-Red II 0 0 0 0 0 0
Code-Red II '.ida?' 1, 'u780' 1, ' HTTP/1.0\r\n' 1, 'GET /' 1, 'u' 2 '.ida?' 1, 'u780' 1, ' HTTP/1.0\r\n' 1, 'GET /' 1, 'u' 2 '.ida?' 1, 'u780' 1, ' HTTP/1.0\r\n' 1, 'GET /' 1, 'u' 2 '.ida?' 1, 'u780' 1, ' HTTP/1.0\r\n' 1, 'GET /' 1, 'u' 2 '.ida?' 1, 'u780' 1, ' HTTP/1.0\r\n' 1, 'GET /' 1, 'u' 2 '.ida?' 1, 'u780' 1, ' HTTP/1.0\r\n' 1, 'GET /' 1, 'u' 2
CLET 0 0.109 0 0.06236 0.06236 0.268
CLET '0\x8b' 1, '\xff\xff\xff' 1,'t\x07\xeb' 1 '0\x8b' 1, '\xff\xff\xff' 1,'t\x07\xeb' 1 '0\x8b' 1, '\xff\xff\xff' 1,'t\x07\xeb' 1 '0\x8b' 1, '\xff\xff\xff' 1,'t\x07\xeb' 1 '0\x8b' 1, '\xff\xff\xff' 1,'t\x07\xeb' 1 '0\x8b' 1, '\xff\xff\xff' 1,'t\x07\xeb' 1
- Single worm with noise
- Suspicious pool size 100 and 200 samples
- Noise ratio 0, 10, 30, 50, 70
- Noise samples randomly picked from the normal
pool - Always get above signatures and accuracy.
- Multiple worms with noises give similar results
22Speed Results
- Implementation with C/Python
- 500 samples with 20 noise, 100MB normal traffic
pool, 15 seconds on an XEON 2.8Ghz, 112MB memory
consumption - Speed comparison with Polygraph
- Asymptotic runtime O(T) vs. O(M2), when M
increase, T wont increase as fast as M! - Experimental 64 to 361 times faster (polygraph
vs. ours, both in python)
23Outline
- Motivation
- Hamsa Design
- Model-based Signature Generation
- Evaluation
- Related Work
- Conclusion
24Related works
Hamsa Polygraph CFG PADS Nemean COVERS Malware Detection
Network or host based Network Network Network Host Host Host Host
Content or behavior based Contentbased Contentbased Behaviorbased Contentbased Contentbased Behavior based Behaviorbased
Noise tolerance Yes Yes (slow) Yes No No Yes Yes
Multi worms in one protocol Yes Yes (slow) Yes No Yes Yes Yes
On-line sig matching Fast Fast Slow Fast Fast Fast Slow
Generality Generalpurpose Generalpurpose Generalpurpose Generalpurpose Protocolspecific Serverspecific Generalpurpose
Provable atk resilience Yes No No No No No No
Information exploited egp egp p egp e eg p
25Conclusion
- Network based signature generation and matching
are important and challenging - Hamsa automated signature generation
- Fast
- Noise tolerant
- Provable attack resilience
- Capable of detecting multiple worms in a single
application protocol - Proposed a model to describe the worm invariants
26Questions ?