The Role of Buffer Overflow Detection in Intrusion Prevention

1
The Role of Buffer Overflow Detection in
Intrusion Prevention

• Andrew Wentzell

2
Overview

• Intrusion Detection Systems
• Host-based vs. Network-based
• Signature-based vs. Anomaly-based
• Intrusion Prevention Systems
• Vulnerability Detection
• Static vs. Dynamic

3
Buffer Overflow Background

• C provides low-level memory access
• Some programs do not properly check input from
  the user
• Can allow malicious user to manipulate program
• Write to unexpected areas of memory
• Alter programs execution

4
The Stack

• Parameter storage
• Dynamic memory allocation
• func(arg1, arg2, arg3)
• int x
• char buf10
• char p

5
The Objective

• Overwrite return addresss
• Execute code at arbitrary location
• Shellcode

6
Detection

• NIDS watches network traffic for signs of attack
• Signatures known attacks
• Anomalies traffic that varies widely from the
  good traffic
• HIDS watches a single system for signs of
  compromise
• Rootkits, trojans, etc.
• Changes to system files

7
Prevention

• Dynamic systems catch a buffer overflow in any
  number of ways
• Canary value
• str() function wrappers
• Local variable re-ordering
• Saved return values
• Static systems analyze source code for
  programming errors
• Sanity checks

8
Example Scenario

• Attacker knows of vulnerability in your web
  server platform.
• Can he exploit it?
• Connected to www.yourcompany.com.GET /index.cgi
  HTTP/1.1Host www.yourcompany.comUser-Agent
  AAAAAAAAAAAAAAAAAAA...

9
Example Scenario (cont.)
10
Thats not the end of the story

• Would provide extra security against
  vulnerabilities in the software itself
• Protect your systems in the critical time before
  a patch is available
• You still have to secure your networks, use
  secure protocols, etc.

11
Thank you!

• Questions?