Security Vulnerability Analysis and Mitigation for RealWorld Systems

1
Security Vulnerability Analysis and Mitigation
for Real-World Systems
Shuo Chen Center for Reliable and
High-Performance Computing Coordinated Science
Laboratory University of Illinois at
Urbana-Champaign Final Exam, August 18th, 2005
Committee Chair Prof. Ravi Iyer
Committee Prof. Vikram Adve
Prof. Ravi Iyer
Prof. Jose Meseguer
Prof. David Nicol
2
Prelim Exam Recap

• Analyzed security vulnerability reports in
  Bugtraq and CERT advisories
• Most vulnerabilities can be modeled as a series
  of simple logic predicates.
• Used FSM models to reason about many categories
  of vulnerabilities.
• A common characteristic of most security
  vulnerabilities pointer taintedness
• Pointer value derived from user input
• Allow users to specify memory addresses. Usually
  due to attacks!
• Developed a theorem proving approach to reason
  about possibility of pointer taintedness
• To uncover potential vulnerabilities.

3
Since Prelim Exam

• Questions focused
• Is pointer taintedness detection just an
  alternative approach to existing defense
  techniques, or is it a significant improvement?
• Is pointer taintedness detection applicable to
  large real-world software?

4
Since Prelim Exam (cont.)

• Contributions
• Demonstrate that a new security attack
  non-control-data attack, is applicable to many
  real-world software, not addressed by many
  current defense techniques.
• Demonstrate that pointer taintedness detection
  can naturally defeat non-control-data attacks as
  well as traditional attacks.
• Demonstrate that pointer taintedness detection
  can be deployed in large systems.
• By building into processor architecture
• By combining theorem proving and runtime
  assertions

5
Summary of My Research

• Start from the analysis of a large volume of
  security data
• Extract common characteristics of security
  vulnerabilities and attacks
• Propose new defense techniques (supported by
  real-world attack models)

6
Publications

• S. Chen, J. Xu, E. C. Sezer, P. Gauriar and R. K.
  Iyer. "Non-Control-Data Attacks Are Realistic
  Threats," USENIX Security Symposium, 2005.
• S. Chen, J. Xu, N. Nakka, Z. Kalbarczyk, R. K.
  Iyer. Defeating Memory Corruption Attacks via
  Pointer Taintedness Detection, DSN, 2005.
• S. Chen, J. Dunagan, C. Verbowski and Y.-M. Wang,
  A Black-Box Tracing Technique to Identify Causes
  of Least-Privilege Incompatibilities, NDSS,
  2005.
• S. Chen, J. Xu, Z. Kalbarczyk, R. K. Iyer.
  Security Vulnerabilities From Analysis to
  Detection and Masking Techniques, Proceedings of
  the IEEE, 2005.
• S. Chen, K. Pattabiraman, Z. Kalbarczyk, R. K.
  Iyer, "Formal Reasoning of Various Categories of
  Widely Exploited Security Vulnerabilities Using
  Pointer Taintedness Semantics," IFIP SEC, 2004
• S. Chen, J. Xu, Z. Kalbarczyk, R. K. Iyer and K.
  Whisnant. Modeling and Evaluating the Security
  Threats of Transient Errors in Firewall
  Software, Performance Evaluation, 2004.
• S. Chen, Z. Kalbarczyk, J. Xu, R. K. Iyer. "A
  Data-Driven Finite State Machine Model for
  Analyzing Security Vulnerabilities," DSN, 2003.
• S. Chen, J. Xu, R. K. Iyer, K. Whisnant.
  "Modeling and Analyzing the Security Threat of
  Firewall Data Corruption Caused by Instruction
  Transient Errors," DSN, 2002.
• J. Xu, S. Chen, Z. Kalbarczyk, R. K. Iyer. "An
  Experimental Study of Security Vulnerabilities
  Caused by Errors," DSN, 2001.
• 9 full papers in IEEE DSN, USENIX Security, IFIP
  Security, ISOC NDSS, Proceedings of IEEE and
  Journal of Performance Evaluation

7
Non-Control-Data Attacks Are Realistic Threats

• (Joint work with Jun Xu)
• In USENIX Security Symposium, 2005

8
Control Data Attack Well-Known, Dominant

• Control data attack corrupt function pointers,
  jump targets and return addresses to run
  malicious code
• Currently the most dominant form of memory
  corruption attacks CERT and Microsoft Security
  Bulletin
• By exploiting many vulnerabilities such as buffer
  overflow, format string bug, integer overflow,
  double free, etc.
• Many current defense techniques to enforce
  control data integrity to provide security.
• Monitor system call sequences (Intrusion
  detection systems)
• Protect control data (Secure Program Execution,
  Minos)
• Non-executable stack and heap (Linux, OpenBSD,
  Windows XP SP2)

9
Non-Control-Data Attack

• Non-control-data attacks attacks not corrupting
  any control flow data
• Currently very rare in reality
• Very few instances documented in literature.
• Several papers possible to construct
  non-control-data attack against synthetic
  programs.
• Not yet considered as a serious threat
• How applicable are such attacks against
  real-world software?
• Why rare ? attackers incapability or lack of
  incentives?
• No focused investigation yet.

10
Our Claim General Applicability of
Non-Control-Data Attacks

• The claim
• Many real-world software applications are
  susceptible to non-control-data attacks.
• The severity of the attack consequence is
  equivalent to that due to control data attacks.
• Goal of our project
• Experimentally validate the claim
• Construct non-control-data attacks to compromise
  the security of widely-used applications
• Discuss limitations of current defense techniques
• Show that pointer taintedness detection can
  defeat both control-data attacks and
  non-control-data attacks.

11
Non-Control-Data Attack against WU-FTP Server
(via a format string bug)
int x FTP_service(...) authenticate() x
user ID of the authenticated user seteuid(x)
while (1) get_FTP_command(...) if
(a data command?) getdatasock(...)
getdatasock( ... ) seteuid(0)
setsockopt( ... ) seteuid(x)
When return to service loop, still runs as EUID 0
(root). Allow me to upload /etc/passwd I can
grant myself the root privilege! Only corrupt an
integer, not a control data attack.
12
Non-Control-Data Attack against NULL-HTTP Server
(via a heap overflow bug)

• Attack the configuration string of CGI-BIN path.
• Mechanism of CGI
• suppose server name www.foo.comCGI-BIN
• Requested URL http//www.foo.com/cgi-bin
• The server executes
• Our attack
• Exploit the vulnerability to overwrite CGI-BIN to
  /bin
• Request URL http//www.foo.com/cgi-bin/sh
• The server executes

/usr/local/httpd/exe
/usr/local/httpd/exe
/bar
/bar
/bin
/sh
The server gives me a root shell! Only overwrite
four characters in the CGI-BIN string.
13
Non-Control-Data Attack against SSH
Communications SSH Server (via an integer
overflow bug)
void do_authentication(char user, ...) int
auth 0 ... while (!auth) / Get a
packet from the client / type
packet_read() switch (type) ...
case SSH_CMSG_AUTH_PASSWORD if
(auth_password(user, password)) auth
1 case ... if (auth) break
/ Perform session preparation. /
do_authenticated()
14
More Non-Control-Data Attacks

• Against NetKit Telnet server (default Telnet
  server of Redhat Linux)
• Exploit a heap overflow bug
• Overwrite two strings/bin/login h foo.com -p
  (normal scenario) /bin/sh h p
  -p (attack scenario)
• The server runs /bin/sh when it tries to
  authenticate the user.
• Against GazTek HTTP server
• Exploit a stack buffer overflow bug
• Send a legitimate URL http//www.foo.com/cgi-bin/b
  ar
• The server checks that /.. is not embedded in
  the URL
• Exploit the bug to change the URL to
  http//www.foo.com/cgi-bin/../../../../bin/sh
• The server executes /bin/sh

15
What Non-Control-Data Attacks Imply?

• Control data integrity is not sufficient to
  ensure software security for real-world software.
• Many types of non-control data critical to
  security
• User identity data, configuration data, user
  input text string and decision-making Boolean
• Once attackers have the incentive, they are
  likely to succeed in non-control-data attacks.

16
Runtime Pointer Taintedness Detection at
Processor Level

• Joint work with Jun Xu and Nithin Nakka
• In IEEE International Conference on Dependable
  Systems and Networks (DSN), 2005

17
Recap Pointer Taintedness

• The root cause of many memory corruption attacks
  pointer taintedness
• No matter whether they overwrite control-data or
  non-control-data
• Many type of vulnerabilities e.g., buffer
  overflow, format string, heap corruption, integer
  overflow, and globbing attacks.
• Pointer taintedness a pointer value is derived
  from user input
• In prelim, I showed a theorem proving technique
  to reason about possibility of pointer taintedness

18
Is a Format String Attack Due to Pointer
Taintedness?
Vulnerable code recv(socket,filename)
sprintf(buf,s not found,filename)
printf(buf) / should be printf(s,buf)
/ Suppose user ID, CGI-BIN or critical flag in
0x1002bc20
High
n d d d 0x1002bc20
Stack growth
Low
In vfprintf(), if (fmt points to n) then
ap (character count)
ap is the tainted value 0x1002bc20.
19
Runtime Pointer Taintedness Detection

• A processor architectural level mechanism to
  detect pointer taintedness
• On SimpleScalar processor simulator
• Implemented a taintedness-aware memory system
• One-bit extension for each byte, similar to the
  parity bit, to indicate the taintedness of this
  byte
• Taintedness tracking
• Taintedness is propagated by ALU instructions
• Taintedness initialization
• read and recv system calls tag every byte of
  receiving buffer as tainted
• Attack detection
• When a tainted value is dereferenced (i.e., used
  as a pointer).

20
ID/EX
EX/MEM
MEM/WB
Opcode
store path
Compare specific logic
Shift specific logic
AND specific logic
XOR specific logic
4 bits
4 bits
Bitwise OR
4 bits
Register File
M U X
alert
MUX
Data Memory
MUX
load/store?
jr?
M U X
M U X
alert
0
0
4 bits
M U X
36 bits
32 bits
A L U
32 bits
8-bit byte
32 bits
Taintedness bit
load path
36 bits
21
Evaluation

• Effectiveness of attack detection
• Synthetic vulnerable programs
• Real-world network applications
• Evaluation of false positives
• Real-world network applications
• SPEC 2000 benchmarks
• Potential false negative scenarios
• A few attack scenarios that are not detected

22
Effectiveness of Attack Detection

• First, test on synthetic vulnerable programs
• All attacks are detected and terminated

23
Attack Detection Effectiveness (cont.)

• Test on real network applications
• All attacks are detected
• No difference between control-data attack and
  non-control-data attack from the viewpoint of
  pointer taintedness

24
Evaluation of Transparency and False Positives

• Transparent precompiled binary executables can
  run
• Test on network applications
• No attack ? no alert
• Test on SPEC benchmarks
• Execute 15,139 million instructions without any
  alert
• Conclusion No known false positive

25
Potential False Negative Scenarios

• Incorrect array index boundary check
• Determining correct array size requires source
  code analysis very hard at binary level
• Buffer overflow within the local frame
• If no pointer is tainted, no alert is raised
• Unlikely to cause severe security damage because
  attacker-controllable location is very limited
• Format string attack causing information leak
• This attack allows peeking a few words on the top
  of the stack.
• Cause security compromises if these words contain
  security-critical secret, e.g., key and password

26
Combining Static Analysis and Runtime Detection
27
Towards An Easier Deployment of Pointer
Taintedness Detection

• Advantage/limitation of static analysis
• to derive assertions (when satisfied, eliminate
  pointer taintedness)
• No need for hardware modification, but not easy
  to deploy in large programs
• Advantage/limitation of runtime detection
• Easy to deploy in large programs, but needs
  modification of the processor
• Can we combine the two?
• Static analysis to extract security
  specifications of critical functions
• Enforce these specifications by runtime
  assertions
• Purely a software approach (of course, we can
  also design hardware to enforce runtime
  assertions)

28
Verification Condition (VC) Generation
char p, q if (a 1) p p 10 q
p - 2 q 12
29
Case Study free()
typedef struct _HEAP_BLOCK int
Size int Busy struct
_HEAP_BLOCK Fwd, Bak HEAP_BLOCK,
PHEAP_BLOCK char BlockSizes void free(char
p) int BlockSize,i char
BuddyBlock, FreedBlock int
FreeBlockListIndex,MergeExit
FreedBlockp-sizeof(HEAP_BLOCK) // Mark this
block free. FreedBlock-gtBusy0
BlockSizeFreedBlock-gtSize FreeBlockListIndex
CalculateFreeBlockLis
tIndex(BlockSize) FreeBlockListIndex0
while (BlockSize gt (BlockSizesFreeBlockListIndex
)) BlockSize BlockSize / 2
FreeBlockListIndex MergeExit0
while (FreeBlockListIndex lt 6 MergeExit0)
BuddyBlock HEAP_BASE
(FreedBlock- HEAP_BASE) BlockSize
if (BuddyBlock-gtBusy BuddyBlock-gtSize
! BlockSize) MergeExit1
else // Make a bigger block and
free it. BlockSize2
FreeBlockListIndex if
(BuddyBlockltFreedBlock)
FreedBlock BuddyBlock
BuddyBlock-gtFwd-gtBakBuddyBlock-gtBak
BuddyBlock-gtBak-gtFwdBuddyBlock-gtFwd
FreedBlock-gtSize BlockSize \
FreedBlock-gtBusy 0 InsertTailList(FreeBloc
kListIndex, FreedBlock)
Compile
inst(1) mov FreedBlock lt- ( p - 16)
. inst(2) mov FreedBlock 4 lt- 0
. inst(3) mov BlockSize lt- (( FreedBlock
0)) . inst(4) mov FreeBlockListIndex lt- 0
. inst(5) no-op . inst(6) branch (( ((
BlockSizes FreeBlockListIndex)) lt
BlockSize)) 10 . inst(7) mov BlockSize
lt- ( BlockSize / 2) . inst(8) mov
FreeBlockListIndex lt- ( FreeBlockListIndex)
1 . inst(9) branch true 5 . inst(10)
no-op . inst(11) mov MergeExit lt- 0
. inst(12) no-op . inst(13) branch ((
FreeBlockListIndex lt 6 MergeExit is 0)) 28
. inst(14) mov BuddyBlock lt- ((HEAP_BASE
(((( FreedBlock - HEAP_BASE)) xor
BlockSize)))) . inst(15) branch (((
(( BuddyBlock 4)) is 0) ( (( BuddyBlock
0)) is BlockSize))) 18. inst(16)
mov MergeExit lt- 1 . inst(17) branch
true 26 . inst(18) no-op . inst(19)
mov BlockSize lt- 2 . inst(20)
mov FreeBlockListIndex lt- (
FreeBlockListIndex) 1 . inst(21)
branch (( BuddyBlock lt FreedBlock)) 23
. inst(22) mov FreedBlock lt-
BuddyBlock . inst(23) no-op . inst(24)
mov ( BuddyBlock 8) 12 lt- (
BuddyBlock 12) . inst(25) mov (
BuddyBlock 12) 8 lt- ( BuddyBlock 8)
. inst(26) no-op . inst(27) branch
true 12 . inst(28) no-op . inst(29) mov
FreedBlock 0 lt- BlockSize . inst(30) mov
FreedBlock 4 lt- 0 . inst(31) no-op .
VC generation
VC(1) T ( p) false ? T ( ( x 8))
false ? T ( ( x 12))
false x (((p-16) - HEAP_BASE) xor (p-16))
HEAP_BASE
30
Case Study free() (cont.)

• Runtime enforcement of VC using a runtime
  assertion

void free(char p) HEAP_BLOCK
x(HEAP_BLOCK)
(HEAP_BASE (((p-16) - HEAP_BASE)
((UINT)(p-16)))) assert (x-gtFwd-gtBak x
x-gtBak-gtFwd x) ( the
original source code of free() )

• Effectiveness

/ try to hijack f() to buffer p / int
main() char p void (f)() p
malloc(40) (UINT)(p60)(UINT)p
(UINT)(p56)((UINT)f)-12 free(p)
Heap corruption attack. Assertion is violated!
31
Case Study vfprintf()
int vfprintf (char s, char format, char ap)
char p, q int done,state,data,n char
buf10 pformat done0 if (p0) return
0 state1 while (p ! 0) if
(state1) if (p) state0
else done
else if (p)
done else if (pd)
dataap if
(datalt0) done
data-data
n0 while (datagt0
nlt10)
(bufn)data100
data/10 n

while (ngt0) n--
done
else if (ps)
qap if (q0) break
while (q!0)
done q
else if (pn)
q ap (int
) q done done
else done state1
p return done
32
Case Study vfprintf() (cont.)

• Extracted VC

VC(8) ( ( state 1) p n) -gt (T(
ap) false)

• Runtime enforcement of VC using a runtime
  assertion

int vfpintf (FILE s, const char format, va_list
ap) while (p ! 0) assert
(!(state ! 1 pn !UNTAINTED(ap)))
int printf (const char format, ...)
return vfprintf (stdout, format, arg)
void main() mov esp, stack_top
ADD_UNTAINTED_ADDR (stack_top-4)
printf("strings\nid\nn",buf,i,j)
REMOVE_UNTAINTED_ADDR (stack_top-4)
scanf(s,buf) printf(buf)
Legitimate call. Assertion holds
Format string attack. Assertion is violated
33
Conclusions
34
Conclusions

• Most security vulnerabilities (in Bugtraq and
  CERT) can be modeled as a series of violations of
  logic predicates
• Promising to apply formal method to analyze
  software security (shown in prelim exam)
• Many real-world software can be compromised by
  corrupting non-control data.
• Need a more comprehensive defense technique
• Pointer taintedness is a unifying perspective to
  reason about most memory corruption
  vulnerabilities/attacks.
• Effective for defeating both control-data attacks
  and non-control-data attacks
• Detecting about pointer taintedness is a
  promising direction to enhance security on
  real-world systems
• Techniques explored
• theorem proving (shown in prelim exam)
• runtime detection
• combination of automatic VC generation and
  runtime assertion