INTELLIGENT INTRUSION DETECTION RESEARCH

1
INTELLIGENT INTRUSION DETECTION RESEARCH
ATTACKING A HIGH PERFORMANCE COMPUTER CLUSTER
by Miguel Torres, Rayford B. Vaughn, German
Florez, Zhen Liu, Susan M. Bridges ltmt98,vaughn,gf
24,zliu,bridgesgt_at_cs.msstate.edu May 2003

2
Outline

• Introduction
• IIDS
• Daemon Attacks
• Interposition Library Attacks
• Device Driver Attack
• Conclusions

3
Introduction

• Linux High Performance Clusters
• Supports the research in the intrusion detection
  area which captures function or system calls to
  detect anomalous behavior
• The implemented attacks will be presented here

4
Why This Work Is Needed

• Creates anomalous data sets for testing our IIDS
  AI techniques
• Assist us in providing an established data set
  with known attacks for community research

5
Structure of the Cluster Microcosm
6
The Architecture of the IIDS
7
Daemon Attacks

• daemon process A Non-interactive process running
  in the background that cannot maintain an
  association with a terminal
• can not write error messages on standard error
• can write files and do any other operation
  allowed to a normal process

8
Daemon Attacks (Cont.)

• Trojan horse implemented in a trusted
  application, that can execute the daemon process
  at any moment
• Can be implemented in a C and/or MPI environment
• Example DoS (buffer overflow, resource attack)

9
Daemon Attacks (Cont.)
Steps to Follow
Trusted Process

• fork() Trusted process
• setid() Make the new process a group leader,
  session leader, and lose control of tty
• signal(SIGHUP, SIG_IGN) Ignore SIGHUP that will
  be sent to a child of the process
• umask(0) lose file creation mask inherited from
  parent
• chdir(/) Change to working dir
• fork() Child process (grandchild)
• signal(SIGPIPE, SIG_IGN) Ignore for reading and
  writing to non-opened pipes.

10
Interposition Library Attacks

• Used for tuning performance, collecting runtime
  statistics, function/parameter information or
  debugging applications.
• Is a library that is loaded into the system to
  intercept any function call that an application
  makes to any selected shared library (libc, MPI
  PRO, etc)
• Once the function call is intercepted, the
  behavior of the function can be altered or
  changed to perform an attack on the system.
• Linux LD_PRELOAD environment variable

11
Interposition Library Attacks (Cont.)
Steps to Follow
MPI or C Executable Program

• Create library
• Function headers
• Mount the library
• LD_PRELOAD

Libc
libmpipro
System Calls
Linux Kernel
12
Interposition Library Attacks (Cont.)

• Linux Dynamically Loaded (DL) Libraries Package
  provides specific functions that assist in the
  implementation of the interposition library
• dlsym provides the address of a function symbol.
• dlopen opens a library
• dlclose closes a library

13
Interposition Library Attacks (Cont.)
FILE fopen(const char filename, const char
mode) // data referent to the real
function typedef FILE(function_type) (const
char filename, const char mode) static
function_type functionNULL static char
function_name"fopen" FILE retvalue //
Search for the next function that fits the
specification of function_name if
(!function) function (function_type)
dlsym(RTLD_NEXT,function_name) // If the
profiling is active if (DoProfile) //
Malicious code here!!! else //do not profile,
only execute retval ((function)(filename,mode
)) return (retval)
14
Interposition Library Attacks (Cont.)

• Examples
• LIBC
• fopen (file open function libc) the requested
  file can be copied in another directory with the
  user permissions changed so it can be accessed
  later by the attacker.
• fread, fwrite (read/write operation on files
  libc) can corrupt the information before the
  real function is called.
• malloc, memcpy (libc) memory management.

15
Interposition Library Attacks (Cont.)

• Examples
• MPI
• MPI_INIT, MPI_FINALIZE randomly generates a
  daemon attack on each processor that is running
  the distributed application to slow down the
  system.
• MPI_Recv, MPI_Send corrupting the transmitted
  data between processes.
• The LIBC attacks can be applied here too.

16
Device Driver Attack

• Implemented for a Linux environment
• Trojan horse inserted in a device driver,
  installed on the system
• Is loaded into the system to intercept any system
  call that an application makes to the kernel of
  the OS (sys_open, sys_close, etc)
• Change the current pointers of the system calls
  of the kernel, for other functions
• Can be used for DoS, stealing information or
  hijacking the system.

17
Device Driver Attack (Cont.)
MPI or C Executable Program
System Calls
Interposed Calls
Linux Kernel
18
Detection of Cluster Attacks

• Longer goal design and building of a prototype
  Intelligent Intrusion Detection System (IIDS)
  that demonstrates the effectiveness o Artificial
  Intelligence approaches in a HPC.
• MPI (Message Passing Interface)
• Problem Large amount of data to present and
  analyze.

19
Detection of Cluster Attacks (Cont.)

• Current mechanisms
• Detection with system calls
• Neural Networks
• Detection with function calls
• Hidden Markov Models
• Creation of an attack database

20
Conclusions

• The reported attacks were tested in a Linux OS
  and MPI environment
• We have shown some vulnerabilities in a cluster
  environment using MPI
• We created anomalous data sets where none
  previously existed