Software Security Vulnerability Testing in Hostile Environment

1
Software Security Vulnerability Testing in
Hostile Environment

• Herbert H. Thompson
• James A. Whittaker
• Florence E. Mottay

2
Introduction

• Functionality and Security Are Often in
  Contention as Development Goals
• Increased Functionality Leads to Decreased
  Security
• Increase Security requires less Software
  Interaction with It Environment
• Security Issues are left under-exercised or not
  traversed at all due to the difficulty in
  simulating a hostile environment
• Network Failures during aremote transaction
• Disk Write errors, Memory Failures
• Such Failures Need to be Integrated into the Test
  Cases

3
Placing a System under Stress

• A Security defect, in a general sense, is any
  attribute of the software that violates policies
  regarding access to resources
• Sensitive information is written out to an
  unencrypted file
• Allows attackers to Deny access to a web server
  by authorized users
• Some Security vulnerabilities can be discovered
  using conventional testing techniques
• Many others security defects only surface when
  the application is placed under stressed
  environments
• Sources of Stress load, memory or resource
  deprivation, etc
• When stress is present error handling routines
  are executed

4
Continuation

• Three Basic Approaches to Introduce Environmental
  failure
• Code-Based Fault Injection
• External Stress Simulation
• Runtime Fault Injection

5
Symptoms of Security Defects

• Security failures derived from the interaction
  with the environment
• Reading/Writing Files
• Writing to the Registry
• Sending information across the network

6
Monitoring for Symptoms

• An application interaction with its environment
  is its most critical behaviour
• External Activities can be observed by monitoring
  the System Calls the Application makes
• Application can be monitored by getting in
  between the application and the environment when
  a system call is performed
• To determine if there exist a security risk a
  special tool can be used to provide system-level
  design detail
• Holodeck an application tool intercepts systems
  calls and allows to view the interaction between
  the application and its environment
• Holodeck allows the tester to carefully examine
  the interaction to find potential points of
  security break

7
Different types of approaches for security
testing

• Code-Based Fault Injection
• Forces error conditions
• Error handling is white box in nature
• Involves modifying the source code and hard
  coding return values
• Forces the application to traverse a particular
  path
• Although efficient it has some problems
• Tester do not always have access to all the parts
  of source code
• Even with source code testers might lack
  expertise of knowledge of the codes design and
  structure to effectible implement the branch
  testing
• It is time consuming implementing those
  situations at code level one instance per
  situation

8
Continuation

• Example of Code-based Fault Injection
• hModule
• LoadLibraryEx(TEXT(msrating.dll),NULL,
• LOAD_LIBRARY-AS-DATAFILE)
• hModule NULL // Hard-Coded failure of
• //LoadLibraryEx

9
Different types of approaches for security
testing (cont)

• External Stress Simulation
• Involves simulating a high volume of activity on
  the system
• Accomplished by using an external application
  that does not interact directly with the
  application under test
• Or by limiting disk or memory resources
• Large files
• Background processes, etc

10
Different types of approaches for security
testing (cont)

• Runtime Fault Injection
• Black box fault injection
• System calls are monitored and values returned by
  these calls are controlled
• Faults are simulated without changing the code of
  the application
• The application is placed in a realistically
  hostile environment by inserting a runtime fault
• A single system call can be isolated to
  manipulate its response during stress situation

11
Conclusion