SIMULATING ERRORS IN WEB SERVICES

1
SIMULATING ERRORS IN WEB SERVICES

• International Journal of Simulation Systems,
  Sciences and Technology 2004
• Nik Looker, Malcolm Munro and Jie Xu

2
Outline

• Introduction
• Quality of Service
• Fault Injection
• Fault Model
• Test Case
• Conclusion and Future Work

3
Introduction

• Web Service technology is a key factor in the
  success of any e-Science or e-Commerce project.
• High Quality of Service (QoS) to allow the
  production of highly reliable systems.
• Fault injection is a well-proven method of
  assessing the dependability of a system

4
Introduction

• This fault injection method is a modified version
  of Network Level Fault Injection.
• This method differs from standard Network Level
  Fault Injection techniques
• Fault injector decodes the SOAP messages and can
  inject meaningful faults into a message
• This enables API-level parameter value
  modification to be performed as well as standard
  Network Level Fault Injection.

5
Quality of Service

• Availability
• the quality aspect of whether a Web Service is
  present and ready for use.
• Accessibility
• the quality aspect that represents the degree the
  Web Service is capable of servicing a request.
• Integrity
• the quality of the Web Service maintaining the
  correctness of any interaction.
• Performance
• the quality aspect that is defined in terms of
  the throughput of a Web Service and the latency.

6
Quality of Service

• Reliability
• the quality aspect that represents the capability
  of maintaining the service and service quality.
• Regulatory
• the quality aspect that the service corresponds
  to rules, laws, standards and specifications.
• Security
• the quality aspect that defines confidentiality
  for parties using a service

7
Fault Injection Techniques

• Network Level Fault Injection
• Corruption, loss or reordering of network
  messages at the network interface.
• Runtime injection technique that injects faults
  into captured network packets.
• Software Implemented Fault Injection (SWIFI)
• instrumenting the operating system protocol stack
• Perturbation
• Modify program states without mutating existing
  code statements.
• testing fault tolerance mechanisms

8
Fault Injection Techniques

• Corruption of Memory Space
• Platform Specific
• Code Mutation
• Certification Problems because code changed
• Syscall Interposition
• Platform Specific

9
Fault Injection Tools

• Ferrari (Fault and Error Automatic Real-Time
  Injection)
• uses a system based around software traps to
  inject errors into a system.
• traps are activated by either a call to a
  specific memory location or a timeout.
• FTAPE (Fault Tolerance and Performance Evaluator)
  
• inject faults into memory, registers and disk
  accesses

10
Fault Injection Tools

• Xception
• processors exception handling capabilities are
  used to trigger fault injection.
• DOCTOR (IntegrateD sOftware Fault InjeCTiOn
  EnviRonment)
• allows injection of memory and register faults,
  and network communication faults.
• combination of timeout, trap and code
  modification.

11
Fault Injection Tools

• Orchestra
• a script driven fault injector which is based
  around Network Level Fault Injection.

12
WS-FIT

• Web Service Fault Injection Technology
• Method and tools implement mechanism of fault
  injection and also provide a framework for the
  creation and execution of test cases
• Fault injector that allows network level fault
  injection
• Dependability Assessment of SOAP based SOA
• SOAP Based
• WSDL Defined Interfaces
• Combined with conventional specification

13
Fault Model

• Fault Types that can effect a SOAP based system
• Physical Faults
• Effecting memory or processor registers
• Software Faults
• Programming errors and design errors
• Resource Management Faults
• Memory leakage and exhaustion of resource
• Communication Faults
• Message deletion, duplication, delay, reordering
  or corruption
• Life-Cycle Faults
• Premature object destruction through starvation
  of keep-alive messages and delayed asynchronous
  responses

14
Enhanced Fault Model
15
Failure Modes

• Crash of a service
• Crash of a hosting web server (or the host
  itself)
• Hang of a service
• Corruption of input data
• Corruption of output data
• Duplication of messages
• Omission of messages
• Delay of messages

16
Why WS-FIT testing?

• Particularly interested in
• Assessing Quality of Service
• Assessing Fault Tolerance of Systems
• Developing Methods and Tools
• So that we can
• Detect defects in platform code and design
• Gather dependability metrics on platforms to
  allow comparisons
• Use WS-FIT to test individual Web Services

17
WS-FIT

• Consists of 3 parts
• Hook code in SOAP Library
• Fault Injector Framework
• Script
• Trigger
• Injection
• Results

18
SOAP API Hooks

• SOAP API Hooks
• One for sending messages.
• One for receiving messages.
• Hooks consisted of simple socket code
• To pass messages to fault injector.
• Receive (possibly) modified message from injector.

19
Instrumenting a Service
20
What the Framework Provides

• Logging Function (XML format for easy analysis)
• Detects faults and logs faults
• Logs injected fault packets
• Trigger and Injection mechanism
• Trigger on Message Type and parameter.
• Inject Fault through User Script.
• Implemented
• Framework implemented in Java for portability
• Scripts implemented in Python
• Executed in the JVM by Jython

21
User Script

• Result class
• One instance of this for duration of test.
• Used to store static results.
• Currently only tag count but will be enhanced in
  later experiments.
• Trigger Class
• One instance per packet.
• Returns Boolean value to indicate if injector
  class should be run.
• Use data from result class to determine if the
  correct point in message stream has been reached.
• Injector Class
• One instance per packet
• Inject fault into packet

22
WS-FIT
23
GUI

• The GUI provides
• An easy to use environment to create and execute
  tests
• Ability to create a skeleton test script from
  WSDL definitions
• Populate tests through user defined Fault Model
• Execution of test scripts from tool
• Real-time visualization of RPC message parameters

24
Quality of Service

• Experiment to demonstrate WS-FIT can be used to
  assess QoS criteria
• WS-FIT will be used to modify the performance of
  a system by injecting a latency (without code
  modification being required)

25
Test Scenario

• The test system implements a simple, self
  regulating, heater unit.
• The hardware used is simple and relies on the
  software drivers to prevent failure states being
  reached.
• Under normal operation the Unit will give a
  logarithmic temperature rise so the Controller
  uses timers to modify to a linear temperature
  rise.
• Will be used in a chemical experiment
• A substance must be heated to 60C
• Must reach 60ºC in a specified period of time
• Must have a linear temperature rise
• The test will introduce a latency to decrease the
  throughput and hence the performance of the
  system.

26
Test System
27
Natural Operation of Heater
28
Model
29
Applying a Fault Model to aParameter
30
Injecting Latency
31
Summary

• WS-FIT can be effectively used to
• Introduce SOAP protocol faults
• Introduce both random and non-random protocol
  faults
• Assess Quality of Service
• Such as throughput modification
• WS-FIT
• Relatively non-invasive
• Can be used to assess both middleware and
  individual services
• Gives real-time feedback through visualization of
  RPC parameters
• http//www.wsfit.org/