SOA Testing SISE Approach

1
SOA Testing SISE Approach Scelta Test
Automation Framework
Sathyan S Nair Senior Technical Analyst December
13,2007
2
Agenda

• Needs Solutions
• Focus Areas Challenges
• SOA Lifecycle SISE Approach
• Scelta The Automation Framework
• Key Benefits
• Future Research
• Open Discussion

3
Needs Solutions
4
Business Testing Needs

• Business Needs
• Evolve IT Solutions
• Avoid costly integration and re-engineering
  efforts
• Testing Needs
• How do we validate services for functional
  requirements?
• How do we manage the complexity of testing
  Integration of multiple services?
• How do we validate shared services across
  multiple applications, as services change and
  evolve?

5
Address the Needs

• How to address the needs?
• Various stages of testing - Required to address
  function as well as interaction complexities
• Testers to wear additional hats and follow an
  evolved approach
• Automation Framework for multiple test execution
  cycles

6
Our Solution
SISE Approach Scelta The Framework
7
Focus Areas Challenges
8
How different is SOA Testing

• Head-fewer services, without a UI
• Services spread across different boundaries
• Look beyond FT PT Integration, System
  Workflow Testing
• Requirements change impact Across multiple
  application boundaries
• Need for multiple end-to-end business
  process/workflow testing

9
Addressing Challenges
10
SOA Lifecycle SISE Approach
11
SOA Lifecycle SISE Approach
12
SISE Approach Service/Component Testing

• What we do?
• Validation - Light-weight service Implementation
• Scripts Development - Assertions and Boundary
  Conditions
• Good known data based validation technique

• Test Results
• Ensure the proper functioning of individual
  web-methods/services

13
SISE Approach Integration Testing

• What we do?
• Validation Communication between different
  systems
• Scripts Development Messaging formats
  (namespace, SOAP compliance etc), Communication
  policies (parameters, return type etc), Security
  policies (Authentication, WS-Security etc)
• Good known data based validation technique

• Test Results
• Different types of services across multiple
  systems seamlessly integrate and communicate with
  each other

14
SISE Approach System Testing

• What we do?
• Validation Communication between different
  systems
• Scripts Development Messaging formats
  (namespace, SOAP compliance etc), Communication
  policies (parameters, return type etc), Security
  policies (Authentication, WS-Security etc)
• Mock services to handle Asynchronous Messaging
  communications
• Message Checkpoints Route messages to
  appropriate services
• Performance Monitors Study the response times ,
  uptimes etc
• Helps the team with known limitations,
  re-engineer and hardware recommendations

• Test Results
• Different types of services across multiple
  systems seamlessly integrate and communicate with
  each other, under production env
• Overall transactions response time calculation

15
SISE Approach End-To-End Testing

• What we do?
• Validation Complete business cases/workflow
• Different types of business systems - Types of
  protocols include HTTP, HTTPS, SMTP, SOAP,
  XML-RPC, IRC etc.
• Mock services , Messages checkpoints
  performance monitors deployed for appropriate
  usage
• Service change undergoes a complete end-to-end
  testing cycle
• Study of response time delays and key
  performance/response issues

• Test Results
• Complete business workflow completes without any
  errors
• System satisfies the requirements

16
Scelta The Automation Framework
17
Test Automation Framework
18
Scelta Input Data Driver Components

• Input Store
• XML driven Test Cases
• Business and technical users can create test
  cases
• Elements and schema guidelines for file creation
• Configuration file for implementation information
• Workflow Manager
• Test execution - User choice (Business/Integratio
  n/security cases etc)
• Priority for test workflow/scenario/case
• Test execution Environment configuration
• Policy Manager
• Assign appropriate policy Test cases

19
Scelta Service Validation Components

• Service Validation
• Validate individual services/web-methods
• Assertions for different test-conditions
• Xpath query to fetch information
• Response Manager
• Manages request and directs response
• Invokes mock services - For transactions with
  response delay time
• Message Routers To deliver responses
  appropriately
• Logs Reports
• Centralized logs and reports collector
• Customized reports based on customer requirements

20
Scelta Performance Monitors Metadata

• Distributed Monitoring
• Service level monitors and agents
• Customized agent for collecting transactional
  information
• Load Manager
• Workflow driven load testing
• Configurable load Monitor performance of
  service
• Run periodically - Ensure Web service stability
• Service Metadata
• Collects Historical information - Measure
  service reliability and quality
• Study service behavior (up/down time, time since
  last failure, service reliability history etc)

21
Key Benefits Approach Framework

• Multi-level validationDefects fixes-Minimal
  effort cost
• Test Coverage - Different requirements validation
• Minimal Defect leakage in production
• Asynchronous test script execution
• Service response facts , reliability and quality
  reports
• Re-usable across n of test cycles

22
Future Research

• Support security related aspects
• Finding services Addressing testing pain-points

23
Open Discussion
24
Thank YouReach Mesathyan.sethumadhavan_at_csscor
p.com