SOA Testing using Keyword Driven Test Automation

1
SOA Testing using Keyword Driven Test Automation

• Peter Gillogley

2
Overview

• The SOA Challenge
• 7 principals for SOA testing

3
Service Oriented Architecture
4
Another view
5
The System to be covered
6
7 Principals for SOA Testing

• Use a flexible automation framework
• Adopt a keyword driven test approach
• Apply data driven testing when appropriate
• Hide mundane data setup
• Use risk and requirements driven test design
• Test the testware
• Maximize the chance of surprises

7
The Journey
Manual
QTP
FIT
WATIR
Model Based (UML)
Ruby
DIY Model
Java Net Beans
JUnit
Keyword Framework
8
Next Generation Open Source Testing Tool
Elisabeth Hendrickson, Quality Tree Software,
Inc. (inspired by the work of the Agile Alliance
Functional Testing Tools Program)
9
Use a flexible automation framework
10
Adopt a Keyword Driven Test Approach

• Action words / Action Based Testing (Hans
  Buwalda)
• Modular, reusable test components that are built
  by test architects and then assembled into test
  scripts by test designers (Forrster 2005)
• An application-independent automation framework..
  independent of the test automation tool used to
  execute them and the test script code that
  "drives" the application-under-test and the data
  (IBM 2003).

11
KJRA Action Word Framework
Keyword Interpreter
XML log
XSL translation
SOAP
12
Example Keyword Test
13
Keyword Test Result
14
Apply Data Driven Testing
15
Hide Mundane Data Driven Setup

• Creates a person with random content for approx
  100 demographic fields (e.g. surname, first name,
  middle name, DOB, sex, religion,
  country_of_birth)
• Ensures valid and unique of names and identifiers

• Creates and sends a HL7 Message populated with
  the data generated above

16
Develop Keywords for Web service

• Query the web service (store the response)
• Parameterised checks of the content of the
  response
• Specific keywords for regularly used checks
• Specific keywords for repeating fields

17
Develop Keywords for HL7
18
Use Risk and Requirements to Drive Test Case
Design

• Test management tool used to document risks and
  requirements (Quality Centre)
• Keyword tests linked to relevant risks and
  requirements
• Risk and requirements coverage managed by test
  management tool.
• Test execution and reporting managed by test
  management tool (manually initiated)

19
Test the Testware

• Keywords and underlying methods / classes
• Testware regression suite
• Potential false positives
• Test framework needs to be stable
• Testware bugs sap tester morale

20
Maximise Surprises

• Simple test cases are easier to read and
  maintain but
• Unexpected outcomes may be discovered by
• Checking outside the focus of the test case
• Not immediately blame the testware for
  intermittent errors
• Sufficient logging to support analysis of
  incomplete test case execution
• Randomised input data
• Frequent execution of test cases

21
Lessons Learnt

• Automation is a powerful way to check complex
  interfaces
• Test scenarios may identify specification gaps
• Possible to get high requirements coverage

• But

• Dont underestimate the number of keywords
  required to be developed
• Framework bugs effect morale and schedule
• Need domain and automation knowledge
• Business scenarios may identify specification
  gaps

22
Questions
Peter Gillogley ( http//www.gillogley.com )