A Case Study on Automated Generation of Integration Tests

1
A Case Study on Automated Generation of
Integration Tests

• Giuseppe Della Penna, Alberto Tofani
• Dipartimento di Informatica, Università di
  LAquila
• Marcello Pecorari, Orazio Raparelli
• Technolab SRL, LAquila, Italy
• Igor Melatti, Enrico Tronci
• Dipartimento di Informatica, Università di Roma
  La Sapienza

FDL06, September 19-22, 2006, Darmstadt
University, Germany
2
Background
GDMO ASN.1
NE developed according to the TMN
(Telecommunication Management Network) framework.
For each NE submodule S ia avilable (from the
design flow) a formal description based on GDMO
(Guidelines for the Definition of Managed Object)
and ASN.1 (Abstract Syntax Notation number one).
Test engineers are familiar with GDMO, ASN.1.
3
Our Problem
GDMO ASN.1

• Generate tests to check integration between
  Hardware and Software in TLC systems. Presently
  tests planning and test generation are manual
  processes (altough they start from formal GDMO,
  ASN.1 system descritpions).
• GOAL Automate test generation, subject to the
  following constraints
• Do not alter too much the design flow
• No heavy retraining
• Only use open source software

4
Manual Test Environment
Network Inputs
file_1.h
GDMO, ASN.1 File
NE
include
. . .
file_n.h
Parser
Trigger
IST-Simulator
asntbl.h
read
Human Intervention to design and submit test
cases. Namely, from GDMO, ASN.1 definitions test
enginners define test cases that submit to the NE
using the IST graphical interface.
5
Proposed Automated Integration Test Environment
Defined using a GDMO-ASN.1 like notation
6
Sketch of the Proposed Approach

• Status
• Company Engineers are familiar wth GDMO and ASN.1
  notations.
• GDMO and ASN.1 are used to define the external
  view of the NE.
• Communication between NE and the external
  environment is organized in triggers.
• A trigger is an assignment to some of the
  variables (attributes) of an NE submodule.
• The structure of triggers is defined via GDMO ans
  ASN.1
• A test case is just a trigger (i.e. a variable
  assignment).

• Our contribution
• Define a GDMO/ASN.1 like notation to define
  triggers. This avoids the use of the IST
  graphical interface (manual approach).
• Define a GDMO/ASN.1 like notation to define test
  plans.
• Implement glue software . as needed.

• Rationale
• Company engineers are familiar with GDMO, ASN.1
  notation. No retraining needed.
• No change in the design flow.
• Many company tools (already based on GDMO, ASN.1)
  can be reused.
• No need to buy/learn new software products. Al
  software used is open source.

7
Sequence Choice Trees (SCTs)
Plan Test Root
SCT for test plan of MSI-FP
ConnectionChoice
T4Selectedreference
crossConnUni
Type(sN) 1..5 Type(inst) 1..2
FromTo subtree
FromTo subtree
Sequence Node
Choice node
Leaf node
FromTo subtree
8
Triggers (1)
9
Triggers (2)
10
Test Plan Specification Example
ConnectionChoice (crossConnBI.FromTo1.from.sN
1..5), (sN 1..3), (crossConnBI.FromTo2.fro
m.inst 1..4) (inst 1..2)
Here is a trigger (test case) satisfying the
above spec-line (test plan)
ConnectionChoice
Trigger
crossComb1
FromTo2
from
to
sN 3
inst 2
sN 1
inst 2
11
Generation of Test Cases
We have implemented a tool that given a test plan
specification L (as a set of spec-lines) returns
all triggers stafying L.
Let L ltA1(r1), Ak(rk)gt. Using the info in
the asntbl.h file we can build k SCTs S1, Sk
representing triggers A1(r1), Ak(rk) . A1(r1)
? S1 Ak(rk) ? Sk We then build the SCTs S
Finally we generate from S all sequences of k
triggers in the test plan by instantiating the
leaves of S (I.e. by assigning them a value
compatibile with the corresponding spec-line in
L).
12
Conclusions
Our main contributions can be summarized as
follows Define a GDMO/ASN.1 like notation to
define triggers (i.e. test cases). This avoids
the use of the IST graphical interface (manual
approach) and allows us to automate submission of
test cases to the system. Define a GDMO/ASN.1
like notation to define test plans. Using such
test plan specifications engineer can easily
define the set of intresting test to carry
out Implemented the software needed to
effectively integrate our approach in the company
design flow. Found bugs the went undiscovered
with manually (informally) generated test cases.