Title: UCMBased Generation of Test Goals
1UCM-Based Generation of Test Goals
- Daniel Amyot, University of Ottawa
- (with Michael Weiss and Luigi Logrippo)
- damyot_at_site.uottawa.ca
- RDA Project (funded by NSERC)
2UCM-Based Testing?
- Can we derive test goals and test cases from UCM
models? - Scenario notation ? good basis!
- Reuse of (evolving) requirements model for
validation - UCMs are very abstract
- Very little information about data and
communication - Difficult to derive implementation-level test
cases automatically - Deriving test goals is more realistic
- How much value? How much effort?
3Test Generation Approaches
- Based on UCM Testing Patterns
- Grey-box test selection strategies, applied to
requirements scenarios - Manual
- Based on UCM Scenario Definitions
- UCM simple data model, initial values and start
points, and path traversal algorithms - Semi-automatic
- Based on UCM Transformations
- Exhaustive traversal
- Mapping to formal language (e.g., LOTOS)
- Automated
4UCM Testing Patterns
5UCM-Oriented Testing Pattern Language
HandleStubs
HandleStartPoints
HandleConstructs
UCM
Test Goals
6TP6 Causally-Linked Stubs
- Handles combinations of plug-ins bound to
causally linked stubs. - 6A Default behavior (when no feature is active)
- 6B Individual functionalities (when one feature
is active at a time) - 6C Functionality combinations (when several or
all functionalities are active)
7TP5 Single Stub
a
Plug-in 1
OUT1
IN1
S
EP1
OUT2
OUT1
b
IN1
SP
c
Plug-in 2
IN2
EP2
OUT1
OUT2
OUT2
d
- Flattens one stub from a hierarchical UCM.
- 5A Static flattening (when only one plug-in in
the static stub) - 5B Dynamic flattening, some plug-ins (when
several plug-ins in the dynamic stub) - 5C Dynamic flattening, all plug-ins (when
several plug-ins in the dynamic stub)
8UCM-Oriented Testing Pattern Language
HandleStubs
HandleStartPoints
HandleConstructs
UCM
Test Goals
9TP4 Multiple Start Points
SP1
EP
SP2
SP3
Strategies based on necessary, redundant,
insufficient, and racing subsets (8 strategies
based on path sensitization, not shown here)
10UCM-Oriented Testing Pattern Language
HandleStubs
HandleStartPoints
HandleConstructs
UCM
Test Goals
11TP1 Alternatives
a
c
SP
EP
b
d
- 1A All results (end points)ltSP, a, c, EPgt
- 1B All segmentsltSP, a, c, EPgt, ltSP, b, d, EPgt
- 1C All pathsltSP, a, c, EPgt, ltSP, a, d, EPgt,
ltSP, b, c, EPgt, ltSP, b, d, EPgt - 1D All combinations of sub-conditions (for
composite conditions, e.g., (X OR Y) AND Z )
12TP2 Concurrency
a
SP
EP
b
c
- 2A One combinationltSP, a, b, c, EPgt
- 2B Some combinationsltSP, a, b, c, EPgt, ltSP, b,
a, c, EPgt - 2C All combinationsltSP, a, b, c, EPgt, ltSP, b,
a, c, EPgt, ltSP, b, c, a, EPgt
13TP3 Loops
b
a
SP
EP
- 3A All segments
- 3B At most k iterations
- 3C Valid boundaries low, highTests low,
low1, high-1, and high - 3D All boundaries low, highTests valid ones
(3C) and invalid ones (low-1 and high1, for
rejection tests)
14Complementary Strategies
- Strategies for value selection
- Equivalence classes, boundary testing
- Typical approaches in traditional testing
- Strategies for rejection test cases
- Forbidden scenarios
- Testing patterns
- Incomplete conditions
- Off-by-one value
15UCM Scenario Definitions
16Scenarios Definitions (all start at
Order) NormalLargeQty LargeQtyTrue,
NeedMoreFalse. NormalSmallQty
LargeQtyFalse, NeedMoreFalse. UndefinedNeedMo
re LargeQtyFalse, NeedMoreUndefined. Interest
ingCase LargeQtyFalse, NeedMoreTrue.
17Process Overview Tool Support
2. UCM File
2. UCM File
4. UCM
E
(
scen
.
defs
)
(
scen
.
defs
)
XPORTER
8.
Telelogic
5. MSC
5. MSC
TAU
3. XML File
3. XML File
3. XML File
3. XML File
6. Sequence
6. Sequence
9.
Rational
(scenarios)
(scenarios)
(scenarios)
(scenarios)
Rose
Diagram
Diagram
1. UCMN
AV
with traversal
7.
TTCN
-
3
10.
TTthree
18NormalLargeQty Scenario,in XML and in MSC
lt?xml version'1.0' standalone'no'?gt lt!DOCTYPE
scenarios SYSTEM "scenarios1.dtd"gt ltscenarios
design-name "WITUL04" ...gt ltgroup name
"WitulTests" group-id "1" gt ltscenario name
"NormalLargeQty" scenario-definition-id "1"
gt ltseqgt ltdo name"Order"
type"Start" comp "Customer" ... /gt
ltcondition label"LargeQty" expression
"LargeQty" /gt ltdo name"PrepRequest"
type"Resp" comp "Retailer" ... /gt
ltcondition label"DeliverAndRequest" expression
"NeedMore" /gt ltdo name"in1"
type"Connect_Start" comp "Retailer" .../gt
ltpargt ltseqgt ltdo
name"out2" type"Connect_End" comp "Retailer"
.../gt ltdo name"RequestForMore"
type"End_Point" .../gt lt/seqgt
ltseqgt ltdo name"out1"
type"Connect_End" comp "Retailer".../gt
ltdo name"Delivered" type"End_Point" comp
"Customer".../gt lt/seqgt lt/pargt
lt/seqgt lt/scenariogt lt/groupgt lt/scenariosgt
19UCM Transformations
20Automatic Test GoalsLOTOS and TTCN-2 (Charfi)
- Path traversal algorithm
- Fixed subset of testing patterns
- No data model
- Exhaustive coverage of UCM model
- Unfeasible scenarios
- Ucm2LotosTest
- Based on older version of UCMNav
- Test goals as LOTOS processes
- Converted to TTCN-2 via TGV
21Automatic Test GoalsLOTOS and MSC (Guan)
- Generation of LOTOS specification and test
scenarios - Fixed subset of testing patterns
- Exhaustive coverage of UCM model
- Unfeasible scenarios
- Can be verified against spec
- Still, no data model
- Tools (in Java) independent of UCMNav
- LOTOS model could be used as basis for test
generation
22Summary
23Towards Test Cases
- Communication
- Communication mechanisms between pairs of
components connected by a UCM path must be
specified (e.g., messages, parameters and data
values, protocols). - Hiding
- UCM responsibilities and start/end points located
inside components may be hidden and hence left
out of the test goals. The interface used to test
needs to be specified. - Data values
- Data values need to be selected such that the
various conditions in the test goal are
satisfied. Conventional techniques are
applicable. - Time information
- UCM timers do not have a quantitative notion of
time intervals or constraints. - Set-up and clean-up
- Preambles and postambles may be needed for each
test case. - Target
- Tests need to be re-targetable and readable by
test equipment, something that is supported by
languages such as TTCN-3. - Should some of these aspects have an impact on
the source scenario language?
24Several Opportunities
- UCM ? TTCN-3
- Using scenario definitions
- UCM ? MSC ( SDL) ? TTCN-2
- Using scenario definitions and Tau Autolink
- UCM ? (H)MSC ? TTCN-3 (text or GFT)
- Using scenario definitions with additional static
information suggested by Michael Ebner - UCM ? Web-service testing languages
- Preserve alternatives, loops, and sub-scenarios
during UCM traversal - Right now only sequence and concurrency is
preserved - UCM ? UML 2.0 activity/sequence diagrams
- UCM and testing profile?
25Tools
- UCMNav 2.2http//www.usecasemaps.org/tools/ucmnav
/ - UCMExporterhttp//ucmexporter.sourceforge.net/
- UCM2LOTOSAvailable upon request
26(No Transcript)
27Test Case GenerationUCM/Requirements Validation
Scenario definition MSC/SDL
Scenario definition LOTOS
SPEC-VALUE
28Test Case GenerationConformance Testing
UCM-SDL indirect (validated UCM tests)
SDL direct (conventional)
UCM direct
LOTOS direct (conventional)
UCM- LOTOS indirect (validated UCM tests)