Automated functional test objectives generation from requirements

1
Automated functional test objectives generation
from requirements

• Clémentine Nebut
• Yves Le Traon
• Franck Fleurey

2
Objectives -1-

• Generating automatically test objectives from the
  functional requirements
• Within the UML functional requirements expressed
  with use cases
• Functional testing from use cases Binders
  problematic
• How do I choose test cases ?
• In what order should I apply my tests ?
• How do I know when Im done ?

3
Objectives -2-

• Idea Benefiting from the sequential dependencies
  of the requirements
• Expressing the sequential dependencies a
  contract language for the requirements
• Generating test objectives ensuring test covering
  criteria

4
Outline

• A contract language for functional requirements
• Test generation from requirements with contracts
• Results
• Conclusion

5
Method
Requirements (use cases and contracts)
Interpreter
Simulator
Execution model (UCTS)
Test objective generator
Test objectives and verdicts
Test case template
Mapper
Set of test cases and their verdicts
6
Expressing use case sequential dependencies

• Adding contracts to the use cases
• Based on Catalysis and Meyers design by
  contract approach
• Constraints
• Use case contracts do not rely on the model
• but deal with the use case parameters
• Contracts must be easy to write
• Which language for contracts ?

7
Use case parameters

• Parameters Actors or Business concepts
• Used to represent the inputs of the use case
• May be reified during the design process

8
Example a virtual meeting server
VirtualMtg
enter
plan
consult
open
leave
manager
close
speak
user
hand over
moderator
connect
ask
9
Virtual Meeting Use case parameters

• Types participant, meeting
• Examples
• UC Enter (p participant, m meeting)
• UC HandOver(p1 participant, p2 participant, m
  meeting)
• NB. Participant and Meeting types will be reified
  into User class and Meeting class in the design
  process

10
Contracts logical expressions on predicates

• Predicate a name, an arity, and a semantic set
  of typed parameters
• Ex created(mmeeting)
• manager(uparticipant,mmeeting)
• First-order logic classical boolean operators
  (and, or, implies, not) quantifiers (forall,
  exists)
• Restrictions on post conditions

11
Contracts example
use case OPEN UC open(uparticipantmmtg) pre
created(m) and moderator(u,m) and not closed(m)
and not opened(m) and connected(u) post opened(m)
use case CLOSE UC close(uparticipant mmtg)
pre opened(m) and moderator(u,m) post not
opened(m) and closed(m) and forall(vparticipant)
not entered(v,m) and not asked(v,m) and not
speaker(v,m)
12
First Conclusions on Requirements by contract

• High level contracts
• Not linked with other UML views
• Easy to write
• Allows the requirements analyst to express kinds
  of dependencies
• How to generate test objectives from them ?

13
Test generation from enhanced use cases

• A structure A use case transition system
• Definition
• Building algorithm
• Coverage criteria
• 4 structural criteria
• 1 semantical criterion
• Robustness tests

14
Use Case Transition System (UCTS) definition

• Quadruple M(Q,q0,A,?)
• Q non empty set of states
• State set of instantiated predicates
• q0 initial state
• A alphabet of actions
• action instantiated use case
• ? ? Q x A x Q transition function

15
UCTS example
0 -gtmoderator(p2, m1), connected(p1),
connected(p2), created(m1), manager(p2, m1),
opened(m1) 1-gtmoderator(p2, m1), connected(p1),
connected(p2), created(m1), manager(p2, m1),
opened(m1), entered(p2, m1) 2-gtmoderator(p2,
m1), connected(p1), connected(p2), created(m1),
manager(p2, m1), opened(m1), entered(p1,
m1) 3-gtmoderator(p2, m1), connected(p1),
connected(p2), created(m1), manager(p2, m1),
closed(m1) 4-gtmoderator(p2, m1), connected(p1),
connected(p2),created(m1), manager(p2, m1),
opened(m1), entered(p2, m1), entered(p1,
m1) 5-gtmoderator(p2, m1), connected(p1),
connected(p2), created(m1), manager(p2, m1),
opened(m1), entered(p2, m1), asked(p2,
m1) 6-gtmoderator(p2, m1), connected(p1),
connected(p2), created(m1), manager(p2, m1),
opened(m1), entered(p1, m1), asked(p1, m1)
16
UCTS example
0 -gtmoderator(p2, m1), connected(p1),
connected(p2), created(m1), manager(p2, m1),
opened(m1) 1-gtmoderator(p2, m1), connected(p1),
connected(p2), created(m1), manager(p2, m1),
opened(m1), entered(p2, m1) 3-gtmoderator(p2,
m1), connected(p1), connected(p2), created(m1),
manager(p2, m1), closed(m1)
17
Building algorithm principles

• Need of the enumeration of each type
• For example, 3 participants and 2 meetings
• Instantiated use case/predicate use case/
  predicate with its operational parameters
• The algorithm tries to apply successively all the
  instantiated use cases
• Q ? 2ip, ipnumber of instantiated predicates

18
UCTS in the testing context

• Path of the UCTS correct sequence of use cases
• Find a sub-set of UCTS paths covering the
  requirements
• Necessity to find test criteria

19
Test criteria

• Structural criteria
• All edges
• All vertices
• All instantiated use cases
• All instantiated use cases and all vertices
• Semantical criterion
• All precondition terms
• Implementation breadth-first search of the UCTS

20
All precondition terms criterion

• Find all the combinations of terms that make the
  precondition true
• Examples

21
Robustness tests

• Generate paths leading to an invalid application
  of the use case
• Exercize correctly the system, then make a non
  specified action
• Criterion similar to all precondition term (all
  the terms that makes the precondition fail)

Correct path
Non specified action
22
Results virtual meeting case study
23
Experimental protocol

• Mapping from test objective to test case
• Ad hoc to the case study
• By extending the input format of the tool
• Suppression of the dead code
• Use of Jtracor tool to measure code coverage

24
Results
25
Comparaison between criteria efficiency
26
Relative efficiency of the test cases
27
Conclusion

• Test objectives generated automatically from
  enhanced use cases (for sequential systems)
• 2 relevant test criteria
• 1 relevant robustness test criterion
• Prototype tool

28
Future work

• Applying the method on other case studies
• Enhance the approach to deal with product lines
• Use scenarios to map test objectives into test
  cases
• Give test priorities to use cases, and adapt the
  criteria

29
Contracts Grammar
BOOLEXPR -gt DISJONCTIONDISJONCTION -gt
CONJONCTION (or CONJONCTION)CONJONCTION -gt
UNARYEXPR (and UNARYEXPR)UNARYEXPR -gt (
BOOLEXPR ) NEGATION FORALL IMPLIES EXISTS
PREDICATE DIFF EQUALITYPREDICATE -gt IDENT
( IDENT (, IDENT) ) IDENT() EQUALITY -gt IDENT
IDENTDIFF -gt IDENT / IDENTNEGATION -gt not
BOOLEXPRFORALL -gt forall ( LISTFORMALPARAMS )
BOOLEXPR EXISTS -gt exists ( LISTFORMALPARAMS )
BOOLEXPR IMPLIES -gt BOOLEXPR implies
BOOLEXPR
30
Building algorithm
algorithm buildUCLTS param initState STATE
useCases SETACTION var result UCLTS
to_visit STACKSTATE currentState
STATE newState STATE init
result.initialState initState
to_visit.push(initState) body while
(to_visit??) currentState to_visit.pop
? uc in useCases currentState ?uc.pre do
newState apply(currentState, uc) if
newState ? result then result.Q
result.Q ? newState to_visit.push(newSta
te) fi result. ? result. ?
?(currentState,uc,newState)
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)