Coveragedriven Automatic Test Generation for UML Activity Diagrams

1
Coverage-driven Automatic Test Generation for
UML Activity Diagrams
Dhrubajyoti Kalita Intel Corporation Folsom, CA,
USA

• Mingsong Chen, Prabhat Mishra
• CISE Department
• University of Florida, USA

2
Motivation
Hardware Components

• Advantages of UML
• Intuitive
• Friendly
• Semi-formal
• Tool support

Hardware
Design (Synthesis, Layout, )
HW/SW Partitioning
Specification

• UML Validation
• Major Design Bottleneck
• Error prone
• Time consuming

Design (Compilation, )
Estimation - Exploration
Software Components
Software
Automated Techniques
3
Related Work

• There are two kinds of methods to validate the
  UML activity diagrams.
• Model checking based method.
• Consistency checking (Eshuis, TOSEM 2006).
• Timing verification (Guelfi et al., ASPEC 2005
  Das et al., ASPEC 2006).
• Model driven testing.
• Gray-Box method (Wang et al., ASPEC 2004)
• Random method (Chen et al., TCJ 2007)
• Currently, there are no approaches that can
  automatically generate the directed tests from
  the UML activity diagrams.

4
The Framework of Our Approach
Testing criteria
UML activity diagram Parser and Analyzer
translate
extract
Coverage data
Formal Model (NuSMV input)
Fault model generator
Model checker
transform
Test cases (Counter example)
Properties (CTL, LTL formulas)
5
An Example of ATM
start
t1
Verify access code
a

• 1. Activity
• e.g. Dispense_cash
• 2 Transition
• e.g. t7 with guard amount available
• 3. Key path
• e.g. key path 2
• start -gt a-gtc -gt
• d,f-gte,f -gt g -gt
  end

b
incorrect
Handle incorrect access code
t2
correct
t3
resolved
Ask for amount
c
t4
t6
amount not available
amount available
t7
not resolved
Dispense cash
Prepare to print receipt
d
f
t9
t5
Generate receipt content
t8
e
t10
Finish transaction and print receipt
g
t11
end
6
Extraction and Translation

• Activity Diagram Extraction
• Prepare necessary information for translation.
• Collect all the graphic elements.
• Parse the relation between the graphic elements.
• Analysis the data in activities and conditions.
• Translation to NUSMV
• Specify transitions for each element.
• Activity has 3 states 0-unvisited, 1-visiting
  and 2-visited.
• Transition has 2 states 0-unfired, 1-fired.
• Testing oriented translation rules.
• We define 7 rules to describe various cases.

7
Skeleton of Translated ATM Model
MODULE state VAR access_code
A1, B1, C1 access_code_input A1,
B1, C1, D1 ...... start
0..2 syn_1 0..2
verify_access_code 0..2 ......
t2_cond 0..1 t3_cond 0..1
...... ASSIGN
init(start)1 init(syn_1)0
init(verify_access_code)0
...... MODULE ATM(st) ASSIGN
next(st.start) next(st.t2_cond)
...... MODULE prepare_print(st)
ASSIGN next(st.prepare_print_receipt)
...... MODULE check_amount(st)
ASSIGN next(st.dispense_cash)
next(st.t7_cond)
...... MODULE main VAR st
state atm ATM(st) print process
prepare_print(st) check process
check_amount(st)
8
Fault Models and Examples

• Activity fault model. For each activity of AD,
  the model assumes that such activity is not
  reachable.
• Can be used to check the reachability of each
  activity.
• e.g. The activity dispense_cash can not be
  visited.
• CTL formula ! EF (
  st.dispense_cash 2 )
• Transition fault model. For each transition of
  AD, the model assumes that such transition can
  not be fired.
• Can be used to check the execution order of the
  activities.
• It also can be used to check whether the
  condition guard is satisfied.
• e.g. The transition with condition amount
  available can not be fired.
• CTL formula !EF( t71
  st.amount_input lt st.amount_available )
• Key path fault model. For each key path of AD,
  there is no corresponding executable path.
• Can be used to check the dynamic behavior of the
  system.
• e.g. Key path 2 can not be covered.
• CTL formula ! EF ( st.start 2
  st.verify_access_code2 st.ask_for_amount
  2 st.prepare_print_receipt 2
  st.dispense_cash 2 st.generate_receipt_conte
  nt 2 st.finish_transction_print_receipt 2
  st.end 2 st.t3_cond1 st.t7_cond1)

9
Case Study A Control System

• The first case is a small control system. The
  activity diagram consists of 17 activities, 23
  transitions and 6 key paths.
• Specification Coverage.
• Implementation Coverage.
• This case study shows an order of magnitude
  improvement compared with the random method. The
  generated tests have a good functional coverage
  in the implementation.

10
Case Study On-line Stock Exchange System

• The second case is a on-line stock exchange
  system. The activity diagram consists of 27
  activities, 29 transitions and 18 key paths. The
  implementation consists of 7 packages, 39
  classes, 372 methods and 2510 lines.
• Specification Coverage.
• Implementation Coverage.
• Our approach reduce the validation effort by 4
  times compared to the random method.

11
Conclusions

• Automatic test generation and design validation
  from UML activity diagram is promising .
• Time efficient .
• Guarantee the correctness of the specification.
• Test reusing
• Consistency checking between different
  abstraction levels.
• Good implementation level coverage.
• Three contributions of our work
• Propose a set of testing oriented translation
  rules.
• Derive the properties from the specification
  based on fault model.
• Develop a model checking based test generation
  methodology.
• Future work
• Design and property decomposition.
• Property compaction.

12
References

• OMG. UML2.0 Superstructure Specification.
  Available at http//www.omg.org/UML2.0, October
  2004.
• Grant Martin. UML for Embedded Systems
  Specification and Design Motivation and
  Overview. DATE, 2002.
• Rik Eshuis. Symbolic model checking of UML
  activity diagrams. ACM Transactions on Software
  Engineering and Methodology , 15(1), 2006.
• Dipankar Das, Rajeev Kumar and P. P. Chakrabarti.
  Timing Verification of UML Activity Diagram Based
  Code Block Level Models for Real Time
  Multiprocessor System-on-Chip Applications.
  APSEC, 2006.
• N. Guelfi and A. Mammar. A Formal Semantics of
  Timed Activity Diagrams and its PROMELA
  Translation. APSEC, 2005.
• L. Wang, J. Yuan, X. Yu, J. Hu, X. Li, and G.
  Zheng. Generating Test Cases from UML Activity
  Diagram based on Gray-Box Method. APSEC, 2004.
• M. Chen, X. Qiu, W. Xu, L. Wang, J. Zhao and X.
  Li. UML Activity Diagram Based Automatic Test
  Case Generation for Java Programs. The Computer
  Journal, 2007.
• A. Cimatti, E. M. Clarke, F. Giunchiglia, M.
  Roveri. NUSMV A New Symbolic Model Verifier.
  CAV, 1999.