ASTA Aspect Software Testing Assistant

1
ASTAAspect Software Testing Assistant

• Juha Gustafsson,
• Juha Taina,
• Jukka Viljamaa
• University of Helsinki

2
MotivationCross-Cutting Concerns in Software
Implementation modules
3
(No Transcript)
4
An ExampleMoney Transfer Logical Level
void transfer(Account from, Account to, int
amount) if (from.getBalance() lt amount)
throw new InsufficientFundsException()
from.withdraw(amount) to.deposit(amount)
5
An ExampleMoney Transfer Implementation Details
void transfer(Account from, Account to, int
amount) if (!getCurrentUser().canPerform(OP_TR
ANSFER)) throw new SecurityException()
Transaction tx database.newTransaction()
if (from.getBalance() lt amount)
tx.rollback() throw new InsufficientFundsExce
ption() from.withdraw(amount)
to.deposit(amount) systemLog.logOperation(OP
_TRANSFER, from, to, amount) tx.commit()
6
SolutionAspect-Oriented Programming (AOP)

• Separating cross-cutting concerns from thecore
  concerns (i.e. from the main class model)
• Aspect a definition of a cross-cutting concern
• pointcut a pattern that selects join
  points(i.e. elements that appear in method
  bodies)
• advice a method-like construct that is
  executedin the selected join points (before,
  after, around)
• weaving the process of attaching advice to the
  appropriate join points

7
An example AOP programHelloWorld in the AspectJ
language

• Go to http//www.eclipse.org/ajdt/
• and watch the demo!

8
A new question arisesHow to test AOP?

• Do we test what we intend to test?
• How to prioritise our tests?
• How to pick a good testing strategy?
• When should we stop testing?
• In AOP these are especially difficult questions
  because
• the execution context of an aspect depends on the
  (internal implementation of the) class it is
  woven to
• relating a failure to the fault that causes it is
  difficult
• the control and data flows are not explicit
• a particular aspect weave order can cause problems

9
A new question arisesHow to test AOP?

• Visualising AOP testing is not straightforward
• what kinds of test coverage criteria to use and
  visualise (is it sufficient to show that we have
  visited an aspect once, how to show the weaving
  context)?
• current control flow graphs are insufficient to
  visualise the test execution paths in the
  presence of aspects
• high-level coverage visualisation for AOP testing
  is a completely open question
• Yet, visualisation of AOP testing is (at least)
  as important as visualisation of traditional (OO)
  testing

10
Testing AOPFault Sources

• Fault in the core concern code
• Fault in the (context independent) aspect code
• Fault created by interactions between the aspect
  and the core concern code
• Fault created when more than one aspect is woven
  into the core concern code

11
Testing AOP Alexander, Bieman,
Andrews, 2004 Fault Model Additional Testing
Required

• Incorrect strength in pointcut patterns
• test the aspect
• Incorrect aspect precedence
• test all weave orders
• Incorrect focus of control flow
• use a form of condition coverage of pointcut
  designators
• Failure to establish expected postconditions
• use the original test set to re-test all methods
  that have the aspect
• Failure to preserve state invariants
• same as in previous
• Incorrect changes in control dependencies
• same as in previous

12
AOP Testing Theory

• Coverage criteria
• Coupling metrics
• Control flow formalism

13
Tool Support

• Tool support is essential for systematic testing
• Not many tools exist for AOP testing
• coverage tools
• visualisation tools
• scripting languages environments to automate
  AOP testing

14
Research Background

• Due to project RITA (fRamework Intgration and
  Testing Application) we have a strong background
  in white-box test execution and analysis
• RITA is intended for product family testing, but
  similar ideas and issues arise in AOP testing
• RITA as a prototype is not suitable for
  large-scale testing but its ideas are relevant

15
RITA Functionality

• Rita supports the following
• Java program testing
• Product family or single application testing
• JUnit and private test execution
• Full support for data flow diagrams
• Visualisation and white-box coverages on four
  levels
• Method level traditional coverages
• Object level object and class coverages
• Package level object coupling, reference
  coverages
• Application level multiple application coverages

16
Lessons learned from RITA

• A White-Box Testing Software (WBTS) must support
  traditional code
• RITA not all applications are in product
  families
• ASPECTS not all applications use aspects
• WBTS should be integrated to a software
  development environment
• While stand-alone software can have similar
  functionality than integrated software, usability
  issues favor the latter approach
• Code coverage is not sufficient in WBTS
• The higher is the abstraction of testing, the
  more complex coverages are needed.
• Even at code level code coverage gives weak
  results.

17
Aspect-Oriented Testing Tool

• Based on our experiences in RITA we propose the
  following requirements for an Aspect-Oriented
  Testing Tool (AOTT)
• It has to support testing of traditional software
• It must be integrated to a well-known and
  actively updated programming environment such as
  Eclipse
• It has to support several levels of white-box
  testing including program-level, package-level,
  object-level, aspect-level, and method-level
• Aspect-level testing and visualisation are
  inside a single aspect.

18
AOTT Requirements Continue

• More AOTT requirements
• It has to calculate static mesurements of the
  software
• Lines of code (LOC)
• Aspects of code (AOC)
• Coupling between objects
• Depth of inheritance tree
• Depth of aspect tree (aspects in aspects)
• Coupling between code and aspects etc.
• It has to support integration between dynamic
  test execution results and static measurements
• How? This is an open research question
• Something else? Definitely! This is a
  work-in-progress list

19
Research directions Alexander, Bieman,
Andrews, 2004 Open Questions

• Can we measure test coverage after weaving?
• also visualisation
• How do we test aspects that interact with the
  core concern code?
• coupling between an aspect and core concern code
  as a measure of testability of the aspect
• How do we test aspects that interfere with each
  other?
• inter-aspect coupling
• How do we test aspects whose effects must span
  more than one concern?
• Are there ways to test aspects on their own?
• probably not ?
• (Can we reverse engineer the weave process?)
• how does this relate to testing

20
Validation

• How to validate that
• Theory is correct?
• Application is effecient?
• Alternatives
• Formalism
• Empirical controlled experiments
• Practical experiments in industrial settings
• All alternatives are
• Relevant
• Needed.

21
Summary

• To be written...