Tool Support for Testing Concurrent Java Components

1
Tool Support for Testing Concurrent Java
Components

• Paul Strooper
• pstroop_at_itee.uq.edu.au
• School of IT EE
• (Roger Duke, Dan Hoffman, Brad Long, Luke Wildman)

The University of Queensland
2
Overview

• Issues in testing concurrent Java components
• producer-consumer example
• ConAn testing tool
• generates Java test driver from test script
• progression of threads in test sequences
  controlled by internal clock
• Conclusions

3
Unit and System Testing

• Unit testing
• test unit in isolation using stubs and drivers
• Integration testing
• test combinations of unit or subsystems
• System testing
• test entire system
• Difficult to test unit in its production
  environment
• due to poor controllability and observability
• tool support exists for sequential unit testing
• for example, JUnit and Roast for Java
• Will not discuss integration/system testing issues

4
Java Threads and Monitors

• A concurrent Java program has multiple threads of
  control
• threads are created and deleted dynamically
• A Java monitor encapsulates data and access
  procedures
• synchronized methods provide mutually exclusive
  access
• condition synchronization allow threads to
  suspend to wait for certain conditions
• wait suspends a thread
• notify/notifyAll wake up one/all suspended
  thread(s)

5
Example Bounded Buffer

• Class Buffer
• public synchronized void put(Object o)
• while (/ buffer is full /) wait()
• / add item to the buffer /
• notifyAll()
• public synchronized Object get()
• ...

6
Testing Concurrent Components

• A component is a unit of composition with
  contractually specified interfaces and explicit
  context dependencies (Szyperski,1998)
• typically one or more Java classes (monitors)
• assume it will be accessed by multiple threads
• Need a driver to execute/test the component
• how many threads?
• how do we control the interaction between these
  threads and the component under test?
• Example driver
• 5 producer and 5 consumer threads access a shared
  buffer that can hold up to 3 items
• each thread produces/consumes 10 items

7
Testing Issues

• Concurrency leads to many interleavings and
  non-determinism
• test cases may have to be run multiple times
• traditional notions of test coverage may not be
  sufficient
• automated checking of outputs may be difficult
• Specific properties/problems of concurrent
  software
• safety issues interference, deadlock/livelock
• liveness issues starvation

8
Testing Java Monitors

• Extend Brinch-Hansens method for testing
  Concurrent Pascal monitors
• 1. Identify set of preconditions that exercise
  each monitor method in desired way.
• 2. Construct a set of test sequences of monitor
  calls to satisfy the test conditions identified
  in step 1.
• 3. Construct a test driver that starts a number
  of threads that call the component in the order
  prescribed in step 2.
• 4. Execute test program and compare output to
  expected output.
• Steps 3 and 4 have been automated with ConAn

9
1. Identify Test Conditions

• put
• C1 0 iterations of the loop
• C2 1 iteration of the loop
• C3 many iterations of the loop
• get
• C4 0 iterations of the loop
• C5 1 iteration of the loop
• C6 many iterations of the loop
• threads suspended on queue
• C7 no threads suspended
• C8 1 producer suspended
• C9 many producers suspended
• C10 1 consumer suspended
• C11 many consumers suspended
• C12 1 producer and 1 consumer suspended
• C13 many producers and consumers suspended

public synchronized void put(Object o)
while (/ buffer full /) wait() /
add item to buffer / notifyAll()
10
2. Construct Test Sequences
11
3. Implement Sequences in Driver
begin test C4 C5 C10 tick // Time 1
thread ltproducer1gt
excMonitor m.put(a) end
valueCheck time() 1 end end end
tick // Time 2 thread ltconsumer1gt
valueCheck m.get() a end
valueCheck time() 2 end end
end tick // Time 3 thread
ltconsumer2gt valueCheck m.get()
b end valueCheck time() 4
end end end tick // Time 4
thread ltproducer2gt
excMonitor m.put(b) end
valueCheck time() 4 end end end end
12
4. Execute Driver
TEST DRIVER GENERATION
ConAn Test Script
Roast
ConAn
Test Driver
TEST DRIVER EXECUTION
Test cases 84 Value errors 0
Exception errors 0 Liveness errors 0
13
ConAn Features Limitations

• ConAn features
• reduces testing of concurrent components to
  something familiar
• allows for testing of non-deterministic output
• detects liveness errors
• Limitations
• tester must define test conditions and test
  sequences
• difficult to detect problems with interference
• can control some non-determinism, but not all
• no control over order in which JVM grants locks
• no control over order in which JVM removes
  threads from wait set

14
Concluding Remarks

• Summary
• issues in testing concurrent components
• method and tool support for testing Java monitors
• TestCon group
• http//www.itee.uq.edu.au/testcon/
• tools, publications, staff, etc.
• testing interrupts and timed waits
• model-based testing of concurrent Java components
• method for verifying concurrent Java components

15
ASWEC 2005

• Australian Software Engineering Conference
• 29 March 1 April 2005
• Carleton Crest, Brisbane
• http//aswec2005.itee.uq.edu.au
• Research Papers
• abstracts due 1 Oct
• full papers due 15 Oct
• Industry Experience Reports
• extended abstracts due 4 February