Reachability Testing of Concurrent Programs1

1
Reachability Testing of Concurrent Programs

• Richard Carver, GMU
• Yu Lei, UTA

2
Outline

• Introduction
• The Reachability Testing (RT) Process
• Computing Race Variants
• An Example Scenario
• Conclusion and Future work

3
A concurrent program ...

• Contains a set of cooperative threads/processes
• multithreaded programs - threads synchronize by
  accessing shared memory
• distributed programs threads synchronize by
  exchanging messages
• Is pervasive in modern software development
• better resource utilization
• increases computing efficiency
• provides simple solutions to certain problems
• Is notoriously difficult to test due to their
  non-deterministic behavior

4
Testing Strategies

• Non-deterministic testing - execute the same
  program with the same input for multiple times
• easy, but inefficient, and some errors cannot be
  detected
• Deterministic testing - select a set of test
  sequences and then force them to be exercised
• can detect more subtle errors, but requires
  additional effort for test sequence selection and
  runtime control
• Prefix-based testing a combination of
  non-deterministic and deterministic testing
• a test run executes deterministically at the
  beginning and then proceeds non-deterministically

5
Reachability testing

• ... is based on prefix-based testing, and has the
  following main features
• Dynamic generation of test sequences - avoids
  constructing any static model
• Dealing with partial orders directly avoids
  generating redundant interleavings
• Complete coverage on synchronization behavior
  has important applications on program
  verification
• Can be implemented in a platform-independent
  manner

6
Program Execution Model

• In our paper, a general representation of
  concurrent executions is provided, which allows
  RT to be applied to different types of programs,
  including semaphore-based, monitor-based, and
  message-passing programs.

7
Outline

• Introduction
• The RT Process
• Computing Race Variants
• An Example Scenario
• Conclusion and Future work

8
SYN-sequence

• Records the relative ordering of events that
  occur on a synchronization object
• e.g., P/V events on a semaphore, entry/reentry
  events on a monitor, lock/unlock events on a
  lock, send/receive events on a channel
• Used to characterize the synchronization behavior
  of an execution
• the result of an execution is determined by the
  program text, the input data, and the
  SYN-sequence exercised by the execution

9
Concurrent Path
Port M T1 T2 T3 (1) M.send(10) (2) M.send(4) (3)
x M.receive () (4) y M.receive () (5) z
x - y
Assume that an execution of the above program
exercises the sequence of statements (1, 2, 3,
4, 5). Can we determine the value of z?
10
Space-Time Diagram
T1
T2
T3
s1
s2
r1
r2
z -6
z 6
11
The RT Framework

• Execute a program P with input I
  non-deterministically to collect a SYN-sequence Q
• Compute the race variants of Q, each of which is
  a prefix of one or more other SYN-sequences
• For each variant, conduct a prefix-based test run
  to collect a new SYN-sequence Q
• Repeat 2, 3, and 4 for each newly collected
  SYN-sequence Q.

12
Tracing and Replay

• Additional runtime control is needed to conduct a
  prefix-based test run
• Synchronization library with semaphores,
  monitors, and message passing
• Library has built-in tracing and replay
  capabilities
• No modifications to the thread scheduler

13
Outline

• Introduction
• The RT Process
• Computing Race Variants
• An Example Scenario
• Conclusion and Future work

14
Race Condition

• Refers to the phenomenon where a receiving event
  can be synchronized with two or more sending
  events (in different executions)
• e.g., the messages sent by two send events can be
  received by the same receive event
• The race set of a receive event r consists of all
  the send events that r could possibly be
  synchronized with (in different executions).

15
Example
Note that s1 is not in the race set of r2,
because r2 may not even exist if r1 receives a
message other than s1.
16
Identifying Races

• Assign each event a vector timestamp to indicate
  the happens-before relation
• Whether there is a race between two events can be
  determined by comparing their timestamps
• Two timestamp assignment schemes described in
  paper
• Thread-centric keeps a logic clock for each
  thread
• Object-centric keeps a version number for each
  synchronization object

17
Race Variant

• Represents the beginning portion of one or more
  SYN-sequences that could have happened but didnt
• Derived from a SYN-sequence by changing one or
  more race outcomes in the sequence
• If we change a race outcome, then all the events
  that could potentially be affected by the new
  outcome need to be removed

18
Race Table

• Consists of one column for each receiving event
  with a non-empty race set
• Each row can be used to derive a unique,
  partially-ordered race variant of Q.

• Value v indicates how receiving event r in Q is
  changed to create variant V
• v -1 r is removed from V
• v 0 the send partner of r is left unchanged
• in V and
• v gt 0 in V, the send partner of r is changed
• to the v-th event in race_set(r, Q),

19
Example
20
Outline

• Introduction
• The RT Process
• Computing Race Variants
• An Example Scenario
• Conclusion and Future work

21
An Example Scenario
22
Outline

• Introduction
• The RT Process
• Computing Race Variants
• An Example Scenario
• Conclusion and Future Work

23
Conclusion

• RT generates test sequences dynamically, which
  is more scalable than many model-based
  approaches. Our latest tool has almost constant
  memory requirements.
• Compared to JPF and VeriSoft, which try to
  reduce the chances of generating redundant
  interleavings, RT deals with partial orders
  directly.
• While most tools for concurrent programs testing
  rely on access to the thread scheduler and are
  thus platform dependent, our RT tool is platform
  independent.

24
Future Work

• Test oracle automatically evaluate test runs
  against properties specified in some logic
  formalism
• Coverage-based RT - use coverage measures to
  selectively exercise a subset of the
  SYN-sequences
• Model extraction use RT to extract a complete
  model of the synchronization/communication
  behavior of a program
• Real-time testing add explicit time into the RT
  framework