Coverage Based Testing for Concurrent Software

1
Coverage Based Testing for Concurrent Software

• Report Wei-Yu Chen

2
Reference

• Juichi Takahashi, Hideharu Kojima and Zengo
  Furukawa
• The 28th International Conference on Distributed
  Computing Systems Workshops, 2008. ICDCS08.
  pages 533 538.

3
Outline

• Introduction
• Modeling
• Concurrent module flow graph and coverage
  criteria
• Case study
• Conclusion

4
Introduction

• Concurrent programming is playing key roles for
  improving software performance since multi-core
  processes are used on various computing system.
• In this paper, focus software testing method that
  detects concurrent software specific defects by
  using modeling and its coverage method.
• Model concurrent software by enhancing ordinary
  modeling method, and define blocks which are
  subset of concurrent software.

5
Introduction Concurrent defects pattern

• Before modeling for concurrent software, consider
  the implication of typical race condition
  defects.
• A race condition shall be defined as
• A race condition occurs when more than two
  programs access a variable simultaneously, and
  more than one access is write.
• In the case, there are two types of confliction
  happened.
• Read-write conflict
• Write-write conflict
• Concurrent software is supposed to executed
• based on happens-before relation.
• Definition of happens-before
• If A and B events in the same process, and A
  occurs before B, then we can state that A
  happen-before B is true.

6
Introduction Local happen-before defect

• A number of recent programming languages support
  lock and unlock mechanisms in order to protect
  unexpected memory access.
• In terms that the mutex usage are correct,
  testers are expected to test the happen-before
  relation.
• In this case the state aa5 is never accessed
  by program 2.

7
Introduction Program logic defect

• Happen-before can be tested by a testing method
  which makes sure sequence of happens.
• But, in case that two and more modules are
  complicatedly access shared resources, then
  encounter difficulties when testing it, such as
  race condition.
• Likely case below is inappropriate usage of mutex
  and is called program logic defect.

8
Modeling

• To accomplish solving those difficulties, this
  propose a modeling methodology which is block
  based.
• The block based modeling aims for measuring
  concurrent coverage rate same as code coverage
  for simultaneous software.
• First, define blocks.
• Second, model the blocks.
• Third, measure concurrent coverage rate by using
  the model.

9
Blocks

• Atomic Block (AB)
• The atomic block is abstracted block through
  combining the blocks.
• The atomic block is meaningful block that is
  either local block or external operation block.
• Local Block (LB)
• When there are not shard resources and data, the
  block is called LB.

10
Blocks

• External Operation Block (EOB)
• When there is shared resource and data within
  blocks, it is called EOB.
• EOB would be divided two types.
• Mask-able EOB
• From just starting the mutex command and just
  after finishing release state.
• Non Mask-able EOB

11
Concurrent module flow graph and coverage criteria

• Define the Concurrent Module Flow Graph (CMFG),
  and it expresses as
• CMFG can be more complicated than general flow
  graph since concurrent modules are executed
  parallelize.

12
CMFG and coverage criteria All concurrent
path (ACP)

• When testing non-concurrent software, there has
  been a lot of research to define coverage
  criteria.
• For the non-concurrent testing path, Rapps and
  Weyuker defined the all-path criterion, such
  that
• When define concurrent software for covering
  blocks as all-path, then define all-concurrent
  path (ACP).

13
All concurrent path (ACP) - the number of test
cases

• When there are n threads, and each thread are
  defined as
• And the number of blocks within each thread is
  defined as thn. The number of cumulative
  blocks between th1 and thn is defined as Sk. And
  total test cases TC is.

The number of test cases for all-blocks-permutatio
n.
14
CMFG and coverage criteria All concurrent
binominal path (ACBP)

• Because the number of test cases can be exploded
  when using the coverage criteria of ACP, so this
  paper propose another criterion in order to
  reduce the test case.
• ACBP only verify two unique block path.
• Definition

15
All concurrent binominal path (ACBP)

• Three type of test cases which are aiming for
• th1, th2 and th3.
• Each test case would be
• expth21 is the number of th2s test case.
• expth32 is the number of th3s test case.
• expother3 is the number of th1s test case.
• The number of test cases of from th3 is
  shall be
• The number of test cases of from th1. total
  test cases
• Total number of test case for th2 is
• When calculating same method, have concluded a
  whole test cases is

16
Case study - A simple race condition

• Testing the sample code by ACP, there are test
  cases, such as

17
Case study - A simple race condition

• All test case are executed, summarize the number
  of blocks and test cases.

18
Conclusion

• In this paper, have succeeded to propose
  efficient coverage criteria for concurrent
  software.
• Describe that typical concurrent defect patters
  that are local happens-before and program logic
  defect.
• Proposed a way of calculating number of required
  test cases for concurrent software.