Testing Distributed Software on .NET Platform

1
Testing Distributed Softwareon .NET Platform

• Victor Kuliamin
• ISP RAS, Moscow
• kuliamin_at_ispras.ru

2
Outline

• Introduction
• Some Theory
• UniTesK Method
• UniTesK Tools and Practice

3
Distributed Software Blessing and Damn

• Distributed software is a need
• it helps more people to get more services
• Distributed software is very complex
• human mind operates mostly sequentially
• errors are getting much more subtle
• .NET makes its development much easier
• making the future more splendid
• making the future more dangerous

very complex
4
How to Mitigate Risks?

• Development technologies should be augmented with
  software quality assurance
• There are a lot of ways
• Correctness by construction
• Formal proofs
• Model checking
• Search for fault patterns and vulnerabilities
• Testing
• Testing is necessary anyway

5
Outline

• Introduction
• Some Theory
• UniTesK Method
• UniTesK Tools and Practice

6
Testing Fundamentals

• How to test?
• We act upon the system under test
• We watch its reaction
• We check whether that reaction is what should be
• We repeat this until all the reasonable
  situations are exhausted

7
Testing of Distributed System

• Organize distributed action
• harder then for non-distributed system, but
  possible
• Watch distributed reaction
• harder then for non-distributed system, but
  possible
• Check whether it is expected
• how to learn what is expected?
• Ensure reasonable exhaustiveness
• what does this mean at all?

Main sources of complexity
8
How to Describe Expectations?

• The description should be
• Sufficiently expressive
• As clear as possible
• Scalable to rather complex systems
• preferably, component-wise
• Suitable to distributed systems
• include several sides
• How do people describe their expectations in
  complex cases?
• By means of contracts!

9
Contract Specifications

• Pre- and postconditions (Hoare, 1969)
• means for reasoning about program behavior
• Design by Contract (Meyer, 1992)
• pre- and postconditions are defined for
  interface operations of a component
• constraints on data integrity are stated in
  invariants
• together they form software contract between
  a component and its environment
• Insufficient for distributed systems does not
  consider asynchronous behavior
• Event contracts (7, 2005)

10
Event Contracts
Obligations of the environment
Obligations of the system
Precondition
says in what states such an event is possible
Pre-state
Input event
Pre-state
Output event
System
System
Environment
Environment
Post-state
Post-state
Postcondition
says what post-states can follow such an event in
such a pre-state
Obligations of the system
Obligations of the system
11
Relation with Classic Contracts
These two events are considered together as call
of an operation and return of result
If no actions are possible in the intermediate
states, we can exclude them from consideration
But in distributed systems such actions are
possible
12
Example

• public specification class Barrier
• int awaitedThreads 0
• int waitingThreads 0
• invariant CountersAreNonnegative() return
  awaitedThreads gt 0 waitingThreads gt 0
• public specification void Init(int n)
• post
• if(n lt 0 waitingThreads gt 0)
  branch NoChanges
• return awaitedThreads pre
  awaitedThreads waitingThreads pre
  waitingThreads
• else
  branch NewHeightSet
• return awaitedThreads n
  waitingThreads 0
• public specification void Wait()
• post
• if(awaitedThreads lt 1)
  branch Immediate
• return awaitedThreads 0
  waitingThreads pre waitingThreads
• else deferred
  branch Waiting

13
Testing Adequacy

• We can describe what we expect
• How can we define reasonable exhaustiveness?
• Possibilities
• Fault-based approaches
• Percent of faults detected by tests to all faults
  reported
• Mutants percent of mutants detected
• Source code coverage-based approaches
• Requirements coverage-based approaches
• UniTesK method requirements-based
• Try to cover structure of postconditions

14
Outline

• Introduction
• Some Theory
• UniTesK Method
• UniTesK Tools and Practice

15
Test Coverage Goals

• post
• if ( f(a, b) g(a) )
• else if( h(a, c) !g(b) )
• else

16
Construction of Abstract States
parameters
operation domain
2
3
coverage goals
1
states
17
Test Data Generation

• Computation of single call arguments

parameters
Test data generation is based on simple
generators and coverage filtering
2
3
1
states
current abstract state
current state
18
The Whole Picture
Testing Model
Behavior Model
System under Test
Coverage Model
On-the-fly Test Sequence Generation
Single Input Checking
19
Testing Distributed Software
11
21
21
s11
s12
11
Target System
r12
r11
12
s21
31
12
22
r22
r21
s31
Time

• Multisequence of stimuli is used
• Stimuli and reactions form a partially ordered set

20
Checking The Behavior

• Plain concurrency behavior of the system is
  equivalent to some sequence of the actions

Plain concurrency axiom
?
21
Outline

• Introduction
• Some Theory
• UniTesK Method
• UniTesK Tools and Practice

22
UniTesK Tools

• C / Visual Studio 6.0, gcc 2002
• Java / NetBeans 2002
• C / NetBeans MS Visual Studio
  2003specifications in Java extension
• Specialized tool for compiler testing 2003and
  complex data generation
• C / Visual Studio .NET 7.1 2003
• Java / Eclipse 2005

23
Tool Demonstration
24
Case Studies

• ISP RAS Nortel Networks 1994-1997functional
  test suite development for Switch Operating
  System kernel
• IPv6 implementations 2001-2003
• Microsoft Research
• Mobile IPv6 (in Windows CE 4.1)
• Oktet
• Intel compiler optimization units 2001-2003
• AVS-IPMP Standard 2004
• IPSec 2004-
• Pilot projects
• Enterprise application development
  framework 2003
• Components of TinyOS 2003
• Web-based banking client management system
  (Luxoft) 2004
• Components of billing system (Vympelkom) 2005
• http//www.unitesk.com

25
References

1. V. Kuliamin, A. Petrenko, I. Bourdonov, and A.
   Kossatchev. UniTesK Test Suite Architecture.
   Proc. of FME 2002. LNCS 2391, pp. 77-88,
   Springer-Verlag, 2002.
2. V. Kuliamin, A. Petrenko, N. Pakoulin, I.
   Bourdonov, and A. Kossatchev. Integration of
   Functional and Timed Testing of Real-time and
   Concurrent Systems. Proc. of PSI 2003. LNCS 2890,
   pp. 450-461, Springer-Verlag, 2003.
3. V. Kuliamin, A. Petrenko. Applying Model Based
   Testing in Different Contexts. Proceedings of
   seminar on Perspectives of Model Based Testing,
   Dagstuhl, Germany, September 2004.
4. A. Kossatchev, A. Petrenko, S. Zelenov, S.
   Zelenova. Using Model-Based Approach for
   Automated Testing of Optimizing Compilers. Proc.
   Intl. Workshop on Program Undestanding,
   Gorno-Altaisk, 2003.
5. V. Kuliamin, A. Petrenko, A. Kossatchev, and I.
   Burdonov. The UniTesK Approach to Designing Test
   Suites. Programming and Computer Software, Vol.
   29, No. 6 , 2003, pp. 310-322. (Translation from
   Russian)
6. S. Zelenov, S. Zelenova, A. Kossatchev, A.
   Petrenko. Test Generation for Compilers and Other
   Formal Text Processors. Programming and Computer
   Software, Vol. 29, No. 2 , 2003, pp. 104-111.
   (Translation from Russian)
7. V. Kuliamin, N. Pakoulin, A. Petrenko. Extended
   Design-by-Contract Approach to Specification and
   Conformance Testing of Distributed Software.
   Proc. of 9-th World Multi-Conference on
   Systemics, Cybernetics, and Informatics, Model
   Based Testing Session, July 2005, to be published.

26
Contacts

• RedVerst group web page http//www.ispras.ru/grou
  ps/rv/rv.html
• UniTesK projects web site http//www.unitesk.com
• Group leaderAlexander Petrenkopetrenko_at_ispras.ru
  

27
Thank You!