White Box Testing and Symbolic Execution

1
White Box Testing and Symbolic Execution

• Written by Michael Beder

2
Agenda

• What is White Box Testing?
• Flow Graph and Coverage Types
• Symbolic Execution
• Formal Definition
• Examples
• Questions

3
What is White Box Testing?

• Software testing approach that uses inner
  structural and logical properties of the program
  for verification and deriving test data
• Also called Clear Box Testing, Glass Box Testing
  and Structural Testing
• Manual Inspection, Walkthrough
• Automatic Syntax Parser, Symbolic Execution

4
Pros and Cons

• Pros
• Usage of more information on the tested object
  (than BlackBox)
• Inference of real Equivalence Partitioning
• Structural Coverage Assurance
• Cons
• Expensive
• Limited Semantic Coverage

5
Example I

• Sin(x)
• if (x lt eps) return x
• if (x lt 5eps) return x (x3)/6
• Black Box Testing Test 0, Pi/2, -Pi/2
• White Box Testing Test 0.5eps, eps, 3eps,
  5eps, 7eps,

Usage of logical properties makes better coverage

6
Example II Unit Testing

• f(x) g(x) h(x)
• if (x gt 0) return g(x)
• return h(x)
• Black Box Testing Test f(x), g(x), h(x) for
  every x
• White Box Testing Test g(x) for x gt 0, h(x) for
  x lt 0 and verify f(x)

Usage of structural properties makes fewer,
qualitative tests
7
Flow Graph

• Abstraction of the program
• Defines the data and control flow in the program
• Uniform representation of the program, language
  independent
• Simple basic elements assignment and condition
• Further analysis is performed using graph
  algorithms

8
Flow Graph cont.

• G (V, E) where
• - V is a set of basic blocks. start, end in V
• - E is a set of control branches
• Example
• 1 a Read(b)
• 2 c 0
• 3 while (a gt 1)
• 4 If (a2 gt c)
• 5 c c a
• 6 a a - 2

F
T
Input b 2
F
T
Output a 0, c 2
9
Basic Path Set

• Let p1, p2 be paths from start to end. Then, p1 lt
  p2 if there exists a vertex v that belongs to p2
  and not to p1
• A basic path set is a maximal set of paths p1,
  p2, , pk such that pi lt pj for i lt j

10
White Box Coverage Types

• Statement Coverage Every statement is executed
• Branch Coverage Every branch option is chosen
• Path Coverage Every path is executed
• Basic Path Coverage Every basic path is executed

Loops?
11
Basic Path Coverage

• Basic path set is of size E N 2 (Linear
  Complexity)
• Each path is called basic path
• Example
• p1 start 1,2 3 end
• p2 start 1,2 3 4 6 3 end
• p3 start 1,2 3 4 5 6 3 end
• E N 2 8 7 2 3

F
T
F
T
12
Path Function

• A function , when D is
  the working domain
• Represents the current values of the variables as
  function of their initial values
• Each variable X is represented by a projection
  function
• Function composition
• For example

13
Path Condition
For Example p start 1,2 3
end. 1 a Read(b) 2 c 0 3 while (a gt 1)
4 If (a2 gt c) 5 c c a 6 a a - 2 The
path condition is B lt 1, when B is bs value at
start

• A condition that should be fulfilled for going
  along the path
• A constraint on the initial values of the
  variables

14
Symbolic Execution

• A method for deriving test cases which satisfy a
  given path
• Performed as a simulation of the computation on
  the path
• Initial path function Identity function,
  Initial path condition true
• Each vertex on the path induce a symbolic
  composition on the path function and a logical
  constraint on the path condition
• If an assignment was
  made
• If a conditional decision was made
• path condition path condition
  branch condition
• Output path function and path condition for the
  given path

15
Example Symbolic Composition

• x x y
• y y x
• end
• The final path function represents the values of
  X, Y, Z after both assignments as a function of
  their initial value

16
Concatenation and Associativity

• If is the path function of path
  and is the path function of
• path then
  is the path function of path
• The composition is associative

17
Example Symbolic Execution

• 1 a Read(b)
• 2 c 0
• 3 while (a gt 1)
• 4 If (a2 gt c)
• 5 c c a
• 6 a a - 2
• Find test case for path
• p start 1,2 3 4 5 6 3 4 5 6
  3 end

input b output c
18
Example Symbolic Execution

• 1 a Read(b)
• 2 c 0
• 3 while (a gt 1)
• 4 If (a2 gt c)
• 5 c c a
• 6 a a - 2
• p start 1,2 3 4 5 6 3 4 5 6
  3 end
• vertex path function path condition
• start (A, B, C) true
• 1,2 (A, B, C) true
• 3 (B, B, 0) true
• 4 (B, B, 0) (true ? Bgt1) ? Bgt1
• 5 (B, B, 0) (Bgt1 ? B2gt0) ? Bgt1

input b output c
F
T
F
T
19
Example Symbolic Execution

• 1 a Read(b)
• 2 c 0
• 3 while (a gt 1)
• 4 If (a2 gt c)
• 5 c c a
• 6 a a 2
• p start 1,2 3 4 5 6 3 4 5 6
  3 end
• vertex path function path condition
• 6 (B, B, B) Bgt1
• 3 (B-2, B, B) Bgt1
• 4 (B-2, B, B) (Bgt1 ? B-2gt1) ? Bgt3
• 5 (B-2, B, B) (Bgt3 ? (B-2)2gtB) ? Bgt4
• 6 (B-2, B, 2B-2) Bgt4
• 3 (B-4, B, 2B-2) Bgt4
• end (B-4, B, 2B-2) (Bgt4 ? B-4lt1) ? B5

input b output c
F
T
F
T
20
Example Symbolic Execution

• 1 a Read(b)
• 2 c 0
• 3 while (a gt 1)
• 4 if (a2 gt c)
• 5 c c a
• 6 a a 2
• p start 1,2 3 4 5 6 3 4 5 6
  3 end
• end (B-4, B, 2B-2) B5
• Hence the test case is B 5 and the expected
  result is
• 2B-2 8.

input b output c
Is there a test case for p start 1,2 3 4
5 6 3 4 5 6 3 4 5 6 3 end
?
21
Question (from exam)

• 1 d b c
• 2 if (d gt 20)
• 3 a 3 a d
• if (b lt a)
• 5 a 1
• 6 if (d lt 2 b)
• 7 b 2
• 8

• Draw programs Flow Graph
• Find minimal number of test cases for the
  following coverage types
• Statement Coverage
• Path Coverage
• Branch Coverage
• Basic Path Coverage

22
White Box Testing vs. Black Box Testing

• Given a function f(X1, X2, , X10) with the
  following preconditions
• Every parameter is odd
• Every parameter is less or equal to M
• Some parameter is equal to M
• The function should report about every
  precondition that is not fulfilled
• f examines each parameter in turn using if
  statements (without else)
• and handles differently the following cases
• Exactly one parameter is higher than M
• Two or more parameters are higher than M
• Check fs correctness using White/Black Box
  Testing methods