CUTE: A Concolic Unit Testing Engine for C

1
CUTE A Concolic Unit Testing Engine for C

• Koushik Sen
• University of Illinois Urbana-Champaign

Joint work with Darko Marinov and Gul Agha
2
Outline

• Motivation
• Introduce CUTE through an Example
• CUTE in a nutshell
• Experimental Results
• Advantages of CUTE over existing approaches
• Internal details
• Conclusion

3
Software Testing

• Testing accounts for 50 of software development
  cost
• Software failures cost USA 60 billion/year
• Improvement in software testing infrastructure
  can save one-third of this cost
• The economic impacts of inadequate
  infrastructure for software testing, NIST, May
  2002
• Currently, software test generation is mostly
  manual automation can lower the costs

4
CUTE

• Extends the previous work DART Directed
  Automated Random Testing by Patrice Godefroid,
  Nils Klarlund, and Koushik Sen (PLDI05)
• Supports C with
• pointers, data-structures
• Highly efficient constraint solver
• arithmetic, pointers
• Provides Bounded Depth-First Search and Random
  Search strategies
• Tool support

5
Example

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

• Random Test Driver
• random memory graph reachable from p
• random value for x
• Probability of reaching abort() is extremely
  low

6
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

7
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

8
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

x0gt0
9
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

x0gt0
!(p0!NULL)
10
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

solve x0gt0 and p0?NULL
x0gt0
p0NULL
11
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

solve x0gt0 and p0?NULL x0236, p0
NULL
x0gt0
p0NULL
12
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

13
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

x0gt0
14
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

x0gt0
p0?NULL
15
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

x0gt0
p0?NULL
2x01?v0
16
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

x0gt0
p0?NULL
2x01?v0
17
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

solve x0gt0 and p0?NULL and 2x01v0
x0gt0
p0?NULL
2x01?v0
18
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

solve x0gt0 and p0?NULL and 2x01v0 x01, p0
NULL
x0gt0
p0?NULL
2x01?v0
19
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

20
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

x0gt0
21
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

x0gt0
p0?NULL
22
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

x0gt0
p0?NULL
2x01v0
23
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

x0gt0
p0?NULL
2x01v0
n0?p0
24
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

x0gt0
p0?NULL
2x01v0
n0?p0
25
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

solve x0gt0 and p0?NULL and 2x01v0 and
n0p0 .
x0gt0
p0?NULL
2x01v0
n0?p0
26
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

solve x0gt0 and p0?NULL and 2x01v0 and
n0p0 x01, p0
x0gt0
p0?NULL
2x01v0
n0?p0
27
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

p , x1
pp0, xx0,p-gtv v0, p-gtnextn0
28
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

x0gt0
29
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

x0gt0
p0?NULL
30
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

x0gt0
p0?NULL
2x01v0
31
CUTE Approach
Concrete Execution
Symbolic Execution

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

Program Error
x0gt0
p0?NULL
2x01v0
n0p0
32
Pointer Inputs Input Graph

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

33
Pointer Inputs Input Graph

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

34
CUTE Approach

• typedef struct cell
• int v
• struct cell next
• cell
• int f(int v)
• return 2v 1
• int testme(cell p, int x)
• if (x gt 0)
• if (p ! NULL)
• if (f(x) p-gtv)
• if (p-gtnext p)
• abort()
• return 0

p!NULL
N
Y
Error
35
Explicit Path (not State) Model Checking

• See all executions of a program on all possible
  inputs as a binary tree
• Internal nodes are branching points
• edges are then or else branches
• possibly infinite

36
Explicit Path (not State) Model Checking

• See all executions of a program on all possible
  inputs as a binary tree
• Internal nodes are branching points
• edges are then or else branches
• possibly infinite
• Traverse all execution paths one by one to detect
  errors
• check for assertion violations
• check for program crash
• combine with valgrind to discover memory leaks
• detect invariants

37
Explicit Path (not State) Model Checking

• See all executions of a program on all possible
  inputs as a binary tree
• Internal nodes are branching points
• edges are then or else branches
• possibly infinite
• Traverse all execution paths one by one to detect
  errors
• check for assertion violations
• check for program crash
• combine with valgrind to discover memory leaks
• detect invariants

38
Explicit Path (not State) Model Checking

• See all executions of a program on all possible
  inputs as a binary tree
• Internal nodes are branching points
• edges are then or else branches
• possibly infinite
• Traverse all execution paths one by one to detect
  errors
• check for assertion violations
• check for program crash
• combine with valgrind to discover memory leaks
• detect invariants

39
Explicit Path (not State) Model Checking

• See all executions of a program on all possible
  inputs as a binary tree
• Internal nodes are branching points
• edges are then or else branches
• possibly infinite
• Traverse all execution paths one by one to detect
  errors
• check for assertion violations
• check for program crash
• combine with valgrind to discover memory leaks
• detect invariants

40
Explicit Path (not State) Model Checking

• See all executions of a program on all possible
  inputs as a binary tree
• Internal nodes are branching points
• edges are then or else branches
• possibly infinite
• Traverse all execution paths one by one to detect
  errors
• check for assertion violations
• check for program crash
• combine with valgrind to discover memory leaks
• detect invariants

41
Explicit Path (not State) Model Checking

• See all executions of a program on all possible
  inputs as a binary tree
• Internal nodes are branching points
• edges are then or else branches
• possibly infinite
• Traverse all execution paths one by one to detect
  errors
• check for assertion violations
• check for program crash
• combine with valgrind to discover memory leaks
• detect invariants

42
Explicit Path (not State) Model Checking

• See all executions of a program on all possible
  inputs as a binary tree
• Internal nodes are branching points
• edges are then or else branches
• possibly infinite
• Traverse all execution paths one by one to detect
  errors
• check for assertion violations
• check for program crash
• combine with valgrind to discover memory leaks
• detect invariants

43
CUTE in a Nutshell

• Generate concrete inputs one by one
• each input leads program along a different path

44
CUTE in a Nutshell

• Generate concrete inputs one by one
• each input leads program along a different path
• On each input execute program both concretely and
  symbolically

45
CUTE in a Nutshell

• Generate concrete inputs one by one
• each input leads program along a different path
• On each input execute program both concretely and
  symbolically
• Both cooperate with each other
• concrete execution guides the symbolic execution

46
CUTE in a Nutshell

• Generate concrete inputs one by one
• each input leads program along a different path
• On each input execute program both concretely and
  symbolically
• Both cooperate with each other
• concrete execution guides the symbolic execution
• concrete execution enables symbolic execution to
  overcome incompleteness of theorem prover
• replace symbolic expressions by concrete values
  if symbolic expressions become complex
• resolve aliases for pointer using concrete values
• handle arrays naturally

47
CUTE in a Nutshell

• Generate concrete inputs one by one
• each input leads program along a different path
• On each input execute program both concretely and
  symbolically
• Both cooperate with each other
• concrete execution guides the symbolic execution
• concrete execution enables symbolic execution to
  overcome incompleteness of theorem prover
• replace symbolic expressions by concrete values
  if symbolic expressions become complex
• resolve aliases for pointer using concrete values
• handle arrays naturally
• symbolic execution helps to generate concrete
  input for next execution
• increases coverage

48
Experiments Needham-Schroeder Protocol

• Tested a C implementation of a security protocol
  (Needham-Schroeder) with a known attack
• 600 lines of code
• Took less than 13 seconds on a machine with
  1.8GHz Xeon processor and 2 GB RAM to discover
  middle-man attack
• In contrast, a software model-checker (VeriSoft)
  took 8 hours
• DART took 5 hours

49
Testing Data-structures of CUTE itself

• Unit tested several non-standard data-structures
  implemented for the CUTE tool
• cu_depend (used to determine dependency during
  constraint solving using graph algorithm)
• cu_linear (linear symbolic expressions)
• cu_pointer (pointer symbolic expressions)
• Discovered a few memory leaks and a couple of
  segmentation faults
• these errors did not show up in other uses of
  CUTE
• for memory leaks we used CUTE in conjunction with
  Valgrind

50
More on Error found in CUTE

• typedef struct
• int count
• int symid
• int coeff
• int constant
• culinear
• 2x34x7 5 will be
• represented as
• count 2
• symid 3,7
• coeff2,4
• constant -5

• cu_linear
• cu_linear_add(cu_linear c1, cu_linear c2, int
  add)
• int i,j,k,flag
• cu_linear ret (cu_linear )malloc(sizeof(cu_l
  inea))
• // skip 18 lines of code
• if(cu-gtcount0) return NULL
• .

51
More on Error found in CUTE

• typedef struct
• int count
• int symid
• int coeff
• int constant
• culinear
• 2x34x7 5 will be
• represented as
• count 2
• symid 3,7
• coeff2,4
• constant -5

• cu_linear
• cu_linear_add(cu_linear c1, cu_linear c2, int
  add)
• int i,j,k,flag
• cu_linear ret (cu_linear )malloc(sizeof(cu_l
  inea))
• // skip 18 lines of code
• if(cu-gtcount0) return NULL
• .
• CUTE constructed the following sequence
• l1 cu_linear_create(0)
• l2 cu_linear_create(0)
• l1 cu_linear_negate(l1)
• l1cu_linear_add(l1,l2,1)

Memory Leak
52
SGLIB popular library for C data-structures

• Used in Xrefactory a commercial tool for
  refactoring C/C programs
• Found two bugs in sglib 1.0.1
• reported them to authors
• fixed in sglib 1.0.2
• Bug 1
• doubly-linked list library
• segmentation fault occurs when a non-zero length
  list is concatenated with a zero-length list
• discovered in 140 iterations ( lt 1second)
• Bug 2
• hash-table
• an infinite loop in hash table is member function
• 193 iterations (1 second)

53
Statistics of testing SGLIB 1.0.1
54
Simultaneous Symbolic Concrete Execution

• void again_test_me(int x,int y)
• z xxx 3xx 9
• if(z ! y)
• printf(Good branch)
• else
• printf(Bad branch)
• abort()

• Let initially x -3 and y 7 generated by
  random test-driver

55
Simultaneous Symbolic Concrete Execution

• void again_test_me(int x,int y)
• z xxx 3xx 9
• if(z ! y)
• printf(Good branch)
• else
• printf(Bad branch)
• abort()

• Let initially x -3 and y 7 generated by
  random test-driver
• concrete z 9
• symbolic z xxx 3xx9
• take then branch with constraint xxx 3xx9
  ! y

56
Simultaneous Symbolic Concrete Execution

• void again_test_me(int x,int y)
• z xxx 3xx 9
• if(z ! y)
• printf(Good branch)
• else
• printf(Bad branch)
• abort()

• Let initially x -3 and y 7 generated by
  random test-driver
• concrete z 9
• symbolic z xxx 3xx9
• take then branch with constraint xxx 3xx9
  ! y
• solve xxx 3xx9 ! y to take else branch
• Dont know how to solve !!
• Stuck ?

57
Simultaneous Symbolic Concrete Execution

• void again_test_me(int x,int y)
• z xxx 3xx 9
• if(z ! y)
• printf(Good branch)
• else
• printf(Bad branch)
• abort()

• Let initially x -3 and y 7 generated by
  random test-driver
• concrete z 9
• symbolic z xxx 3xx9
• take then branch with constraint xxx 3xx9
  ! y
• solve xxx 3xx9 ! y to take else branch
• Dont know how to solve !!
• Stuck ?
• NO CUTE handles this smartly

58
Simultaneous Symbolic Concrete Execution

• void again_test_me(int x,int y)
• z xxx 3xx 9
• if(z ! y)
• printf(Good branch)
• else
• printf(Bad branch)
• abort()

• Let initially x -3 and y 7 generated by
  random test-driver

59
Simultaneous Symbolic Concrete Execution

• void again_test_me(int x,int y)
• z xxx 3xx 9
• if(z ! y)
• printf(Good branch)
• else
• printf(Bad branch)
• abort()

• Let initially x -3 and y 7 generated by
  random test-driver
• concrete z 9
• symbolic z xxx 3xx9
• cannot handle symbolic value of z

60
Simultaneous Symbolic Concrete Execution

• void again_test_me(int x,int y)
• z xxx 3xx 9
• if(z ! y)
• printf(Good branch)
• else
• printf(Bad branch)
• abort()

• Let initially x -3 and y 7 generated by
  random test-driver
• concrete z 9
• symbolic z xxx 3xx9
• cannot handle symbolic value of z
• make symbolic z 9 and proceed

61
Simultaneous Symbolic Concrete Execution

• void again_test_me(int x,int y)
• z xxx 3xx 9
• if(z ! y)
• printf(Good branch)
• else
• printf(Bad branch)
• abort()

• Let initially x -3 and y 7 generated by
  random test-driver
• concrete z 9
• symbolic z xxx 3xx9
• cannot handle symbolic value of z
• make symbolic z 9 and proceed
• take then branch with constraint 9 ! y

62
Simultaneous Symbolic Concrete Execution

• void again_test_me(int x,int y)
• z xxx 3xx 9
• if(z ! y)
• printf(Good branch)
• else
• printf(Bad branch)
• abort()

• Let initially x -3 and y 7 generated by
  random test-driver
• concrete z 9
• symbolic z xxx 3xx9
• cannot handle symbolic value of z
• make symbolic z 9 and proceed
• take then branch with constraint 9 ! y
• solve 9 y to take else branch
• execute next run with x -3 and y 9
• got error (reaches abort)

63
Simultaneous Symbolic Concrete Execution

• void again_test_me(int x,int y)
• z xxx 3xx 9
• if(z ! y)
• printf(Good branch)
• else
• printf(Bad branch)
• abort()

• Let initially x -3 and y 7 generated by
  random test-driver
• concrete z 9
• symbolic z xxx 3xx9
• cannot handle symbolic value of z
• make symbolic z 9 and proceed
• take then branch with constraint 9 ! y
• solve 9 y to take else branch
• execute next run with x -3 and y 9
• got error (reaches abort)

Replace symbolic expression by concrete value
when symbolic expression becomes unmanageable
(i.e. non-linear)
64
Simultaneous Symbolic Concrete Execution

• void again_test_me(int x,int y)
• z xxx 3xx 9
• if(z ! y)
• printf(Good branch)
• else
• printf(Bad branch)
• abort()

• void again_test_me(int x,int y)
• z black_box_fun(x)
• if(z ! y)
• printf(Good branch)
• else
• printf(Bad branch)
• abort()

65
Handling Pointer Casting and Arithmetic

• struct foo
• int i
• char c
• void memset(void s,char c,size_t n)
• for(int i0iltni)
• ((char )s)i c
• return s
• bar(struct foo a)
• if(a a-gtc1)
• memset(a,0,sizeof(struct foo))
• if(a-gtc! 1)
• ERROR

• Reasoning about dynamic data is easy
• Due to limitation of alias analysis static
  analyzers cannot determine that a-gtc has
  been rewritten
• BLAST would infer that the program is safe
• practical
• CUTE finds the error

66
Library Functions With Side-effects

• struct foo
• int i
• char c
• bar(struct foo a)
• if(a a-gtc1)
• memset(a,0,sizeof(struct foo))
• if(a-gtc 1)
• ERROR

• Reasoning about function with side-effects
• Sound static tool will give false alarm
• CUTE will not report the error
• because it cannot generate a concrete input that
  will make the program hit ERROR.

67
Underlying Random Testing Helps

• 1 foobar(int x, int y)
• 2 if (xxx gt 0)
• 3 if (xgt0 y10)
• 4 ERROR
• 5
• 6 else
• 7 if (xgt0 y20)
• 8 ERROR
• 9
• 10
• 11

• static analysis based model-checkers would
  consider both branches
• both ERROR statements are reachable
• false alarm
• Symbolic execution
• gets stuck at line number 2
• or warn that both ERRORs are reachable
• CUTE finds the only error

68
Instrumentation
69
Instrumented Program

• Maintains a symbolic state
• maps each memory location used by the program to
  symbolic expressions over symbolic input
  variables
• Maintains a path constraint
• a sequence of constraints C1, C2,, Ck
• each Ci is a symbolic constraint over symbolic
  input variables
• C1 Æ C2 Æ Æ Ck holds over the path currently
  executed by the program

70
Instrumented Program

• Maintains a symbolic state
• maps each memory location used by the program to
  symbolic expressions over symbolic input
  variables
• Maintains a path constraint
• a sequence of constraints C1, C2,, Ck
• such that Ci s are symbolic constraints over
  symbolic input variables
• C1 Æ C2 Æ Æ Ck holds over the path currently
  executed by the program

• Arithmetic Expressions
• a1x1anxnc
• Pointer expressions
• x

71
Instrumented Program

• Maintains a symbolic state
• maps each memory location used by the program to
  symbolic expressions over symbolic input
  variables
• Maintains a path constraint
• a sequence of constraints C1, C2,, Ck
• such that Ci s are symbolic constraints over
  symbolic input variables
• C1 Æ C2 Æ Æ Ck holds over the path currently
  executed by the program

• Arithmetic Expressions
• a1x1anxnc
• Pointer expressions
• x
• Arithmetic Constraints
• a1x1anxnc 0
• Pointer Constraints
• x y
• x ? y
• x NULL
• x ? NULL

72
Constraint Solving

• Path constraint
• C1Æ C2Æ Æ Ck Æ Cn

73
Constraint Solving

• Path constraint
• C1Æ C2Æ Æ Ck Æ Cn
• Solve
• C1Æ C2Æ Æ Ck
• to generate next input

74
Constraint Solving

• Path constraint
• C1Æ C2Æ Æ Ck Æ Cn
• Solve
• C1Æ C2Æ Æ Ck
• to generate next input
• If Ck is pointer constraint then invoke pointer
  solver
• Ci1 Æ Ci2 Æ Æ Ck where ij 2 1..k-1
  and Cij is pointer constraint
• If Ck is arithmetic constraint then invoke linear
  solver
• Ci1Æ Ci2 Æ Æ Ck where ij 2 1..k-1
  and Cij is arithmetic constraint

75
Optimizations

• Fast un-satisfiability check
• C1Æ C2Æ Æ Ck

76
Optimizations

• Fast un-satisfiability check
• C1Æ C2Æ Æ Ck
• Eliminate same constraints
• (ygt2) Æ (ygt2)Æ (y4)
• (ygt2)Æ (y4)
• that is
• C1Æ C2ÆCi Cj Æ Ck C1Æ C2Æ Ci Æ Ck
  if Ci Cj

77
Optimizations

• Fast un-satisfiability check
• C1Æ C2Æ Æ Ck
• Eliminate same constraints
• (ygt2) Æ (ygt2)Æ (y4)
• (ygt2)Æ (y4)
• that is
• C1Æ C2ÆCi Cj Æ Ck C1Æ C2Æ Ci Æ Ck
  if Ci Cj
• Eliminate non-dependent constraints
• (x1) Æ (ygt2)Æ (y4)
• (ygt2)Æ (y4)

78
Data-structure Testing

• Solving Data-structure Invariants
• int isSortedSlist(slist head)
• slist cur, tmp
• int i,j
• if (head 0) return 1
• ij0
• for (cur head cur!0 cur cur-gtnext)
• i
• j1
• for (tmp head jlti tmp tmp-gtnext)
• j
• if(curtmp) return 0
• for (cur head cur-gtnext!0 cur
  cur-gtnext)
• if(cur-gti gt cur-gt_next-gti) return 0
• return 1

• testme(slist L,slist e)
• CUTE_assume(isSortedSlist(L))
• sglib_slist_add(L,e)
• CUTE_assert(isSortedSlist(L))

79
Data-structure Testing

• Generating Call Sequence
• for (i1 ilt10 i)
• CU_input(toss)
• CU_input(e)
• switch(toss)
• case 2 sglib_hashed_ilist_add_if_not_member(
  htab,e,m) break
• case 3 sglib_hashed_ilist_delete_if_member(
  htab,e,m) break
• case 4 sglib_hashed_ilist_delete(htab,e)
  break
• case 5 sglib_hashed_ilist_is_member(htab,e)
  break
• case 6 sglib_hashed_ilist_find_member(h
  tab,e) break

80
Discussion

• In comparison to existing testing tools, CUTE is
• light-weight
• dynamic analysis (compare with static analysis)
• ensures no false alarms
• concrete execution and symbolic execution run
  simultaneously
• symbolic execution consults concrete execution
  whenever dynamic analysis becomes intractable
• real tool that works on real C programs
• completely automatic
• Software model-checkers using abstraction (SLAM,
  BLAST)
• starts with an abstraction with more behaviors
  gradually refines
• Time consuming in comparison to static analysis
• Requires actual code that can be fully compiled
• Can sometime reduce to Random Testing
• Complementary to Static Analysis Tools

81
Current Work

• Use static analysis to find branches that can
  lead to assertion violation
• use this info to prune search space
• Concurrency Support
• dynamic pruning to avoid exploring equivalent
  interleaving
• Application to find Dolev-Yao attacks in security
  protocols
• Combine with BLAST model-checker