Evolutionary Testing Using an Extended Chaining Approach

1
Evolutionary Testing Using an Extended Chaining
Approach

• For paper presentation
• By Jared DeMott
• Harvest Day 2006

2
My Background

• I find bugs in software for a living
• So Im into testing (and more)
• Testing is an NP-Hard problem
• Youll here more testing during my project preso
• Wed like to create a tool that can
  auto-magically find bugs wonder if EC could
  help us? ?

3
White box testing using Evolutionary Testing

• Current method for deriving fitness
• approach_level norm(branch distance)
• Example a10, b20, c30, d40
• Answer fitness 2 norm(10). Zero weve
  found test data.
• (s) void example(int a, int b, int c, int d)
• (1) if (a gt b)
• (2) if (b lt c)
• (3) if (c d)
• //target

4
Problem 1(First to guess what it is gets a candy)

• (s) void flag_example(int a, int b)
• (1) int flag 0
• (2) if (a 0)
• (3) flag 1
• (4) if (b ! 0)
• (5) flag 0
• (6) if (flag)
• (7) //target
• (e)

Figure 1
5
Problem 2(test your wit/preparation again)

• (s) double function_under_test (double x)
• (1) if (inverse(x) 0 )
• (2) //target
• (e)
• double inverse (double d)
• (3) if (d 0)
• (4) return 0
• else
• (5) return 1 / d

6
Chaining Saves the Day!

• The problem node on Figure 1 is node 6.
• The constraint or (branch predicate at node 6) is
  flag.
• Find the definition-clear (flag isnt modified)
  paths from flag to node 6.
• The following paths or event sequences are
  created
• E1 lt (s, 0), (1, flag), (6, 0), (7, 0) gt
• E2 lt (s, 0), (3, flag), (6, 0), (7, 0) gt
• E3 lt (s, 0), (5, flag), (6, 0), (7, 0) gt

7
Summing is fun for you and me

• To make use of this new information, sum the
  fitness's at each node to get a new overall
  fitness.
• Notice we use same fitness function, so were
  unlikely to any worse
• They also take the chaining technique another
  step forward by developing an algorithm to
  include influencing nodes (thus the term extended
  chaining).
• Deals with intermediate steps that affect the
  constraint.
• Ill spare you the pain

8
Experimental Results

• 7 toy problems
• Designed to showcase the weakness of current
  technology
• 1 real problem
• Drawn from open source file format code (libpng)
• The results are impressive and prove their point

9
Customary Slide
10
My Random Thoughts

• Good paper
• Sorry the scan stunk
• Impressive results but how does this get applied?
• Coverage doesnt security
• It needs to be covered right
• Source isnt usually available to me, how about
  grey boxing? ?