CS 240 Section 2

1
CS 240 Section 2

• Computer Science I
• Dr. Dale E. Nelson

2
Chapter 8 Testing and Debugging
3
Chapter Goals

• To learn how to design test harnesses for testing
  components of your programs in isolation
• To understand the principles of test case
  selection and evaluation
• To be able to use assertions to document program
  assumptions
• To become familiar with the debugger
• To learn strategies for effective debugging

4
Unit Tests

• In a unit test, a function or procedure is
  compiled outside the program in which it will be
  used, together with a test harness that feeds
  arguments to it.
• Once you have confidence that the function is
  working correctly, you can plug it into your
  program.
• Test data can be entered manually (see
  sqrtest1.cpp), in a loop (see sqrtest2.cpp), or
  using random number generation (see sqrtest3.cpp).

5
Unit Tests (sqrtest1.cpp)
include include using
namespace std /---------------------------------
------------------------- Tests whether two
floating-point numbers are approximately
equal. ///////////////////////////////////////////
///////////////// bool approx_equal(double x,
double y) const double EPSILON 1E-14 if
(x 0) return fabs(y) 0) return fabs(x) y) / max(fabs(x), fabs(y)) --------------------------------------------------
Function to be tested Computes the square
root using Heron's formula ///////////////////////
////////////////////////////////////// double
squareroot(double a) if (a 0) return 0
double xnew a, xold do xold xnew
xnew (xold a / xold) / 2 while
(!approx_equal(xnew, xold)) return
xnew /-----------------------------------------
--------------------/ / Test harness / int
main() double x while (cin x)
double y squareroot(x) cout "squareroot of " return 0
6
Unit Tests (sqrtest2.cpp)
include include using
namespace std /---------------------------------
--------------------------- Tests whether two
floating-point numbers are approximately
equal. ///////////////////////////////////////////
/////////////////// bool approx_equal(double x,
double y) const double EPSILON 1E-14 if
(x 0) return fabs(y) 0) return fabs(x) y) / max(fabs(x), fabs(y)) --------------------------------------------------
--- Function to be tested Computes the square
root using Heron's formula ///////////////////////
/////////////////////////////////////// double
squareroot(double a) if (a 0) return 0
double xnew a double xold do
xold xnew xnew (xold a / xold) / 2
while (!approx_equal(xnew, xold)) return
xnew /-----------------------------------------
----------------------- Test harness / int
main() double x for (x 0 x 0.5) double y squareroot(x) cout
return 0
7
Unit Tests (sqrtest3.cpp)
include include include
include using namespace
std /-------------------------------------------
------------------ Sets the seed of the random
number generator. /------------------------------
------------------------------ Compute a
random floating point number in a
range /------------------------------------------
-------------------- Tests whether two
floating-point numbers are approximately
equal. /-----------------------------------------
-------------------------- Function to be tested
Computes the square root using Heron's
formula //////////////////////////////////////////
//////////////////////////// double
squareroot(double a) if (a 0) return 0
double xnew a, xold do xold
xnew xnew (xold a / xold) / 2
while (!approx_equal(xnew, xold)) return
xnew /-----------------------------------------
---------------------------/ int main()
rand_seed() int i for (i 1 i i) double x rand_double(0, 1E6)
double y squareroot(x) cout "squareroot of " return 0
8
Selecting Test Cases

• Positive tests consist of legitimate inputs that
  you expect the program handles correctly.
• Boundary cases are also legitimate inputs that
  you expect the program to handle in a trivial
  way.
• Negative cases are inputs that you expect the
  program to fail on.
• Keeping test data in a file is smart because you
  can use it to review every version of the
  program.
• When you find a bug, you should record the data
  that caused the bug, and retest your program with
  that data after you have tried to fix the bug.
• The phenomenon of fixing a bug, only to have it
  reappear after further program modifications is
  called cycling.
• The process of testing against a set of past
  failures is called regression testing.

9
Test Case Evaluations

• Once you have determined what inputs are needed
  to test the program, you need to decide if the
  outputs are correct.
• Sometimes you can verify output by calculating
  the correct values by hand.
• Sometimes a computation does a lot of work, and
  it is not practical to do the computation
  manually.
• double squareroot(double a)
• if (a 0) return 0
• double xnew a
• double xold
• do
• xold xnew
• xnew (xold a / xold) / 2
• while (not approx_equal(xnew, xold))
• return xnew
• We could write a test program that verifies that
  the output values fulfill certain properties.
• Here we test the squareroot() function by
  comparing the square of the returned number with
  the original input. (See sqrtest4.cpp).
• Another method is to use a slower but reliable
  procedure (called an oracle) to verify the
  results. (In this case, we will use pow() as an
  oracle). (See sqrtest5.cpp)

10
Test Case Evaluations (sqrtest4.cpp)
include include include
include using namespace
std /-------------------------------------------
-------------------------/ / Sets the seed of
the random number generator.
/ /---------------------------------------------
-----------------------/ / Compute a random
floating point number in a range
/ /---------------------------------------------
-----------------------/ / Tests whether two
floating-point numbers are approximately
equal./ /---------------------------------------
-----------------------------/ / Function to be
tested Computes the square root using Heron's
formula/ double squareroot(double a) if (a
0) return 0 double xnew a, xold do
xold xnew xnew (xold a / xold)
/ 2 while (!approx_equal(xnew, xold))
return xnew / Test harness / int main()
int i for (i 1 i x rand_double(0, 1E6), y squareroot(x)
if (not_equal(y y, x)) cout " else cout return 0
11
Test Case Evaluations (sqrtest5.cpp)
01 include 02 include
03 include 04 include
06 using namespace std 08
/------------------------------------------------
------------------/ 09 // Sets the seed of
the random number generator. 10
/------------------------------------------------
------------------/ 11 // Compute a random
floating point number in a range 19
/------------------------------------------------
------------------/ 30 //Tests whether two
floating-point numbers are approximately
equal. 35 /-------------------------------------
-----------------------------/ 44 // Function
to be tested Computes the square root using
Heron's formula 50 /----------------------------
--------------------------------------/ 51
double squareroot(double a) 52 if (a 0)
return 0 55 double xnew a, xold 58
do 59 xold xnew 61 xnew (xold
a / xold) / 2 63 while (!approx_equal(xnew
, xold)) 65 return xnew 68 / Test
harness / 70 int main() 71 rand_seed() 73
int i 74 for (i 1 i double x rand_double(0, 1E6), y
squareroot(x) 78 if (!approx_equal(y,
pow(x, 0.5))) cout else cout "squareroot of " return 0
12
Assertions

• Functions often contain implicit assumptions
  (denominators should be nonzero, salaries should
  not be negative).
• Such illegal values can creep into a program due
  to an input or processing error - with
  distressing regularity!
• Assertions provide a valuable sanity check.
• void raise_salary(Employee e, double by)
• assert(e.get_salary() 0 )
• assert(by -100)
• double new_salary e.get_salary() (1
  by / 100)
• e.set_salary(new_salary)
• If an assertion is not satisfied, the program
  terminates with a useful error message showing
  the line number and the code of the failed
  assertion
• assertion failed in file finclac.cpp line 61 by
  -100

13
Program Traces

• To get a printout of the program flow, you can
  insert trace messages into the beginning and end
  of every procedure.
• It is also useful to print the input parameters
  when a procedure is entered and to print return
  values when a function is exited.
• string int_name(int n)
• cout "\n"
• ...
• cout "
• return s
• To get a proper trace, you must locate each
  function exit point.
• Sample output for n 12305.
• Inside int_name. Thousands.
• Entering int_name. n 12
• Inside int_name. Teens.
• Entering teen_name. n 12
• Exiting teen_name. Return value twelve
• Exiting digit_name. Return value twelve
• Inside int_name. Hundreds.
• Entering digit_name. n 3

14
Program Traces (Problems with Trace Messages)

• Program traces can be time-consuming to find out
  which trace message to insert.
• If you insert too many messages, you produce a
  flurry of output that is hard to analyze.
• If you insert to few, you many not have enough
  information to spot the cause of the error.
• When you are done with the program, you need to
  remove all the trace messages.
• When you find a new error, you need to stick the
  print statement back in.
• Many profession programmers use a debugger, not
  trace messages, to locate errors in their code.

15
The Debugger

• Modern development environments contain special
  programs, called debuggers, that help you locate
  bugs by letting you follow the execution of a
  program.
• You can stop and restart your program and see the
  contents of variables whenever your program is
  temporarily stopped.
• Using a debugger is not cost free - it takes time
  to set up and carry out an effective debugging
  session.
• Larger programs are harder to debug without using
  a debugger, so learn how to use one is time well
  spent.

16
The Debugger

• Debuggers vary wildly from one system to another.
  
• If you use an integrated development environment,
  which contains an editor, compiler, and a
  debugger, finding and starting the debugger is
  usually very easy.
• On a UNIX system, you must manually build a debug
  version of your program and invoke the debugger.
• You can go a long way with just three commands
• Run until this line.
• Step to next line.
• Inspect Variable.
• The exact names for these commands will vary from
  debugger to debugger, but all debuggers support
  them.

17
The Debugger

• The "run until this line" command is the most
  important.
• Select a line with the mouse or cursor, then hit
  a key or select a menu command to run the program
  to the select line.
• It's possible that the program terminates
  normally without reaching that line, but that
  could be informative.

18
The Debugger

• The "step to next line" command executes the
  current line and stops at the next program line.
• Once the program has stopped, you can look at the
  current values of variables.
• Some debuggers require you select the variable
  the choose a command such as "inspect variable."
• Some debuggers require you to type the name of
  the variable into a dialogue box.
• Some variables automatically show the values of
  all variables.

19
The Debugger

• Many debuggers also have "step over" and "step
  into" commands.
• The "step over" command advances to the next
  program line
• Example If the current line is
• r future_value(balance, p n)
• cout
• "step over" advances the program to
• r future_value(balance, p n)
• cout
• The "step into" command advances into function
  calls.
• Example if the current line is
• r future_value(balance, p n)
• cout
• "step into" advances the program to
• double future_value(double initial_balance,
  double p, int n)
• double b initial_balance pow((1 p
  /100), n)
• return b

20
The Debugger

• You should not "step into" system functions like
  setw.
• Some debuggers have a "run to end of function" or
  "step out" command to get out of a function.
• Most debuggers can show you a call stack a
  listing of all currently pending function calls.
• My selecting the function in the middle of the
  call stack, you can jump to the code line
  containing that function call.

21
The Debugger

• All debuggers support a navigational approach by
  inserting breakpoints in the code.
• When the program reaches any breakpoint,
  execution stops.
• Breakpoints are particularly useful when you know
  at which point your program starts doing the
  wrong thing.
• Some debuggers let you set conditional
  breakpoints - the program stops only when a
  certain condition, such as n 0, is met.

22
The Debugger

• When inspecting an object variable, all the
  fields are displayed.

• With some variables you must "open up" the
  object, usually by clicking on a tree node. Here,
  we have opened up the name field of the harry
  object..

23
Strategies

• Reproduce the error.
• What numbers did you enter?
• Where did you click the mouse?
• Run the same program again type in exactly the
  same input.
• Divide and Conquer
• Step over features using debugger, but don't step
  into them.
• Locate the last procedure called before failure.
• Step into that procedure, and repeat.

24
Strategies

• Know what your program should do.
• Does your program do what's expected?
• Before you inspect a variable, decide what value
  is should have.
• If value is correct, look elsewhere for the bug.
• If value is incorrect, double check your
  computation, then try to find out why your
  program comes up with a different value.
• Look for off by one errors.
• Look for computation errors (is the formula typed
  in correctly).
• Make "sign calculations" when the numbers get
  nasty. ("Should this number be positive or
  negative?")

25
Strategies

• Avoid quick fixes - they tend to create problems
  elsewhere.
• Develop a thorough understanding of the problem
  and the program before you try to fix it.
• Sometimes fix and fix only causes the problem to
  move around. Possibly the program logic is
  incorrect (does the program need redesign?).

26
Debugger Limitations

• Break points in recursive procedures interrupt
  the program each time through the procedure.
• When using the debugger on a recursive procedure,
  watch the call stack carefully.
• Sometimes the compiler generates faster code by
  keeping a variable in a processor register rather
  than reserving a memory location for it.
• The debugger cannot find that variable or
  displays a wrong value for it.
• You can try turning off all compiler
  optimizations and recompile.
• You can open a special register window that shows
  all processor registers (advanced).
• Some errors show up when you run the program
  normally, but go away under the debugger.
• Usually an uninitialized variable is the cause.
• Inspect all variables manually and check that
  they are initialized.
• Insert print statements if you are desperate.