CS 141a Distributed Computation LaboratorySoftware Testing - PowerPoint PPT Presentation
1 / 27
Title:
CS 141a Distributed Computation LaboratorySoftware Testing
Description:
CS 141a - Distributed Computation Laboratory Software Testing ... Integration testing attempts to locate errors that result from interactions ... – PowerPoint PPT presentation
Number of Views:40
Avg rating:3.0/5.0
Title: CS 141a Distributed Computation LaboratorySoftware Testing
1
Recap
- Design by Contract
2
Today
- Software Testing
3
Software Testing
- Testing is an important part of the software
engineering process, because there are almost
always differences between what is specified and
what is initially implemented. - Misunderstanding of specifications.
- Incorrect choices of algorithms or data
structures. - Pathological cases that arent considered.
- Typographical errors that happen to compile.
4
Types of Software Testing
- There are various types of software testing
- Unit testing attempts to locate errors in
isolated pieces of a software system. - Integration testing attempts to locate errors
that result from interactions between pieces of a
software system. - Interactive testing attempts to locate problems
that arise when humans must interact with a
software system - such as flaws in its human
interface, or in the assumptions made about
users knowledge or experience. - Well mainly concern ourselves with the first
two, because our user interfaces consist
primarily of command line parameters.
5
Unit Testing
- A unit test is a test that attempts to validate
one particular piece of a software system. - The size of the piece being tested can vary
- Method
- Class
- Package
- A set of unit tests is called a test suite.
- There are two main forms of unit test, whitebox
and blackbox.
6
Unit TestingWhitebox
- Whitebox tests access the internal structure of
the component being tested - Assertion checking at runtime is a common form of
whitebox testing. - A test method on a class that accesses the
classs private fields or methods is also a
whitebox test. - Whitebox tests are almost always incorporated
into the component being tested. - This is necessary in most object-oriented systems
because of information hiding.
7
Unit TestingBlackbox
- Blackbox tests use only the public interfaces of
the components being tested. - A separate class that exercises one or more other
classes would be an example. - Blackbox tests are typically outside the
component (class, package) being tested. - Ensures that they dont accidentally use inside
information that they shouldnt use. - Allows the component to be built without the test
being included, if necessary.
8
Unit Testing
- The world of blackbox and whitebox testing isnt
entirely black and white - some situations
require hybrid, or greybox, tests that observe
the internal workings of a component. - Most systems require both blackbox and whitebox
testing - neither one alone is enough. - Testing isnt a substitute for good design and
programming methods.
9
Unit TestingResponsibility
- The people who design and build software systems
are not necessarily the people who write the test
suites for those systems. - Whitebox tests are almost always written by the
same people as the component being tested,
because theyre internal to the component or rely
on internal information. - Blackbox tests are sometimes written by the same
people, but are often written by special test
designers whose job it is to try to find things
that the software designers and builders didnt
account for.
10
Unit Test Coverage
- What should be covered in unit tests?
- Whitebox tests should primarily cover consistency
conditions (preconditions, postconditions,
invariants). - Blackbox tests should exercise every feature of
every class in a system. - This may involve invoking multiple features as
part of a single test, to set up an environment
or establish some particular condition.
11
Unit TestingExample
- Assume we have a Point class, with the following
methods - x, y, rho, theta - accessors for the coordinates
of the point. - distance - takes a point as a parameter and
calculates the distance between the two points. - translate, scale, rotate - change the position of
the point. - Constructors for polar and Cartesian coordinates.
- Other standard methods (equals, hashCode, clone).
- Assume further that a reasonable set of whitebox
tests are included in the class as assertions. - What blackbox tests can we do to satisfy
ourselves that all these methods work properly?
12
Unit Testing Example
- Constructors, equals, hashCode, x, y, rho, theta
- Create a few points and verify that they get
created correctly, by examining x, y, rho, theta. - Ensure that points created with equivalent polar
and Cartesian coordinates end up equal to each
other, and have the same hashCode. - Degenerate conditions create points with theta
gt 2p, or theta lt 0, and ensure that theyre
created correctly (this only applies if we dont
have a precondition restricting the values of
theta).
13
Unit TestingExample
- distance
- Check that the distance between a point and
itself is 0. - Check that the distance between a point and a
clone of that point is 0. - Create two points a known distance away from each
other and check to see that distance gives the
known distance. - Call distance on both objects and ensure the
result is identical. - Do this with enough pairs of points to convince
yourself that distances work between points in
different quadrants, and when the origin is one
of the points.
14
Unit TestingExample
- translate
- Boundary condition attempt to translate a point
by (0, 0). - Attempt to translate a point by (, ), (, -),
(-, ), (-, -) amounts. - Attempt to translate a point between quadrants,
through the origin, etc. - Perhaps attempt to induce a numeric overflow and
see how its handled
15
Unit TestingExample
- scale
- Boundary conditions scale by a factor of 0
(should make the point (0, 0), regardless of the
original point) and a factor of 1 (should leave
the point unchanged). - Scale by positive and negative factors.
- Perhaps attempt to induce a numeric overflow and
see how its handled
16
Unit TestingExample
- rotate
- Boundary condition attempt to rotate the point
(0, 0) by various amounts. - Boundary condition attempt to rotate a point by
an angle of 0. - Attempt to rotate a point by various amounts and
check the results (probably cover all 4
quadrants). - Degenerate conditions attempt to rotate points
with angle gt 2p, or angle lt 0, and ensure that
theyre rotated correctly (this only applies if
we dont have a precondition restricting the
values of angle).
17
Unit Testing
- Thats a whole lot of tests for one little class,
and in many cases you wouldnt need to be quite
so exhaustive. - Every class for which it makes sense to test in
isolation should have its own individual suite of
unit tests (in addition to its assertions). - Sometimes this requires a test harness, a special
environment in which to conduct a test of a
particular component. - For instance, if youre testing a class that
implements a dealer in a card game, you need to
fool it into thinking that it has players to
deal to
18
Unit TestingImplementation
- Tests can get pretty complex.
- Our tests for Point were all very simple, most of
them involving only one or two method calls, but
sometimes it takes a lot more effort to set up a
particular situation to be tested. - For example, to test the behavior of a Stack
class when it reaches its maximum capacity, youd
have to put in a whole lot of distinct objects,
then remove them again and ensure that the order
you got them back in is exactly what it should be.
19
Unit TestingAutomation
- Automation can help a lot, both with creating and
with running unit tests. - One thing you absolutely never want to do is to
sit at a keyboard and interact with a unit test. - Waste of time - you should be working on your
design and code, not interacting with some test
program. - Error-prone - youre not necessarily guaranteed
to provide the same input over and over again,
which makes tests less useful. - Unit tests should be automated to the maximum
extent possible.
20
Unit TestingAutomation
- Jtest and JUnit are unit testing automation tools
for Java, and well be using them this term. - JUnit is free, and lets you create unit tests
just by implementing some classes with specially
formatted methods. - Jtest is commercial, and can automatically
generate a unit test suite based on your
Jcontract assertions. It has the added benefit of
automatically generating JUnit skeletons for you,
which you can fill in with your own more complex
blackbox tests.
21
Integration Testing
- Integration testing can also be automated, in
much the same ways as unit testing. - For example, if you write a class that sends and
receives a bunch of messages through your router,
that is effectively an integration test. - Integration testing should only be done once
youre convinced that every component works on
its own. - Integration can cause interesting things to
happen, especially in concurrent systems, and if
the individual components dont work correctly in
isolation its a lot harder to figure out why the
interesting things happen
22
Interactive Testing
- Interactive testing should usually be the last
phase of testing. - Once the system is working according to its
specification, then you can see how it works when
subjected to the various abuses that users can
put on it. - Interactive testing can often reveal the need for
more unit or integration tests, when a user does
something unpredictable. - User interfaces are typically the components most
affected by interactive testing.
23
Test Failures
- So we have all these different kinds of tests -
what happens when one fails? - Blackbox tests and integration tests are only
going to fail during explicit testing runs, so
those failures are handled by tracking down the
problem and re-running the tests. - Whitebox tests can fail at runtime even after the
software has been delivered to clients (assertion
checks).
24
Test Failures
- When a whitebox test fails, its sometimes
appropriate to stop the entire system, but there
are often better ways to handle it. - Attempt to fix the problem automatically, through
exception handling, retries, etc. - Keep the system running but limit its
functionality (the requested function is
currently unavailable, please try again later).
25
Homework 3 Overview
- Homework 3 is the first design assignment.
- Goal To carry out a design using Design by
Contract, and work with more threads. - Design and implement a thread-safe active router.
- Functionality just like the passive router,
except that it has its own thread(s) that handle
message delivery, and messages may be sent to
multiple destinations rather than a single
destination. - Must explicitly copy message objects, so they
cant be modified after being sent. This will
likely affect your design.
26
Homework 3 Overview
- What you have to turn in
- A set of class skeletons, containing full Javadoc
comments and full contracts for the entire public
API of your router. - Textual documentation (text, HTML, its up to you
as long as its not a Word file) describing
aspects of your design that dont necessarily
belong in Javadoc comments. - Homework 4 will be the implementation of this,
along with a test suite. - Its extremely unlikely that you will get a grade
for Homework 3 before having to work on Homework
4, but well take that into account when grading
both of them.
27
Next Class
- Formal Reasoning about Concurrent Systems
Recommended
CrystalGraphics Presentations
