Kein Folientitel

1
Testing - a Definition Errors are part of Human
Nature Spectactular Errors The Testing
Dilemma The Origins of Testing Organisational
Aspects of Testing Legal Aspects of
Testing Economical Aspects of Testing Test Costs
and Benefits Test Approaches Test Methods Test
Levels
Structural Testing Functional Testing Test
Objectives Test Processes Test Planning Test
Techniques Test Effort Distribution Error
Categories Error Distribution Error Detection
Probability Testing as a Profession - Minus
Points Testing as a Profession - Plus Points
1
13
2
14
3
15
4
16
5
17
6
18
19
7
8
20
9
21
10
22
11
23
12
24
2
Testing ...
... Implies empirically demonstrating that a
given piece of software fulfills a given set of
requirements in a given environment at a specific
moment in time. Testing is intended to both
validate and to verify system behaviour. To
verify is to confirm that the software matches
its specs. To validate is to confirm that the
software functions in a real live environment.
... Testing is only one of several activities
directed toward ensuring the quality of a
software system, activites which include, among
others, reviews, inspections, audits, source
analysis and simulation. These are all regarded
as Validation und Verification (VV) activities.
3
The human being is unfortunately an imperfect
creature, which carries over its own
imperfections into the works it itself creates.
Humans, in their current state of evolution, are
unsuitable for complex software development, no
matter in what language and at which level of
abstraction. Humans say what they do not mean and
write what they do not intend to write. The human
communication pattern is imprecise. Even the
communication between mind and mouth, as well as
between mind and the extremities, is distorted.
Thus every human is to a certain extent
handicapped, some more, others less. Transmission
disorders occur in what the human being perceives
in his environment. He confuses what he thinks he
hears or sees with what really is heard or seen.
His sensors are biased by his expectations and
prevent him from being totally objective. (See
Kant "Kritik der reinen Vernunft"). It can not be
expected from a human being, as a biologically
imperfect creature, to produce software without
errors. Only a biologically altered organism,
i.e. a genetically reengineered creature, could
be expected to produce error free software..
4
Spectacular faults in the past have lead to the
emergence of testing as as an unavoidable
activity within the life-cycle of a software
system. Without testing, there is no way to
ascertain the presence or the absence of such
errors.The Venussonde was lost because of a
comma instead of a point.
DO 1000 I 1,15 DO 1000
I 1.15The rising moon was detected as being an
incoming enemy rocket.The
second Space-Shuttle-Start was interrupted
because of a synchronizsation error in the five
shuttle processors just before takeoff.The
Ariane-5 Spacecraft exploded at takeoff time
because of a register overflow in an untested
reused component.The Deutsche Bank lost 8
Million German marks because of a rounding error
in the Euro Conversion.
5
Why havent you fed the horse - customer - yet???
First I have to remove the needles, otherwise
it will choke.
6
The first technical conference on software
testing occured at the Univ. of North
Carolina in July 1972 Other confernces followed
in both the US.A. and in Europe. The first major
European test conference took place in London in
1978.By 1980, test technology had become a
recognized branch of software engineering. The
U.S. Ballistic Missle Defense system was the
first to be systematically tested using automated
test tools.Testing became the empirical
approach to software-quality assurance as
opposed to program proving which remained the
theoretical appraoch.The first commercial test
laboratory was established by Ed Miller and Harry
Sneed in Budapest in the spring of
1978.
7
From the viewpoint of the organisation there are
three tests the test by the developer
himself, the test by an independent
tester and the test by the user..Each
Test consists of the following steps
Organising the test resources Selecting
the proper test strategy Planning the
test activities Specifing the test
cases Executing the tests
Evaluating the test results Testing is a project
in its on right, i.e., a special project within
the development, maintenance, migration or
integration project.
8
The overall standard for evaluating the quality
of software products is the ISO Standard
9126. The IEEE Standard 829 prescribes the proper
system test documentation. . The IEEE Standard
1008 prescribes procedures for software unit
testing. Responsible standardisation gremiums
are ISO/EMC - int. Standards
Organisation ANSI - American National
Standard institute IEEE - Int. Electronic
Electrical Engineers NBS - National
Bureau of Standards ASQF- American
Standard Quality FoundationThe ASQF is
authorized to examine and to certify testers.
Only certified testing companies can be held
liable for software reliability.
9
No Testing
With Test
1 Erroneous
Zero Test Costs
100 Test Costs
25Erroneous
24corrected
75Correct
75Correct
6 of the Results will be wrong
0,25 of the Results will remain wrong
10
100
25
Error Probability
of Development Effort
50
10
5
5
Test effort
1
0
11
Developer Test
Tester Test
User Test
Systemtype Programtest Systemtest
Acceptance TestStandalone-PC Trial Error
- -Mainframe Dialoge Test System
Test Live Test Batch TestEmbedded Module
Test Integration Test Hardware TestProcess
Control Module Test Integration Test Process
TestClient/Server Client Test Frontend
Test Live Test Server Test BackendTest Internet
Webclient Test Network Test Usage Test
12
Test Methods according to the American National
Bureau of Standards Publication for
"Software-Quality Assurance"
Review of the Req. Definition Audit of the Req.
Specs Design Review Testplan Review Code
Inspection Module/Class Test Component
Test Funktion Test Black-Box Test White-Box Test
Object Integration Test Component Integration
Test Interface Test Load Test Regression
Test System Test Acceptance Test Installation
Test Beta Test
13
Testing requires knowledge of the software test
object at some level. One can acquire that
knowledge ? by using the software without
knowing how it is constructed (corresponds
to a trial run, e. g. the test drive of a new
automobile). This level of testing is
termed Functional Testing Black-Box
Testing. One can also acquire that knowledge ?
by examining the inner structure of the test
object (corresponds to the inspection of
the motor when buying a new automobile).
This level of testing is termed
Structural Testing White-Box-Testen. One can
also acquire that knowledge ? by discovering
how the parts of the system interact with one
another This level of testing is termed
Integrational Testing Grey-Box-Testen.
14
Structured Testing is the most wide spread
method of White-Box Testing. This method is
based on the extraction of test paths and test
parameters from the structure of the source
code.There are various intermediate approaches
between the pure Black-Box and the pure White-Box
test approach.
Debugging and Testing
There is a fine distinction between debugging and
testing although both activities are
similar. Debugging is directed toward the removal
of errors Testing is directed toward the
identification of errorsBoth activities are
performed by the developer.
15
Functional Testing (Black-Box-Testing) takes
place without any knowledge of how the software
is constructed. It is focused on the usage of the
product.For functional testing, the proper test
data must be selected.The preparation of the test
data must be planned from the beginning.Function
al Testing is not sufficient for Systems with a
complex internal construction (e. g. self made
technical services). Each subsystem must be
tested independently of the others.Every Test
is a test against somethimg, a comparison with
another object. There must be an oracle to
determine whether the behaviour of a software
system is correct or not.Functional testing
must be made by an independent tester.
16
Each Test is a test against something. The actual
behaviour of a program is compared with the
expected behaviour. For that, the expected
behaviour must be defined.
Program
Oracle
As a rule, a program can be tested against
the specification, the users Handbook,
the previous program and the operational
profile.
17
Test Management
Test Planing
Test Design
Test Specification
Test Execution
Test Evaluation
Test Documentation
18
Which data?Where from?
Where to?What is correct?
Test paths
Output data
Input data
Program
Faults
With what probability?
Which paths are to be traversed? (cases)How long
may a transaction last? (Time)What data are
required? (Data sets)When can one stop? (End
Criteria)
Testing without specific test objectives is
similar to shooting in the dark. Finding an error
is purely coincidental. Random testing produces
random results, which cannot be controlled.
19
Technique Manu/Auto.
Stat/Dyn Struct/Funct Proving
both static bothWalkthroughs
manu dynamic
bothInspections manu
static bothDesign Reviews Audits
manu static bothSimulation
auto dynamic
functionalDesk Checking manu
both structuralPeer Review
manu both
structuralExecutable Specifications auto
dynamic functionalAutomated Unit
Testing auto dynamic
functionalLoad Testing manu
dynamic functionalError Prediction
manu dynamic
functionalCause Effect Graphs both
dynamic functionalDesign-based
Testing manu dynamic
functionalCoverage directed Testing auto
both structuralComplexity
directed Testing auto both
structuralCompiler supported Analysis auto
static structuralData Flow Analysis
auto static
structuralControl Flow Analysis auto
static structuralSymbolic Execution
auto static
structuralInstrumentation auto
dynamic structuralCombined
Techniques auto both
both
From the NBS Spec. Publ. 500-75 Validation,
Verification and Testing of Computer Software,
U.S. Dept. of Commerce, 1981
20
System-design
20
40
Program Design
Module Design
Coding
60
20
Module Test
Integration Test
System- test
20
40
Test Effort
21
Main
Categories Controlflow-Error Program flow goes
in an unexpected directionData-Error Program
produces unexpected resultsAbend-Fehler
Program is abnormally terminated
1.
2.
3.
Sub categories of Errors Computional
errors (COMP)Logical errors
(LOGIC)Data output errors
(RESULT)Initialization errors
(INIT)Data definition errors
(DEF)interface errors
(INTER)Input/output errors (I-O)
1.
2.
3.
4.
5.
6.
7.
22
ErrorTypeCOMPLOGIC
Frequency
1214
2428
RESULTINIT
8 9
1113
DEFINTER
7 8
910
I-OOther
1618
12
TRW-Experience GRC-Experience
According to "Error Detection Using Path Testing
And Static Analysis" in Computer Vol. 12. Nr. 8.
23
E r r o r D e t e c t i o n R a t e
Test Method
E r r o r T y p e s
Average
COMP LOGIC RESULT INIT DEF
INTER I-O LOADCode Inspection
25 20 25 25
12Static
Analysis 25 20
50 40
18Path Analysis 25
40 50
14Result Comparison
25 50
25 25
16Integration Test
25
25 25 9Function Test
20 50
50 50 21Load
Test

25 50 9
24
Testing is a dirty job, one has to clean up the
mess of others. Testing is not creative and
offers no possibilty of self fullfillment. Testing
is a dependent activity and encompasses many
detailed tasks. Testing is not a highly respected
job, testers are looked down upon. Testing tools
are complicated and difficult to use. Testing
methods are often project specific and not
portable. Testing is a destructive activity - one
is out to destroy the system. Testing can become
terribly boring!
25
Testing is a challenge, a difficult task which
requires creativity and lots of self discipline.
Testing is important, because managers and
users need to gain confidence in the product.
Developers should be thankful that someone finds
there errors before the users do.Testing is an
activity, which demands both domain knowledge and
technical savay.Testing can be interesting,
since the technology is still highly
underdeveloped, thus providing the opportunity
for innovation. There is a tremendous need for
better tools.Testing is unaviodable - someone
has to do it.Testing offers an employment
opportunity for old programmers, who are no
longer able to keep pace with the youngsters.