Software Testing | Aravali College of Engineering & Management

1
Department ofComputer Science Engineering
SOFTWARE TESTING
2

• Software testing can be stated as the process of
  verifying and validating that a software or
  application is bug free, meets the technical
  requirements as guided by its design and
  development and meets the user requirements
  effectively and efficiently with handling all the
  exceptional and boundary cases.
• The process of software testing aims not only at
  finding faults in the existing software but also
  at finding measures to improve the software in
  terms of efficiency, accuracy and usability. It
  mainly aims at measuring specification,
  functionality and performance of a software
  program or application.

• Testing is the process of evaluating a system or
  its component(s) with the intent to find whether
  it satisfies the specified requirements or not.
  In simple words, testing is executing a system in
  order to identify any gaps, errors, or missing
  requirements in contrary to the actual
  requirements.
• According to ANSI/IEEE 1059 standard, Testing can
  be defined as - A process of analyzing a software
  item to detect the differences between existing
  and required conditions (that is
  defects/errors/bugs) and to evaluate the features
  of the software item.

DEFINITION OF TESTING

• Software Testing is a process of evaluating the
  functionality of a software application to find
  any software bugs. It checks whether the
  developed software met the specified
  requirements  and identifies any defect in the
  software in order to produce a quality product.
  It is basically executing a system in order to
  identify any gaps, errors, or missing
  requirements in contrary to the actual
  requirements.

3

• Who does Testing?
• It depends on the process and the associated
  stakeholders of the project(s). In the IT
  industry, large companies have a team with
  responsibilities to evaluate the developed
  software in context of the given requirements.
  Moreover, developers also conduct testing which
  is called Unit Testing. In most cases, the
  following professionals are involved in testing a
  system within their respective capacities -
• Software Tester
• Software Developer
• Project Lead/Manager
• End User
• Different companies have different designations
  for people who test the software on the basis of
  their experience and knowledge such as Software
  Tester, Software Quality Assurance Engineer, QA
  Analyst, etc.
• It is not possible to test the software at any
  time during its cycle. The next two sections
  state when testing should be started and when to
  end it during the SDLC.
• When to Start Testing?
• An early start to testing reduces the cost and
  time to rework and produce error-free software
  that is delivered to the client. However in
  Software Development Life Cycle (SDLC), testing
  can be started from the Requirements Gathering
  phase and continued till the deployment of the
  software.
• It also depends on the development model that is
  being used. For example, in the Waterfall model,
  formal testing is conducted in the testing phase
  but in the incremental model, testing is
  performed at the end of every increment/iteration
  and the whole application is tested at the end.
• Testing is done in different forms at every phase
  of SDLC -
• During the requirement gathering phase, the
  analysis and verification of requirements are
  also considered as testing.
• Reviewing the design in the design phase with the
  intent to improve the design is also considered
  as testing.
• Testing performed by a developer on completion of
  the code is also categorized as testing.
• When to Stop Testing?
• It is difficult to determine when to stop
  testing, as testing is a never-ending process and
  no one can claim that a software is 100 tested.
  The following aspects are to be considered for
  stopping the testing process -
• Testing Deadlines
• Completion of test case execution

4

• Lets break down the basic definition of Software
  testing into the following parts
• 1)  Process  Testing is a process rather than a
  single activity.
• 2)  All Life Cycle Activities Testing is a
  process thats take place throughout the Software
  Development Life Cycle (SDLC).
• The process of designing tests early in the life
  cycle can help to prevent defects from being
  introduced in the code. Sometimes its referred
  as verifying the test basis via the test
  design.
• The test basis includes documents such as the
  requirements and design specifications.
• 3)  Static Testing  It can test and find defects
  without executing code. Static Testing is done
  during verification process. This testing
  includes reviewing of the documents (including
  source code) and static analysis. This is useful
  and cost effective way of testing.  For example
  reviewing, walkthrough, inspection, etc.
• 4)  Dynamic Testing  In dynamic testing the
  software code is executed to demonstrate the
  result of running tests. Its done during
  validation process. For example unit
  testing, integration testing, system testing,
  etc.
•  5)  Planning  We need to plan as what we want
  to do. We control the test activities, we report
  on testing progress and the status of the
  software under test.
• 6)  Preparation  We need to choose what testing
  we will do, by selecting test conditions
  and designing test cases.
• 7)  Evaluation  During evaluation we must check
  the results and evaluate the software under test
  and the completion criteria, which helps us to
  decide whether we have finished testing and
  whether the software product has passed the
  tests.
• 8)  Software products and related work products 
  Along with the testing of code the testing of
  requirement and design specifications and also
  the related documents like operation, user and
  training material is equally important.

5

• Why do we need Software Testing?
• The software applications success rate controls
  the growth of our business. It plays an important
  role for the development of software applications
  and products.
• We need software testing for the following
  reasons
• Cost Effective  Testing has many benefits and
  one of the most important ones is
  cost-effectiveness. Testing our project on time
  can save money in the long run. Software
  development consists of many stages and if bugs
  are caught in the earlier stages it costs much
  less to fix them.

2. Security  This is the most sensitive and
vulnerable part of software testing. Users are
always looking for trusted products that they can
rely on. It helps in removing problems and risks
beforehand.
3. Product Quality  In order to make your
product vision come to life, it has to work as
planned. It is important to follow the product
requirements because it helps you get the
required end results.
4. Customer Satisfaction  The ultimate goal for
a product owner is to give the best customer
satisfaction.  Software should be tested in order
to bring the best user experience possible. Being
the best product in this saturated market will
help you gain trustworthy clients which will have
great long-term effects.
6
Difference between Testing and Debugging
What is Testing? Testing Fundamentally, testing
is a process to check if the system is working
same as it was supposed to do, and not working as
it was not supposed to do. It is done by the
tester to identify the defects in the system
(actual result of test case execution is not
matching with expected result). It can be done
using Manual and automated way. The issues are
logged against the all failed cases and
communicate to the dev team for debugging and
fixing. After the bug fixes then tester retest
the bug and checks if the issue is no more
exists. You may interest in Defect Life
Cycle. Testing involves identifying software
bug/error/defect without its correction. Usually
professionals with a background of quality
assurance are involved in the process of bugs
identification. This process is performed in the
testing phase.    
7
What is Debugging? Debugging is the activity
performed by developers to fix the bug found in
the system. This is manual step by step
unstructured and unreliable process to find and
removes a specific bug from the system. It is
done by the developers to fix the defect in the
system. Based on the defects reported, the
developer starts analyzing the root cause of the
defect to make system defect free. The developers
need to fix the code to make actual result same
as expected result. With the help of details
added in the defect developer follow steps
mentioned and check the loopholes in the
particular code area. They may use a different
tool for debugging. On fixing the defect
developer send back the defect to a tester for
retesting.

Debugging involves identifying, isolating, and
fixing the bugs/problems. Developers who code the
software carry out debugging in order to
encounter an error in the code. This process is a
part of Unit Testing or White Box Testing.
Debugging can be performed either while
conducting Unit Testing, i.e. development phase
or while fixing the reported bugs. Debugging is
deeper process than the  Testing. Debugging
involves identifying, isolating and fixing the
bugs.
8
(No Transcript)
9
Following are the important differences between
Testing and Debugging.
Sr. No. Key Testing Debugging
1 Definition Technically Testing is a process to check if the application is working same as it was supposed to do, and not working as it was not supposed to do. On other hand Debugging is the activity performed by developers to fix the bug found in the system.
2 Objective Main objective of Testing is to find bugs and errors in an application which get missed during the unit testing by the developer. On other hand the main objective of Debugging is to find the exact root cause at code level to fix the errors and bugs found during the testing.
3 Perform As Testing is mainly to find out the errors and bugs is mainly performed by the testers. Also if testing is at developer end known as unit testing then it is performed by the Developer. While on other hand Debugging is to find the missing or de-faulty code in an application hence major performed by the developers only.
4 Knowledge Required As Testing covers the functional and behavioural flow of an application so only functional knowledge is required for the tester to perform the testing. On other hand Debugging is to find the error at code level so technical and code level knowledge is required for the developer to perform debugging.
5 Automation Testing can be manual or made automated with the help of different tools. On other hand Debugging can't be get automated it is always be the manual.
6 Level Testing on basis of level of performing is at different level i.e., unit testing, integration testing, system testing, etc. On other hand no such level of Debugging is possible.
10
What are Software Testing Objectives?
11

• To evaluate the work products such as
  requirements, design, user stories, and code
• The work products such as Requirement document,
  Design, and User Stories should be verified
  before the developer picks it up for development.
  Identifying any ambiguity or contradicting
  requirements at this stage saves considerable
  development and test time. The static analysis of
  the code (reviews, walk-thru, inspection, etc.)
  happens before the code integrates/is ready for
  testing.  This idea of testing is known
  as Verification. It is the process of evaluating
  the product during the development phase of each
  work product.
• To verify the fulfillment of all specified
  requirements
• This objective reveals the fact that the
  essential elements of testing should be to
  fulfill the customers needs. Testers test the
  product and ensure the implementation of all the
  specified requirements have. Developing all the
  test cases, regardless of the testing technique
  ensures verification of the functionality for
  every executed test case. The Tester should also
  create a requirement traceability matrix (RTM),
  which will ensure the mapping of all the test
  cases to requirements. RTM is an effective way to
  ensure that test cases have got the right
  requirement coverage.
•  
• To validate if the test object is complete and
  works as per the expectation of the users and the
  stakeholders
• Testing ensures the implementation of
  requirements along with the assurance that they
  work as per the expectation of users. This idea
  of testing is called Validation. It is the
  process of checking the product after
  development. Validation can be a manual or
  automation. It usually employs various types of
  testing techniques, i.e., Black Box, White Box,
  etc. Generally, testers perform validation,
  whereas customers can also validate the product
  as part of User acceptance testing. Every
  business considers the customer as the king. Thus
  the customers satisfaction is a predominant need
  for any business. 
• For example, customer satisfaction and loyalty in
  online shopping and e-commerce environments is a
  useful indicator for long-term business success.

12

• To build confidence in the quality level of the
  test object
• One of the critical objectives of software
  testing is to improve software quality.
  High-Quality software means a lesser number of
  defects. In other words, the more efficient the
  testing process is, the fewer errors you will get
  in the end product. Which, in turn, will increase
  the overall quality of the test object. Excellent
  quality contributes to a significant increase in
  customer satisfaction as well as lower
  maintenance costs. 
•  
• To prevent defects in the software product
• One of the objectives of software testing is to
  avoid the mistakes in the early stage of the
  development. Early detection of errors
  significantly reduces the cost and effort. The
  prevention of defects involves doing a root cause
  analysis of the defects found previously and
  after that, taking specific measures to prevent
  the occurrence of those types of errors in the
  future. Efficient testing helps in providing an
  error-free application. If you prevent defects,
  it will result in reducing the overall defect
  count in the product, which further ensures a
  high-quality product to the customer.
•  
• To find defects in the software product
• Another essential objective of software testing
  is to identify all defects in a product. The main
  motto of testing is to find maximum defects in a
  software product while validating whether the
  program is working as per the user requirements
  or not. Defects should be identified as early in
  the test cycle as possible.
• E.g., a defect found in the UAT phase will be
  much costlier to fix than the same defect found
  in the Sprint testing phase. 
•  

13

• To provide sufficient information to stakeholders
  to allow them to make informed decisions,
  especially regarding the level of quality of the
  test object
• The purpose of testing is to provide complete
  information to the stakeholders about technical
  or other restrictions, risk factors, ambiguous
  requirements, etc. It can be in the form of test
  coverage, testing reports covering details like
  what is missing, what went wrong. The aim is to
  be transparent and make stakeholders fully
  understand the issues affecting quality.
•  
• To reduce the level of risk of insufficient
  software quality
• The possibility of loss is also known as risk.
  The objective of software testing is to reduce
  the occurrence of the risk. Each software project
  is unique and contains a significant number of
  uncertainties from different perspectives, such
  as market launch time, budget, the technology
  chosen, implementation, or product maintenance.
  If we do not control these uncertainties, it will
  impose potential risks not only during the
  development phases but also during the whole life
  cycle of the product. So, the primary objective
  of software testing is to integrate the Risk
  management process to identify any risk as soon
  as possible in the development process.
•  
• To comply with contractual, legal, or regulatory
  requirements or standards, and to verify the test
  objects compliance with such requirements or
  standards
• This objective ensures that software developed
  for a specific region must follow the legal rules
  and regulations of that region. Moreover, the
  software product must be compatible with the
  national and international standards of testing.
  We have ISO/IEC/IEEE 29119 standards that deal
  with the software testing concept.
• E.g., each country has laws specific to
  accessibility requirements which must be
  fulfilled to avoid legal implications. The
  European Union has strict rules on how the
  Personal Identifiable Information (PII) like
  Social security number etc. should be handled.
  Failure to adhere to such requirements will lead
  to failure of the product, no matter how
  defect-free it has been working!
•  

14
Software Testing - Types of Testing
1) Manual Testing Manual testing includes testing
a software manually, i.e., without using any
automated tool or any script. In this type, the
tester takes over the role of an end-user and
tests the software to identify any unexpected
behavior or bug. There are different stages for
manual testing such as unit testing, integration
testing, system testing, and user acceptance
testing. Testers use test plans, test cases, or
test scenarios to test a software to ensure the
completeness of testing. Manual testing also
includes exploratory testing, as testers explore
the software to identify errors in it. 2)
Automation Testing Automation testing, which is
also known as Test Automation, is when the tester
writes scripts and uses another software to test
the product. This process involves automation of
a manual process. Automation Testing is used to
re-run the test scenarios that were performed
manually, quickly, and repeatedly.

• What to Automate?
• It is not possible to automate everything in a
  software. The areas at which a user can make
  transactions such as the login form or
  registration forms, any area where large number
  of users can access the software simultaneously
  should be automated.
• Furthermore, all GUI items, connections with
  databases, field validations, etc. can be
  efficiently tested by automating the manual
  process.
• When to Automate?
• Test Automation should be used by considering the
  following aspects of a software -
• Large and critical projects
• Projects that require testing the same areas
  frequently
• Requirements not changing frequently
• Accessing the application for load and
  performance with many virtual users
• Stable software with respect to manual testing
• Availability of time

15

• How to Automate?
• Automation is done by using a supportive computer
  language like VB scripting and an automated
  software application. There are many tools
  available that can be used to write automation
  scripts. Before mentioning the tools, let us
  identify the process that can be used to automate
  the testing process -
• Identifying areas within a software for
  automation
• Selection of appropriate tool for test automation
• Writing test scripts
• Development of test suits
• Execution of scripts
• Create result reports
• Identify any potential bug or performance issues
• Software Testing Tools
• The following tools can be used for automation
  testing -
• HP Quick Test Professional
• Selenium
• IBM Rational Functional Tester
• SilkTest
• TestComplete
• Testing Anywhere
• WinRunner

16
Software Testing - Myths
Myth 1 Testing is Too Expensive Reality - There
is a saying, pay less for testing during software
development or pay more for maintenance or
correction later. Early testing saves both time
and cost in many aspects, however reducing the
cost without testing may result in improper
design of a software application rendering the
product useless. Myth 2 Testing is
Time-Consuming Reality - During the SDLC phases,
testing is never a time-consuming process.
However diagnosing and fixing the errors
identified during proper testing is a
time-consuming but productive activity. Myth 3
Only Fully Developed Products are
Tested Reality - No doubt, testing depends on the
source code but reviewing requirements and
developing test cases is independent from the
developed code. However iterative or incremental
approach as a development life cycle model may
reduce the dependency of testing on the fully
developed software. Myth 4 Complete Testing is
Possible Reality - It becomes an issue when a
client or tester thinks that complete testing is
possible. It is possible that all paths have been
tested by the team but occurrence of complete
testing is never possible. There might be some
scenarios that are never executed by the test
team or the client during the software
development life cycle and may be executed once
the project has been deployed. Myth 5 A Tested
Software is Bug-Free Reality - This is a very
common myth that the clients, project managers,
and the management team believes in. No one can
claim with absolute certainty that a software
application is 100 bug-free even if a tester
with superb testing skills has tested the
application.
17
Myth 6 Missed Defects are due to
Testers Reality - It is not a correct approach to
blame testers for bugs that remain in the
application even after testing has been
performed. This myth relates to Time, Cost, and
Requirements changing Constraints. However the
test strategy may also result in bugs being
missed by the testing team. Myth 7 Testers are
Responsible for Quality of Product Reality - It
is a very common misinterpretation that only
testers or the testing team should be responsible
for product quality. Testers responsibilities
include the identification of bugs to the
stakeholders and then it is their decision
whether they will fix the bug or release the
software. Releasing the software at the time puts
more pressure on the testers, as they will be
blamed for any error. Myth 8 Test Automation
should be used wherever possible to Reduce
Time Reality - Yes, it is true that Test
Automation reduces the testing time, but it is
not possible to start test automation at any time
during software development. Test automaton
should be started when the software has been
manually tested and is stable to some extent.
Moreover, test automation can never be used if
requirements keep changing. Myth 9 Anyone can
Test a Software Application Reality - People
outside the IT industry think and even believe
that anyone can test a software and testing is
not a creative job. However testers know very
well that this is a myth. Thinking alternative
scenarios, try to crash a software with the
intent to explore potential bugs is not possible
for the person who developed it. Myth 10 A
Tester's only Task is to Find Bugs Reality -
Finding bugs in a software is the task of the
testers, but at the same time, they are domain
experts of the particular software. Developers
are only responsible for the specific component
or area that is assigned to them but testers
understand the overall workings of the software,
what the dependencies are, and the impacts of one
module on another module.
18
Estimation Techniques in Software Testing
19

• One of the most important factors while
  estimating testing efforts is the hands-on
  experience on varied projects for the software
  test life cycle. No longer can one just take a
  guessing approach about the number of days for
  any task or working on the old-time formula of
  one-third of the development effort.
• Although this method is not based on any
  scientific principle or technique, it is one of
  the most widely used estimation technique
  by companies offering software testing services.
  Unfortunately, the development versus testing
  effort method has given many failures in software
  projects in the past, thereby compromising the
  software or mobile apps on quality.
• In recent years there have been many techniques
  that have been developed for estimating the
  software testing timeframe. These techniques are
  3-Point Software Testing Estimation Technique,
  Use-Case Point Method and Wide Band Delphi
  Method.
•  
• 3-Point Software Testing Estimation Technique
• 3-Point Software Testing Estimation Technique is
  based on statistical methods in which each
  testing task is broken down into sub-tasks and
  then three types of estimation are done on each
  tasks.
• The formula used by this technique isTest
  Estimate P (4N) E / 6Where,
• P Positive Scenarios
• N Negative Scenarios
• E Exceptional Scenarios
•  
• Standard deviation for the technique is
  calculated as,
• Standard Deviation (SD) (N E)/6
•  

20

• Use Case Point Method
• Use-Case Point Method is based on the use cases,
  where we calculate the unadjusted actor weights
  and unadjusted use case weights to determine the
  software testing estimation.
• The formula used for this technique is
• Unadjusted actor weights Total no. of actors
  (positive, negative and exceptional)
• Unadjusted use case weight Total no. of use
  cases
• Unadjusted use case point Unadjusted actor
  weights Unadjusted use case weight
• Adjusted use case point Unadjusted use case
  point 0.65 (0.01 50
• Total Effort Adjusted use case point 2
•  
• Wideband Delphi
• In Wideband Delphi Method, work structure is
  broken down for each task and is distributed to a
  team comprising of 3-7 members for re-estimating
  the task. The final estimate is the result of the
  summarized estimates based on the team consensus.
  This method speaks more on experience rather than
  any statistical formula. The Wideband Delphi
  testing estimation technique logically estimates
  the group iteration efforts required in a visual
  manner for the testing team. This test was coined
  by Barry Boehm and is widely accepted software
  testing estimation technique to solve complex
  problems.
• How to evaluate a software testing estimation
  technique?
• For any software testing estimation technique, it
  is highly recommended that the following factors
  should be taken into account
• Domain knowledge and core requirements
• Risks and complexity of the application
• Team knowledge on the subject/skills
• Historical data for the previous estimation for
  improvement and accuracy
• Estimation should include buffer time
• Bug cycles for the project

21
Aravali College of Engineering And
Management Jasana, Tigoan Road, Neharpar,
Faridabad, Delhi NCR Toll Free Number 91-
8527538785 Website  www.acem.edu.in