Title: DataDriven Automation Testing
1- Data-Driven Automation Testing
2Table of contents
- Cost-Effective Automated Testing
- Record/Playback Myth
- Viable Automated Testing Methodologies
- Managing Resistance to change
- Staffing Requirements
- Conclusion
3Cost-Effective Automated Testing
- Automation testing more expensive and it doesnt
replace the complete manual testing. Instead, it
reduces the effort of manual testing by
automating the repeatable test cases. - It doesnt make sense in automating some test
cases. Over complex tests are more trouble than
they are worth to automate. Concentrate mainly on
the straight forward test cases and leave the
complex cases to manual testing. - Automate only the repeated test cases. One time
running test cases are not worth automating.
Besides, it is useless to automate one time
running test cases. Which will be of no use in
the future. - Avoid always record/playback approach of
automation. It consumes most of time and cost to
maintain the scripts. Most of the concerns who
started with record/playback methodology of
scripts closed automation and started continuing
manual testing.
4- Adopt Data-Driven approach for testing. In this
methodology, only inputs and expected results
needs to be updated mostly in maintenance. Its
ease in maintaining the automation scripts.
5Record/Playback Myth
- The script resulting from this method contains
hard-coded values. - The cost of maintaining the scripts are
astronomical and unacceptable. - The scripts are not reliable and robust enough.
They may fail even if the application has not
been changed, in the replay. - Its time consuming process. For ex. If a tester
enters wrong data, he needs to re-record again. - The scripts must be re-recorded even if the
application changes. - While recording test cases, if the application
has a bug, the tester could not continue
recording until the bug is fixed. Then why we
need automation testing for ? - Note Normally after 2 to 3 months of this
nonsense, the testers put these scripts in
recycle bin and get backs to manual testing again.
6Viable Automated Testing Methodologies
- There are two best automation testing
methodologies. They are - Functional Decomposition Method
- Test Plan / Key-Word Driven Method
- Functional Decomposition Method
- The main concept behind the "Functional
Decomposition" script development methodology is
to reduce all test cases to their most
fundamental tasks, and write User-Defined
Functions, Business Function Scripts, and
"Sub-routine" or "Utility" Scripts which perform
these tasks independently of one another. In
general, these fundamental areas include - Navigation (e.g.
Navigating from Login page") - Specific Business Function (e.g. Submit an order
or Payment") - Data Verification (e.g. "Verify
Payment Updates Current Balance") - Return Navigation (e.g. Logout")
7- In order to achieve this, we need to separate
data from function. A hierarchical is employed,
using a structured or modular design. - Driver Scripts Perform Initialization (if req.),
then calls the test case in desired order. - Test case Scripts Perform the application test
case logic using business functions. - Business Functions Perform the business
functions with the application. - Subroutine scripts Perform application specific
tasks, which in turn called from 2 or more
business function scripts. - User-Defined functions These can be called as
Utility functions which in turn is general and is
not functionality related. - The "Business Function" and "Subroutine" scripts
invoke "User Defined Functions" to perform
navigation. The "Test Case" script would call
these two scripts, and the Driver Script would
call this "Test Case" script some number of times
required to perform all the required Test Cases
of this kind. In each case, the only thing that
changes are the data contained in the files that
are read and processed by the "Business Function"
and "Subroutine" scripts.
8- Advantages
- Utilizing a modular design, and using files or
records to both input and verify data, reduces
redundancy and duplication of effort in creating
automated test scripts. - Scripts may be developed while application
development is still in progress. If
functionality changes, only the specific
"Business Function" script needs to be updated. - Since scripts are written to perform and test
individual Business Functions, they can easily be
combined in a "higher level" test script in order
to accommodate complex test scenarios. - Data input/output and expected results is stored
as easily maintainable text records. The users
expected results are used for verification, which
is a requirement for System Testing. - Functions return "TRUE" or "FALSE" values to the
calling script, rather than aborting, allowing
for more effective error handling, and increasing
the robustness of the test scripts. This, along
with a well-designed recovery routine, enables
unattended execution of test scripts.
9Keyword/Test Plan Driven Method This method
uses the actual Test Case document developed by
the tester using a spreadsheet containing special
"Key-Words". This method preserves most of the
advantages of the "Functional Decomposition"
method, while eliminating most of the
disadvantages. In this method, the entire
process is data-driven, including functionality.
10- How to Work With this
- The above table can be created as a template in a
excel sheet. - A controller script can be written to fetch the
data as a list. This can be passed to an utility
program and where in it identifies the keyword
and commands the business functions to perform
the intended action. - The data can be given as comma-delimiter or else
tab-delimiter as specific to the verification in
the project. - The controller script should be written such that
it fetches all the required data until the
end-of-file (EOF) is reached. - Architecture
- The architecture seems to be same for both
Functional Decomposition and as well as Test
Plan/Key-Driven methods. - Driver Script It calls the application specific
controller script and passes it to the test
cases. - Test case scripts Reads and executes the data
from the driver scripts and also calls utility
scripts based on the keywords from Controller
scripts. - Utility Scripts Performs actions based on the
parameters from Controller scripts by calling
user-defined functions and returns back the
control to controller scripts.
11- User-Defined functions It contains general or
application-specific functionalities. It can be
called from any script to perform specific
actions or tasks. - Preparation is the key
- Preparation have to made by the organization
before initializing automation process. If
preparation has not been made adequately, then
the ramp-up time for developing scripts will
increase dramatically. - An adequate test-environment must exists that
accurately replicates the production environment.
It can be a small-scale replica. But it should
contain all the hardware, database and operating
systems that development environment has. - The test environments database must be restored
to a known baseline, otherwise the tests
performed against the database could not be
repeated, since it is altered. - Dedicated high speed machines should be set up to
run the scripts. Its vague to run the scripts in
the testers machine parallel to developing
scripts. - Detailed test cases should be converted into
automation format. The tool cannot think and we
should script everything to tell what to do.
12Managing Resistance to Change
- The main fall of automation in lots of
organizations is mainly because of the wrong
choice in choosing automation tool for the
applications. The main downfall happens is that
the management chooses the tool without
consulting the testers, who is responsible for
automation and who have to cope up all these. - There is a myth in between almost all testers
that Tool replaces human. The testers should
approach a testing tool such like another tool
that makes their testing in a more efficient way
by means of - Relaxing them from being executing the same
boring kind of test cases. - It helps to save time by making them to create
value added activities for the testing projects. - The common pitfalls in introducing a new
automation tool in the manual testing projects
are - It will take so much of the time for all testers
to learn to use the tool. - The tool may be difficult for the testers to use.
13- This pitfalls can be removed by hiring a Test
Tool Specialist or else the tool vendors will
also offer some free Training Programs for
learning more about the tool. A senior-level
programmer who can learn easily the tool or else
an expert in that particular tool can reduce
those pitfalls. - Mentoring the team with the above mentioned
specialists will increase the efficiency in
automating approach.
14Staffing Requirements
- Testing tools normally have a standard approach
for programming. The tester who uses automation
testing tool is nothing but a programmer.
Programming skills in any language with a good
approach is needed for a good automation tester. - He/She should have Logical, technical and as well
as analytical skills. If these skills are there
for an automation tester, they can master any
tool. - The automation tester is responsible for
- Developing standards and procedures for
automation scripts/functions. - Developing change management procedures for
automation scripts/functions implementation. - Developing the details and infra-structure for a
data-driven method. - Testing, managing and implementing the scripts
done by testers. - Running the automated scripts and provides
results in the form of reports to the
organization.
15Conclusion
- Its a better way to switch rather than following
the traditional functional approach. - Select a tool that will allow you to implement
all logics what you can implement with
Data-Driven approach and as well as it can
support long-term testing strategies. - Before getting a tool for the project, make sure
that the vendor can provide you support and as
well as training. - Its the right way to work as a team in planning
together and selecting a right tool for the
application. - Lets see how to select a good tool for the
application under test in upcoming presentations.