Test Automation

1

• Test Automation
• Using
• SilkTest

2
Test Automation Rules

• Product feature understanding, test planning,
  test case documentation, test bed setup, defect
  tracking, progression tests are all manual
• Test automation success depends on test cases
• Every manual step has commands in test script
• If it is not worth, do NOT automate

3
Tools - Comparison

• WinRunner, QTP are from Mercury Interactive USD
  1bn company and market leader Robot from
  Rational IBM - USD 100bn company Silk from Segue
  USD 50mn company
• Script here is object oriented and close to QTP
  script. Winrunner TSL is like C and Robot SQA
  Basic is like Visual Basic
• GUI to create verification points are better in
  WinRunner, QTP and Robot
• Silk is more towards moving scripts on MS Windows
  and Unix whereas Robot or WinRunner scripts
  cannot be moved to other OS as those tools do not
  support
• Rational does not provide GUI specification in a
  separate file WinRunner, QTP and Silk provide
  that
• You need to buy Java Plugin for WinRunner Silk
  and Robot come with Java plugin
• Record, replay, verification method, data driven
  tests, database connectivity, bitmap verification
  are all same at concept level and differ only at
  script syntax level
• Wizards are less in SilkTest, compared to other
  tools
• Cost-wise, QTP and WinRunner are most expensive
  Silk stands in between WinRunner and Rational
  Robot

4
(No Transcript)
5
Architecture

• Project .vtp (for maintenance)
• Testplan .pln (organizing the tests)
• Test Frame .inc (AUT Window details)
• Scripts .t (test scripts)
• Data driven tests .g.t (scripts with many data)
• Include file .inc
• Suite .s (for batch runs)
• Result .res

6
Test Automation Principle

• Make the tool learn about AUT GUI
• Record steps based on mouse and key events
• Organize steps into test case
• Organize test cases into test plans
• Run test cases
• Batch different sets and run
• Analyze results
• Do NOT try to code application logic into testing
  code
• Whenever possible, generate code thru the tool
  and not hand-code
• Read STUSER Help file (user guide for SilkTest)

7
Test Process

• Manage requirements using SilkCentral Test
  Manager
• Manage test plans and cases using Silk test
• Associate test plans/cases to requirements using
  test manager
• Declare GUI declarations using SilkTest
• Develop scripts for atomic activities first so
  that they can be reused later
• Develop scripts using SilkTest and run them
• Track defects using Defect Tracker

8
Test Frames

• Before test case recording, do this step
• This creates an .inc file that has GUI
  declarations
• Identifies GUI controls as classes
• Label or caption is the best way to identify
  objects if not available, object index is used
  (available tags are caption, prior text, index,
  window id, coordinates)
• Class and tag are the key elements in
  declarations
• Record window declarations is the way to store
  test frames
• When objects are not identified properly, use
  different tags to uniquely identify the objects.
  This will vary from application to application
• If object X,Y coordinates are to be used, declare
  constants and then use those constants for
  further declarations
• To declare new objects within a window, it is
  better to use AnyWin or CustomWin classes. Use
  ActiveApp to specify the coordinates

9
(No Transcript)
10
Test Frames

• Along with window declarations, we can declare
  methods that are specific to that window and
  objects
• The method declarations within a window object is
  similar to object oriented programming
• Uses can use these method declarations as wrapper
  classes for executing a series of steps that are
  recorded
• To the extent possible, do not have any hard
  coded values in the GUI Window methods
• While recording new test cases, ensure that you
  update window declarations as well but check the
  frame.inc file, not to have duplicate
  declarations for same window
• Default is frame.inc but we can have any number
  of include files
• Unless the window declarations are in scope, the
  test script will not recognize the controls

11
Recording

• Recording will automatically generate statements
  on every user action
• To capture the mouse click coordinates and
  keyboard events, we can use low level enabler for
  every control in recording options
• We can record the base state as well as test
  steps
• Soon after recording, ensure the test frame
  declarations, for any changes or duplicates
• Recording can be done for window declarations or
  test case or actions
• Every recorded script must start from a known
  base state and must return back to the same known
  base state
• BaseState and DefaultBaseState can also be
  specified using recording (appState is a function
  that ensures the pre-condition state for a test
  case)

12
(No Transcript)
13
(No Transcript)
14
Test Cases

• Each test case must be independent
• Initialize (pre-requisite setup), run and verify,
  cleanup
• One test case must have One purpose
• Establish application state for every test case.
  Record the base state
• Test Case contains commands for every test step.
  Record the steps and edit if needed
• Establish verification points for every test case
• Record the clean up steps as well
• Sequencing of test cases is very important when
  it comes to automation
• These are also functions, with the type as
  testcase
• We can use parameters for testcase as well
• A testcase is also a function by itself, but it
  can have base states

15
4Test Elements

• Variables INTEGER, STRING, BOOLEAN, ANYTYPE,
  REAL, LIST
• Variables do have scope public or private
• Const is for constants
• Regular arithmetic and string operations hold
  good
• 2 types of for statements
• Foreach and While loops are also available
• IF, select-case and SWITCH-case are available for
  branching
• Do-except is for exception handling
• Exception functions will be useful for tracing
  exception number and details
• Test steps will have window.object.method or
  window.object.property style of commands
• User defined functions with in, out, inout
  parameters
• Functions can return values as well
• Use built-in functions for list and array
  manipulations
• Print and Printf commands are used for sending
  output to results file

16
Built-in Functions, Classes

• For debugging purpose, we can use print function.
  It helps in documentation as well as to track the
  test run status, at any given point of time
• Refer to General Core Classes section in SilkTest
  Help (PARTNER file) AnyWin, MainWin, CheckBox,
  ComboBOx, DialogBox, ListBox, Menu, PushButton,
  RadioButton, StaticText, TextField, StatusBar are
  the most used classes
• Refer to Functions by Programming Task section in
  SilkTest Help (PARTNER file) Datatype commands,
  Database commands, List and string Manipulations,
  Window Information commands are the most used
  ones
• System related functions can be used to carry out
  any DOS commands to set the test environment
  (Sys_Execute, Sys_Kill)
• String functions such as ltrim, rtrim, etc are
  used extensively to manipulate test data
  arithmetic functions are not used often
• User defined functions must be called from main
  function

17
(No Transcript)
18
(No Transcript)
19
AnyWin Properties Methods

• CaptureBitmap
• ClearTrap
• Click
• DoubleClick
• Exists
• GenerateDecl
• GetAppId
• GetArrayProperty
• GetBitmapCRC
• GetCaption
• GetChildren
• GetClass
• GetContents
• GetEverything
• GetHandle
• GetHelpText
• GetIDGetIndex
• GetInputLanguage

• GetManyProperties
• GetName
• GetNativeClass
• GetParent
• GetProperty
• GetPropertyList
• GetRect
• GetTag
• InvokeMethods
• IsActive
• IsArrayProperty
• IsDefined
• IsEnabled
• IsOfClass
• IsVisible
• InvokeJava
• MenuSelect
• MoveMouse
• MultiClic

20
AnyWin Properties Methods

• MultiClick
• PopupSelect
• PressKeys
• PressMouse
• ReleaseKeys
• ReleaseMouse
• ScrollIntoView
• SetArrayProperty
• SetInputLanguage
• SetProperty
• SetTrap
• TypeKeys
• VerifyActive
• VerifyBitmap
• VerifyEnabled
• VerifyEverything
• VerifyText
• VerifyProperties
• WaitBitmap

• bActive
• AppId
• sCaption
• lwChildren
• Class bEnabled
• bExists
• sID
• iIndex
• sName
• wParent
• Rect
• hWnd
• WndTag

21
Method examples

• Find.OK.Click
• Find.MatchCase.Check
• Find.FindText.SetText(abc)
• Notepad.CloseWindows
• Find.Up.VerifyValue(TRUE)
• Fonts.FontName.verifyContents()
• Notepad.Editor.SelectAll() // this is a custom
  method to select all contents. This needs to be
  declared in the windows declarations
• User defined functions can have in, out, inout
  and optional parameters
• ClipBoard functions are used to manipulate
  clipboard text
• Cursor class methods can be used to manipulate
  coordinates
• Property prefixes b for boolean, I for int, ls
  for list, s for string

22
Get, Set, Verify for Controls

• For every object, Get methods are used to obtain
  attribute values into a variable. After getting
  to a variable, we need to manipulate
• For every object, Set methods are used to set
  attribute values on the screen. This will work
  only when the object cooperates with SilkTest
  operations
• For every object, Verify methods are used to
  check the attribute values implicit Get and an
  if-then-else condition. It must have an expected
  value as argument
• Whatever properties that can be used for get
  statement, can be used for set and verify
  statements
• Verify methods evaluate to pass or fail status
• Any other custom methods can be used, if they are
  already declared in window declarations
• Most of the properties and methods are inherited
  from AnyWin to all other GUI classes
• Dlls can be loaded using dll declaration and the
  functions can be invoked from 4Test
• Capture and Compare bitmap commands are used for
  image checks

23
Utility Elements

• Array functions are used to manipulate arrays
  (ArraySize, ArrayFind)
• List functions are used to manipulate lists
  (ListSort, ListAppend)
• DateStr, TimeStr, Sleep, StartTimer, StopTimer
  are used for timer functions
• FileOpen, FileClose, FileWriteLine, FileReadLine
  are used for file operations
• Wild cards - ? And MatchStr command is used
  for expressions
• Multitags enable an object to be identified with
  many labels
• Registry Operations are available not used
  widely (Reg_CreateKey)
• LogError, LogWarning, Print to write to results
  file
• Timer commands to create, start, stop, destroy
  and get the interval
• Date format and functions to manipulate date
  fields
• DLLs can be loaded and then functions can be
  executed but this needs to be kept at minimum
• To get script, environment and browser related
  information, we can use the respective built-in
  functions (GetArgs, GetBrowserType)
• Date related functions are available but it is
  advised not to use inside script to manipulate
  dates

24
Database Test

• Database tests follow the verification of GUI
  verifications
• Database commands start with DB_
• Connect to DB (DSN needed)
• Execute Query (give SELECT)
• Fetch the records and compare with baseline
• Finish DB operation (release resources)
• Disconnect after every database operation
• Do not keep the database connections for a long
  time in 4Test scripts

25
Verification

• Object properties are the key elements
• Control state, value, label, text, count, size,
  location are the most common properties
• Feed the expected result to the verify command
• Bitmaps can be verified using CaptureBitmap and
  VerifyBitmap
• Use Bitmap tool to capture baseline image
• Is methods evaluate to a boolean value
  (IsActive, IsEnabled)
• LinkTester checks the links in web applications

26
Data Driven Tests

• Same script, variety of data, less coding
• Use Data Driven work flow for a standard recorded
  test case
• Create a test database and configure DSN
• Replace hard coded recorded values to data driven
  variables (using find/replace option)
• When you run the test, it runs for all rows of
  data in the data set
• A new test case with DD_ prefix will be created

27
(No Transcript)
28
Run Tests

• One can run a test case or test plan or test
  suite
• Suite contains many test cases in it
• Using distributed run mode, one can run scripts
  in another machine that has Silk Agent
• We can run in run mode or debug mode
• Debug mode enables us to step, step into, step
  out, declare breakpoints etc
• All other hosts run the Agent
• Spawn, Parallel, Rendezvous are used in
  distributed testing but unless required,
  distributed testing is not widely used
• One must connect and disconnect to target
  machines

29
Traceability

• Requirements to test traceability is thru Test
  Manager
• Test case to tester traceability is again part of
  test manager
• We need to define naming conventions by ourselves
  for test plans, test cases, requirements
• Suggested guideline is to name the test case with
  the test case ID itself and have the test case
  description as comments
• Use base states for capturing test pre-requisites
• Version control is integrated with PVCS for
  check-in, check-out and labeling features

30
Recovery Scenarios

• Recovery system enables the test to come to the
  Base State
• Record application state is used to specify the
  recovery scenarios
• Exception handler itself is script recovery
  feature
• StopRunning() method bypasses all recovery
  settings

31
Test Extensions

• These are used to help SilkTest to identify new
  objects and their properties
• Silktest can automatically enable extensions for
  open applications
• Java, .Net, AOL Browser, Netscape Navigator,
  Internet Explorer require extensions
• For browsers, enabling DOM or Virtual Object mode
  would be configured to identify objects

32
(No Transcript)