Capture and Replay

1
Capture and Replay

• Often used for regression test development
• Tool used to capture interactions with the system
  under test.
• Inputs must be captured outputs may also be
  recorded and (possibly) checked.
• Examples
• Telelogic Tau save command history in simulator
• GUI testing tools
• Capture requires a working system to be available
  already!

2
Testing on Replay

• During the replay of events, there are various
  output checking possibilities
• Manual user has to watch the system for
  anomalies
• Complete all outputs were recorded during
  capture, and system must reproduce them
  exactly.
• Exactly to the level of detail of the
  recording.
• Checkpoints system output is only checked at
  certain points for specified values.
• These could be inserted manually or automatically.

3
The capture record

• Inputs, outputs, and other information needed to
  reproduce a session with the system under test
  need to be recorded during the capture process.
• Examples
• General information date/time of recording,
  etc.
• System start-up information
• Events from test tool to system
• Point of control, event
• Events from system to test tool
• Checkpoints / expected outputs
• Time stamps

4
Event-driven architecture

• Basis of GUI frameworks
• Various controls create events when they are
  created, activated, modified, deactivated, or
  disposed.
• Input devices create events as per their
  functions key pressed, key released, mouse
  moved,
• Events are sent to an event dispatcher
• Listeners register with the event dispatcher to
  receive events
• Listeners specify the type of events in which
  they are interested, and which method should be
  called when the event occurs
• When an event is received by the event
  dispatcher, notifications are sent to each
  registered listener for the type of event.
• Each listener has their event notification method
  called in turn.

5
Event-driven architecture
registration
Event dispatcher
Event listener 1
Event source 1
notifications
Event listener 2
Event source 2
event instances
6
Integrating a Capture and Replay tool

• During the capture process, the tool will
  register as an event listener
• Event notification method for the tool will
  record the details of all events that occurred.
• During the replay process, the tool will register
  as an event source (possibly also as a listener)
• For mouse and keyboard events, the tool has to
  substitute for the actual devices as the event
  source.
• Replay events should be initiated at the same
  relative time as during the capture.
• Other controls issue events as usual (e.g. GUI
  button deactivated)

7
Capture
Event dispatcher
Event listener 1
Event source 1
Event listener 2
Event source 2
Capture and record
8
Replay
Event dispatcher
Event listener 1
Event source 1
Event listener 2
mouse keyboard replaced
Replay tool input
9
Integration Alternatives

• Create native system events
• Example The Robot class in Java will, for
  example, actually move the mouse cursor on the
  underlying platform, instead of just recreating
  the mouse event that would be generated.
• Analyze screen images
• A screen capture of a tools GUI can be stored
  while the tool is running, analyzed for widgets,
  and then used for replay.

10
Tests

• Alternatives
• During the capture process, the tool may record
  events, and then compare the results during
  playback.
• At test checkpoints, the tool can ask various
  controls for their state and record it.
• Examples
• Contents of text fields
• GUI radio button properties
• enabled / disabled
• selected / not selected

11
Integration options

• Compile tool classes and application together
• Similar to JUnit approach
• User has to determine object instances that
  should be monitored, and then register instances
  of tool classes as event source and listeners
• Connections are also set up to query controls as
  to their states.
• Provides better integration, more stable replays
• Tester has to learn how to use tool classes
• Use virtual machine registration (when there is a
  VM)
• Tool has to discover objects of interest

12
Example Jacareto

• Java-based open source GUI capture replay tool
• Authors C. Spannagel et. al.
• Web site http//jacareto.sourceforge.net
• Tool sub-parts
• Jacareto event record and playback
• Picorder command line interface
• CleverPHL graphical user interface (for
  Jacareto)

13
Jacareto

• Uses virtual machine registration
• Batch file and XML script used to start Java
  virtual machine, start the test tool, and load
  the applications classes.
• Main method of application called by test tool as
  required.
• Jacareto has to discover all instances of
  subclasses of Component (including JComponent for
  Swing)

14
Event records

• Stored as XML
• Information recorded
• Type of event mouse move, key pressed, etc.
• Includes event-specific information X-Y
  location of mouse cursor, which key was pressed,
  
• Relative time from previous event
• Needed to keep events synchronized
• Example
• ltKeyEvent procTime"0" duration"240"
  source"SampleDialog_(1).JRootPane_(1).JLayeredPan
  e_(1).JPanel_(1).JTextField_(1)"
  class"javax.swing.JTextField"uuid"89a95465-8cae
  -467b-bc48-6422a63bcfc4" ID"401"
  component"null" root"SampleDialog_(1)" xPos"0"
  yPos"0" width"0" height"0" when"1175219983088"
  isConsumed"false"gt
• ltKeyInfo keyCode"65" keyChar"A"
  modifiers"1" /gt
• lt/KeyEventgt

15
XML record displayed for user
16
Tests

• At various user-defined checkpoints, the state of
  a control can be tested.
• Example Test if a text field contains the
  specified text
• Also specify that we want to stop if there is an
  error (i.e. ignore error is false), and that we
  do not want to correct the state of the control
  if it does not conform to the record.
• ltJTextComponentTest component"MainFrame_(1).JRo
  otPane_(1).JLayeredPane_(1).JPanel_(2).JTextField_
  (1)" isRegExp"false" isCorrecting "false
  isEnabled"true" isIgnoring"false"
  hasFocus"false"gtHello, Alanlt/JTextComponentTestgt

17
UI for Test creation
18
Test reports
19
Example Marathon

• Capture and playback tool for testing Java/Swing
  graphical user interfaces.
• www.marathontesting.com
• Uses the Jython (Python for Java) scripting
  language to store test cases.
• Test cases can be captured interactively, or
  written as Jython scripts.
• Assertions can be added to check the state of
  various components.
• Uses test runner that is a non-standard version
  of JUnit.

20
Sample Jython test script

• useFixture(default)
• def test()
• java_recorded_version '1.5.0_11'
• if window('GUI demo')
• click('Click me')
• assert_p('TextField', 'Text', 'Hello, Alan')
• close()

21
Running a Marathon test
22
Test report
23
Eclipse TPTP Auto GUI recorder

• Capture and replay of GUI events and actions
  within the Eclipse IDE.
• Test suite is set up as an Eclipse plug-in
  project that integrates itself with the IDE.
• Any application that is also an Eclipse plug-in
  can be tested with the auto GUI recorder.
• The plug-in mechanism exposes how Eclipse can
  interact with the plug-in, and the auto GUI
  recorder uses this information to send and
  receive events to/from the plug-in.

24
Pros and cons

• Advantages
• Requiring the test application to be an Eclipse
  plug-in allows for better knowledge of the
  environment for the record / replay tool.
• Less difficulty and more reliability for locating
  user interface widgets.
• Tool can call methods directly, instead of
  simulating mouse operations and keystrokes.
• Disadvantages
• Cannot be used for stand-alone applications
• Requires the use of Eclipses Standard Widget
  Toolkit (SWT), instead of Javas Abstract Widget
  Toolkit (AWT) or the Swing toolkit.

25
Additional features

• Verification hooks
• Points at which tests can be inserted to check
  the state of various widgets.
• Variable substitution
• Allows script to substitute variable values for
  variable names during run-time.
• Example specification of temporary directory,
  relative file names

26
TPTP Auto GUI recorder
27
Action script

• ltmacro version"1.0"gt
• ltshell descriptive"Test - AutomatedGUISuite -
  Eclipse SDK"
• referenceId"0" return-code"-1"gt
• ltcommand descriptive"Project..."
  type"select" referenceId"1"/gt
• ltshell descriptive"New Project"
  referenceId"2" return-code"0"gt
• ltcommand descriptive"Java Project
• type"item-select"
  referenceId"3"gt
• ltitem referenceId"4"/gt
• lt/commandgt
• ltcommand type"focus" referenceId"3"/gt
• ltcommand descriptive"Next ampgt"
• type"select" referenceId"5"/gt
• ltcommand type"modify" referenceId"6"gt
• lt!CDATAA Java Projectgt
• lt/commandgt
• ltcommand descriptive"Next ampgt"
• type"select" referenceId"5"/gt
• ltcommand descriptive"Finish"
  type"select" referenceId"7"/gt
• lt/shellgt

28
TPTP Auto GUI recorder test setup
29
TPTP Auto GUI recorder execution log
30
TPTP Auto GUI recorder execution log details
31
TPTP API recorder

• Another tool being developed for Eclipse is an
  API recorder.
• Essentially, this performs capture and replay
  without having a user interface.
• Uses the TPTP profiler.
• As methods are called, the parameter values are
  stored on entry, and the values that are returned
  are stored as well.
• After execution, the saved parameter values are
  used to create JUnit test cases.
• Still in the technology preview phase.