Employing User Profiles for Regression Testing of GUIs

1
Employing User Profiles for Regression Testing of
GUIs
Penelope Brooks and Jaymie Strecker CMSC838P
Project Report May 9, 2006
2
Overview

• Need to test changes to a GUI-based app
• Regression testing - program families!
• Find failures before users do
• Point of view

Observe
General, automated technique
Model
Simulate
3
Problem GUI Regression Testing

• GUIs are important
• GUI testing is important
• GUI testing requires new techniques
• Event-driven paradigm
• Visual output
• Rapid evolution
• Which subset of state space to test?

4
Cost of Failures
_at_!
File -gt Open
Copy -gt File -gt Open -gt OK -gt Paste
Open -gt OK
Copy -gt Paste
5
Failures in GUIs

• Event interactions
• Modes
• Widget enabled/disabled
• Visual output
• Inconsistent with internal state
• GUI says you've saved your document... Have you?

6
Background GUI Input Domain
Input domain and test cases consist of event
sequences
Menu
Button
Text area
7
Background GUI Structure Model
Event flow graph (EFG)
8
Extension Probabilistic Model
Probabilistic event flow graph (PEFG)
0.02
1.0
P(ManageHelp,Bookmarks) 0.1 P(ManageTools,Book
marks) 0.3
9
PEFG Context
Automated process
User Profiles
Test Cases
Pass/Fail Results
PEFG Traversal
Replayer
Profiler
Verifier
vers.1
vers.2
vers.2
10
Capturing User Profiles

• Profiler tool
• Java reflection
• Adds event handlers to visible components
• Buttons
• Menus and menu items
• Text areas
• Records event sequences

11
Traversing the PEFG
Input Profiles, maxHistory for h from 1 to
maxHistory Add each length-h subsequence in
Profiles to Histories for each event sequence s
in Histories for each unique event e in
Profiles Distributions(e, s) lt- P(e s) for
each event sequence s in Histories e_max lt-
argmax Distributions(e, s) Add an event sequence
containing (s,e_max) to TestSuite Output
TestSuite
All subsequences in Profiles
Probability distribution of each (history,event)
pair
For each history, most likely event to follow
12
Running the Test Cases

• GUI Testing frAmewoRk (GUITAR)
• Replayer tool
• Verifier tool
• http//www.cs.umd.edu/atif/GUITARWeb/

13
Empirical Evaluation

• Compare 3 types of test suites
• Event interaction-coverage adequate
• All EFG edges
• User profiles, unaltered
• Simulated user profiles
• PEFG traversal

14
Application Tested

• TerpWord 3.0
• 'Correct' version (CMSC435)
• Fault-seeded versions
• How many can test suites detect?
• Real profiles
• 15 CMSC435 students
• Pretend profiles
• Test cases written by CMSC435 students

15
User Profile Test Cases vs. PEFG Test Cases

• Test cases directly from user profiles found no
  failures
• Test cases from PEFG model failed approximately
  10-20 of the time

16
Test Suite Size

• PEFG Test Suites averaged 193 test cases
• User Profile Test Suites averaged 536 test cases

17
Summary and Conclusions

• Regression testing
• Testing GUIs
• Testing with user profiles

Observe
General, automated technique
Model
Simulate
18
Questions?
19
PEFG Simple Cases
0.5
0.3
0.5
0.09
0.4
0.01
0.1
0.1
h 1 Edge probabilities
h 0 Node probabilities
20
PEFG Simple Cases
0.2 0.2 0.2
0.4
h infinite Zero probability for event sequences
that have never been observed
21
(No Transcript)