Test Driven Development - PowerPoint PPT Presentation

1 / 48
About This Presentation
Title:

Test Driven Development

Description:

Like testing any other kind of codes, graphical user interface ... The Burke Porter Dyno Host Project. http://atomicobject.com/pages/Burke Porter Dyno Host ... – PowerPoint PPT presentation

Number of Views:430
Avg rating:3.0/5.0
Slides: 49
Provided by: Mak73
Category:

less

Transcript and Presenter's Notes

Title: Test Driven Development


1
Test Driven Development Testing the Graphical
User Interface
  • Prepared by
  • Yibo Sun
  • Soha Makady

2
Agenda
  • What is GUI testing?
  • Generation of test cases
  • Running test cases
  • Design approaches for TDD of GUI
  • GUI TDD problems
  • Programmatic GUI Testing
  • Conclusion

3
Definition
  • The process of testing a software product with
    graphical user interface
  • Like testing any other kind of codes, graphical
    user interface testing requires test cases
  • All GUI functionalities of the SUT should be
    covered by test cases

4
Importance
  • GUI accounts for as much as 60 of the total
    application code
  • Use in safety critical systems

5
Benefits
  • Safety
  • Robustness
  • Usability

6
Difficulty
  • Scope huge number of GUI operations, states
    which need to be tested
  • Sequence GUI operations are dependent and rely
    on other operations, when the sequence is very
    long, the test case will be very long and error
    prone.
  • Regression Test GUI may change significantly
    across versions, test cases need to be changed
    repeatedly

7
Coverage Criteria
  • Conventional software testing uses coverage
    criteria which define a set of rules telling the
    type and amount of code to test
  • It is hard to map GUI states and events to
    underlying code
  • GUI states
  • It is hard to determine the most important subset
    of all possible test cases

8
Regression A.F. Memon and M.L. Soffa,
Regression testing of GUIs, Software
Engineering Notes, Sep. 2003
  • Test cases become obsolete
  • Recognize GUI components by names or IDs instead
    of their placements
  • 74 of test cases are broken per version (study
    on Adobe Reader 4.0 and 5.0)

9
Generation of Test Cases
  • Manual
  • Automated
  • Other Monkey Testing

10
Automated Test Case Generation
  • Approaches for automated test case generation
  • State machine modelling
  • Automated Planning System
  • Genetic Algorithm

11
Automated Test Case Generation - State Machine
Modelling
Event/Transition 1
State 1
State 2
Event/Transition 2
Event/Transition 3
State 3
State 4
12
State Machine Modelling
  • Modelling GUI as a state machine with states and
    events (transitions)
  • FSM (Finite State Machine)
  • Scaling problem
  • VFSM (Variable Finite State Machine)
  • Assertion for GUI states to be inserted manually,
    at points determined by test designers

13
Automated Planning System
  • Based on an artificial intelligence algorithm
  • Defines
  • an initial state
  • a goal state
  • a set of operations
  • A test plan is determined as a path from an
    initial state to a goal state
  • Testers focus on what function to test instead of
    how to test a function which is determined by the
    test plan path
  • Limitation Manually decide the tasks (initial
    and goal states) to be tested and poorly chosen
    tasks can produce a test suite without adequate
    GUI testing coverage

14
Genetic Algorithm
  • Generate tests simulating novice usage, a more
    random path achieves more system states
  • A gene is a path through GUI states transitions
    generated for some task
  • Choose an initial gene to mimic a novice path.
  • More genes are then generated by modifying and
    extending the initial gene
  • Genes that complete tasks best are kept, scored
    by some criteria as best novice paths.
  • Surviving genes are replicated and the natural
    selection process is repeated with more genes.
  • Eventually, multiple test cases to mimic novice
    paths (genes) are produced
  • Limitation The initial gene has to be chosen
    manually, and thus this approach largely relies
    on the human experts ability to choose the
    initial gene.

15
What is a Good GUI Suite? Xie and Memon,
ISSRE, 2006
  • Three experiments were conducted to answer these
    questions
  • How many test cases should be generated?
  • What should be the length of each test case?
  • What events should be put into a test case?
  • Tested applications TerpWord, TerpSpreadSheet,
    TerpPaint, and TerpCalc
  • The primary measured variable is the
    fault-detection effectiveness of a test suite

16
Used Terminology in Experiments
  • Test case A sequence of events
  • Fault A mismatch between the expected and actual
    GUI states (asserted for every GUI property in a
    certain GUI state)
  • Fault-detection effectiveness the number of
    unique faults detected by the test cases in the
    suite

17
Effect of Test Suite Size
For all suites Fixed test case length Fixed
event composition
Largest observation
Upper quartile
Median
Lower quartile
Smallest observation
Fault Detection Effectiveness vs. Test Suite Size
for TerpCalc
18
Effect of Test Case Length
For all suites Fixed suite size Fixed event
composition
Fault Detection Effectiveness vs. Test case
length Size for TerpCalc
19
Effect of Event Composition
  • Fixed test suite size
  • Fixed test case length
  • Results revealed that absence of an event (that
    interacted with a functional unit ) in a test
    suite directly effects the detection of a fault
  • Example When running a test suite that does not
    contain a click event on a form full of
    buttons, not all faults are detected

20
Running GUI Test Cases
  • Generated test cases (manually/automatically) can
    be run in two ways
  • Playing scripts using Record playback
    technique
  • Programmatically send commands/events using API
    functions

21
Record and Playback
  • Capture events and put states assertions in
    scripts
  • Playback events in scripts
  • Compare expected state in scripts to actual UI
    state to determine the correctness of execution
  • Scripts can be edited when UI is changed

22
Record and Playback Tools
  • Java AWT and Swing
  • Marathon
  • Jacareto
  • Abbot(Costello)
  • Web
  • Selenium

23
Record and playback tools - Comparison
24
Selenium
  • Runs directly in a browser
  • Supports web page testing

25
Demo
26
Drawbacks of Record playback
  • Requires the GUI to be built first
  • Assumes the GUI under test can already be
    functional
  • The generated test scripts cannot keep up with
    design changes

27
GUI Architectures for TDD
  • Following, are two proposed GUI architectures to
    enable GUI unit testing
  • The Humble Dialog Box
  • Presenter First

28
The Humble Dialog BoxMichael Feathers, Object
Mentor, Inc. 2002
  • Creates smart objects that contain the functional
    behavior of a particular GUI object
  • Adds a thin GUI view object (a humble dialog) to
    display the smart objects information (and has
    no application logic)
  • Tests the smart objects mainly
  • As a result, the application behavior will be
    covered by unit tests, and the display code (GUI)
    will not require complex GUI tests

29
The Humble Dialog -ExamplePaul Hamill, Unit Test
Frameworks, OReilly , 2004
  • Consider a Library GUI application that allows
    the user to add new books.
  • GUI design sketch for the AddBook dialog Window
  • The smart object AddBook and its humble dialog
    (view class) AddBookView

30
The Humble Dialog Box Example (Contd)
  • Unit test for the smart object AddBook
  • public class AddBookTest extends TestCase
    public void testAddBook( )
  • Library library new Library( )
  • AddBook addBook new AddBook(library)
    addBook.add(" Java ", "Carl Sagan")
  • assertNotNull( library.getBooksByTitle(" Java
    ") )

31
The Humble Dialog Box Example (Contd)
  • Unit test for the humble dialog
    AddBookViewTest.java
  • public class AddBookViewTest extends TestCase
  • public void setUp( )
  • public void testControlValues( )
  • AddBookView view new AddBookView( )
    assertEquals( "Add Book", view.getTitle( ) )
    assertEquals( "", view.titleField.getText( ) )
    assertEquals( "", view.authorField.getText( ) )
    assertEquals( "Add", view.addButton.getText( )
    ) assertEquals( "Cancel", view.cancelButton.getT
    ext( ) )

Testing application GUI
32
The Humble Dialog Box Example (Contd)
  • Unit test for the humble dialog
    AddBookViewTest.java
  • public class AddBookViewTest extends TestCase
  • public void testAddButton( )
  • view.titleField.setText(Java")
    view.authorField.setText("Carl Sagan")
    view.addButton.doClick( )
  • assertEquals(1, library.getBooksByTitle(Java").
    size( ))

Testing application functionality
33
Drawbacks of the Humble Dialog Box Approach
  • Tested GUI components have to be visible as
    non-private instance variables, or through getter
    methods, which conflicts with the normal design
    convention, and increases GUI code.
  • GUI components do not operate over the actual
    event mechanism, thus not all errors will be
    revealed.
  • Example Invisible and disabled components can be
    addressed in the unit tests.

34
Presenter First (PF)Micah Alles et al.,
Presenter First, AGILE06
  • A design and development technique to produce
    fully tested GUI applications following TDD
  • A variant of the Model View Presenter (MVP)
    design pattern

35
Model View Presenter Pattern
  • Model The data and logic for business aspects
  • View what the user sees and interacts with
  • Presenter Behavior that corresponds to customer
    stories (application writing)

Any interaction between View and Model takes
place through Presenter
36
MVP Variant for PF
Dashed lines are events, solid lines are object
messages
  • Model and View are isolated from each other
  • Model and View are members of Presenter, and do
    not know of its presence
  • Presenter works with Model and View interfaces
  • Testing the View can be automated using tools, or
    done manually

37
MVP Variant for PF (Contd)
38
PF Example Presenter Class
  • public class TBPresenter
  • private ITBModel m_model
  • private ITBView m_view
  • public TBPresenter(ITBModel model, ITBView view)
  • m_model model
  • m_view view
  • m_view.SubscribeItemDrag(new
    UpdateDelegate(ItemDrag))
  • m_model.SubscribeToolboxItemsChanged(new
  • UpdateDelegate(ToolboxItemsChanged))

39
The Burke Porter Dyno Host Project
http//atomicobject.com/pages/BurkePorterDynoH
ost
  • Done in Presenter First Style
  • Status
  • 6 developers for 9 months
  • One week iterations
  • 99 estimation accuracy over project lifetime
  • Full test driven development (5200 automated
    unit tests)
  • Project A Visually configured automotive test
    and measurement application
  • But
  • The level of experience of the developers is not
    mentioned as a factor in the estimates

40
GUI TDD Unit Testing Problems
  • Difficulty in verifying some behaviors
  • Example Check that the dialog box pops up
    modally showing the alert icon and the warning
    message in red.)
  • Using ONLY JUnit has its limitations
  • Tests may start before the application starts as
    they run in a separate thread
  • When starting the application from within the
    unit test, it may take time loading, thus the
    test may fail due to the time out waiting for the
    GUI
  • There is no easy way to find and interact with
    GUI components except through Java Robot class

41
java.awt.Robot Class
  • Emulates end-user interactions using the keyboard
    and/or mouse.
  • Limitations
  • Takes control of the mouse and keyboard, so the
    user cannot do anything else on the computer
    while the test is running
  • Tests are fragile, where any change in the layout
    will break them
  • The user would have to add getter methods, or
    loop through the component hierarchy to reference
    a certain component.

42
Programmatic GUI Testing Tools (Comparison)
43
Abbot Java GUI Test Framework
  • Supports record and playback (Costello) and
    programmatic handling
  • Supports both AWT and Swing components
  • Based on Java AWT Robot
  • Main components
  • A single finder class
  • A tester class for one/several components

44
Demo
45
Conclusion
  • Record and playback tools cannot be used when
    developing a GUI following TDD
  • Designing an application following the PF style,
    makes it easier to follow a TDD approach.
  • Testing the application GUI (view) can be done
    through a programmatic GUI testing tool that can
    find components regardless of their
    layout/position
  • There is still no clear criteria for determining
    the GUI testing coverage (e.g. no tool like
    Coverlipse for GUI testing coverage)
  • Unit testing GUI scenarios is complicated due to
    the need to simulate many actions to reach the
    initial state for a certain test case

46
References
  • A.F. Memon, GUI testing pitfalls and process,
    Computer, v 35, n 8, Aug., pp. 87-88, 2002
  • A.F. Memon and M.L. Soffa, Regression testing of
    GUIs, Software Engineering Notes, v 28, n 5,
    Sept., pp. 118-27, 2003
  • R.K. Shehady and D.P. Siewiorek, A Method to
    Automate User Interface Testing Using Variable
    Finite State Machines, Proc. 27th Ann. Int'l
    Symp. Fault-Tolerant Computing (FTCS '97), pp.
    80-88, June 1997
  • A.F. memon, M.E. Pollock, and M.L. Soffa,
    Hierarchical GUI test case generation using
    automated planning, IEEE Transaction on Software
    Engineering, Vol. 27, Issue 2, Feb., pp. 144-55,
    2001.

47
References (Contd)
  • D.J. Kasik and H.G. George, Toward Automatic
    Generation of Novice User Test Scripts, Proc.
    Conf. Human Factors in Computing Systems Common
    Ground, M.J. Tauber, V. Bellotti, R. Jeffries,
    J.D. Mackinlay, and J. Nielsen, eds., pp.
    244-251, Apr. 1996
  • Jacareto, http//jacareto.sourceforge.net/
  • Abbot, http//abbot.sourceforge.net/doc/overview.s
    htm
  • Selenium, http//www.openqa.org/selenium/.
  • Marathon, http//www.marathon.com
  • JFCUnit, http//jfcunit.sourceforge.net/
  • MVC and MVP patterns http//www.martinfowler.com

48
Thank you
Write a Comment
User Comments (0)
About PowerShow.com