Automated Testing in an Agile Environment

1
Automated Testing in an Agile Environment

• Presented by Ken Arneson
• Developed by Bob Crews and Ken Arneson
• Checkpoint Technologies, Inc.

2
About Checkpoint Technologies

• Checkpoint Technologies, headquartered in Tampa,
  FL, is a solutions provider which focuses
  strictly on Quality Assurance and Software
  Testing with an emphasis on Functional,
  Performance, and Security testing.
• We provide leading edge software as VARs as well
  as consulting, training, mentoring, and staff
  augmentation.
• We are an HP Reselling Partner and Certified
  Training Partner with a new training facility
  located in St. Petersburg, FL.

3
What is Agile?

• In the late 1990s several development
  methodologies began to get increasing public
  attention.
• They all emphasized
• Close collaboration between the programmer team
  and business experts
• Frequent delivery of new deployable business
  value
• Tight, self-organizing teams
• Ways to craft the code and the team such that the
  inevitable requirements (documentation) churn was
  not a crisis.

4
Agile Development Methodologies

• Extreme Programming (XP)
• Crystal
• Adaptive Software Development (ASD)
• Scrum
• Feature Driven Development (FDD)
• Dynamic Systems Development Method
• (DSDM)
• XBreed

5
The Agile Manifesto

• We are uncovering better ways of developing
  software by doing it and helping others do it.
  Through this work we have come to value
• Individuals and interactions over processes and
  tools
• Working software over comprehensive documentation
  
• Customer collaboration over contract negotiation
• Responding to change over following a plan
• SOURCE http//www.agilemanifesto.org/

6
What is Agile Testing?

• Its a testing practice
• that adheres to the Agile Manifesto treating
  the development team as the customer of testing
• for projects using agile methodologies

7
Testing in Agile

• While Agile was a methodology that was created
  from a development point of view, testing was not
  ignored
• The majority of the focus, however, was placed on
  unit testing and exploratory testing using the
  assembled team
• Some Agile proponents did not see (or argued
  against) the need for testing specialists on an
  assembled team

8
A Few Years Later

• Most organizations implementing Agile soon came
  to value and see the need for a specialized
  testing skillset
• Testers are now almost always included in the
  cross-functional Agile teams
• In some instances, these testers will also take
  on non-testing tasks when cross-skill set
  participation is encouraged

9
Why Testing Matters in Agile

• Testing is a discipline, even developers
  following methodologies like test driven
  development often lack good testing skills
• The success or failure of code to meet
  requirements or user stories must still be
  executed and documented in an auditable fashion
• Even with business value and cake slices being
  added in small increments, code still needs to be
  regression tested

10
Some Agile Practices

• Test-First Programming Developers write unit
  tests before coding
• Acceptance Testing Acceptance tests are used to
  verify the completion of user stories
• Short Iterations and Releases Quick
  rapid-fire development and testing
• User Stories Short descriptions of features
  that need to be coded and tested
• SOURCE Bret Pettichord 2002 Agile Testing.
  What is it? Can it work?

11
More on User Stories

• A user story is a very high-level definition of a
  requirement
• Just enough information for developers and
  testers to produce a reasonable estimate of the
  effort to implement/test it.
• Small and can be implemented quickly
• Often written on a paper medium (index cards,
  post-its, etc.)

12
User Story Examples

• Customers can book flights online.
• Air fare can be paid online via credit cards.
• Customers can print boarding pass online.
• Ticket agent can update customers reservation via
  terminal.
• Gate agent can modify customers seat assignment
  via terminal.

13
User Story Card
14
Automated Testing In Agile

• Automated testing still fills an important role
  in the world of Agile Development
• The benefits of automation still apply
  repeatability, consistency, better coverage
  through data driving
• There is also the regression testing that will be
  required as an iterative approach is developed

15
Issues Implementing Automation in Agile

• Automation is a development exercise, and
  therefore requires planning and time to develop
• Agile projects or iterations are often less
  than a month in length (less planning and even
  less time)
• Agile Developers often face the same issues when
  they try to determine how much architecture and
  design to do up front
• Agile projects are often developed based on a
  system metaphor or very loose design of the
  product architecture which evolves through
  iterations (solid rqts are not considered
  essential up-front)

16
The System Metaphor and Automated Testing

• When Agile teams are developing the object model
  or system metaphor, automated testers should be
  using the same time to develop an automated
  system testing metaphor which will be used to
  create a design approach to test scripts so they
  can later be put together for regression
• As iterations are planned, time for development
  of underlying infrastructure for automated tests
  should be included in the stories

17
Quick Hit Automation

• One way automated testers can add immediate
  benefit to an Agile project is to look for
  opportunities for quick hit tasks
• These tasks may include data creation scripts,
  scripts which expand on developers test driven
  development examples and expand the testing
  through data driven techniques
• Any other task, which will be repeated, that
  looks quickly automatable

18
A Philosophy of Reacting

• Agile is built on a philosophy of being able to
  quickly adapt and react to changes in
  requirements
• Automated test scripts should be developed with
  the same thoughts in mind
• Automated testers should look for ways to
  abstract code into functions and external data
  sources to allow for quick changes to the
  execution steps and data
• Data drive expected results as well as input and
  output data

19
Use the Stories Negative Testing

• Often when stories are developed in Agile, they
  are written with acceptance criteria
• These criteria are often simple and based on the
  language of the original story
• Sometimes lost in the Agile mix are the negative
  scenarios that testers conduct every day
• Having an underlying data-driven framework that
  can react to changes in interface and
  requirements can also allow for injection of
  negative data and highlight these issues

20
Regression Testing

• Agile development teams need to communicate and
  coordinate when building a system
• Automated testers on the teams should coordinate
  as well, to agree on design principles which
  allow for scripts to be integrated for regression
  testing
• Regression testing should take place in the
  integrated application environment and run as
  cake slices are pushed into production

21
Agile versus Documentation

• Agile often eschews documentation, but test
  assets (test cases, test data, test results,
  defect reports, etc.) are a large part of audits
  and required for liability reasons (CMMI
  compliance)
• Automated test repositories can be customized to
  help capture and store these assets and consume
  less enterprise time and resources while still
  allowing for regulatory satisfaction

22
Quality Center and Agile

• Quality Center TestDirector (or any test
  management solution) can be utilized for Agile
  projects
• Define user stories as
• Requirements
• Test Cases
• Keep high-level and small (dont lose the
  rapid-fire benefits of Agile)
• Use customization features of solution to add
  user-defined fields for stakeholders, estimate,
  priority, etc

23
Benefits

• Reusability
• Traceability
• Requirements ? Tests ? Execution ? Defects
• Impact analysis If a user story changes how
  many tests will that affect and whats the
  estimate to implement it?
• Audit trail (for CMMI compliance)
• Metrics Analysis

24
Summary

• Agile Development Its all about individuals,
  working software, customer collaboration, and
  responding to change (rapidly)
• Agile evolved over the years from we dont need
  testing specialists to OK maybe we do and,
  thus, Agile Testing came to be
• User stories are an important element of Agile
• There is a place for automation in Agile (the
  benefits still exist)
• Test planning tools can be utilized with the
  Agile approach

25
Questions?
26
Thank you!
Ken Arneson Checkpoint Technologies, Inc. Email
karneson_at_checkpointech.com Cell [phone]