IPMA and SolutionsIQ

1
Welcome

• IPMA and SolutionsIQ
• Professional Event
• Testing, Testing, 123
• Improving software quality -- one bug at a time

2
Agenda

• Building the Test Framework
• Jan McCollum, SolutionsIQ
• Break
• Practical Panel Discussion
• Cheryl Hainje AFRS Product Manager, OFM
• Dotti Lane QA Project Manager, OFM
• Tim Vessey POS Project Manager, LCB
• Stein Wang Quality Assurance Lead, SolutionsIQ
• Break
• Testing Templates Checklists

3
SolutionsIQ Overview

• SolutionsIQ is a full-spectrum IT services
  company
• 25 years of technology services and solutions
• 400 consultants
• Corporate headquarters (Bellevue, WA)
• Professional Services (Bellevue, WA)
• Oregon Branch Office (Lake Oswego, OR)
• 8 years of serving the State of WA
• DOC, AOC, LCB, DNR, DOL, LNI, DSHS

4
SolutionsIQ Expertise

• Professional Services Division
• Consulting and Analytical Solutions
• Project management
• Assessments and feasibility studies
• Design and architecture roadmaps
• Development and Test Solutions
• Full life cycle development projects
• Custom application development
• EAI, portals, and business intelligence
• Quality assurance and testing solutions

5
Building the Testing Framework

• Jan McCollum
• Manager, Quality Assurance and Testing Solutions

6
Setting Goals

• Knowing WHAT you want is as important as knowing
  how to get it
• Defining the vision
• Defining the timeline
• Gaining acceptance and buy in

7
Defining the Vision

• To define the vision look at what came before
• What went well
• What went badly
• What now
• Where do you want to go

8
Testing vs. Quality Assurance

• Testing is about finding bugs
• Quality Assurance is about preventing them!

9
Quality Assurance

• Takes time
• Is about the overall effort including
  development
• Methodologies can be very formal

10
QA Applied to Testing

• Quality assurance principals applied to the
  testing effort will produce higher quality work

11
Establishing a Timeline

• The 6 month / 1 year / 3 year plan
• Implement processes and strategies that give the
  best return on investment

12
Quality Testing Roadmap

• After the goals and objectives are complete, make
  them real by publishing the quality testing
  roadmap

13
Quality Testing Roadmap

• Roadmap should include
• Test team structure
• Communications plans
• Test processes
• Test procedures

14
Quality Testing Roadmap

• Test scope
• Test dependencies and impacts
• Automation transition plan
• Test deliverables

15
Gaining Acceptance and Buy In

• Development
• Business management
• Project management
• IT management
• Customer/product support

16
Making it Happen!

• Organizational structure
• Qualified candidates
• Roles and responsibilities

17
Test Planning

• The master test plan a one-stop shopping guide
  for your project
• Contents
• Contributing documents
• Sign-off procedures

18
Test Planning

• Test matrix and test suites
• Detailed test steps
• Pass/Fail results
• Tester who performed tests

19
Test Planning

• Test case design what is a good test case?
• Accurate tests what its designed to test
• Repeatable, reusable has a life after this
  release
• Economical no unnecessary steps

20
Test Planning

• Test case design
• Traceable to a requirement
• Appropriate for test environment, testers
• Self-standing has enough information for anyone
  to run

21
Test Planning

• Test case design How to make good test cases
  better
• Setup, environment, data
• Steps, actions and expected results
• Use active voice in expected results
• System displays this, does that
• Simple, conversational language

22
Test Planning

• Test case design Why work to improve test cases?
• Productivity less time to write and maintain
  cases
• Testability less time to execute them
• Scheduling better reliability in estimates

23
Defect (Bug) Management

• Deciding upon a tool
• Easy of configuration
• Ability to add/change fields
• Reporting capabilities
• Integrated solution

24
Defect (Bug) Management

• The bug lifecycle
• Who can create bugs
• Who can assign bugs
• Who can close bugs

25
Defect (Bug) Management

• The bug triage meeting
• Purpose and who should go
• Reporting
• Determining a trend
• Bug metrics
• Number of bugs found
• Bugs found in production vs. test cycle

26
Moving On

• Improving the process Requirements traceability
• Test cases for each requirement
• Requirements matrix
• Tracing requirements to defects

27
Moving On

• Improving the process Risk-based testing
• You cant test everything so test what is
  important
• The risk list and how to use it to drive test
  strategy

28
Broadening Your Scope

• Build verification testing
• Also called smoke or acceptance tests
• Is a subset of the major functional areas
• Integration testing
• Testing the entire system

29
Broadening Your Scope

• Compatibility testing
• How application works with other apps
• Configuration testing
• Testing on different configurations
• Setup testing
• Testing the installation
• Regression testing
• Verify if bug fixes are successful

30
Broadening Your Scope

• Black box testing
• White box testing
• Grey box testing

31
Improving Quality

• Testing metrics measure your success
• Bug tracking metrics
• Number found
• Number found per component
• Daily bug find rate

32
Improving Quality

• Test case effectiveness
• Metric Test case effectiveness test case
  effectiveness bugs found in test/total found
  100
• Test coverage
• Metric Test coverage (absolute) tests
  conducted/total tests 100

33
Improving Quality

• Test team performance
• Metric Test process effectiveness test process
  effectiveness bugs fixed/bugs found 100
• Metric Planned days vs. actual days in test

34
Improving Quality

• QA and test involvement early!
• Design reviews
• Why testers should attend
• Develop and use checklists
• Project closeout meetings
• You should have them

35
Questions?

• For additonal information, email
• JMcCollum_at_SolutionsIQ.com