Software Quality Management Today

1
Software Quality Management Today

• Becky Howard
• Tivoli Systems
• July 1999

2
My Background

• 22 years in high tech quality management and
  project management
• Liberal arts degree from Michigan State
• Certified Quality Manager and Certified Software
  Quality Engineer with the American Society for
  Quality
• 5 years Board of Directors of Austin Quality
  Council

3
Progression of Software Quality

• Ad hoc efforts
• Process oriented approach to testing
• focus on building quality in product through
  testing
• Process oriented approach to software development
• Corporate approach to quality in all key business
  processes

4
Pyramid of Quality Efforts
Main tasks Debugging, functional testing Test
planning, test groups/lab, functional testing,
release processes Verification and Validation
focus, quality assurance planning
Company examples Software Artistry Datapoint Ti
voli Raytheon Northern Telecom
Start-Up
Gaining Momentum gt 3000 employees
Big Business 5000 employees
5
Minimum Efforts

• Testing is a big investment for a small company
  betting on new product revenue
• Emphasis is on black-box testing to determine
  points of failure
• Configuration management of product is loose
• Accidents happen, and so do fixes

6
Moderate Efforts

• A too-small group of testing warriors facing
  throughput problems -- more code than time to
  test
• Push back on developers with some quality
  requirements
• A few basic measurements, usually defect-focused
• Growing product support costs

7
Comprehensive Efforts

• Working accountability for product quality
  through the entire product development
  organization
• Emphasis on planning, planning, measuring,
  improving
• Problem prevention rather than corrective action
  focus for the company
• Driven by customer requirements
• Internal service level agreements to assure key
  business processes are optimized

8
Software Quality Practices

• Success in any competitive marketplace requires
  planning for quality initiatives
• Each project should have a quality plan
• addressing all phases of the project
• driven from customer requirements
• focusing on problem prevention
• The software quality function should add value to
  the overall project effort
• The key word is ACCOUNTABILITY

9
Quality Assurance Tasks

• Preparing Software Quality Assurance Plans and
  associated documentation
• Providing review feedback on project plans,
  documents and specifications
• Auditing the processes used by development
• Monitoring the implementation of project plans
• Inspecting code and documentation
• Identifying preventive and corrective actions
• Testing, or witnessing testing
• Communicating overall status to management

10
Quality Planning

• Standards - even if company isnt compliant,
  SEIs CMM and ISO 9001 can add value and clarity
  to business ops
• System requirements
• Evaluating requirements for correctness,
  consistency, completeness and testability
• Evaluating interfaces
• Problem trend analysis
• Inspection, test, and review of deliverables
• Physical traces, source code, documentation
• Can also address performance, design architecture
  and re-engineering

11
Software Quality Requirements

• Reliability and performance
• Test case pass/fail criteria
• Requirements traceability
• Software baseline control
• Consistency from a user perspective

12
Software Inspection

• A best practices investment
• Code reviews driven by
• peer review
• mapping to requirements
• use of best practices
• continuous improvement
• documented agenda and results
• error detection and prevention

13
Robust Software Testing

• Unit testing performed by developer on smallest
  unit of code
• Functional testing focusing on robustness of a
  component
• Black box and white box testing
• Usability testing suitability for the customers
  needs
• System testing looking at how the components
  work together, evaluating packaging of all
  components included in a customer software
  product
• Integration testing with other products smooth
  interoperability with application environment
• Certification with new product releases
  ensuring it still works with new operating
  systems, databases, comm drivers, etc.
• Benefits of documented plans and results

14
Comprehensive Verification and Validation

• Assures that quality is built into the product
  through each stage of the project
• Goal is to identify defects as early as possible
  in the project
• definition phase
• development phase
• maintenance phase
• Plan for any type/size of development effort
• Include measurable criteria, inputs and outputs
  required
• Identify tools and techniques
• Identify roles and responsibilities
• Evaluate anomalies
• Assess modifications

15
The PDCA Cycle

• Plan - identify the area of improvement, collect
  and review underlying process data
• Do - implement process changes
• Check - gather and review data to assess effect
  of improvements
• Act - recycle process, continually implementing
  improvements document process, roles
  responsibilities

16
Understanding Internal Customer Needs

• Objective smarter, cheaper, faster product
  development
• Inputs and outputs to development processes
• Articulating expectations and requirements
• Cycle time and quality commitments
• Reduce or eliminate redundancy
• Improve testing approach with understanding of
  process hand-offs

17
Gathering Customer Feedback

• Methods for gathering feedback
• Surveys, Site visits, Focus groups, Support
  problem reports
• Value added
• Ability to prioritize platform testing
• Drive test cases based on real-life scenarios
• Understand the business impact of reliability,
  performance, and usability issues
• Improve basic inspection, review and test methods

18
Software Metrics

• A best practices investment
• Process Measures
• How effective is the test process?
• Cycle time management, cost management
• Product Measures
• Measure defects at all stages of detection
• Estimate defects that will appear in field
• Cost of Quality
• Activity-based cost accounting
• Cost of identifying problems after release

19
Problem-Solving

• A key skill for every role in product development
• Deciding when software product quality is
  acceptable to ship
• Business ramifications - bookings billings
• Customer impacts and opportunities
• First impressions
• Managing problems
• Planning patches and maintenance releases

20
Inter-Group Communication

• Working with the customer support organization
• Internationalization of software
• Hand-offs between development and testing
• Hand-off to manufacturing
• Accountability for integrity of product

21
Whats good about software quality management?

• Process-reliance
• Customer advocacy
• Emphasizes communication and analytical skills
• Broad product perspective
• Focuses on continuous improvement, growth and
  product/company success