Automating Software Quality Control

1
Automating Software Quality Control

• www.qa-systems.com

2
QA Systems Mission and Focus

• mission..
• is to improve our customers software health.
• focus is to
• develop software tools to assess, support and
  monitor the health (quality) of software
  applications developed by our customers
• provide automated support of aspects of software
  quality processes such as the SEI Capability
  Maturity Model (CMM)

3
Software Quality Control an Emerging Market
Integrated Software Quality Control
Software Product Quality
Manual Testing
Automated Formal Testing
Debugging
1970s 1980s
1990s 2000s
Scope of solution
Integrated Applications
Point Product
Product Suite
Productivity focus
Teams of Teams
Individual
Team
Geography
Virtual Teams
Single Location
Multiple Sites
Life Cycle Focus
End to end Delivery
Coding
Development
Process Procedures
Automated
Manual
Management of
Software Artifacts
All eAssets
Source Code
4
How to improve Software Quality - 1

• Use modern mainstream programming languages
• Advantages
• third party tooling available
• large community to discuss problems
• relatively easy to acquire new personnel or to
  subcontract
• implicitly safer and more productive
• Examples
• C, C, Java

5
How to improve Software Quality - 2

• Define and measure software quality and
  incorporate quality control in the software
  process
• Advantages
• creates uniformity among different parties
• improves quality attributes such as reliability
  and portability
• reduces independence on individual developer
  discipline
• establishes quality awareness among developers
• provides quantitative quality data enabling
  manage and control

6
Software Quality Control Positioning
7
QStudio - Integrated Software Quality Control
Requirements Management
Requirement
Analysis
Design
Implement.
Test
Change, Configuration Test Management
8
QStudio Approach
9
Personal Quality Control

• Extends developers IDE to help individual
  software developers to implement better code in
  less time
• Facilitates automated early life cycle
• testing
• Includes
• Coding standards conformance checking
• Identification of bad programming practices
• Knowledge transfer

10
Team Quality Control

• Provides applications to help software
  development teams to develop better source code
  in less time and in a uniform way
• Facilitates centralized automated source code
  inspection
• Includes
• Coding standards conformance checking
• Identification of bad programming practices
• Knowledge transfer

11
Process Quality Control

• Applications for cross-project, cross-team,
  cross-functional software quality information,
  analysis, and reporting
• Includes
• Process Managers dashboard
• Programming practices reporting
• Metrics reporting and analysis
• Quality benchmarking
• Realtime quality status information

12
QStudio Concepts
13
Static Analysis

• Automated technique to walk through the source
  code and detect constructs non complying with
  pre-defined rules.

14
Static Analysis Diagnoses for

• Quality aspects such as maintainability,
  reliability, understandability and complexity
• Testing issues
• Coding standard compliance issues
• Best programming practices and unsafe programming
  constructs and coding defects

15
Static Analysis Benefits

• Reduces time to market by cutting testing time
  due to earlier detection of software errors
• Significantly reduces review effort by automating
  a major portion of the inspection process
• adherence to coding standards
• usage of best programming practices
• Improves quality control by enforcing adherence
  to improved programming practices and corporate
  coding standards

16
Software Manager Findings on Static Analysis

• We proved the tight relationship between static
  analysis and the reduction of support efforts on
  released software products.
• Dr. Thomas Liedtke and Dr. Christian
  EbertAlcatel AG in Stuttgart, GermanyOn the
  Benefits of Reinforcing Code Inspection
  Activities, EuroStar 1995

17
Analyst Findings on Static Analysis

• 60 of the software faults that were found in
  released software products could have been
  detected by means of static analysis
• Bloor Research Ltd., UK CAST Tools report of 1996

18
Researcher Findings on Static Analysis

• On average, 40 of the faults that could be found
  through static analysis will eventually become a
  defect in the field.
• Professor Dr. Les Hatton,
• University of Kent

19
Static Testing

• Static testing is using static analysis as part
  of the test trajectory
• Static and Dynamic testing are supplementary
  static analysis does not replace dynamic testing
  but can significantly reduce dynamic testing
  effort
• Static testing achieves 100 statement coverage
• Including explicit static analysis in test
  coverage
• Improves overall test quality and test planning
• Results in shorter dynamic testing time
• Allows stronger focus testing on complex and
  crucial modules

20
Defect Removal Cost

• Cost of defect removal rises
• exponentially for defects found
• later in the development cycle

Dynamic Testing
Static Testing
21
Time (cost) required for ASA is low
22
Impact (benefit) of ASA is high
Static Analysis may reduce defects by a factor of
6!

• Source Capers Jones, Software Productivity
  Group, Inc.

23
Java Patterns

• A Pattern Language is a popular and well-accepted
  method for documenting best (Java) practises.
• A pattern is a structured piece of text
  containing specific practical knowledge.
• Used as an input rules are based on patterns
• Used as an output knowledge transfer to
  developer

24
Java Patterns

• Patterns Structure
• Problem Description
• Sample Code
• Forces
• Solution
• Improved Code
• Consequences
• Examples of QStudio for Java Patterns
• Exception Handling, Naming Conventions, No Magic
  Numbers, Unused Declarations, Multithreading,
  Unnecessary Names

25
Total Quality Management

• Quality (definition)
• Multi Customer Focus
• Quality Control
• Continuous Quality Improvement

26
Quality (definition)

• Quality is not just Freedom of deficiencies but
  also Meeting customer needs
• TQM states that quality should be specified using
  objective quantitative indicators.
• Quality Attribute Tree
• Rules assigned to quality (sub)attributes
• Metrics assigned to quality (sub)attributes
  (enterprise version only)
• Quantitative indicators
• rule hits per (sub)attribute
• metrics values

27
QStudio ISO 9126 Quality Model
28
Quality Attribute Tree Objective
MetricsProgramming Practice Mapping
Attribute
Sub-Attribute
Rule
1n
11
Failure Liability
Reliability
Complexity
Volume
(fragment) ... ... 55 too many comments150
private field name not used157 local
variable not used163 import declaration not
used (fragment)
Maintainability
Conciseness
Clarity
Testability
Modularity
Style Conformance
Re-Usability
Structuredness
Dev. Env. Conformance
Portability
User Platform Conformance
Time Behavior
Efficiency
Resource Behavior
29
Quality Attribute Tree Objective Metrics Code
Metric Mapping
Attribute
Metric
Sub-Attribute
1n
11
Failure Liability
(fragment) ... ... Static Path CountCyclomatic
ComplexityNesting DepthDepth of
inheritance (fragment)
Reliability
Complexity
Volume
Maintainability
Conciseness
Clarity
Testability
Modularity
Style Conformance
Re-Usability
Structuredness
Dev. Env. Conformance
Portability
User Platform Conformance
Time Behavior
Efficiency
Resource Behavior
30
Quality Sub-Attribute Analysis
31
Quality Attribute Tree
32
Observations-Rules-Patterns
Observation
Notice

• Observation
• Impact Level
• Quality Attribute

Inform
Learn
Rule Description

• Observation
• Rationale
• Related Pattern
• Check data

Pattern
11

• Problem Description
• Sample Code
• Forces
• Solution
• Improved Code
• Consequences

n1
33
Rules-Checks-Observations
Rule dont overload names
Code
Observation!
Check
34
Rules-Checks-Observations
35
Multiple Stakeholder Focus

• 5 - Reliable and fail safe
• 4 - Expected functionality
• 3 - Integrates well with little debugging and
  rework
• 2 - Reusable and maintainable
• 1 - Adheres to programming best practices

Quality Demanding Parties
36
Multiple Stakeholder Focus
37
Impact Levels

• 5 - Leads to a software product that is reliable
  and does not fail operation during use
• 4 - Offers features and functionality that is
  expected and required by the software user
• 3 - Can be integrated with other software parts
  and made operational with little debugging and
  rework
• 2 - Can easily be used and maintained by others
• 1 - Adheres to programming best practices

Increased Impact
38
Code Quality Control

• Programming Standards Enforcement
• Code Quality Analysis and Reporting
• Trend Analysis
• Outsourcing Supplier Quality Control
• Benchmarking
• Risk Assessment

39
Continuous Quality Improvement

• Code Quality Assessment
• analyse code quality
• compare to standards (benchmarking)
• report results
• Quality Planning
• define quality milestones
• make improvement plan
• define coding standard
• Quality Improvement Projects
• enforce programming standard
• perform code improvements
• analyse results

40

• Early life cycle programming error detection
  saves testing time by detecting programming risks
  prior to compiling source code
• Significantly reduces review effort by automating
  a major portion of the inspection process -
  direct support of coding standards, usage of best
  programming practices
• Assesses the quality of the software delivered by
  software development suppliers
• Demonstrates to customers the quality of the
  software delivered (for software development
  organizations)
• Assesses the quality of an existing code base in
  order, for example, to establish maintenance
  budgets or for establishing in-sourcing or
  outsourcing contracts

41

• Automatic multiple coding standards enforcement
  Automatic standards enforcement reduces manual
  review effort and improves programming practices.
  
• ISO 9126 quality model conforming objective
  software quality measurement Objective
  measurement in terms of high level management
  relevant quality attributes
• Flags weak/erroneous programming practices
  Identifies these constructs and gives you
  guidelines for improvement
• Descriptive pattern based reporting Empower best
  practices and improve software quality and
  productivity by reporting on software quality
  issues using descriptive Java patterns.
• Quality trend analysis based on formal milestone
  creation and reporting showing the quality
  evolution
• Leading IDE Integrations QStudio for Java is
  seamlessly integrated with leading IDEs
  (including JBuilder, JDeveloper, Eclipse,
  WebSphere Studio and Visual Age and in the
  future NetBeans and IntelliJ)
• Extensive platform support QStudio for Java is
  supported on the leading Java platforms
  Windows, Solaris and Linux, runs on JRE 1.2,
  1.3 and 1.4.