Inspections III

1
Inspections III
EE491.07
Maj JW Paul
From a presentation by Diane Kelly Dr Terry
Shepard
2
Education is what you get from reading the fine
print.Experience is what you get from not
reading it.
3
Outline

• What is a defect?
• Inspection in Industry
• The 3 Ms of Inspection
• Eight Maxims of Inspection OH
  case study

Diane Kelly
4
3 Ms of inspection
Goals, Culture, Resources, Process
Management
Inspection
Mechanics
Metrics
Unless a Thing can be defined by measurement, it
has no place in Theory. Feynman
Who, When, What, What, How
5
Mechanics

• 3 Ms of Inspection

The right person
who when
editor, detective, expert
Anytime (goals)
dont let it bottleneck process
6
What to inspect

• Everything (all work products)
• Efficiency gains
• design, architecture or requirements inspections
  can replace code inspections
• code inspections can replace unit testing
• architecture and design inspections have high
  payback (XP challenges this)
• start small even one requirements page can
  contain 50 or more defects

Inspection Testing are complementary activities
7
What to look for (goals)

• Usability issues
• Consistency of User Interface
• Defaults for switch statements
• Coding Standards (variable names)
• Numerical analysis
• etc...

Aside, inspection activity can help jr
programmers become familiar with the system
8
Do you need to inspect it all...

• Sampling an excellent alternative to complete
  checking. 4
• get an estimate error rate (per page, kloc...)
• - find systematic errors/corrections
• - do we need to inspect more

• Assumption sample document
• may miss specific issues on unchecked page
• still useful for quality measurement
• - teaching the product author
• - insights for process improvement

9
How to Inspect

• Ad-hoc (paraphrasing)
• Checklists

Unstructured

• Desk checking
• Fagan style
• SBR (e.g. test case)
• TDI
• Modeling

Structured
Goal is to combine structure rigour with
freedom to act effectively and so encourage
participants to add value
10
Unstructured Techniques
captures expertise style manual/standards predi
ctable/old project helps junior members used
independently based on past wont find new
errors blinder effect can be long/outdated migh
t be wrong one

• quick and easy
• no focus on issue
• no guidance
• inspectors expertise
• not repeatable
• inconsistent
• easy hard?
• start end?

Ad-hoc (paraphrasing)
Checklists 2,9,12,23
11
Structured Techniques

• Desk Checking 32
• do it yourself or team up
• Pair Programming 15, Testing Buddies 23
• structured - formal - track time/defects
• Fagan style
• defined process
• defined roles
• feedback

12
Scenario Based Reading

• A technique that is used individually in order
  to analyze a product or a set of products. Some
  concrete instructions are given to the reader on
  how to read or what to look for in a document.
  10
• Defines a goal/path for inspection
• defect-based
• perspective-based

13
Defect-based reading 11

• Have categories or types of errors
• precision
• language artifact (pointers)
• timing
• syntax
• Inspection focus on the error

14
Test case based reading 27

• Tests are directed at requirements
• build a complete test suite
• Caution
• can you isolate requirement with single test
• different requirements may generate similar test
  cases, but contradictory result
• make sure you know what the result will be
• is the requirement unambiguous/correct
• Inspection focus is on solution

15
Modeling the Inspection Artifact

• Re DaimlerChrysler inspections
• independently created a model of the requirements
  specifications (eg UML)
• Two inspectors read specifications
• prepare slides about the created model for the
  subsequent inspection meeting
• focus on qualities such as correctness,
  consistency, testability, maintainability
• inspectors dont need inspection specific training

16
Task Directed Inspection (TDI)

• While creating next work product inspectors check
  current product
• based on observation that creating careful
  documentation often shows up defects in product
  being documented
• used with a light-weight inspection process
• Case Study at OHN

17
Tools (aka automating inspections)

• Aimed at certain tasks (mindless)
• enforcing coding style, syntax checking,
  restructuring, code clean-up
• automatic generation of documentation
  (flowcharts, UML diagrams, call trees)
• Problems
• some style errors slip past
• cannot find errors that result in correct syntax
• tools can get confused in complex code
• report false positives/miss real issues
• still need someone to read the code

18
Metrics

• 3 Ms of Inspection

19
Metrics

• SWEng as an Experimental Discipline
• Role of Inspection in Measuring
• Software Measurement Maturity Model
• Inspection Metrics
• Defect Classification

Model Building, Measuring Learning
Models are required to have a basis for
measurement
20
How to check your software...

• Experimental Method
• Build a model
• (business, process, application, product)
• Test the model
• Measure the results
• Compare
• Refine the model
• Models are at different levels of detail
• hierarchy ? measurement results for one model are
  input data for higher models

21
SE as an Experimental Discipline

• We are an emerging discipline
• Current set of models is insufficient for the
  diversity of software projects
• Every project needs to be treated as an ongoing
  experiment
• may need to develop their own models for their
  own purposes
• required for meaningful experimentation (i.e.
  feedback on how well we did)

22
Model Software Entropy

• Belady Lehman 70s
• redesign is needed in cases when entropy grows
  too much
• Basili Turner 75
• incremental approach results in increase in
  structure, decrease in entropy
• Need a stronger definition of entropy (i.e. model
  needs to be improved)

23
Model Size ? Defect Rate

• Basili and Perricone 84
• defect rates of modules shrink as modules get
  larger (up to certain size)
• 40 of defects were interface defects, meaning
  that there was a need to know about the behaviour
  of other modules in order to make changes in this
  module
• larger is better because there is less chance of
  interface error?

24
What can be collected?

• Almost anything
• time used, counts, location, sources, impact,
  time to fix, phase, role (user, developer,
  inspector,...), etc
• Findings (may generate change requests)

25
Software Measurement Maturity Model
Levels of Maturity (using SEI CMM terms)
Initial
Little or no Measurement
Repeatable
Project Measurement (costs, functions)
Defined
Product Measurement (quality)
Managed
Process Measurement (quality)
Continuous Feedback for Process Improvement
Optimized
26
Data collection caveats

• effort must be proportional to the value of the
  data
• should be an integral part of the software
  process
• minimize human collection effort
• minimize the number/type of data
• if there is no payback, spend time elsewhere

27
Why Classify and Track Defects

• Project management
• resources, priorities, priorities
• Project execution
• develop test cases, inspection goals
• Responsiveness to customers
• accountability
• Process improvement
• goal of inspection
• Product improvement
• defect rates, statistical analysis
• Research
• future evaluation

IBM ODC, GQM
Beizers taxonomy Parnasian methods
IEEE 1044
Hewlet-Packard SBR (defects)
ODC-CC CISL
28
IBM ODC Categories

• Defect Removal Activities
• Triggers
• Impact
• Target
• Defect Type
• Qualifier
• Age
• Source

Assignment/Initialization Error Checking
Algorithm/Method Function/Class/Object Timing/Se
rial Interface/O-O Messages Relationship
29
HPs Categorization Scheme 1987
30
Beizers taxonomy (statistical framework)

• Requirements Features
• defects related to the requirements
  specification
• Functionality Features
• completeness, correctness, input domain, error
  messages
• Component Structure
• control flow, case selection, initialization,
  processing
• Data
• definition, structure or use of data
• Implementation
• typographic errors, standards violation,
  documentation
• Integration
• integration and interfaces between components
• System Software Architecture
• architectural erros that affect entire system
• Test Definition Execution
• definition, design, execution of test or data
  used in tests

31
ODC-CC Defect Classification for Computational
Codes
Where is it

• documentation
• calculation/logic
• error checking
• support code
• external resources

• missing
• wrong
• superflous
• inconsistent
• obscure

What is it
How hard is to find
32
Why so many classifications?

• Depends on what you are looking for
• Inspection goals
• Lord Kelvin checking your model...
• the first essential step in the direction of
  learning any subject is to find principles of
  numerical reckoning and practicable methods for
  measuring some quality connected with it when
  you cannot measure it, when you cannot express it
  in numbers, your knowledge is of a meagre and
  unsatisfactory kind

33
Next Class

• OH Guest Lecture
• Then
• Module Decomposition
• Intro to Lab2
• Read HS 6.1-6.3

34
Questions?