IEEE Standard 10281997

1
IEEE Standard 1028-1997

• Software Reviews
• Guy Bedette
• EEL 6883

2
Abstract

• Defines five types of software reviews and
  procedures for conducting each review type
• Does not define procedures for determining the
  necessity of reviews of disposition of results
• Review types management, technical, inspections,
  walk-throughs, audits

3
Overview

• Purpose to define systematic reviews applicable
  to software acquisition, supply, development,
  operation and maintenance.
• Scope defines minimal acceptable requirements
  for systematic reviews. Systematic attributes
  include
• Team Participation
• Documented Results
• Documented Review Procedures
• Reviews that do not meet requirements considered
  non-systematic
• Standard not intended to discourage or prohibit
  non-standard reviews

4
Overview

• Provides definitions, requirements, and
  procedures applicable to reviews of software
  development products throughout lifecycle
• Intended to be used with other standards to
  determine reviewed products, timing, and
  necessity of reviews
• Conformance to standard can be claimed when all
  mandatory actions (shall) are carried out.
• Make use of standard by deciding objective of
  review and selecting appropriate review type

5
Management Reviews

• Purpose To monitor progress, determine status,
  confirm requirements and allocation, evaluate
  effectiveness of management approaches
• Support decisions about corrective actions,
  changes in resource allocation or scope of
  project
• Carried out by or on behalf of management
  personnel with direct system responsibility
• Examples of products subject to management review
• Shall be performed by personnel best qualified to
  evaluate the software product

6
Management Reviews

• Following roles shall be established
• Decision maker person for whom the review is
  conducted
• Review leader responsible for review
  administrative tasks
• Recorder documents anomalies, action items,
  decisions recommendations
• Management staff responsible for active
  participation
• Technical Staff provide information necessary
  for management staff
• May also include customer or user representative
• Participants may act in more than one role

7
Management Reviews

• Required inputs
• Statement of objectives
• Software product being evaluated
• Software project management plan
• Status of the product relative to plan
• Current anomalies or issues
• Documented review procedures
• Entry criteria
• Authorization project plan or management
  authority
• Preconditions statement of objectives and
  required inputs

8
Management Reviews

• Management preparation procedures
• Plan time and resources
• Provide funding and facilities
• Provide training and orientation on review
  procedures
• Ensure reviews are conducted and recommendations
  acted upon in timely manner
• Planning procedures
• Identify review team
• Assign responsibilities
• Schedule meeting and distribute review materials
• Set timetable for return of comments

9
Management Reviews

• Preparation procedures
• Examine product prior to meeting, document
  anomalies and send to review leader
• Review leader classify reported anomalies and
  forward to author of product for disposition
• Examination procedures
• Review objectives
• Evaluate software product against objectives
• Evaluate project status including plans and
  schedules
• Review identified anomalies
• Generate list of action items, emphasizing risk
• Document the meeting

10
Management Reviews

• Rework/follow-up
• Assure action items in meeting are closed
• Exit criteria
• Examination procedure activities accomplished
• Output artifacts exist
• Output documented evidence identifying
• Project being reviewed
• Review team members
• Software product reviewed
• Specific inputs to review
• Action item status
• List of anomalies

11
Technical reviews

• Purpose Evaluate a software product to determine
  its suitability for intended use and identify
  discrepancies from specification and standards
• Provides management with evidence to confirm
  whether
• Product conforms to specification
• Product adheres to regulations, standards,
  guidelines, plans, procedures
• Changes to the product are properly implemented
• Examples of products subject to technical review
• SRS, SDD, test docs, user docs, maint. manual,
  build/install procedures, release notes

12
Technical reviews

• Following roles shall be established
• Decision maker person for whom the review is
  conducted
• Review leader responsible for review
  administrative tasks
• Recorder documents anomalies, action items,
  decisions recommendations
• Technical Staff responsible for active
  participation
• Other roles may include
• Management staff identify issues requiring
  management resolution
• Customer or user representative

13
Technical reviews

• Required inputs
• Statement of objectives
• Software product being examined
• Software project management plan
• Current anomalies or issues
• Documented review procedures
• Entry criteria
• Authorization according to project plan,
  management or engineering request per approved
  procedures
• Preconditions statement of objectives and
  required inputs

14
Technical reviews

• Management preparation procedures
• Plan time and resources
• Provide funding and facilities
• Provide training and orientation on review
  procedures
• Ensure review team members possess appropriate
  expertise
• Ensure reviews are conducted and recommendations
  acted upon in timely manner
• Planning procedures
• Identify review team
• Assign responsibilities
• Schedule meeting and distribute review materials
• Set timetable for return of comments

15
Technical reviews

• Preparation procedures
• Examine product prior to meeting, document
  anomalies and send to review leader who
  classifies reported anomalies and forwards to
  author of product for disposition
• Review leader verifies team members are prepared
  and reschedule meeting if not
• Examination procedures
• Determine agenda
• Determine if product is complete, conforms to
  applicable documents
• Determine if changes properly implemented
• Determine if product is suitable for intended use
• Determine if product is ready for next activity
• Identify anomalies
• Generate list of action items
• Document meeting

16
Technical reviews

• Rework/follow-up
• Assure action items in meeting are closed
• Exit criteria
• Examination procedure activities accomplished
• Output artifacts exist
• Output documented evidence identifying
• Project being reviewed
• Review team members
• Software product reviewed
• Specific inputs to review
• Review objectives and whether they were met
• List of anomalies
• List of management issues
• Action item status

17
Inspections

• Purpose to detect and identify software product
  anomalies
• Peer examination that
• Verifies product meets specifications
• Verifies product satisfies quality attributes
• Verifies product conforms to applicable standards
• Identifies deviations from specifications
• Collects defect and process metrics
• 3-6 participants lead by facilitator
• Mandatory determination of remedial or
  investigative action for anomalies
• Examples of products
• SRS, SDD, code, test docs., user docs, maint.
  manual, build/install procedures, release notes

18
Inspections

• Following roles shall be established
• Inspection leader responsible for administrative
  tasks
• Recorder documents anomalies, action items,
  decisions recommendations
• Reader leads inspection team through product in
  comprehensive and logical fashion
• Author responsible for product meeting entry
  criteria, contributing to inspection, and
  performing rework to meet exit criteria
• Inspectors Identify anomalies in product.
  Different inspectors required to meet different
  viewpoints (requirements, design, code, test
  etc.)

19
Inspections

• Required inputs
• Statement of objectives for the inspection
• Software product being inspected
• Documented inspection procedure
• Inspection reporting forms
• Current anomalies or issues list
• Entry criteria
• Authorization according to project plan, project
  or quality management request per approved
  procedures
• Preconditions statement of objectives and
  required inputs
• Software product is complete and conforms to
  content and format standards
• Automated error-detection tools are available
• Prior milestones are satisfied
• Required supporting documentation available
• All anomalies resolved prior to re-inspection

20
Inspections

• Management preparation procedures
• Plan time and resources
• Provide funding and facilities
• Provide training and orientation on review
  procedures
• Ensure review team members possess appropriate
  expertise
• Act on recommendations in timely manner
• Planning procedures
• Identify inspection team
• Assign responsibilities to team members
• Schedule meeting and distribute inspection
  materials
• Set timetable for return of comments

21
Inspections

• Author should present an overview of product to
  be inspected
• Preparation procedures
• Examine product prior to meeting, document
  anomalies and send to inspection leader who
  classifies reported anomalies and forwards to
  author of product for disposition
• Specify suitable order in which to inspect
  product
• Examination procedures
• Introduce meeting
• Establish preparedness
• Review general items
• Review software product and record anomalies
• Review the anomaly list
• Make exit decision no or minor rework, rework
  verification, re-inspect

22
Inspections

• Rework/follow-up
• Assure action items in meeting are closed
• Exit criteria
• Examination procedure activities accomplished
• Output artifacts exist
• Output documented evidence identifying
• Project being inspected
• Inspection team members
• Inspection meeting duration
• Software product inspected
• Size of materials inspected
• Inputs to the inspection
• Inspection objectives and whether they were met
• Anomaly list
• Anomaly summary
• Disposition of software product
• Estimate of rework effort

23
Inspections

• Anomaly classes
• Missing
• Extra
• Ambiguous
• Inconsistent
• Improvement desirable
• Not conforming
• Risk-prone
• Factually incorrect
• Not implementable
• Editorial
• Anomaly ranking
• Major results in failure of product
• Minor deviates from relevant specification

24
Walk-throughs

• Purpose to evaluate a software product for the
  purpose of finding anomalies, improving the
  product, consider alternatives, evaluate
  conformance.
• May also be used for exchange of techniques,
  training, and pointing out readability or design
  problems in code
• Examples of products
• SRS, SDD, code, test docs., user docs, maint.
  manual, build/install procedures, release notes

25
Walk-throughs

• Following roles shall be established
• Walk-through leader responsible for
  administrative tasks
• Recorder documents decisions, action items,
  comments on anomalies, questions of style,
  suggestions for improvements.
• Author presents the software product in the
  walk-through

26
Walk-throughs

• Required inputs
• Statement of objectives for the walk-through
• Software product being examined
• Documented inspection procedure
• Standard in effect
• Entry criteria
• Authorization according to project plan, project
  or quality management request or author request
  per approved procedures
• Preconditions statement of objectives and
  required inputs

27
Walk-throughs

• Management preparation procedures
• Plan time and resources
• Provide funding and facilities
• Provide training and orientation on walk-through
  procedures
• Ensure walk-through team members possess
  appropriate expertise
• Ensure planned walk-throughs are conducted
• Act on recommendations in timely manner
• Planning procedures
• Identify walk-through team
• Schedule meeting and select meeting place
• distribute materials and allow time for
  preparation

28
Walk-throughs

• Preparation procedures
• Examine product prior to meeting and prepare list
  of items for discussion.
• Discussion items classified as general and
  specific
• Specify suitable order in which to inspect
  product
• Examination procedures
• Introduce participants and describe roles
• State purpose and remind team that focus is
  anomaly detection not resolution
• Comment only on product not its author
• May pose questions to author regarding product
• Make overview presentation and discuss general
  anomalies
• Present product and discuss specific anomalies
• Recorder notes recommendations and actions

29
Walk-throughs

• Rework/follow-up
• Walk-through leader assures assigned action items
  are closed
• Exit criteria
• Entire software product has been examined
• Recommendations and required actions have been
  recorded
• Walk-through output has been complete
• Output documented evidence identifying
• Walk-through team members
• Software product being examined
• Statement of objectives met/not-met
• List of recommendations regarding each anomaly
• List of actions, actionees and due dates
• Team recommendations on how to dispose of
  deficiencies and unresolved anomalies
• Proposals man by team for follow up walk-throughs

30
Walk-throughs

• Anomaly classes
• Missing
• Extra
• Ambiguous
• Inconsistent
• Improvement desirable
• Not conforming
• Risk-prone
• Factually incorrect
• Not implementable
• Editorial
• Anomaly ranking
• Major results in failure of product
• Minor deviates from relevant specification

31
Audits

• Purpose provide an independent evaluation of
  conformance of software products and processes to
  applicable regulations, standards, guidelines,
  plans, procedures
• Examples of products subject to audits
• Plans, reports, procedures, code, development
  folders, manuals, test docs., deliverable media

32
Audits

• Following roles shall be established
• Lead auditor assemble manage team, make
  decisions, prepare report, recommends actions
• Recorder documents anomalies, action items,
  decisions, recommendations
• Auditor(s) examine products, document
  observations and recommend corrective actions
• Initiator Decide need for audit, purpose,
  products audited
• Audited organization provide liaison to auditors
  and all information requested

33
Audits

• Required inputs
• Purpose and scope of audit
• Background information about audited organization
• Software products to be audited
• Evaluation criteria
• Entry criteria
• Authorization initiator decides need based on
  project milestone of suspicion of
  non-conformance. Initiator can be supplier,
  customer, or third party (e.g. regulatory agency)
• Preconditions audit authorized by appropriate
  authority, statement of objectives established,
  required inputs available

34
Audits

• Management preparation procedures
• Plan time and resources
• Provide funding and facilities
• Provide training and orientation on audit
  procedures
• Ensure audit team members possess appropriate
  expertise
• Ensure planned audits are conducted
• Act on recommendations in timely manner
• Audit plan describes
• Purpose and scope of audit
• Audited organization
• Audited products and evaluation criteria

35
Audits

• Audit plan
• Auditors responsibilities
• Examination Activities
• Schedule
• Requirements for confidentiality
• Required follow-up activities
• Opening Meeting between audit team and audited
  organization. Agenda to include
• Purpose and scope
• Products being audited
• Expected contributions
• Schedule
• Access to facilities and information

36
Audits

• Preparation procedures
• Initiator shall notify audited organization in
  writing prior to audit
• Define purpose and scope of audit what will be
  audited auditors and audit schedule
• Audited organization ensures people and material
  to be examined in audit are available
• Examination procedures
• Evidence collection by interviews, examination of
  documents and witnessing processes
• Undertake additional activities required to
  define conformance/non-conformance
• Document observations of non-conformance and
  exemplary performance

37
Audits

• Closing meeting
• Review extent of audit plan implementation
• Problems experienced
• Observations made
• Preliminary conclusions and recommendations
• Overall assessment
• Reporting
• Lead auditor shall prepare an audit report as
  soon as possible. Any communication should pass
  through lead. Audit report delivered to initiator

38
Audits

• Follow-up
• Determine and initiate corrective action if
  required
• Exit criteria
• Audit report submitted to initiator
• Auditing organizations follow-up actions
  performed, reviewed and approved
• Output
• Audit report containing purpose and scope,
  audited organization, products audited,
  applicable regulations, evaluation criteria,
  summary of findings and recommendations