Pre-decisional

1
Basic Terminology and Concepts

• CONFORMANCE the fulfillment of a product,
  process or service of specified requirements.
  (ISO Guide 2)
• The requirements are specified in a standard or
  specification as part of a conformance clause or
  in the body of the specification
• CONFORMANCE CLAUSE - a section of a specification
  that states all the requirements or criteria that
  must be satisfied to claim conformance

2
Terminology - Conformance Testing

• CONFORMANCE TESTING a way to determine
  directly or indirectly that relevant requirements
  are fulfilled.
• REQUIREMENT a necessary condition of a
  specification imposed on implementations.
  Requirements can have different degrees of
  necessity mandatory, recommended, or optional.
• COMPLIANCE POINT an identified, testable
  requirement (i.e., test assertion with a label)
• referred to as conformity assessment in ISO
  Guide 2

3
Terminology - Testing Process

• VALIDATION - process necessary to perform
  conformance testing in accordance with a
  prescribed procedure and official test suite
• ensures that testing can be repeatable and
  reproducible
• ensures that conclusions are consistent with
  facts presented in the evaluation
• CERTIFICATION - acknowledgement that a validation
  was completed and the criteria established for
  issuing certificates (brands) was met.

4
Testing Ingredients
Certification qualified bodies to do the testing
and certification control board - advisory and
arbiter
Validation Process - policy and procedures for
testing
Conformance Testing Test assertions Test suite
(test software, test scripts, test criteria)
Standard Conformance clause, conformance criteria
5
Profiles, Levels, Modules

• Profile is a subset of the overall specification
  that includes all the functionality necessary to
  satisfy the requirements of a particular
  community of users.
• Levels are nested, hierarchical subsets of the
  specification. Level 1 core that all must
  implement, subsequent levels include lower level
  added functionality
• Modules are discrete collections of
  semantically-related units of functionality that
  do not necessarily fit into a simple hierarchical
  structure.
• Reference
• Subdivide a Specification (extracted from W3C
  Quality Assurance documents)

6
Questions

• What does it mean when someone says they conform?
• What do they actually conform to?
• What are the criteria they need to meet?
• Conformance Clause.
• How do you know if they conform or not?
• Self-declaration, Testing, Certification

7
Why have a conformance clause?

• Communication between standard developers,
  product developers, users/buyers
• Defines the meaning of a conforming
  implementation i.e., what is required
• Provides basis for buyers to express their
  purchasing requirements
• People with claim conformance and others will ask
  for it
• If we dont do it someone else will
• Many will do it, in different ways

8
Conformance Topics

• Conformance Clause should address
• What Needs to Conform
• How to conform and claim conformance
• Subdividing a specification Profiles, Levels,
  Applications (modules)
• Variability ways a specification allows
  variation among conforming implementations
• e.g., optionality, alternative approaches