Metrics Framework draft-morton-perf-metrics-framework-02.txt

1
Metrics Frameworkdraft-morton-perf-metrics-framew
ork-02.txt

• Authors Alan Clark

2
Changes from 01 draft

• Tidied up language in Sections 1-3
• Added detail for metric specification/ template
• Added sections on results organization,
  parameters, audience

3
Introduction (1)

• Original paragraph focused too much on
  uncertainties
• Replacement wording simply says that performance
  monitoring is important for distributed apps
• 1.1, 1.2 no changes

4
Purpose and Scope (2)

• Updated to
• The purpose of this memo is to define a
  framework and a process for developing
  performance metrics for IP-based applications
  that operate over reliable or datagram transport
  protocols, and that can be used to characterize
  traffic on live networks and services.
• The scope of this memo includes the support of
  metric definition for any protocol developed by
  the IETF, however this memo is not intended to
  supercede existing working methods within WGs
  that have existing chartered work in this area.
• This process is not intended to govern
  performance metric development in existing IETF
  WG that are focused on metrics development, such
  as IPPM and BMWG. However, the framework and
  guidelines may be useful in these activities, and
  MAY be applied where appropriate.

5
Metrics Development (3)

• Converted 3.1 Audience from a comment to text
• Updated example of good vs weak metric at end of
  section 3.2
• 3.3 temporarily removed details on composition
  of metrics this is a fairly large and well
  established field and hence may need to refer to
  earlier work than ippm

6
Metric Specification (3.4)

• Added section on metric specification
• Normative parts
• Name
• Measurement method
• Need for clear definition and consideration of
  exception conditions
• Units of measurement
• Timing of measurements
• Measurement intervals/ sampling intervals
• Informative parts
• Description
• Metric use and applications
• Implementation
• Reporting model
• Metric definition template

7
Material added by Al Morton

• Organization of results (3.9)
• Singleton, Sample, Statistic
• ADC overlaps with Section 3.4.2 (iv)?
• Parameters (3.10)
• All options and input variables need to be
  identified and specified
• ADC overlaps with Section 3.4.2 (ii)
• Packet measurement details (3.11)
• Consider wire time when specifying timings
• Identifying and categorizing the audience (3.12)
• Consider who will use metrics and how
• ADC - overlaps with Section 3.1
• Proposal Approval (4.2)

8
Next steps?

• Existing text needs some consolidation (e.g.
  Sections 3.9 and 3.1)
• Continue to add more details
• Other suggestions?