NMWG Normalized Schema

1
NMWGNormalized Schema

• Martin Swany
• Dan Gunter

2
Goal Normalized Schema

• Extensible and flexible
• All current and future measurements can be
  communicated with simple, common elements
• Without this, we have to reconvene every time
  someone adds a data source
• The same is true for statistical transformations

3
Identifying Data Values

• Data can be uniquely identified by a candidate
  key consisting of
• Target (Subject)
• Characteristic/Operation (Verb)
• Canonical characteristic methodology
• Parameters (Adverbs)
• NMWG Timestamp
• Obviously each measurement or performance event
  happened at some time

4
Data Values

• Values should be typed
• A number or a string
• Certain operations only work with the former
• Use XML Schema data types like xsdinteger and
  xsdfloat
• Completely extensible with complex types
• Each value has a (logical) timestamp

5
Extensibility

• Each of the key elements can be extended to
  include things we havent thought of yet
• Most simply, each root element is extensible
• We can produce schema definitions that define
  what Subjects and Parameters are appropriate for
  a given characteristic
• Put these somewhere public
• Allow folks to define their own and then describe
  with a document

6
Data/Metadata Separation

• Metadata blocks can be reused
• Metadata can be passed by reference
• To external documents or previous messages
• Passing by reference allows
• Subsequent data blocks to refer to initial
  metadata
• Should help with the batching issue
• Data can also be passed by reference

7
Consistent Metadata

• Consistent Metadata elements for
• Requests for measurements
• Requests for stored data
• Responses for either
• This allows re-use of schema definitions
• Consistent with the current request/response work

8
Compact Time Series

• Metadata separation allows more efficient
  transfers
• Many data values with metadata information
  factored out

9
Metadata Blocks

• On the one hand, single metadata group should be
  referable
• On the other, elements like a consistent hostpair
  factored out to refer to from various data blocks
• Latency and bandwidth from A to B
• Default parameters to traceroute (38, 3, 255) can
  describe a group of traceroutes

10
Metadata Representations

• Metadata blocks and data blocks
• Metadata with id and data blocks that refer to
  explicit metadata_id
• IDs are unique within an XML document
• Metadata_id (subject_id, eventType_id,
  parameter_id)
• Metadata by reference
• Same host pair
• Same default parameters

11
Encoding Overview
Metadata
Data
Data
12
Encoding Overview - 2
Metadata idmetadata_1
Metadata idmetadata_2
Data metadata_1
Data metadata_2
13
Encoding Overview - 3
Metadata Idmetadata_1
Data Metadata_1
Data http//ggf.org/myService/2
14
Encoding Overview - 4
Target idtarget1
Characteristic idcharacteristic1
Parameters idparameter1
Metadata idmetadata_1 targettarget1,charchar1,
paramsparam1
Data Metadata_1
15
Encoding Overview - 5
Target idtarget1
Characteristic idcharacteristic1
Parameters idparameter1
Metadata idmetadata_1 targettarget1,charchar1,
paramsparam1
Metadata idmetadata_2 targettarget2,charchar1,
paramsparam1
16
Encoding Overview - 6
Metadata idmetadata_1 bandwidth a to b
Metadata idmetadata_2 Query timerange
Metadata idmetadata_3 targetmetadata1,charstat.
av, paramsparam1
Data Idmetadata_3
17
Example Encoding - 1
ltresponse xmlns"http//www.ggf.org/nmwg/2004/09/
NetworkMeasurementsNorm-1.0-draft-0 1.wsdl"
xmlnsbw"http//www.ggf.org/nmwg/NM-WG/characteri
stic/bandwidth.available" scope"implicit" gt ltme
tadatagt ltbwsubjectgt lthostpairgt
ltsource addrtypeipv4gt131.243.2.11lt/sourcegt
ltdestination addrtypeipv4gt
192.168.1.1lt/destinationgt lt/hostpairgt
lt/bwsubjectgt
18
Example Encoding - 2
ltbweventTypegt ltidgt event-id lt/gt
ltcharacteristicgtbandwidth.availablelt/characteristi
cgt ltunitgtmillisecondslt/gt lt/bweventTypegt
ltbwparametergt ltidgt param-id lt/gt
ltpacketsizegt38lt/gt ltnum-queriesgt3lt/gt
lt/bwparametergt lt/metadatagt
19
Example Encoding - 3
ltmetadata id"metadata2"gt ltsubject
metadata_id"metadata1"/gt lteventTypegtnetlogger.s
electlt/gt ltparametergt lttimeIntervalgt
ltstartTimegtXlt/gt ltendTimegtYlt/gt
lt/timeIntervalgt ltorderBy type"ascending"gtYlt/gt
lt/parametergt lt/metadatagt ltmetadata
id"metadata3"gt ltsubject metadata_id"metdata2"/
gt lteventTypegtstat.avlt/eventTypegt
ltparamsgtlt/paramsgt lt/metadatagt
20
Example Encoding - 4
ltdata metadata_id"metadata3" type"simple"gt
ltitem timestamp"XX"gtYYlt/itemgt ltitem
timestamp"X2"gtY2lt/itemgt lt/datagt ltdata
metadata_id"metadata3" type"complex"gt ltitemgt
lttimeIntervalgt
ltstartTimegtXlt/startTimegt
ltendTimegtYlt/endTimegt lt/timeIntervalgt
ltvaluegt1lt/valuegt lt/itemgt ltitemgt ...etc...
lt/itemgt lt/datagt
21
Traceroute Encoding
Metadata Src-dst-timestamp
Ttl1, probe1
Ttl1, probe2
Ttl1, probe3
Ttl2, probe1
Ttl2, probe2
22
Open Issues

• The good news is that there are a number of
  isomorphic representations
• The bad news is that
• The intent is to provide a recommended format
• XSLT should allow automatic translation
• Namespace definition for schema validation
• You dont always want to validate
• You cant preclude validation

23
Differentiation of Object typeswith PortTypes

• The problem with abstract types for target,
  params and timestamp is in runtime parsing
• We can easily allow the client to request a
  specific format with a specific PortType
• Use WSDL to define output of a chain of functions