An Introduction to perfSONAR

1
An Introduction to perfSONAR

• ZHU Siyu

2
Outline

• What is perfSONAR
• What can perfSONAR do
• How does perfSONAR do
• What can we get from perfSONAR

3
What is perfSONAR

• PERFormance Service Oriented Network monitoring
  ARchitecture
• PerfSONAR has 3 contexts
• 1st. A consortium of organisations who seek to
  build network performance middleware that is
  interoperable across multiple networks and useful
  for intra-network and inter-network analysis.
• One of the main goals is to make it easier to
  solve end-to-end performance problems on paths
  crossing several networks.

4
What is perfSONAR

• PerfSONAR has 3 contexts
• 2nd. A protocol. It assumes a set of roles,
  defines the protocol standard by which they
  communicate, and allows anyone to write a service
  playing one of those roles.
• The protocol is based on SAOP XML messages and
  following the Open Grid Forum (OGF Network
  Monitoring Working Group (NM-WG).

5
What is perfSONAR

• PerfSONAR has 3 contexts
• Finally. An example set of code (implementation
  of services) that attempts to implement an
  interoperable peformance middleware framework.
• PerfSONAR partner
• ESnet
• GEANT2-JRA1
• Internet2

6
Outline

• What is perfSONAR
• What can perfSONAR do
• How does perfSONAR do
• What can we get from perfSONAR

7
What can perfSONAR do

• Design Goals.
• The design goals for this project include
  flexibility, extensibility, openness, and
  decentralisation. The resulting architecture
  should be
• 1. A modular system that allows the division of
  the architecture into multiple components that
  can be developed by individuals. Over time, it
  should be possible to evolve and/or replace
  individual components as our understanding of the
  needs of the system changes.
• 2. A dynamic system that allows components and
  Measurement Points, to some extent, to be added
  or removed during the system operation.
• 3. A self-configuring system that allows
  components and Measurement Points to autonomously
  announce their existence and capabilities.
• 4. A decentralised system, that allows each
  administrative authority to limit the system
  capabilities in accordance with locally-specified
  policies and procedures.

8
What can perfSONAR do

• Design Goals
• 5. A scalable system that can a) incorporate
  multiple networks and overlay virtual
  communities, b) handle varying numbers of users
  and servers, and c) handle varying information
  volumes as well as differing types of monitoring
  data and tools.
• 6. A secure system that cannot be exploited for
  uses other than performance monitoring, which is
  not particularly vulnerable to attack.
• 7. A safe system that does not overly congest the
  networks it is trying to monitor.
• 8. A fault-tolerant system that fails gracefully
  in the presence of module failures.
• 9. A self-diagnosing system that provides clear
  and timely exception messages in the case of
  failure.

9
Outline

• What is perfSONAR
• What can perfSONAR do
• How does perfSONAR do
• What can we get from perfSONAR

10
How does perfSONAR do

• perfSONAR overview

11
How does perfSONAR do

• perfSONAR overview

12
How does perfSONAR do

• perfSONAR compoments
• Measurement Point Service Creates and/or
  publishes monitoring information related to
  active and passive measurements
• Measurement Archive Service Stores and publishes
  monitoring information retrieved from Measurement
  Point Services
• Lookup Service Registers all participating
  services and their capabilities
• Authentication Service Manages domain-level
  access to services via tokens
• Transformation Service Offers custom data
  manipulation of existing archived measurements
• Resource Protector Service Manages granular
  details regarding system resource consumption
• Toplogy Service Offers topological information
  on networks

13
How does perfSONAR do

• Measurement Point Service
• The Measurement Point (MP) service is in charge
  of providing measurement data if the
  measurements are currently being stored in an
  archive, it retrieves them. If they are not
  already being collected, the MP "creates" the
  measurement data either by initiating active
  measurement tests or querying passive measurement
  devices. An MP is a standard interface wrapper
  around one or more measurement tool/capabilities.
  An MP is, basically, a place where some data are
  being metered. It can be, for example, a piece of
  hardware located in the core of the network to
  get precise data and deterministic behaviour of
  the network, but also can be software installed
  at the edge of the network on the hosts to
  perform end-to-end measurements.

14
How does perfSONAR do

• Measurement Archive Service
• A Measurement Archive (MA) service is used to
  publish historical monitoring data which are
  stored in an archive. It acts as a wrapper around
  an existing data archive to provide data to the
  outside world. The archive can be, for example, a
  networks Round Robin Database (RRD) or a
  proprietary database of a Network Management
  System (NMS). Additionally, an MA can publish
  information produced by MP services. It does not
  create (generate new raw data) or transform (i.e.
  aggregate/correlate/filter) any data.

15
How does perfSONAR do

• Lookup Service
• The Lookup Service (LS) enables users to discover
  other services (MP, MA, Authentication service,
  etc.) and, in particular, the tools,
  capabilities, or data offered by those services.
  The LS gives the requestor all the information
  required to find and contact the necessary
  services. In essence, the LS acts as a service
  directory, where services can advertise
  themselves (provide their lookup information) and
  requestors are able to find any service they
  need. Some information about services can be
  hidden, but high-level information should be
  exposed, so unauthorized users can determine if
  it is interesting or not.
• The LS is a key element of the perfSONAR
  measurement framework because it allows every
  independent service to be a visible part of the
  system. New services identify themselves to the
  community by registering with an LS and tender
  their capabilities, subject to locally-determined
  policies, or are withdrawn from the community
  without disrupting the interaction between other
  services.

16
How does perfSONAR do

• Authentication Service
• Some domains may wish to restrict access to
  service capabilities for some groups of users.
  The Authentication Service (AS) provides the
  authentication (AuthN) functionality for the
  framework as well as an attribute authority. (An
  attribute authority is a component of the
  authentication server that decides which
  attribute can be disclosed to a resource). With
  information provided by the LS, the local service
  will be able to provide access to a set of
  functionalities to a user based on the group of
  which he/she is a part (indicated by the
  attributes) and the service's local authorization
  policy.

17
How does perfSONAR do

• Transformation Service
• The TS is used to pipeline and modify data
  between the other services within the framework.
  It can be used to perform any function upon data.
  It fits between data producers (e.g. MPs, MAs, or
  other services) and data consumers (e.g. MAs,
  other services, or Clients). There is a variety
  of potential functions that a TS might provide
  including aggregation, correlation, filtering,
  and translation of data.
• For example, a TS might compress datasets from
  the more recent high resolution data to less
  recent lower resolution data and publish that
  data to an MA service. A TS also might read from
  multiple data producers to create a specific
  correlation. A very simplistic data analysis
  example would be a threshold detection operation,
  which then pushes data out for the purposes of
  triggering a NOC (Network Operations Center)
  alarm.

18
How does perfSONAR do

• Resource Protector Service
• The Resource Protector (RP) service is used to
  arbitrate the consumption of limited resources,
  such as network bandwidth. It also has a
  scheduling component to deal with the consumption
  of time-dependant resources. When measurement
  activities are involved, resources may be related
  to the measurement infrastructure or ordinary
  network resources. The RP can allocate portions
  of a resource based upon configuration rules and
  schedule the time-dependent resources.
• Services that consume resources can contact the
  RP to have them allocated. The primary example of
  this is the MP. Because RPs reduce scheduling
  flexibility, by design, an RP should be deployed
  only to protect limited resources. In other
  words, some MP services do not have to contact an
  RP at all or only have to contact it for
  requesting access to a limited number of
  resources.

19
How does perfSONAR do

• Topology Service
• The Topology Service (TopS) is used to make
  network topology information available to the
  framework. Typically, visualization clients can
  make use of such information to retrieve the
  topology for the creation of network maps. The
  TopS is a specific example of a Transformation
  Service (TS). It collects topological information
  from a variety of sources and uses algorithms to
  deduce the network topology. The TopS also
  reflects multiple network layers. That is,
  topology described on the domain level through
  network to wavelengths and the physical level
  should be possible. In addition to the network
  topology, it contains geographic information,
  such as GPS coordinates.

20
How does perfSONAR do

• Interactions of Services

21
How does perfSONAR do

• Protocols and Messages
• perfSONAR uses NMWG schema v2 to express
  measurement data
• Metadata describes the type of measurement data,
  the entity or entities being measured, and the
  particular parameters of the measurement. The
  data itself is simply a timestamp and a vector,
  or array, of values.

22
How does perfSONAR do

• Protocols and Messages
• perfSONAR messages
• perfSONAR services exchange NMWG messages. The
  NMWG schema is not only used to describe
  measurement data, but also to exchange those data
  between services. These messages are carried in a
  SOAP message, carried over HTTP. The whole chain
  of protocols used to exchange messages is
  presented in the figure below

23
How does perfSONAR do

• Service Description
• use Web Service to wrap service
• use XML to description
• use XML-Schema to validate XML
• use standard SOAP to exchange message
• to do

24
How does perfSONAR do

• Service Registration and Lookup
• Lookup Service
• basic Service of perfSONAR
• contains operations able to accept data, perform
  queries, manage resources, and purge expired data
• organize with simple p2p idea
• each domain must have at least 1 lookup service
• each LS considered as a peer

25
How does perfSONAR do

• Service Registration and Lookup
• Organization of LS instances
• 2 ways a flattened group of peers versus that a
  hierarchy where designated leaders communicate
  across the group boundaries
• each domain has a leader LS if more than 1, all
  LS that in the same peer organize a group
• there is an election protocol to determine which
  LS to be the leader (now simplely select the LS
  that has the smallest id)
• all LS in the same group exchange full summary
  information using a token passing protocol
• leader represent the group at the next higher
  level of the hierarchy.

26
How does perfSONAR do

• Service Registration and Lookup
• Leader plays as dual roles

27
How does perfSONAR do

• Service Registration and Lookup
• Data Summarization
• Data expressed in XML
• Transform using XSLT
• LS can get the info it intrested in by providing
  specific XSLT
• Information Exchange
• each LS will cache all the service registered to
  other LS in the same group
• given a query, exam local cache first, if have,
  return if not, ask group leader. and so on.

28
How does perfSONAR do

• Service Registration and Lookup
• LS initialization
• initialize by connecting to well-know/administrato
  r configured LS, get a init peer list
• peer list will be saved when shutting down, and
  LS will try to connect to those in the list next
  time up
• LS ring joint
• each peer will hold a peer list containing all
  peers in the group
• when a new peer join in, it will try to contact
  every peer in the init list
• each peer will forward the token to the next peer
  in the list, at last will return to the new peer

29
How does perfSONAR do

• Service Registration and Lookup

30
How does perfSONAR do

• Service Registration and Lookup
• LS failure
• When received a token, LS will send a reply.
• When an LS cannot seem to contact the next
  available peer(reply timed out), the list of
  peers is re-consulted to find who is next
  eligible
• When an LS recovered, it will join in the ring as
  a new peer

31
How does perfSONAR do

• Service Registration and Lookup

32
How does perfSONAR do

• Service Registration and Lookup
• LS Summary
• To summarize, each LS on boot time will consult a
  listpeers to contact. Once brought into the group
  via the joining protocol, the LS will summarize
  local data and exchange information with the rest
  of the group. Additionally, all LSs maintain a
  global summary in the event they detect
  themselves to be the new leader at the upper
  level of the hierarchy.
• In the event of any form of failure, the ring
  will adjust itself and re-summarize the data
  within a single round. Depending on cache
  coherence, data from an expired LS may remain
  active for some time.

33
Outline

• What is perfSONAR
• What can perfSONAR do
• How does perfSONAR do
• What can we get from perfSONAR

34
What can we get from perfSONAR

• Design idea
• SOA
• Web Service
• Service Reg and Lookup
• What can be improved
• Service description
• Ontology
• DHT network
• efficiency service lookup

35
Thats all

• thank you!