perfSONAR Update

1
perfSONAR Update

• Eric Boyd
• Joe Metzger
• Nicolas Simar
• Martin Swany

2
Agenda

• perfSONAR Overview
• perfSONAR Development Status and Plans
• perfSONAR Demos
• perfSONAR Deployment Status and Plans

3
Vision Performance Information is

• Available
• People can find it (Discovery)
• Community of trust allows access across
  administrative domain boundaries (AA)
• Ubiquitous
• Widely deployed (Paths of interest covered)
• Reliable (Consistently configured correctly)
• Valuable
• Actionable (Analysis suggests course of action)
• Automatable (Applications act on data)
• Easy to produce
• Extensible data-sharing toolkit

4
Getting There Build Empower the Community

• Decouple the Problem Space
• Analysis and Visualization
• Performance Data Sharing
• Performance Data Generation
• Grow the Footprint
• Clean APIs and protocols between each layer
• Widespread deployment of measurement
  infrastructure
• Widespread deployment of common performance
  measurement tools

5
What is perfSONAR?

• Performance Middleware
• perfSONAR is an international consortium led by
  ESnet, GÉANT2, Internet2, and RNP
• perfSONAR is a set of protocol standards for
  sharing data between measurement and monitoring
  systems
• perfSONAR is a set of open source web services
  that can be mixed-and-matched and extended to
  create a performance monitoring framework

6
perfSONAR Design Goals

• Standards-based
• Modular
• Decentralized
• Locally controlled
• Open Source
• Extensible
• Applicable to multiple generations of network
  monitoring systems
• Grows beyond our control
• Customized for individual science disciplines

7
(No Transcript)
8
perfSONAR Integrates

• Network measurement tools
• Network measurement archives
• Discovery
• Authentication
• Data manipulation
• Resource protection
• Topology

9
perfSONAR Credits

• GÉANT2 JRA1 includes
• Arnes
• Belnet
• Carnet
• Cesnet
• CYNet
• DANTE
• DFN
• FCCN
• GRNet
• GARR
• ISTF
• PSNC
• Nordunet (Uninett)
• Renater
• RedIRIS
• Surfnet
• SWITCH

• perfSONAR is a joint effort
• ESnet
• GÉANT2 JRA1
• Internet2
• RNP
• ESnet includes
• ESnet/LBL staff
• Fermilab
• Internet2 includes
• University of Delaware
• Georgia Tech
• SLAC
• Internet2 staff

10
Agenda

• perfSONAR Overview
• perfSONAR Development Status and Plans
• perfSONAR Demos
• perfSONAR Deployment Status and Plans

11
perfSONAR Development Process

• Loosely coordinated development of web services
• Each web service owned by 1 or 2 developers
• Core set of services released as a joint package
• Interoperability testing within the core
• Interoperability testing with common UIs
• Reference implementation in Java some services
  in perl
• Common development resources (e.g. Bugzilla,
  mailing lists, SVN, Wiki)
• Steering committee
• Regular email discussions and conference calls
• Quarterly face-to-face meetings

12
perfSONAR release 1.1 (orange signifies new)

• Production release of core services package v1.1
  is planned for February, 2007
• Single domain LS solution (PSNC)
• RRD MA (PSNC)
• SQL MA (PSNC)
• BWCTL MP (DFN)
• SSH/Telnet MP (Belnet)
• Recommended visualization to make use of those
  services
• perfSONAR UI (ISTF)
• CNM (DFN)
• Quality improvements
• Bug fixes
• Documentation
• Functional testing
• Installation

13
Ongoing and Planned Development Work

• Network measurement tools
• ABW (CESNET)
• BWCTL MP (DFN)
• BWCTL becomes a MA/MP (Internet2)
• Ciena MP (UDel)
• CLI MP (RNP)
• L2 status MP (DFN/JRA4)
• Netflow subscription MP (Surfnet)
• SSH/Telnet MP (Belnet)
• TCMP (Arnes)
• Traceroute MP started (GaTech)
• Topology
• TopS under development (RedIRIS)
• cNIS under development (SA3)
• Extension of Indiana NOC DB (Internet2)
• Unified Information Service started (UDel)

• Authentication
• Semantics defined (MACE, JRA5)
• Authorization
• Discussion has just begin (RedIRIS)
• Discovery
• Single LS released (PSNC)
• Multi-LS developed (UDel), in testing (UDel/PSNC)
• Data manipulation
• Anamoly detection service started (UDel)
• NOC Analysis tools under development (SLAC)
• Network measurement archives
• Hades MA (DFN)
• OWAMP MA started (GaTech)
• RRD MA (PSNC, flow CARNET)
• SQL MA (PSNC, L2 status PSNC)

14
Visualisation Status Update

• Allows diversity on the measurement layer and on
  the visualization layer
• BWCTL webpage (DFN)
• CNM (DFN) Top 10, dashboard.
• ICE/NeTraMet (RNP)
• JRA4 E2E L2 visualisation (DFN)
• Looking glass (BELNET)
• NEMO (UNINET)
• perfsonarUI (ISTF)
• VisualperfSONAR (CARNET)

15
Agenda

• perfSONAR Overview
• perfSONAR Development Status and Plans
• perfSONAR Demos
• perfSONAR Deployment Status and Plans

16
perfSONAR Demos

• Visual perfSONAR
• https//noc-mon.srce.hr/visual_perf/
• perfSONAR UI
• http//wiki.perfsonar.net/jra1-wiki/index.php/Perf
  sonarUI

17
Agenda

• perfSONAR Overview
• perfSONAR Development Status and Plans
• perfSONAR Demos
• perfSONAR Deployment Status and Plans

18
perfSONAR Adoption

• RE Networks
• Internet2
• ESnet
• GÉANT2
• European NRENs
• RNP
• Targeted Application Communities (2007)
• LHC
• GLORIAD Distributed Virtual NOC
• Teragrid

• Distributed Development
• Individual projects (10 before first release)
  write components that integrate into the overall
  framework
• Individual communities (5 before first release)
  write their own analysis and visualization
  software

19
perfSONAR Deployment Status
20
GÉANT2 Deployment Status

• 2 LS
• 15 MA (Renater and GARR)
• Hades (a.k.a. IP Performance Metrics) MA
• 1 service 22 measurement nodes
• 4 BWCTL / OWAMP MP
• 1 Telnet / SSH MP
• Also RNP (Brazil NREN), MREN (Montenegrin NREN),
  SEEREN2

21
Internet2 Deployment Status

• Focus is on development of services for Internet2
  new network and integration with Indiana NOC
• Submitting a proposal to NSF for additional
  funding
• Target July 1, 2007 as new Internet2 network
  goes operation
• OWAMP MA
• BWCTL MA/MP
• IU-based Topology Service
• Multi-LS
• NOC Alarm Transformation Service

22
ESnet Deployment Status

• RRDMA to export our link utilization statistics
• SNMP based link status polling system
• link status for LHCOPN and Service trial circuits
• E2E-MON MP from DFN to export this status
• Deploying active latency and bandwidth monitoring
  probes around the network
• but have not integrated this with perfSONAR yet

23
GÉANT2 Transition to ServiceMulti-Domain
Monitoring (MDM) Service
24
Multi-Domain Monitoring Service

• User role group of people making use of a MDM
  Service.
• There may be several categories of users having
  different needs.
• E2E really means Edge to Edge, not End to End
  (unless end institutions buy into it).
• Must go as close as possible to the
  end-institution regional and metropolitan
  networks should also be involved.
• An NREN has two roles
• Data supplier.
• Data user.

25
Multi-Domain Monitoring Service

• Multi-Domain Monitoring Service
• Access to a set of monitoring functionalities
  (e.g. accessing metric or performing tests)
  offered to a group of users accessible directly
  through an XML SOAP interface (perfSONAR
  protocol) or through a visualisation tools.
• Based on an underlying set of perfSONAR
  web-services.
• perfSONAR web-service
• Web service (providing data or allowing to
  perform an action) using the XML NM-WG. The
  perfSONAR web-services are the basic building
  blocs of a MDM service.

26
Users Segmentation
27
MDM Service Support

• Infrastructure to support the perfSONAR
  web-services and the visualisation tools used by
  the MDM will be set-up.
• For the deployers installation, configuration,
  incident, monitoring.
• For users installation, utilisation.

Users (NOC, PERT, Projects)
Deployers (RENs)
SLA()
Deployer Service Desk
SLA()
ISS
User Service Desk
() Dont get scared here!
28
MDM Service Support

• Level1 Service Desk (ISS)
• Help to install, configure the tools, run
  reachability tests, help on usability, track the
  RFE, forward problem to proper person, log the
  requests, update the documentation, track bugs.
  This is a central function (rotating member or
  group of people - ownership).
• Level2 Administrator (RENs or FMS)
• Administrator of the machines where the services
  are installed. The function lies within the
  providers. They are in charge of taking care of
  the security of the services, of their
  availability (up) and reachability (no firewall,
  etc). The service should be available 24/7.
• Level3 Developers (3 years subcontract).
• The JRA1 developers who have build the services.
  They are in charge of implementing new features
  and fixing bugs and of answering the query
  forwarded by level1.
• The three levels of support will be available to
  both the users and the deployers.

29
MDM Service Support Fully Manage Service (FMS)

• A turn key solution service could be provided for
  the web-services of a MDM service or part of it.
• HW bought.
• Web-services installed, monitored and managed on
  behalves of the REN.
• Level2 Service Support provided.
• REN would still have to do a little bit.
• Physical installation.
• GPS antenna coordination.
• Provide the data.
• Train its staff.
• Save 40 of the installation effort and all the
  support time once installed.

30
Going Operational

• Pre-roll Out define and set-up support
  structure now March 07.
• Pilot April 07 August 07 5 RENs GÉANT2
• For NOC and PERT (no AA)
• Understand the issues of going operational.
• Validate the support structure, get feedback for
  next phase.
• Release in January, deployment training in
  February.
• Test the Fully Manage Service.
• Prototype October 07 February 08 11 RENs
  GÉANT2
• For NOC, PERT and a limited number of projects.
• Verify the MDM SLA.
• Dedicated support team.
• Verify how to provide the service to external
  parties.
• Operation April 08
• More RENs, closer to end-institution.
• More projects supported.

31
Pilot - Objectives

• Pilot
• April 07 August 07
• 5 RENs GÉANT2
• For NOC and PERT (no AA)
• Objectives
• Understand the issues of going operational.
• Validate the support structure set-up.
• Get staff trained, raise awareness, provide
  feedback.
• Use a trustable platform.
• Get feedback for next phase.

32
Pilot - Functionalities
() historical is currently not required by the
Circuit E2E monitoring tool. () List of
command from a list of pre-defined commands for
all the backbone routers. () UDP throughput
tests will be restricted to PERT to discover
packets dropped in case of difficult problem to
solve.
33
Pilot - Portfolio
() L2 status MP or SQL MA
34
More Information

• http//www.perfsonar.net/