Network Monitoring for Grid Network Operations Center

1
Network Monitoring for Grid Network Operations
Center
Seongjin Ahn Sungkyunkwan University sjahn_at_comedu
.skku.ac.kr
2
Contents

• Background
• Grid Network
• Grid NOC
• Related study
• Network Measurement
• Grid Monitoring Architecture
• Monitoring Architecture for the Grid NOC
• Goals
• Monitoring Architecture
• Component
• Summary

3
Grid

• In the mid1990s to denote a proposed distributed
  computing infrastructure for advanced science and
  engineering
• A communication service that collaborates
  dispersed high performance computers, large-scale
  databases and modern equipments so that those can
  be shared and worked together
• Pursing Grid project based Grid network in
  several country

4
Grid Network

• High performance infrastructure connected
  advanced resources for realizing Grid projects
• Cooperating with RD networks and primary ISP
  networks
• High speed network pursuing overseas Grid project
  
• U.S.A STAR TAP, Abilence, vBNS, NREN
• HGP, NASA IPG, SETI_at_HOME, iVDGL, NEEDSGrid
• European Union GEANT .
• European Data GRID, Euro GRID
• Canada CANet3
• Japan SINET/IMnet
• AP GRID, HEP GRID
• Domestic Network
• KREONet, KOREN, HPCNet, AII-Testbed

5
STAR TAP

• The Science, Technology, And Research Transit
  Access Point
• The premier global exchange point for advanced
  international networking, in support of
  applications, performance measuring, and
  technology evaluations
• http//www.startap.net/

6
Alliance

• Advanced backbone network that supports
  deployment of the new applications being
  developed within the Internet2 community
• Alliance Grid high speed network to link virtual
  environments
• http//www.ucaid.edu/abilene/

7
NREN

• IPG Baseline System and High Data-rate Testbed
• http//www.nren.nasa.gov/

8
GEANT

• GEANT(the pan-European Gigabit Research Network)
  taken over TEN-155
• EU DataGrid project
• next generation computing infrastructure
  providing intensive computation and analysis of
  shared large-scale databases
• http//www.dante.net/geant/

9
CANet3

• Canada's Research and Education Internet backbone
• Grid Canada
• partnership between CANARIE(Canada's advanced
  Internet development organization), the National
  Research Council , and C3.ca to grid enable the
  nation
• http//www.canet3.net/

10
Super SINET

• Ultrahigh-speed network intended to develop and
  promote Japanese academic
• Supercomputer interlocking (GRID) working group
• promoting the construction of
• GRID computing to realize
• computing performance
• http//www.sinet.ad.jp/english/

11
TEIN

• Trans Eurasia Information Network
• Contribute to enhancing exchanges and cooperation
  between Asia and Europe
• Enhance and diversify research exchanges and
  cooperation between Asia and Europe
• http//www.transeurasia.org/

12
KREONet2

• Connect with oversea
• http//www.kreonet2.net/

Korea
USA
Europe
Seoul
APII Testbed KREONet2 (STAR TAP)
Taejon
Pusan
TEIN (Trans-Eurasia Information Network)
chonju
Japan
APII Testbed KREONet2
13
Global NOC (1/3)

• Global NOC
• Manage the international network connections from
  advanced research and education networks in the
  Asia/Pacific, Europe, Russia and South America to
  the Science Technology and Research Transit
  Access Point (STAR TAP) and the leading US high
  performance research and education networks such
  as Abilene (the network that supports the
  Internet2 project), the NSFs very high
  performance Backbone Network System (vBNS) and
  the Department of Energys ESNET.

14
Global NOC (2/3)

• Principle international connections managed by
  the Global NOC
• TransPAC
• OC-3 connection between Tokyo and the STAR TAP in
  Cicago connecting Japan, Korea, Singapore, and
  Australia
• Euro-Link
• Multiple high performance connections between The
  Netherlans, France, the Nordic countries, Isreal,
  and CERN, and the STAR TAP
• MIRnet
• Connection between Russia and the STAR TAP
• AmPATH
• Connection to various emerging education and
  research networks in Lain America

15
Global NOC (3/3)

• Ensuring the reliable provision of the persistent
  advanced networking infrastructure that will make
  collaboration possible
• Functions
• Problem reporting, monitoring, resolution
• Report generation, distribution, archiving
• Web page development, maintenance
• Tool development, maintenance
• http//globalnoc.iu.edu/

16
iGOC (iVDGL Grid Operations Center 1/2)

• Global services and centralized monitoring,
  management, and support functions
• NOC manages a network, providing a single,
  dedicated point of contact for iVDGL status,
  configuration, and management, and addressing
  overall robustness issues
• A central monitoring point
• for operational activity

17
iGOC (2/2)

• A unified point of support and coordination
• Performance monitoring
• Grid application development
• Grid system incident reporting
• Problem tracking
• To view site specific monitoring information,
  provide link to the desired location
• http//igoc.iu.edu/

18
Grid NOC

• Grid Network Operations Center
• Coordinate global services and centralized
  monitoring, management, and technical support to
  Grid Network
• Allocation of network resources as well as
  construction and management of Policy server
• Allocation of resources such as CPU and DB
• Collaborative work with international Grid
  Network
• Research of information resources as well as a
  provision education and seminar

19
GNOC-kr

• Overall system of GNOC-kr

MIC
Building Operational Structures
Inspecting GRID Making Policies
Resources
GRID Forum
Supercomputer/Cluster/seeMore(CAVE) Science and
Technology Information DB
C
C
GNOC
B
B

• Bandwidth allocation - User
  connection control
• Network Facility Management - Network
  Engineering
• Coordination of RD networks for GRID

A
A
Interconnection
Interconnection
HPCNet/KREONET
KOREN
APII-Testbed

• National wide backbone
• (15 regions)
• About 300 RD member institutions
• KR-USA, KR-JP link for RD

• KR-JP, KR- SG links
• KR-USA via JP link

• Backbone over 5 regions
• 32 RD institutions

Domestic RD Networks
A Requiring RD networks for cooperation /
Providing advanced science technology resources
B Requiring advanced science technology
resources / Providing GRID network resources C
User connection control management (Providing
connection channels for local RD networks)
20
GNOC-kr

• Connection to overseas GRID networks with
  international cooperation

21
Collaborative Environments
Environment
meteorology
VAN/USER
CFD
Physics
Drugs
Bio
Middleware
META Computing
Access GRID
Scientific Visualization
GLOBUS
KRISTAL
CAVERN
NOS
NES
NIS
NSS
GRID NOC
GRID Network
SONET/ATM
KOREN
Primary ISPs
KREONet2
STARTAP APII-Testbed
RD NOC
National RD Main Backbone
National RD Access Backbone
International RD Networks
Primary ISPs Testbed
22
Network Measurement

• Identifies and characterizes metrics useful to
  grid applications and develops standards to
  ensure the compatibility of metrics across
  measurement systems and applications
• Goals
• Describe the metrics of interest to grid
  application
• Define mappings between available tools and
  measurement
• Discuss issues in measuring each metric
• Works in collaboration with other standards
  groups such as the IETF IPPM WG and the Internet2
  End-to-end initiative
• IPPMs focus is on defining best-practices
  metrics of use of network engineers, whereas the
  NMWG focus in cataloging measurement
  methodologies as mush as metric

23
Network Measurement
24
GMA

• Producing a high-level architecture statement of
  the components and interfaces needed to promote
  interoperability between heterogeneous monitoring
  systems on the Grid
• Component
• Producer
• Providing event data to consumers
• Publish event availability information in the
  directory service
• Consumer
• Any process that requests or accepts event data
• Directory Service
• Publish what event data is available and which
  producer to contact to get it

25
GMA
Consumer

• From http//www-didc.lbl.gov/GGF-PERF/GMA-WG/

Directory Service (LDAP?)
Producer
Plus security!
26
Goals

• For GNOC
• Perform overall monitoring with reducing load
• Visualize configuration of overall Grid network
  resources
• Grasping accurate fault and performance situation
  of Grid network
• Scalablity as facilitated addition of managed
  Grid Network

27
Overall architecture
Producer 1
DB
Manager
Directory Service
Producer 2
DB
Producer N

DB
28
Directory Service

• Store meta-information of Grid network
• LDAP(Lightweight Directory Access Protocol)
  Server
• Scalability about new information as hierarchical
  management of information
• Producer publish location and configuration
  information of its domain
• Reduce overhead about discovery of whole network
  resources

29
Producer

• A managing agent having a network domain
• Register managed resource of a network domain in
  commonly used directory service for Grid NOC
• Collect and analyze fault performance
  information
• Store management information as putting local DB
  in domain

30
Producer

• Resource management
• When new resource add, register resource common
  directory service of Grid NOC
• When resource information delete, delete it from
  directory service
• Visualization topology
• Visualize overall network constitution with map
• Provide analysis information of fault and
  performance on the map
• Analysis
• Fault status
• Real-time response to request analysis of fault
  and performance
• Itemized performance analysis

31
Producer

• Reporting
• Keeping of information
• Collection
• Store local DB as collecting information on a
  managed domain
• Trap daemon
• Store database and real-time forward to
  management system
• LDAP message handle
• Convert LDAP protocol form registration,
  deletion, discovery information
• Extract wanted information from response

32
Producer

• Message analysis
• Query to resource and compose needed information
  from response
• Communication
• Set up connection between Directory service and
  devices , lines

33
Manager

• Operation system that NOCs manager request
  information of Grid network
• Visualization of configuration regarding overall
  network
• Connect management system of each network domain,
  perform monitoring in detail

34
Manager

• Configuration management
• Grasp overall situation of distributed resource
• Effective discovery
• Map Visualization
• Visualize topology of Grid network with map
• Provide analysis information of connection on all
  resources
• Provide connection with producer
• Directory service communication
• Make users command LDAP protocol
• Connection with directory service
• Make response result wanted information

35
Summary

• Effective overall monitoring for Grid NOC
• As producer perform a local management, reduce
  load of whole monitoring in Grid NOC
• Hierarchically keep location and configuration
  information in common directory service ,
  guarantee scalablity
• Efficiently use resources as managing status of
  configuration information about overall network
• Ensure safety and efficiency as fault and
  performance management of Grid network