ICFA SCIC Meeting CERN Dec 01

1
Network Connectivity and Projects a
Perspective from the UKRichard Hughes-Jones
PPNCG

• SuperJANET4 Status
• Connectivity to Europe
• Connectivity to the US
• Grid Networking Projects

2
SuperJANET4 Backbone and Access links

• Worldcom supplied the transmission
• UKERNA layer the IP service
• Core PoP IP router at Worldcom
• Backbone Access Router at MANs
• Access Links
• Large MAN 2.5 Gbit -gt 10-20 Gbit
• Medium MAN 622 Mbit -gt 2.5 Gbit
• 4 node DWDM development net
• Deployment Status
• Backbone Oct 00 Routers Nov 00
• All sites Mar 01
• Proved to be Stable
• Constant growth of traffic

3
SuperJANET4 ping rtt Core routers Jun 01
4
SuperJANET4 ping rtt Core routers

• Glasgow

• Lancaster

• MAN / LAN Issues

• Bristol

• Cambridge

5
London MAN Upgrade

• UDP Packet loss

• UDPmon Tests
• Manchester London
• MAN was 155 Mbit ATM

• UDP Throughput
• Mbit/s

• 1st Oct Time interval in Weeks

Richard HJ
6
Connectivity to Europe Access links (1)
ICFAMON Plot from RAL to CERN for 19th Oct to 1st
Nov 2001

• UK Access link 155 Mbit ATM
• Sustained rate 130 Mbit
• Contract to end of Nov 01
• Bad news for users !

7

Connectivity to Europe Access links (2)
Traceping Oxford to CERN for 31st October 2001
loss around ten155-gw.ja.net router
J. Macallister
8
Connectivity to Europe Geant

• Start mid November 2001
• UKERNA switched off TEN-155 3 Dec 2001
• Monitoring see later

9
Connectivity to US

• 6 155 Mbit links
• US link about to go to 2.5 G bit
• IP commodity attached in London
• Research traffic over 2.5G bit
• Peer in Hudson St.
• 622 Mbit to Esnet
• 622 Mbit to Abilene.

10
Monitoring US Traffic
UKERNA Traffic data Kbit/s. Blue Traffic from US
Maroon Traffic to US 17 Jul 2000 (300kbit/s) 7
Dec 2001 (900kbit/s) peak is 88 of total BW
930 Mbit 10 minute averages 10 minute
averages
Last 7 days 1 hour averages Last 7 days 1
hour averages
11
Monitoring US Traffic - TrendsJan 99 Nov 02
UKERNA US Traffic Daily totals Mbytes
Peak load 88
Weekday
Weekend
J. Macallister
Xmas 99
Xmas 01
12
Grid Network Projects

• DataGrid
• GridPP
• Extending DataGrid to BaBar, D0, CDF, in UK
• DataTAG
• Foci Network resarch, Grid inter-operability
• MB-NG
• UK funded 2.5Gbit Development Network
• Optical Switching

13
Network Monitoring

• Several tools in test plugged into a coherent
  structure
• PingER, RIPE one way times, iperf, UDPmon, rTPL,
  GridFTP, and NWS prediction engine
• continuous tests for last few months to selected
  sites
• DL Man RL UCL CERN Lyon Bologna SARA NBI SLAC
• The aims of monitoring for the Grid
• to inform Grid applications, via the middleware,
  of the current status of the network input for
  resource broker and scheduling
• to identify fault conditions in the operation of
  the Grid
• to understand the instantaneous, day-to-day, and
  month-by-month behaviour of the network provide
  advice on configuration etc.
• Report written on LDAP scheme for publishing the
  network information
• Will be used by UK e-science sites and non HEP
  WPs

14
UDPmon Measuring Throughput with UDP

• Send a burst of UDP frames spaced at regular
  intervals
• Vary the frame size and the frame transmit
  spacing
• Record
• The time to send and the time to receive the
  frames
• The number received, the number lost, number out
  of order
• The received inter-packet spacing

15
Network Monitoring Architecture
LDAP Schema
Grid Apps GridFTP


Backend LDAP script to fetch metrics Monitor
process to push metrics
PingER (RIPE TTB) IperfER UDPmon rTPL NWS etc
Local Network Monitoring Store Analysis of Data
(Access)
local LDAP Server
Grid Application access via LDAP Schema to -
monitoring metrics - location of monitoring
data.
Access to current and historic data and metrics
via the Web, i.e. WP7 NM Pages, access to metric
forecasts
Robin Tasker
16
Network Monitoring Components
Clients
WEB Display
Analysis
Grid Broker
Predictions
LDAP
LDAP
Web I/f
raw
raw
Scheduler
control
control
Cron script
Cron script
Cron script
Tool
Ping
Netmon
UDPmon
iPerf
Ripe
17
Ping UDP throughput MAN-RAL From 20 Oct 01

• PingER rtt (ms)
• dl RAL
• 1000 byte packet
• Forecast
• UDPmon Zero packet loss!
• UDPmon throughput Mbit/s
• man RAL
• 300 1400 byte frames

18
Ping UDP throughput MAN-CERNFrom 20 Oct 01

• PingER rtt (ms)
• dl cern
• 1000 byte packet
• Forecast
• UDPmon throughput Mbit/s
• man cern
• 300 1400 byte frames

19
iperf UDP throughput MAN-CERN From 20 Oct 01

• Iperf throughput
• ucl cern
• 262144 byte buffer
• Forecast
• UDPmon throughput Mbit/s
• man cern
• 300 1400 byte frames

20
iperf UDP throughput MAN-SARA From 20 Oct 01

• Iperf throughput
• ucl sara
• 262144 byte buffer
• Forecast
• UDPmon throughput Mbit/s
• man sara
• 300 1400 byte frames

21
iperf Pinger UK-Bologna From 20 Oct 01

• Iperf throughput
• ucl Bologna
• 262144 byte buffer
• Forecast in green
• PingER rtt (ms)
• dl Bologna
• 1000 byte packet
• Forecast

22
iperf throughput UCL-SARA From 1 Nov 01 Geant
Operational

• Iperf throughput
• ucl sara
• 262144 byte buffer

Geant Enabled
Routing Stable
23
Brief outline of work plan

• High Performance High Throughput
• Understanding the end system HW,
• best way to monitor traffic and protocol packets,
  
• Type and effect of background traffic,
• Throughput vs rtt, throughput vs window-size,
• Throughput using multiple TCP streams, and effect
  on the Network
• GridFTP and BBftp at Gigabit,
• Effect of different TCP stacks,
• Use of non-TCP protocols, and effect on the
  Network
• Show and tell demos
• Bandwidth Reservation and QoS
• Identification of traffic classes and middleware
  components requiring QoS
• Study of traffic differentiation and traffic
  engineering techniques
• demonstration of QoS Bandwith Reservation

24
The EU DataTAG project

• EU Transatlantic Gigabit project.
• Status approved for 4 M ECU start on 1 Dec 2001
• Partners CERN/PPARC/INFN/UvA. IN2P3
  sub-contractor
• The main foci are
• Grid Network Research including
• Provisioning (CERN)
• Investigations of high performance data transport
  (PPARC)
• End-to-end inter-domain QoS BW / network
  resource reservation
• Bulk data transfer and monitoring (UvA)
• Interoperability between Grids in Europe and the
  US
• PPDG, GriPhyN, DTF, iVDGL (USA)

25
DataTAG project
NL SURFnet
MREN
STAR-LIGHT
STAR-TAP
CERN
ESNET
NewYork
IT GARR-B
Abilene
Olivier Martin
26
Optical Switching - DataTAG
2.5 Gbps SDH Lambda 100/1000 Mbps Ethernet
UK
Iwire Abilene ESnet
Amsterdam
StarLight
Italy
Application
7600
? Broker
CERN
27
MB NG SuperJANET Testbed
Leeds
SJ4 Dev C-PoP Warrington
SuperJANET4 Production Network
Gigabit Ethernet 2.5 Gbit POS 10 Gbit POS
SJ4 Dev C-PoP Reading
SJ4 Dev C-PoP London
RAL / UKERNA
RAL / UKERNA
28
E-science core project

• Project to investigate and pilot
• end-to-end traffic engineering and management
  over multiple administrative domains MPLS in
  core diffserv at the edges.
• Managed bandwidth and Quality-of-Service
  provision. (Robin Tasker)
• High performance high bandwidth data transfers.
  (Richard HJ)
• Demonstrate end-to-end network services to CERN
  using Dante EU-DataGrid and the US DataTAG.
• Status approved with requested funds start on 1
  Dec 2001
• PartnersCISCO, CLRC, Manchester, UCL, UKERNA
  plus Lancaster and Southampton (IPv6)
• A technical meeting held draft spec. of the HW
  required
• Would like to use real Grid traffic as
  demonstrator
• CDF UCL-RAL
• BaBar Man-RAL

29
And Finally in case it helps

• GridPP and DataGrid have been working on
  packaging Globus v2 as a .rpm kit.
• Makes installation simpler.
• Available from DataGrid WP6 web
• marianne.in2p3.fr/datagrid/testbed1/globus

30
Conclusion Status

• SuperJANET4 stable working well
• Access links to Campus becoming very inportant
• Network Links and peering to US good overall
• US-UK bandwidth upgrade overdue
• Geant access to Europe OK (at the moment !)
• Considerable UK interest in the Internet2 tests
• Grid Lots of interesting work to do