DataGrid WP7 Meeting CERN April 2002

1
Some Measurementson the SuperJANET 4 Production
Network(UK Work in progress)

2
The Measurements (1)

• Latency
• Round trip times measured using Request-Response
  UDP frames
• Latency as a function of frame size
• Slope gives sum of individual data transfer rates
  end-to-end
• Mem copy pci Gig Ethernet pci mem copy
• Histograms of individual measurements
• UDP Throughput
• Send a burst of UDP frames spaced at regular
  intervals
• Vary the frame size and the frame transmit
  spacing
• Record
• The time to send and the time to receive the
  frames
• The number received, the number lost, number out
  of order
• CPU load, Number of interrupts
• Packet Jitter
• Send a series of UDP frames spaced at regular
  intervals
• Record
• The received inter-packet spacing

3
MAN-RAL SuperJANET4 Topology
Gigabit Ethernet 2.5 Gbit POS Access 2.5 Gbit
POS core 622 Mbit Access
4
Latency UDP Man - RAL Gigabit Interfaces

• Latency 5.2 mS
• Slope 0.0904 us/byte
• For 2 PCs expect
• PCI 0.00188
• GigE 0.008
• PCI 0.00188
• Total 0.0118 us/byte
• 7 routers extra links
• 3 GigE 0.024
• 2 2.5G PoS 0.0064
• 1 622Mbit 0.012
• Total 0.0424
• Structure seen
• Period 88 bytes
• Variation 150 190 us

5
Latency Histograms

• Peak separation 120 us
• Multiple packets have fine structure 10 us

6
UDP Throughput UDP Man - RAL Gigabit Interfaces

• Manc Motherboard SuperMicro 370DLE Chipset
  ServerWorks III LE Chipset
• CPU PIII 800 MHz PCI64 bit 66 MHz
• RedHat 7.1 Kernel 2.4.14
• NIC NetGear
• RAL RedHat 7.1 Kernel 2.2.16-3
• NIC Intel pro 1000

• Max throughput 550Mbit/s
• Some throughput drop for packets lt 20 us
  spacing
• Packet loss similar to PC-PC

7
Throughput Intel Pro/1000 on P4DP6

• Motherboard SuperMicro P4DP6 Chipset Intel
  E7500 (Plumas)
• CPU Dual Xeon Prestonia (2cpu/die) 2.2 GHz Slot
  4 PCI, 64 bit, 66 MHz
• RedHat 7.2 Kernel 2.4.14

• Max throughput 950Mbit/s
• Some throughput drop for packets gt1000 bytes
• Packet loss small 800 1000 byte packets