DataGrid WP7 Meeting Jan 2002

1
Initial Performance MeasurementsGigabit Ethernet
NICs64 bit PCI Motherboards(Work in progress)
2
The Measurements (1)

• Latency
• Round trip times measured using Request-Response
  UDP frames
• Latency as a function of frame size
• Slope gives sum of individual data transfer rates
  end-to-end
• Mem copy pci Gig Ethernet pci mem copy
• Histograms of individual measurements

3
The Measurements (2)

• UDP Throughput
• Send a burst of UDP frames spaced at regular
  intervals
• Vary the frame size and the frame transmit
  spacing
• Record
• The time to send and the time to receive the
  frames
• The number received, the number lost, number out
  of order
• The received inter-packet spacing
• CPU load, Number of interrupts

4
The Measurements (3)

• PCI Activity
• Logic Analyzer with
• PCI Probe cards in sending PC
• Gigabit Ethernet Fiber Probe Card
• PCI Probe cards in receiving PC

Gigabit Ethernet Probe
CPU
CPU
NIC
NIC
chipset
chipset
mem
mem
Logic Analyser Display
5
Latency Alteon AceNIC

• Motherboard SuperMicro 370DLE Chipset
  ServerWorks III LE Chipset
• CPU PIII 800 MHz PCI64 bit 66 MHz
• RedHat 7.1 Kernel 2.4.14

6
UDP Throughput Alteon AceNIC

• Motherboard SuperMicro 370DLE Chipset
  ServerWorks III LE Chipset
• CPU PIII 800 MHz PCI64 bit 66 MHz
• RedHat 7.1 Kernel 2.4.14

7
Latency SysKonnect

• Motherboard SuperMicro 370DLE Chipset
  ServerWorks III LE Chipset
• CPU PIII 800 MHz PCI64 bit 66 MHz
• RedHat 7.1 Kernel 2.4.14

• Latency good
• Slope expected 0.118 us/byte

8
UDP Throughput SysKonnect

• Motherboard SuperMicro 370DLE Chipset
  ServerWorks III LE Chipset
• CPU PIII 800 MHz PCI64 bit 66 MHz
• RedHat 7.1 Kernel 2.4.14

• Packet loss during drop

9
PCI SysKonnect

• Motherboard SuperMicro 370DLE Chipset
  ServerWorks III LE Chipset
• CPU PIII 800 MHz PCI64 bit 66 MHz
• RedHat 7.1 Kernel 2.4.14

• SK300
• 1400 bytes sent
• Wait 100 us

10
PCI SysKonnect

• Motherboard SuperMicro 370DLE Chipset
  ServerWorks III LE Chipset
• CPU PIII 800 MHz PCI64 bit 66 MHz
• RedHat 7.1 Kernel 2.4.14

• SK301
• 1400 bytes sent
• Wait 20 us
• Sk303
• 1400 bytes sent
• Wait 10 us
• Frames are back-to-back
• Cannot go any faster !

11
Latency Intel

• Motherboard SuperMicro 370DLE Chipset
  ServerWorks III LE Chipset
• CPU PIII 800 MHz PCI64 bit 66 MHz
• RedHat 7.1 Kernel 2.4.14

• Latency high
• Slope expected 0.118 us/byte

12
Throughput Intel

• Motherboard SuperMicro 370DLE Chipset
  ServerWorks III LE Chipset
• CPU PIII 800 MHz PCI64 bit 66 MHz
• RedHat 7.1 Kernel 2.4.14

• Packet loss during drop

13
PCI Intel

• Motherboard SuperMicro 370DLE Chipset
  ServerWorks III LE Chipset
• CPU PIII 800 MHz PCI64 bit 66 MHz
• RedHat 7.1 Kernel 2.4.14

• IT66M212
• 1400 bytes sent
• Wait 11 us
• IT66M212
• 1400 bytes sent
• Wait 11 us
• Packets lost
• Action of pause packet?

14
Latency Intel

• Motherboard SuperMicro P4CD6 Chipset Intel
  i860
• CPU Dual Xeon Prestonia (2cpu/die) 2.2 GHz
  PCI64 bit 66 MHz
• RedHat 7.1 Kernel 2.4.2

• Latency high
• Slope expected 0.118 us/byte

15
Throughput Intel

• Motherboard SuperMicro P4CD6 Chipset Intel
  i860
• CPU Dual Xeon Prestonia (2cpu/die) 2.2 GHz
  PCI64 bit 66 MHz
• RedHat 7.1 Kernel 2.4.2

16
PCI Intel

• Motherboard SuperMicro P4CD6 Chipset Intel
  i860
• CPU Dual Xeon Prestonia (2cpu/die) 2.2 GHz
  PCI64 bit 66 MHz
• RedHat 7.1 Kernel 2.4.2

• IT66M220
• 1400 bytes sent
• Wait 1000 us
• IT66M224
• 1400 bytes sent
• Wait 100 us
• Detail
• Chipset limits PCI transfers with STOPs
• Try i870 Chipset

17
PCI Intel

• Motherboard SuperMicro P4CD6 Chipset Intel
  i860
• CPU Dual Xeon Prestonia (2cpu/die) 2.2 GHz
  PCI64 bit 66 MHz
• RedHat 7.1 Kernel 2.4.2

• IT66M221
• 1400 bytes sent
• Wait 11 us