OptIPuter High Performance Transport Protocols

1
OptIPuter High Performance Transport Protocols

• Joe Bannister
• USC ISI
• September 2003

2
Acknowledgements

• Quanta RBUDP Team
• E. He (UIC Electronic Visualization Laboratory
  Graduate Student)
• J. Leigh (UIC Electronic Visualization
  Laboratory)
• SABUL Team
• R. Grossman (UIC Laboratory for Advanced
  Computing)
• Y. Gu (UIC Laboratory for Advanced Computing
  Graduate Student)
• GTP Team
• A. Chien (UCSD Concurrent Systems Architecture
  Group)
• X. Wu (UCSD Graduate Student)
• XCP Team
• J. Bannister (USC Information Sciences Institute)
• E. Coe (USC Information Sciences Institute
  Graduate Student)
• T. Faber (USC Information Sciences Institute)
• A. Falk (USC Information Sciences Institute)
• A. Kapoor (USC Information Sciences Institute
  Graduate Student)

3
High Performance Transport Problem

• OptIPuter is Bridging the Gap Between High Speed
  Link Technologies and Growing Demands of Advanced
  Applications
• Transport Protocols Are the Weak Link
• TCP Has Well-Documented Problems That Militate
  Against its Achieving High Speeds
• Slow Start Probing Algorithm
• Congestion Avoidance Algorithm
• Flow Control Algorithm
• Operating System Considerations
• Friendliness and Fairness Among Multiple
  Connections
• These Problems Are the Foci of Much Ongoing Work
• OptIPuter is Pursuing Four Complementary Avenues
  of Investigation
• Quanta RBUDP Addresses Problems of Bulk Data
  Transfer
• SABUL Addresses Problems of High Speed Reliable
  Communication
• GTP Addresses Problems of Multiparty
  Communication
• XCP Addresses Problems of General Purpose,
  Reliable Communication

4
OptIPuter Transport Protocol Roles
E2e Path
Allocated Lambda
Routed
Managed Group
Enhanced Routers
Standard Routers
Unicast
Quanta RBUDP
GTP
SABUL
XCP
5
Quanta and RBUDP

• Quanta is an Open-Source High-Performance
  Networking Toolkit / API That Requires no Changes
  to Applications
• Reliable Blast UDP (RBUDP)
• Assumes an Over-Provisioned or Dedicated Network
• UDP Data Channel and TCP Control Channel Are
  Combined to Support Payload Scale Reliable
  Transfers
• High Speed and Predictability
• Optimized for Large Payloads, but Being Enhanced
  for Variable Length Payloads Found in
  Visualization Streams
• Performance Results
• Clusters at NCSA and SDSC
• RBUDP Was Able to Achieve a Throughput of 18.62
  Gb/s out of the 20 Gb/s Available
• http//www.evl.uic.edu/cavern/rg/20030817_he

6
Photonic Data Services and SABUL

• Photonic Data Services is Open Source Software
  That Incorporates the Simple Available Bandwidth
  Utilization Library (SABUL) Protocol
• SABUL 2.3
• Combines a UDP Data Channel with a TCP Control
  Channel
• Congestion Signals Include Loss and Delay
• Rate- and Window-Based Flow Control
• Slow Start Connection Ramp-Up
• Selective Acknowledgments
• Striped Mode
• Performance Results
• 10 Gb/s Link Between Chicago and Amsterdam
• 2.7 Gb/s in Striped Mode
• 900 Mb/s in Single-Stream Mode

7
Group Transport Protocol (GTP)

• Objective Develop High Performance Multipoint
  Transport Protocols
• In OptIPuter Network Environment (No/Little
  Internal Network Contention)
• Why Grouping?
• Intragroup Management of Congestion Over Multiple
  Flows
• Support Data Fetching from Multiple Senders
  Concurrently
• Multi-Flow Scheduling Makes a Clean Transition
  when Flows Join or Leave
• Achieves Fairness Among Flows

Comparison between TCP and GTP Two Senders
and One Receiver, Flow 2 Starts at Time6s
8
XCP Design Accomplishments

• Explicit Congestion Protocol (XCP) Implemented
• BSD Unix Hosts and Routers
• Alpha-Grade Code Release
• Draft XCP Spec Completed
• ns-2 Simulation Experiments
• XCP Congestion Header Defined
• Located Between the TCP and IP Headers
• Carries RTT, Current Throughput, and Desired
  Throughput Change

0 1 2
3 0 1 2 3 4 5 6 7 8 9 0 1 2 3
4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
-------------------------
------- versionformat protocol
length unused
-------------------------
-------
rtt
-------------------------
-------
throughput
-------------------------
-------
delta_throughput
-------------------------
-------
feedback
-------------------------
-------
9
XCP Implementation

• Configuration
• BSD version 4.8-STABLE
• XCP Sender and Receiver Functions Have Been
  Implemented as Variants of TCP in the BSD Kernel
• XCP Replaces Only Congestion Control Alogrithm
• Can Integrate With Other Transport Protocols
  (SCTP, DCCP, )
• To Use TCP with XCP, an Application Need Only
  Modify the Transport Protocol Specified in the
  Socket Call socket(AF_INET, SOCK_STREAM,
  IPPROTO_XCP)
• XCP Router Functions Have Been Implemented on a
  BSD-Based PC Router
• Part of Forwarding Code
• Code Publicly Available at http//www.isi.edu/isi-
  xcp

10
XCP Performance Results

• Experimental Setup
• 2.4 GHz Single-Processor
• BSD Unix 4.8
• BDP-Size TCP Buffers
• 50 Packet Buffers in Router
• 100BaseT GigE Interfaces
• DummyNet Delay Inducer

ISI Net Lab
SIO Viz Ctr
11
Conclusions and Plans

• OptIPuter is Demonstrating New Ideas in High
  Speed Transport Protocols
• Prototype of RBUDP Achieves 18 Gb/s in Real Wide
  Area Networks (Chicago to San Diego)
• RBUDP Uses Aggregated Acknowledgements to Avoid
  per Packet Interactions That Slow Transmissions
• Prototype of SABUL Achieves Nearly 3 Gb/s in Real
  Wide Area Networks (Chicago to Amsterdam)
• SABUL Couples a UDP Data Channel With a TCP
  Control Channel to Implement Rate- and
  Window-Based Flow Control, Selective
  Acknowledgements, and Slow Start Connection
  Initiation
• GTP is Used to Provide Multipoint Data Transfer
  in Cluster Environments
• Prototype of XCP (Alpha Release of Host and
  Router Code) Has Been Demonstrated to Achieve
  Rapid Ramp-Up in Laboratory Tests
• XCP Separates Fairness Management from Congestion
  Control by Allowing Routers to Update Congestion
  Windows on the Fly, Thereby Achieving Full
  Throughput and Response to Congestion Rapidly
• OptIPuter Will Further Explore the Optimization
  and Application of Transport Protocols for Lambda
  Grids
• XCP Simulation Validations of Multiple-Connection
  Performance in Real Networks
• Quanta RBUDP Enhancements for Small Payloads
• GTP Experimental Assessment