OptIPuter System Software

1
OptIPuter System Software

• Andrew A. ChienSAIC Chair Professor, Computer
  Science and Engineering, UCSD Director, Center
  for Networked Systems
• June 2004

2
Overview

• OptIPuter System Software Goals
• Year 2 Accomplishments
• A Model of Use for Dynamic Lambdas
• Distributed Virtual Computer
• High Speed Transport Protocols
• Year 3 Plans and Additional Pieces of the Puzzle

3
OptIPuter System Software Goals

• Make On-Demand Lambdas Approachable for
  Applications
• Define Software Architecture Internal, External,
  Infrastructure
• Define Application Abstractions Model of Use
• Deliver the Communication Capabilities of Lambdas
• High Performance Transport Protocols
• Novel Communication Capabilities (multicast,
  multi-endpoint)
• Expose and Exploit Optical Network Control
• Enable and Demonstrate Visualization and
  Data-intensive Applications
• Interactive Communication, Novel
• Direct Access to Distributed Displays and Storage
• Aggregate Wide-Area Distributed Storage
  Efficiently

4
OptIPuter Network Model

• Share underlying Physical Infrastructure with
  Routed Internet
• Employ On-Demand Dedicated Optical Paths to
  realize DVCs
• New End-to-End Capabilities Extraordinary
  Bandwidth, Private Connections

5
OptIPuter Software Architecture for Distributed
Virtual Computers v1.1
OptIPuter Applications
Visualization
DVC 1
DVC 2
DVC 3
Layer 5 SABUL, RBUDP, Fast, GTP
Real-Time Objects
Security Models
Data Services DWTP
Higher Level Grid Services
Grid and Web Middleware (Globus/OGSA/WebServices
/J2EE)
Layer 4 XCP
Node Operating Systems
l-configuration, Net Management
Physical Resources
6
Year 2 Accomplishments

• How is a LambdaGrid Different from a Grid in
  Terms of Middleware?
• Novel Integrated View of Resource and Networks
  for Grid Applications
• Configurable Networks and Integration with
  Shared Internet
• Applications Expect High Capability, Guaranteed
  Performance
• Developed Model of Use (Distributed Virtual
  Computer) which supports Interactive
  Communication and Efficient Network Resource
  Management
• Integration of Configurable Networks and Shared
  Internet
• Resource Specification and Selection
• Communication (Transport and Multi-Point
  Protocols)
• Groups, Security, and Distributed Storage
  Integration
• Defined Real-Time Distributed Virtual Computer
• Developed an Initial Prototype of Distributed
  Virtual Computer
• How do we Control Lambdas and How do Protocols
  Influence Their Utility?
• High Speed Transport Protocols
• Integration Framework in DVC, Uniform
  Presentation
• XCP, UDT, GTP, working on RBUDP/LambdaStream and
  Multicast
• We can fill High BDP Paths in Shared, Routed
  Networks
• XCP and UDT Real Implementations and Promising
  Evaluations
• We support Rich Communication Patterns in
  Dedicated Lambdas

7
A Model of Use for On-Demand Lambdas

• Andrew A. Chien
• OptIPuter Site Visit
• June 2004

8
Models of Use

• 1. Automatic Flow Optimization
• End Systems or Network Detects Large Flows and
  Configures Optical Paths to Optimize Extant Flows
• Intelligent Network, Optimizes for Application
  and Network Flow Performance various,
  BigBangwidth
• 2. Scheduled Transfers Optimized FTP Cheetah,
  Veeraraghavan03
• Applications Request File Transfers
• Network Schedules and Configures Dedicated Paths
• Optimizes Network and End Systems for File
  Transfers
• 3. Distributed Virtual Computer Grid with High
  Performance Private Network DVC,
  TaesombutChien04
• System View of a Grid Resource Collection
• Private Network Constructed and Managed for High
  Performance
• Lambda Grid Dedicated Lambdas Grid Resource
  Collection
• Integrates Resources, Networks in SAN-like Fashion

9
Distributed Virtual Computer

• Application Request Grid Resources AND Network
  Connectivity
• Redline-style Specification, 1st Order Constraint
  Language LiuFoster2002
• DVC Broker Establishes DVC
• Binds Resources and Network
• Encapsulates Access to Optical Signaling and
  Setup (MambrettiYu)
• Leverages Grid Protocols for Security, Resource
  Access
• Application Executes in Private Resource
  Environment

10
DVC Examples
SDSC
UCI or UIC
SIO/NCMIR
UCSD CSE

• TeleMicroscopy Experiment DVC
• Microscope Compute Resources Storage System
• Dedicated Lambdas Switching
• Collaborative Visualization Real-Time DVC
• Grid Resources Real-Time (TMO, Kim)
• Dedicated Lambdas Switching
• Photonic Multicast or LambdaRAM (Leigh)

11
Distributed Virtual Computer Services
Optical Network Control
Novel Transport Protocols

• DVC is a Distributed Resource Abstraction
  (including Network)
• Integrates Transport Protocols and Multi-party
  Communication (group namespace)
• Direct Access Wide-Area Shared Storage and
  Devices

12
BIRN Distributed Virtual Computer

• Biomedical Informatics Research Network (BIRN)
• Geographically Distributed Data Sources
• Sharing of Instruments, Compute Resources,
  Visualization Resources
• DVC Grid Resources Private Network
• Data Servers at NCMIR, Harvard, Duke
• Computing Resources at SDSC
• Displays and Visualization Clusters at UCI, UCLA,
  UNC
• Private Network for High Performance Access,
  Multipoint Protocols

13
Example Dynamic Configuration of Lambda Grid

• Creating a BIRN DVC
• Send ResourceCommunication Specification to DVC
  Manager
• Create Resource Groups (i.e. for Collective Data
  Source and Sink)
• Create Communication Sessions and Properties
  (e.g. Security, QoS, etc.)
• Execute Simplified Application

Duke
Harvard
NCMIR/UCSD
GTP enc auth
TCP Optical Multicast
SDSC
UNC
UCLA
UCI
Physical-Level View of BIRN DVC
Application-Level View of BIRN DVC
14
DVC Advantages

• Applications
• Simplifies View, Hides Complexity
• Automates compute/data resource binding
• Automate dynamic ?-configuration expose novel
  ?-capabilities
• Controllable, Secure, Trusted Environment (direct
  access)
• Aggregates Resources with SAN-like model
• Trusted and Secure Environment
• High Bandwidth, Deterministic (10Gbps, no
  jitter)
• Multi-party Communication
• Interactive, Real-time Applications
• Distributed Resource Context (tie to Web Services
  with WSRF)
• System
• Enables Optimized Resource Selection and Use
• Declarative Specification of Resource and Network
  Configuration
• Optimized End Resource, Dedicated Lambda, and
  Switch Selection
• Coordinated End Resource and Network Binding
• Pragmatics
• Leverages VPN and Typical Grid Distributed
  Application structure

15
Vision -- RT Tightly Coupled Wide-Area
Distributed Computing

• Real-Time Object (TMO) network

Dynamically formed Real-Time (RT) Dist. Virtual
Computer (DVC)

• RT DVC Facilitates
• Message communications with easily determinable
  tight latency bounds
• (2) Computing node operations enabling easy
  guaranteeing of timely progresses of threads
  toward computational milestones.

Kane Kim, UC Irvine
16
For More Information

• L. Smarr, A. Chien, T. DeFanti, J. Leigh, P.
  Papadopoulos, The OptIPuter, Communications of
  the Association for Computing Machinery (CACM),
  47(11), November 2003.
• N. Taesombut and A. Chien, Distributed Virtual
  Computer (DVC) Simplifying the Development of
  Grid Applications, Grids and Advanced Networks,
  2004
• Andrew A. Chien, Xinran (Ryan) Wu, Nut Taesombut,
  Eric Weigle, Huaxia Xia, and Justin Burke ,
  OptIPuter System Software Framework, UCSD
  Technical Report CS2004-0786.
• Kane Kim, Wide-Area Real-Time Distributed
  Computing in a Tightly Managed Optical Grid - An
  Optiputer Vision, Paper and Keynote speech at
  Advanced Information Networking and Applications
  2004, Fukuoka, March, 2004.

17
High Performance Transport Protocols

• Andrew A. Chien
• OptIPuter Site Visit
• June 2004

18
High Performance Transport Problem

• OptIPuter is Bridging the Gap Between High Speed
  Link Technologies and Growing Demands of Advanced
  Applications
• Transport Protocols Are the Weak Link
• TCP Has Well-Documented Problems That Militate
  Against its Achieving High Speeds
• Slow Start Probing Algorithm
• Congestion Avoidance Algorithm
• Flow Control Algorithm
• Operating System Considerations
• Friendliness and Fairness Among Multiple
  Connections
• These Problems Are the Foci of Much Ongoing Work
• OptIPuter Pursuing Range of Transport Protocols
• Shared, Routed Infrastructure XCP, UDT
• Provisioned Lambda RBUDP/LambdaStream, GTP

19
XCP for High BDP Networks Shared, Routed
Environment

• Systematic Implementation and Evaluation
• Build/Test FreeBSD Kernel implementation,
  Performance Evaluation
• Full protocol specification and mature the
  protocol
• Work with the community (researchers,
  applications developers, users, vendors,
  operators, IETF), To Develop Deployment plan
• Initial Results are Promising Match Simulations

XCP Measured
TCP Measured
BannisterFalk, USC-ISI
20
LambdaStream Interactive High Bandwidth

• Visualizations Need High Performance for Small,
  Frequent Payloads
• LambdaStream a Streaming descendant of RBUDP
• Will be integrated into the DVC Communication
  Framework
• Will Data transport for JuxtaView, Vol-a-Tile and
  TeraVision/SAGE Visualization
• Techniques
• Early Acknowledgment Of Loss to Minimize
  Jitter
• Adaptive Rate Control Based on Inter-packet
  Arrival Times Bandwidth Estimation
• Prediction Of CAUSE Of Loss Where is it
  Occurring and Why
• Status
• Simulation Modeling Complete and Shows Predicted
  Outcome.
• Preliminary Prototype Complete and Undergoing
  Experimentation Between Amsterdam and Chicago.
• Integration with Teravision Underway

Leigh, UIC
21
Optical Network Cores Shift Contention to Network
Edge

• Lambda-Grid Dedicated Optical Connections
  Provide Plentiful Core Bandwidth
• Driving Applications Access Many High Data Rate
  Sources
• Multipoint-to-point communication
• gt Congestion moves to the endpoints
• Group Transport Protocol Rate-based Receiver
  Based Management

S
3
S
1
S
2
R
(a) Shared IP Network (b) Dedicated
lambda connections
Wu Chien, UCSD
22
GTP Receiver-based Congestion Management

• Request-response for Reliable Data Transfer
• Single Flow Adaptation and Capacity Estimation
• Receiver-based Flow Scheduling for Fairness and
  Low Loss Rate
• Balance Concurrent Data Fetching from Multiple
  Sources
• Fair across Varied Sender RTTs
• Efficient Transitions under Rapid Changes

R2
R1
Multipoint-to-point contention at receivers
GTP Receiver Architecture
23
Fairness and Convergence

• Multipoint Performance in NS2 Simulations
• Four GTP flows with RTT 20, 40, 60 and 80ms
  starting at time 0, 2, 3, and 4s.

24
Quick Adaptation to Flow Transition

• GTP Simulation, Emulation, TCP Simulation
• Second Flow begins at t10 seconds
• GTP Utilizes Network Efficiently through Flow
  Transitions

Converging Flows
25
For More Information

• X. Wu and A. Chien, GTP Group Transport Protocol
  for Lambda Grids, IEEE Symposium on Cluster
  Computing and the Grid (CCGrid), April 2004.
• X. Wu and A. Chien, Evaluation of Rate-based
  Transport Protocols for Lambda Grids, IEEE
  Conference on High-Performance Distributed
  Computing (HPDC-13), June 2004
• Y. Gu, X. Hong, and R. Grossman, Experiences in
  Design and Implementation of a High Performance
  Transport Protocol, (submitted for publication).
• A. Falk, T. Faber, J. Bannister, A. Chien, R.
  Grossman, J. Leigh, Transport protocols for high
  performance, Communications of the ACM, Volume
  46, Number 11, November 2003, pp. 42-49.

26
Year 3 Plans

• DVC Advance and Experiment with Distributed
  Virtual Computer Interface/Service
  (UCSD/UIC/UCI/SIO)
• DVC Implementation to Unifies Interface to
  Transport Protocols
• Network Planning and Lambda Configuration
  Services
• Extensive Simulations of LDPC-based statistical
  approach, RobuSTore prototype Experiments with
  Larger Integrated Collections of Services, become
  a Grid Service
• Real-time DVC Prototype which integrates the TMO
  middleware subsystem and demonstrate a Real-time
  Distributed OptIPuter application on an RT DVC
• Definition of DVC security Models and Security
  Protocols which implement them
• Communication Experiment and Deploy Transport
  and Novel Communication Protocols Shared
  Networks and Provisioned Lambda
  (UCSD/UIC/USC-ISI)
• SABUL/UDT 10GigE networks and IETF Draft
• XCP Continued Experiments and refined
  Implementation
• LambdaStream support for Visualization/Collaborati
  on
• GTP Implementation, Evaluation, and Integration
  into DVC
• MEP Design, Implementation, Evaluation
• Applications Experiments, Demonstrations, and
  Performance Modeling (UCSD/UIC/TAMU)
• DVC Application, Visualization and Testbed
  Experiments
• Communication Application, Visualization and
  Testbed Experiments
• Measure Performance of Viz Applications and
  Characterize Computation, Communication, and
  Storage access

27
Multi-EndPoint Communication
Uh-oh!

• Network Transfers Faster than Individual Machines
• A Terabit flow? A 100Gbit flow? A 10Gbps flow w/
  1Gbps NICs
• Clusters are Cost-effective means to terminate
  Fast transfers
• Support Flexible, Robust, General N-to-M
  Communication
• Manage Heterogeneity, Multiple Transfers, Data
  Accessibility

28
RobuSTore Gigabytes per Second from
Geographically Distributed Storage

• BIRN Distributed Data, Intensive Analysis 100GB
  1 PB
• Comparative and Collective Analysis,
  Visualization of Multi-Scale Data Objects
• How to Access Data From Many Devices and Sites
  with High Performance?
• How to Share the Devices and Sites with Good
  Performance?
• RobuSTore Statistical Storage
• Systematic Introduction of Redundancy, High
  Efficiency LDPC Codes
• Improve Aggregate Statistical Properties of
  Access gt Better Performance
• High Parallel Performance, Isolatable Performance
  in Shared Environments

29
Students

• Xinran (Ryan) Wu PhD/UCSD Group Transport
  Protocol
• Nut Taesombut PhD/UCSD Distributed Virtual
  Computer
• Justin Burke PhD/UCSD RobuSTore
• Huaxia Xia PhD/UCSD RobuSTore and Low-Density
  Parity Check Codes
• Eric Weigle PhD/UCSD Multi-endpoint Protocol
• Frank Uyeda Cal-(IT)2 Undergraduate Fellow and
  MS/UCSD Low-Density Parity Check Codes
• will be at lunch