Recent Developments in Logistical Networking

1
Recent Developments in Logistical Networking

• Micah Beck, Assoc. Prof. Director
• Logistical Computing Internetworking
  (LoCI) Lab
• Computer Science Department
• mbeck_at_cs.utk.edu
• APAN Meeting, Honolulu, HI Jan 29, 2004

University of Tennessee
2
What is Logistical Networking

• A scalable mechanism for deploying shared storage
  resources throughout the network
• An general store-and-forward overlay networking
  infrastructure
• A way to break long transfers into segments and
  employ heterogeneous network technologies
• P2P storage and content delivery that doesnt
  using endpoint storage or bandwidth

3
The Network Storage Stack
Applications

• Our adaption of the network stack architecture
  for storage
• Like the IP Stack
• Each level encapsulates details from the lower
  levels, while still exposing details to higher
  levels

Logistical File System
Logistical Tools
L-Bone
exNode
IBP
Local Access
Physical
4
IBP The Internet Backplane Protocol

• Storage provisioned on community depots
• Very primitive service (similar to block service,
  but more sharable)
• Goal is to be a common platform (exposed)
• Also part of end-to-end design
• Best effort service no heroic measures
• Availability, reliability, security, performance
• Allocations are time-limited!
• Leases are respected, can be renewed
• Permanent storage is to strong to share!

5
Internet Backplane Protocol (IBP)
allocate!
Na
depot
capability
store!
Nw
data
depot
load!
Nr
6
Software Infrastructure

• Tools open source, multiplatform
• IBP Depot (server) and C client library
• exNode and end-to-end services library
• Logistical Backbone server (LDAP-based)
• Linux/C is primary development platform
• Java client
• Command-line utilities, GUI
• Public L-Bone deployment
• 25 TB deployed globally

7
L-Bone August 2003 (20TB)
8
Current Issues in IBP

• Separate TCP connection for each IBP operation
• High latencies between dependent operations
• Redundant DNS resolution for repetitive IBP ops
• High latency, load on DNS server
• Client access control on IBP by IP address only
• Limiting, easily spoofed
• All allocations on one depot managed uniformly
• Lack of control, inefficient
• No computational capabilities at the depot
• Limiting, inefficient

9
New IBP Depot 1.4 Release (Feb 2004)

• Persistent TCP connections optionally maintained
  between client and depot to reduce overhead
• Optional DNS caching eliminates redundant lookups
  in some OS implementations
• Secure IBP variant supported with X.509
  certificate authentication
• Multiple resources (File Sys, RAM) supported on
  one depot with optimized cut-through transfers
• Yong Zhen Huadong Liu

10
The Network Functional Unit

• Network Functional Unit computational operations
  supported on data stored in depot allocations
• See An End-to-End Approach to Globally Scalable
  Programmable Networking by Beck, Moore Plank
  in Future Directions in Network Architecture, A
  SIGCOMM 2003 Workshop
• Jeremy Millar, Alex Bassi Yong Zheng

11
Current Issues with LoRS

• Replication only means of increasing data
  availability
• Inefficient in use of storage
• Point-to-multipoint data transfer not supported
• Inefficient, cumbersome
• Cumbersome client installation procedures
• Users prefer Web-like ease-of-use

12
Upcoming Tool Releases

• Reed-Solomon encoding as an end-to-end service in
  Logistical Runtime System (James S. Plank
  Stephen Soltesz)
• Reduces use of storage and controls F.T.
• Asynchronous multicast engine with replaceable
  policy module (Ying Ding)
• Generic framework for experimentation
• Java Webstart implementation of LoRS Download
• Close to zero installation, speed matches C

13
Conclusions

• The LoCI Lab continues to improve and expand the
  Logistical Networking toolset
• We are eager for more application groups and more
  collaborative developers
• We are excited about the work you will hear about
  today and hope to hear about your new projects in
  the future
• Contact me at mbeck_at_cs.utk.edu
• http//loci.cs.utk.edu