Logistical Networking Basics

1
Logistical Networking Basics Apps

• Micah Beck
• Associate Professor
• Computer Science Department
• University of Tennessee Knoxville
• Universidade de São Paulo 7/28/2006

2
What is Logistical Networking (LN)?

• A simple, limited, generic storage network
  service intended for cooperative use by members
  of an application community.
• Fundamental infrastructure element is a storage
  server or depot running the Internet Backplane
  Protocol (IBP).
• Depots are cheap, easy to install operate
• Design of IBP is modeled on the Internet Protocol
• Goal Design scalability
• Ease of new participants joining
• Ability for interoperable community to span
  administrative domains

3
Internet Backplane Protocol (IBP)
allocate!
Na
depot
capability
store!
Nw
data
depot
load!
Nr
4
Internet Backplane Protocol (IBP)
capability
copy!
Na
depot
depot
compute!
Nr
depot
5
Sample exNodes
IBP Depots
Network
0
100
200
300
A
B
C
6
ExNode Example
ltexnodemappinggt ltexnodemetadata
namealloc_length typeintegergt200lt/exnodemet
adatagt ltexnodemetadata namealloc_offset
typeintegergt0lt/exnodemetadatagt
ltexnodemetadata nameexnode_offset
typeintegergt0lt/exnodemetadatagt
ltexnodemetadata namelogical_length
typeintegergt200lt/exnodemetadatagt
ltexnodereadgtibp//depot1port/key
string/READlt/exnodereadgt ltexnodewritegtibp//
depot1port/key string/WRITElt/exnodewritegt
ltexnodemanagegtibp//depot1port/key
string/MANAGElt/exnodemanagegt lt/exnodemappinggt
0
1
3
2
3
4
399
ltexnodemappinggt ltexnodemetadata
namealloc_length typeintegergt200lt/exnodemet
adatagt ltexnodemetadata namealloc_offset
typeintegergt0lt/exnodemetadatagt
ltexnodemetadata nameexnode_offset
typeintegergt100lt/exnodemetadatagt
ltexnodemetadata namelogical_length
typeintegergt200lt/exnodemetadatagt
ltexnodereadgtibp//depot2port/key
string/READlt/exnodereadgt ltexnodewritegtibp//
depot2port/key string/WRITElt/exnodewritegt
ltexnodemanagegtibp//depot2port/key
string/MANAGElt/exnodemanagegt lt/exnodemappinggt
7
Logistical Networking Stack
8
LoRS GUI Client Tools
Binaries for Windows and MacOSX. Source for
Linux, Solaris, AIX, others.
9
Failsafe Mechanisms for Data Streaming in Energy
Fusion Simulation
10
A Recent Deployment Proposal
11
ncview using netCDF/L
12
LoDN - Network File Manager
Store files into the Logistical Network using
Java upload/download tools.Manages exNode
warming (lease renewal and migration).
Provides account (single user or group) as well
as world permissions.
13
MPI-IO/L Features

• Direct I/O to any portion of remote file
• High-performance remote I/O
• Multithreaded data transfer
• Intelligent download scheme
• Easy management of replicas and stripes using
  exNode
• Easy file sharing via XML representation of exNode

14
MPI-IO/L Architecture
15
Computing for Distributed Viz Marching Cubes
Raycast (Huang, Liu)
CCGRID 06
16
Dynamic Co-Scheduling of Data Storage,
Processing and Transfer
17
Resource Sharing as a Primary Goal

• Whenever sharing can be implemented with
  practical bounds it enables collaboration,
  progress
• The Internet has problems (DoS, spam, ) but they
  are addressed due to its great value
• Wherever resource sharing is not established it
  is counterintuitive to those who manage resources
  to find a way to maximize the possible sharing
• Logistical networking is finding ways to expand
  the domain of resource sharing
• First storage, which has great uniformity
• Then computation, which is harder