Recovering Internet Symmetry in Distributed Computing

1
Recovering Internet Symmetry in Distributed
Computing

• Sechang Son, Miron Livny
• sschang, miron_at_cs.wisc.edu

2
Contents

• Introduction
• Dynamic Port Forwarding
• Architecture
• Implementation
• Analysis
• Generic Connection Brokering
• Architecture
• Implementation
• Analysis
• Conclusion

3
Fate of Private Network

• Introduced as a short term solution to IPv4
  address shortage problem until the full
  deployment of IPv6
• May be not disappeared
• Easy network planning and maintenance
• Cost nothing
• Grid is a big supporter of private network

4
NAT, Masquerading, and Port Forwarding
Private network
NAT
A
B
B X
5
Condor/Grid Requirements

• No change to public side interoperable with
  (existing) regular sockets
• Minimum changes to network components and no
  change to kernel or having system-wide impact
• Easy deployment is one of key factor of Grid
  system
• Highly scalable
• Clusters with hundreds or thousands machines must
  be supported
• High performance
• Representative requirements of any Grid System

6
Previous Works

• Global Approaches
• TRIAD, IPNL, AVES
• Local/Fill-the-gap Approaches
• Napster, Gnutella Application specific
  connection brokering
• SOCKS
• Realm Specific IP (RSIP)
• No system meets Grid requirements

7
Dynamic Port Forwarding (DPF)
Private network
B
8
Implementation of DPF

• Client
• molded into the communication library of Condor
• Creates and deletes forwarding rule on the fly
• Server
• Uses NAT library to add/list/delete port
  forwarding rule
• Maintains 3 different representations of
  forwarding rules for fault tolerance and
  updates/synchronizes those in a careful way
• Periodically polls clients for garbage collection

9
Analysis of DPF

• Highly Interoperable
• Highly scalable
• The number of proxy addresses leased to clients
  is only limiting factor
• DPF with multiple ip addresses is supported
• Very deployable
• No changes to OS, network component, or NAT
  required
• DPF server runs as privileged user
• Security
• Opens holes under administrators permission and
  as long as necessary

10
Analysis of DPF

• Performance Private-to-public

Regular Regular DPF DPF
TCP UDP TCP UDP
Connection Setup 1656 (258) 10167 (2032) 1703 (552) 12086 (303)
Data Xfer 22952 (3800) 2010 (912) 24863 (2121) 693 (260)
11
Generic Connection Brokering (GCB)
Private network
B
12
Generic Connection Brokering (GCB)
Private network
UDP
UDP
B
13
Generic Connection Brokering (GCB)
Private network
Regular socket
B
14
Implementation of GCB client
socket
bind
connect
execve
fork
accept

dup

fd 0
fd i
fd k
socket
bind
connect
execve
fork
accept

dup

15
Implementation of GCB server

• Composed of Broker and RelayServer
• Broker in charge of arranging the direction of
  connection
• RelayServer creates proxy sockets and handles
  relay between two sockets
• Broker forks new RelayServer on the fly
• Stale status due to server crash or machine
  reboot is handled by reregistration

16
Reliable UDP

• Used for communication between clients and server
• Reliable and in-order delivery
• Simple congestion control
• Connected and unconnected UDP
• Time-wait state

17
Analysis of GCB

• Very interoperable
• Public node needs to be a GCB client to get
  brokered
• Regular sockets can talk to GCB nodes through
  relay service
• Highly deployable
• No changes to OS, network component, or NAT
• No requirement for NAT and GCB server runs as a
  normal user
• Very scalable
• Logically as scalable as DPF, but performance can
  be a limiting factor
• Security
• Opens no hole
• May increase the chance of misuse of
  organizations policy

18
Analysis of GCB

• Performance Private-to-public

Regular Regular GCB GCB
TCP UDP TCP UDP
Connection Setup 1656 (258) 10167 (2032) 31428 (2720) 22868 (5193)
Data Xfer 22952 (3800) 2010 (912) 21051 (1045) 745 (136)
19
Firewall

• Both firewalls and private networks damage
  Internet connectivity
• Connections blocked
• Firewall intentional
• Private network side-effect
• Condors mechanism to restrict the range of ports
  that sockets can bind to can be used with either
  DPF or GCB to support firewalls that block some
  outbound connections too

20
Conclusion
DPF GCB
Scalability More Scalable Less Scalable
Change to Public Not Required Not Required
Deployablity NAT Dependent Independent
Nested Private No Support Support
Promiscuous No Support Support
Performance Faster Slower

• DPF for dedicated and large cluster
• GCB for medium and non-dedicated cluster