An Overlay Routing Scheme For Moving Large Files

1
An Overlay Routing Scheme For Moving Large Files
ltzs_at_cs.wisc.edugt ltxuk_at_cs.wisc.edugt
Su Zhang Kai Xu
2
Outline

• Motivation
• Design and Implementation
• Evaluation
• Conclusions

3
Motivation

• Transferring large amount of data across Internet
  is challenging
• Long period of transferring vs. Problematic
  underlying Internet paths Paxson 96
• Path/node failures
• Temporary path outages
• Rapid route alternation
• Temporary routing loops
• An overlay routing scheme can help

4
Overlay Routing Scheme

• A group of application-layer routers
• Build on existing Internet routing substrate
• Choosing good transferring path
• Avoid problematic underlying paths
• Caching on intermediate routers
• Help on retransferring

5
The Generic Design
Router

Router
Router


6
File Transfer Routers

• Link-state based routing protocol
• Send Hello periodically
• Exchange link-state info.
• Detect degraded path performance and failures
• Build forward table dynamically
• Flexible path metrics
• Latency, available throughput, packet loss rate
• Application-specific metrics
• Network conditions fatal for one application, may
  not acceptable for another one

7
File Transferring

• File transfer servers find the closest router
• Propagate who owns cache queries and get metric
  info.
• Large files are split into chunks
• Each chunk is transferred independently
• Over underlying Internet path directly
• Or, via transfer routers
• Best path under current network situation
• Caching policy enforced

8
Caching on Routers

• Chunks are cached on intermediate routers
• Cache policies decided by application
• Build cache info table on each router
• On retransferring
• Cached chunks transferred from intermediate
  routers
• Caching policies two layer
• How to distribute chunks among routers
• How to share the cache storage on each router

9
Evaluation

• Limited experimental environment
• Tux lab
• Simulated network latency and degraded link
  performance
• Illustrate potential performance advantages of
  this routing scheme
• Experiments
• Overcoming degraded performance
• Caching improvement
• Flexible caching policy

10
Overcoming Degraded Performance

• Transferring a 10MB file
• Underlying links experiencing path outages or
  failures
• 10 of transferring time
• Degraded performance 10 - 100
• Routing through intermediate routers during
  performance failures ( 5 vs. direct link)

Direct link
Source
Destination
Router
11
Overcoming Degraded Performance
12
Caching Performance

• Limited caching capacity on intermediate routers
• Testing 10 - 100 data cached on the way
• Caching improved transferring time greatly
• Need more flexible cache policy!
• Spread cached chunks over multiple routers
• Drag frequently accessed chunks near destination

13
Caching Performance
14
Application Cache Policy Lottery vs.
Round Robin
A-gtB Using Round Robin to leave caches (the
first router caches seq1, the second router
caches seq2 and wrap back) A-gtC 73.9
improvement vs. w/o caching A-gtB Using Lottery
based on the distance to B (according to the hop
number, generate possibility) A-gtC
55.1 improvement vs. w/o caching
15
Conclusions

• Using overlay routing can greatly improve the
  performance and reliability of transferring large
  files over problematic underlying Internet links
• Dynamically selecting path based on different
  metrics to adapt to application requirement
• Using cache to speed up multiple transferring
• Flexible cache policy

16
Future Work

• How to setup the nodes on the Internet?
• Real experiments
• How to get network metrics (bandwidth, loss rate
  etc.) accurately?
• How to share the cache storage on each node for
  files in an efficient way?
• More caching policies
• How to recover transferring big files from
  interruptions?

17
Thank You! Questions?
18
Important Reference

• Resilient Overlay Networks http//nms.lcs.mit.ed
  u/ron
• End-to-End Routing Behavior in the Internet,
  Paxson, 96 Sigcomm
• The End-to-End Effects of Internet Path
  Selection, U of Washington