iDIBS: Reliable and Efficient Distributed Backup

1
iDIBS Reliable and Efficient Distributed Backup

• Tam Chantem, Philip Little and Faruck Morcos

2
Improving Peer-to-Peer Backup Systems

• Objective Improve peer-to-peer backup
  techniques.
• Approach Built on DIBS, an existing peer-to-peer
  backup system.
• Results Improved 3 aspects of DIBS
• Reliability, Network Load, and Computation Time.

3
iDIBS
Peer
File
Encoding
Peer
Peer
Database
4
iDIBS
Peer
File
Encoding
Peer
Peer
Database
5
Backup and Recovery
Encoding and Distributing
Peer
File
Peer
Peer
6
Backup and Recovery
Recovering and Decoding
Peer
File
Dead Peer
Peer
7
Peer-List Backups
Unmodified DIBS after a crash
Do you have my data?
Client
8
Peer-List Backups
iDIBS after a crash
Do you have my peer list?
Client
9
Peer-List Backups
iDIBS after a crash
Client
10
Thresholds

• K Required number of pieces to recover
• N Total number of pieces transmitted
• Redundancy level
• DIBS N 2K
• iDIBS Recovery when up to 1 peer is dead

11
Example
Assume Peers 4 K 5 N 6
Peer 1
Peer 2
Peer 3
Peer 4
Cant recover!!!
So, N 8
12
Reduction in Network Utilization
13
Luby Transform Codes

• Reed-Solomon (RS) codes not scalable
• Luby Transform (LT) codes
• Digital Fountain concept
• Probabilistic nature
• 15 redundancy needed for successful decoding
• Flexible symbol size T, to speed up decoding

14
Implementation LT Codes

• Encoder and Decoder modules
• Encoder
• Calculates redundancy needed
• Encodes and splits file depending on T, number of
  users, and probabilistic parameters
• Decoder
• Decodes pieces
• Reconstructs file

15
Luby Transform Codes Encoder/Decoder Performance
LT codes vs. RS codes
Encoder T256
Decoder T256
Processing Time (sec)
File Size (Kb)
16
iDIBS vs. DIBS network utilization behavior

• DIBS network utilization is flat.
• iDIBS has decreased network utilization.
  (OH when userslt10)
• Tradeoff in the theoretical minium is better in
  iDIBS.
• Results shown for T32, for larger numbers iDIBS
  plots are better!

17
The iDIBS tradeoff
18
Contribution

• We contributed to the improvement of DIBS in the
  following ways
• Increase of performance through the introduction
  of a new encoding technology. (LT codes)
• Introduction of this encoding scheme to the
  application of peer-to-peer backup systems.
• More Reliability through redesign of the system
  recovery scheme. (Peer-Lists)
• Reduces Network utilization by changing the
  philosophy of number of transmitted pieces N and
  LT codes.

19
Questions??
20
Implementation Timeouts Thresholds

• If Peers gt K then N K 1else PiecesPerPe
  er 1 while PiecesPerPeer (Peers - 1) lt K
  do PiecesPerPeer PiecesPerPeer 1 N N
  Kend if
• If a peer is offline, recovery is still possible