Backing Up for Eternity

1

• Backing Up for Eternity
• Dr. Martin Atkins
• Parvat Infotech Private Limited

2
Why don't people do backups?

• They don't understand how to do it
• in a reasonable time, given their setup, etc.
• They don't get any added value from backups
• until the system breaks!
• They don't like 'wasting' media
• I only wrote this CD last week....
• They don't think their data is so important
• right up until they lose it!
• They have better things to do

3
But I do do backups!

• Most backup systems
• are designed primarily to backup to tapes
• tapes are re-cycled (losing history)
• tapes and tape drives are expensive, awkward,
  unreliable, etc.
• can backup to write-once media, but
• each backup supersedes much of the old backup
• old media becomes almost useless (beermats!)
• no help for finding historical copies of data

4
Eternal - Management Overview

• Set-up and forget
• as far as possible
• Not just backup - also history
• What did that file look like yesterday?... Last
  week?... Last year?...
• Eternal history filesystem mounted on /backup
• e.g. cd /backup/2003/aug/21/home/martin/...
• Old archive media is never superseded
• it becomes part of your history record

5
How is this possible?

• Identical data blocks are never stored twice
• uses the 'Venti' algorithm
• developed for Plan 9 (from Bell Labs)
• tremendous 'compression', e.g.
• every machine in a network has the same copy of
  vmunix, etc
• most of my files are the same today, as they were
  yesterday
• makes it feasible to store the total system
  and/or network history

6
Techie Overview

• Client / Server architecture
• both can be on the same machine
• No Full / Incremental backup decision
• all backups are created equal
• every backup contains the entire filesystem state
• Primary operation is 'disk-to-disk'
• backups can be fully automatic
• restore operations do not need access to
  removable media

7
Friendly towards removable-media

• data is batched in CD (or DVD) sized units
• media can be written when it is convenient for
  you!
• data, once written, is never changed
• 2 copies can be made for extra security
• 1 for your shelf
• 1 to go off-site (in a bank vault?)
• media is only read if backup server must be
  restored

8
How does it work?

• Each block of data is indexed by its
  'fingerprint'
• the 160-bit SHA1 hash of the contents
• identical data blocks will hash to the same value
• different data blocks can also hash to the same
  value
• but only with a probability of 1 in 2160 (i.e. 1
  in 1048)
• if we store 106 terabytes, the chance of an
  identical hash somewhere in the data is about 1
  in 1020.
• SHA1 is 'cryptographically secure'
• meaning that 'you can't calculate it backwards in
  a reasonable time'

9
The Server

• Server receives request to store a block
• calculates the block's fingerprint
• if that finger print not in index
• store block in the backup history
• return the block's fingerprint
• Server receives request to retrieve a block
• fingerprint given in request
• if fingerprint is not in index
• return error
• return data from backup history

10
The Client

• E.g., backup
• /d1/d2/f1/d1/d2/f2/d1/f3
• tar-like utility
• files ? 20 bytes

Monday
f1
f2
f3
Tuesday
Key
f1
data block
fingerprint
f2
f3
f3'
11
Status and Information

• Eternal is currently being developed by Parvat
  Infotech in Bangalore
• first release intended for early 2004
• try-before-you-buy licenses
• individual and commercial prices
• For more information, please see http//www.parv
  at.com/products/eternal/