Mondo

1
Mondo

• Bringing Distributed File Systems
• to the People
• Xavid Pretzer

2
Location-Independent Files

• What do we want to be able to do?
• Existing Strategies
• Distributed File Systems
• Peer-to-Peer File Sharing
• Mondo a different approach

3
Whats the problem?

• Sharing files between home and work
• Use existing computers to help distribute data
• Distributing information through
  often-partitioned networks

4
Distributed File Systems

• NFS emulates local access
• All peers trusted
• AFS uses trusted servers, untrusted clients
• Also introduced a global namespace
• Coda disconnected operation

NFS
AFS
Coda
5
Why isnt this enough?

• Need dedicated, trusted servers
• Servers can bottleneck file access
• Need more servers for more clients
• Extra client space not used
• Localized failures disrupt system

6
Peer-to-peer File Sharing
Gnutella

• Composed of untrusted peers
• Popular files easier to obtain
• Gnutella share files without centralized server
• Bittorrent cooperate to share large files with
  low bandwidth

Seeder
Tracker
Bittorrent
7
Limitations of File Sharing

• Limited access control and authentication
• No dynamic files
• No useful directory structure
• Difficult to ensure availability

Seeder
Seeder
F
F
?
?
8
Combining P2P with DFS
Farsite
/farsite

• Farsite serverless
• Uses encryption, Byzantine protocols
• Trusts self-reporting
• Files delocalized
• Ivy log-structured
• Logs changes to distributed hash table
• Must agree on which logs to trust
• Can recover from broken trust

/farsite/user
b9
e1
01
8c
Ivy
25
6a
42
9
Mondo a different approach

• File data served both by primary hosts and
  caching clients
• File location info stored in a Distributed Hash
  Table
• Encryption and cryptographic signing used for
  file permissions
• No central authority needed
• Expandable to large, heterogeneous groups

10
Data Storage

• Primary hosts always keep designated files
• Clients cache used files and share with other
  peers
• Mutual exchanges for file replication
• File blocks requested in parallel from multiple
  peers

H ? 1 ? 2 ? 3 ? 4 ?
P
3
3
H ? 1 2 ? 3 4 ?
H ? 1 ? 2 ? 3 4
1
4
11
Permissions without Trust

• File permissions signed by owner
• File data, version, block checksums signed by
  writer
• Read-restricted files encrypted with unique key
• Read key encrypted in header with readers public
  keys

File 7ce5ab92 Owner Alice Writers Alice,
Bob Alices Read Key 523ea220 Bobs Read Key
9a45bc31 Charlies Read Key efed3238
Signed, Alice
Version 2 File length 3214 bytes Block size
1024 bytes Block 1 checksum a4b23ac4 Block 2
checksum 8bed0123 Block 3 checksum
76f3dc13 Encrypted File Data
Signed, Bob
12
Directories

• Directories are files and use permissions
• Directories store file ids for contents
• Also store public keys for file owners as a web
  of trust
• Directories can be multiply linked

Directory 5ab9217e
Signed, Alice
Version 7 Contents
info.txt 8cd349a3 Owned by Alice
photo.jpeg 29468ecd Owned by Charles
junk ea2bc891 Owned by Joe
Signed, Bob
13
Locating Files
b9

• Peers serve as a Distributed Hash Table
• Each peer tells DHT what files it stores
• Primary host labels signed by file owner
• Changes propagated to all primary hosts clients
  update copies lazily

e1
01
8c
25
6a
42
5ab9217e
Cached by 18.3.11.92
5ab9217e
Signed, Alice
Primary host 18.244.3.61
14
Handling Malicious Peers

• Refuse to store improperly signed files
• DHT pairs stored redundantly
• Block checksums verify file data
• Tit-for-tat data sharing
• Periodically verify mutual replication

File 7ce5ab92 Owner Alice Writers Alice, Bob
Signed, Alice
Signed, Joe

H ? 1 ? 2 ? 3 4 ?
H ? 1 2 3 4
1
15
Example Reading a file

• Start with a root directory id and its owners
  public key
• Look up that id in the DHT to find peers with
  that directory
• Retrieve it and verify with the key
• Read contents and repeat recursively

/mondo 5ab9217e
DHT
18.244.3.61
Contents info.txt photo.jpeg
/mondo/info.txt 8cd349a3
16
Potential Difficulties

• Syncing changes efficiently
• Variable TTL?
• Availability of unpopular files
• Bogus DHT announcements
• Changes in file ownership
• Merging parallel changes

P
?
?
File 7ce5ab92 Owner Alice
Signed, Alice
chown
File 7ce5ab92 Owner Bob
???
Signed, ???
17
Applications

• Location-independent files without dedicated
  servers
• Making better use of existing resources
• Sharing mutable files with a large audience
• Distributing files on failure-prone networks

18
What Mondo Provides

• Scalable and secure distribution of mutable files
  among untrusted computers
• Advantages of DFS without dedicated servers
• Web of trust in directory hierarchy
• Effective distribution over normally-partitioned
  networks

19
References

• Adya, Atul, William J. Bolosky, Miguel Castro,
  Gerald Cermak, Ronnie Chaiken, John R. Douceur,
  Jon Howell, Jacob R. Lorch, Marvin Theimer, and
  Roger P. Wattenhofer. FARSITE Federated,
  Available, and Reliable Storage for an
  Incompletely Trusted Environment. 2002.
• Bolosky, William J., John R. Douceur, David Ely,
  and Marvin Theimer. Feasibility of a Serverless
  Distributed File System Deployed on an Existing
  Set of Desktop PCs. 2000.
• Cohen, Bram. Incentives Build Robustness in
  BitTorrent. 2003.
• Howard, John H., Michael L. Kazar, Sherri G.
  Menees, David A. Nichols, M. Satyanarayanan,
  Robert N. Sidebotham, and Michael J. West.
  Scale and Performance in a Distributed File
  System. 1988.
• Kon, Fabio. "Distributed File Systems Past,
  Present, and Future A Distributed File System
  for 2006". 1996.
• Maymounkov, Petar and David Mazières. Kademlia
  A Peer-to-peer Information System Based on the
  XOR Metric. 2002.
• Muthitacharoen, Athicha, Robert Morris, Thomer M.
  Gil, and Benjie Chen. Ivy A Read/Write
  Peer-to-Peer File System. 2002.
• Stoica, Ion, Robert Morris, David Karger, M.
  Frans Kaashoek, and Hari Balakrishnan. Chord A
  Scalable Peer-to-peer Lookup Service for Internet
  Applications. 2001