Principal Resource

1

• Principal Resource
• OceanStore An Architecture for Global-Scale
  Persistent Storage John Kubiatowicz, Davic
  Bindel, Yan Chen, Steven Czerwinski, Patrick
  Eaton, Dennis Geels, Ramakrishna Gummadi, Sean
  Rhea, Hakim Weatherspoon, Westley Weimer, Chris
  Wells, Ben Zhao. Proceedings of ACM ASPLOS 2000.
• Additional Materials
• http//oceanstore.cs.berkeley.edu/
• www.cs.fsu.edu/awang/courses/cop5611_s2006/oceans
  tore.ppt www.cs.fsu.edu/awang/courses/cop5611_s2
  004/lecture_21_distributed_fs3.ppt
• Presenter
• Stanley Ziewacz
• April 10, 2008

2
Global-Scale Persistent Storage

• Global Scale
• 1010 users
• 10,000 files per user
• Population Clocks
• U.S. 303,807,847
• World 6,660,028,725
• 1612 GMT (EST5) Apr 08, 2008
• Mole
• 6.023 X 1023 atoms in exactly 12 grams of
  carbon-12

3
Persistent Data Storage

• Connectivity to all types of computing devices
• Desktop, laptop, palmtop, cellphone, etc.
• Information Safety
• Avoid prying eyes and survive malicious hands
• Durable1000 years?
• Redundancy with continuous repair and
  redistribution for long-term storage
• Uniform and highly-available access to data
• Data divorced from location
• Servers close to clients

4
Cooperative Utility Model

• OceanStore Service Providers
• Client pays one monthly bill to one company
• Clients can use resources on other OSPs
• OSPs buy and sell capacity among themselves
• Millions of servers

5
Design Highlights

• Infrastructure is only trusted in the aggregate
• Servers may crash without warning
• Servers may leak data
• Data can be cached anywhere, anytime
• Promiscuous caching
• Nomadic Data

6
Applications

• Groupware
• Personal information managers
• Digital libraries
• Scientific data repositories

7
System Architecture

• Naming
• Data location and routing
• Update model and conflict resolution
• Deep archival storage
• Introspection

8
Naming Globally Unique Identifiers GUIDs

• Object GUID
• Secure hash of owners key and human-readable
  name
• Server GUID
• Secure hash of servers public key
• Archival fragment GUID
• Secure hash over the data it stores

9
Access Control

• Restricting Readers
• If data is not completely public
• Encrypt
• Distribute key to users with read permission
• To revoke read permission
• Request that replicas be deleted or re-encrypted
• Restricted at clients
• Restricting Writers
• All writes are signed
• Owner can provide access control lists for
  objects
• Restricted at servers

10
Data Location and Routing

• Support location-independent routing
• Message routes to discover a destination
• Then message routes directly to destination
• Two routing strategies
• Bloom filters, fast probabilistic algorithm
• Plaxton-style routing

11
Bloom Filter

• A Bloom filter
• Represents a set S S1, Sn
• Is depicted by a m bit array, filterm
• Uses r independent hash functions
• h1hr
• for i 1n
• for j 1r
• filterhjSi 1
• www.cs.fsu.edu/awang/courses/cop5611_s2004/lectur
  e_21_distributed_fs3.ppt

12
Bloom Filter Example

• filter 1, 1, 0, 1, 0, 1
• Does x belong to the set?
• filterh1(x) filter0 1
• filterh2(x) filter3 1
• filterh3(x) filter5 1
• Does z belong to the set?
• filterh1(z) filter2 0 ? no
• filterh2(z) filter3 1
• filterh3(z) filter5 1
• www.cs.fsu.edu/awang/courses/cop5611_s2004/lectur
  e_21_distributed_fs3.ppt

13
Attenuated Bloom Filters
www.cs.fsu.edu/awang/courses/cop5611_s2006/oceans
tore.ppt
14
Variation on Plaxton Routing

• Each object GUID has root node
• Root ID matches GUIDs hash in the most bits
• But replicas can be placed anywhere
• Publishing process for replicas
• Do Plaxton hops from replica location to root
• Place a pointer to replica locale at each hop

15
Update Model

• Client generates updates
• Primary tier of replicas commit
• Evaluate updates predicates in time order
• Perform action with earliest predicate

16
Updating Ciphertext

• Only 4 predicates available
• Compare-version
• Compare-size
• Compare-block
• Search
• Actions available
• Replace-block
• Insert-block
• Delete-block
• Append

17
Serializing Updates

• Primary tier of replicas
• Byzantine agreement protocol
• Final commit order
• Multicast committed updates
• Tentative Updates
• Sent to several random replicas
• Tentative commits spread by epidemic

18
Deep Archival Storage

• Objects exist in both active and archival form
• Archival form is permanent read-only
• Form treated as series of fragments of data
• Fragments spread over the network structure
• Use any n fragments to reconstruct data
• In principle every version is archived

19
Introspection

• Cluster recognition
• Each client machine has an event handler
  triggered by each data access
• Replica Management
• Event handlers monitor client requests and system
  load
• Detect periodic migration of clusters from site
  to site
• OceanStore can monitor my work/travel routine