SHARCnet

1
Introduction
2
Readings

• Lessons from Giant-Scale Services, Eric Brewer,
  IEEE Internet Computing '01.

3
Data Centers

• Companies like Amazon, Google, eBay are running
  data centers with tens of thousands of machines
• Rate of growth is staggering

4
Data Centers

• A data center has a physical structure (racks of
  machines) and a logical structure (the one we
  just saw)
• Must launch the applications needed on them
• Monitor and relaunch if crashes ensue
• Poses optimization challenges
• We probably will have multiple data centers
• Must control the external DNS, tell it how to
  route
• Answer could differ for different clients

5
Data Centers

• A data center has a physical structure
  characterized by
• Workstation-class nodes
• Nodes connected by dedicated, low-latency network
• There may be multiple networks

6
Anatomy of a Cluster-Based Service

• IP network
• Load manager
• DNS round robin
• Layer-4/7 switches
• Servers
• Web
• Business logic
• Data store
• Internal network
• Backplane

7
Basic Model Components

• Clients e.g, browsers initiate queries to
  services
• IP network Could be the Internet or a private
  network provides access to service
• Load Manager Provides a level of indirection
  between the services external name and the
  servers physical names
• Servers Combining CPU, memory, and disks into
  an easy-to-replicate unit
• Datastore This is a replicated or partitioned
  database that is spread across the servers
  disks or network-attached storage (e.g., external
  DBMS or RAID).
• Backplane Handles interserver traffic such as
  redirecting client queries (optional)

8
Load Management

• Round-robin DNS
• Distributes different IP addresses for a single
  domain name among clients in a rotating fashion
• Good balancing
• Does not hide inactive servers e.g.,
• A client with a down nodes address will continue
  to try to use it until the DNS mapping expiries
• Could take several hours for this to happen
• Vendors now sell layer-4 switches to solve the
  problems with DNS

9
Load Management

• Level 4 Balancer Understand TCP connections
• Differentiation based on port numbers
• Web request routed differently than SMTP request
• Level 7 Balancer Understand application level
  information
• Can parse URLs
• Look at parameter of GET
• The balancers (switches) typically come in pairs
  so that failover is supported
• Used to address single point of failure

10
High Availability

• For high availability we should aim for
  something similar to the aims of other
  infrastructures e.g., telephone, rail or water
  systems
• Two possible approaches
• Partitioning
• Replication

11
Partitioning

• Lets then partition the data so that groups of
  servers handle just a part of the inventory (or
  any other data)
• Router needs to be able to extract keys from
  request
• Hashing is one strategy for doing this
• Based on the key you then determine the server to
  handle the request
• Need for deep packet inspection
• Example Amazon

12
Amazons Architecture
13
Partition Consistent Hashing

• Consistent hashing the output range of a hash
  function is treated as a fixed circular space or
  ring.
• Each node in the ring deals with a part of the
  output range
• One can think of each node in the ring as a
  virtual node. Multiple virtual nodes can be
  assigned to a physical node
• routers take request and apply a hashing
  function to determine where to send the request

14
Partition vs Replication

• The advantage of partitioning is that the load of
  handling requests is spread out.
• This suggests that requests can be processed
  faster
• Partitioning does not provide high availability
  though.
• Replication is needed for this
• Systems like Amazon use both replication and
  partitioning

15
Replication

• Each data item is replicated at N hosts.
• Amazon maintains a preference list The list of
  nodes that is responsible for storing a
  particular key.
• Each key is assigned a coordinator node which
  coordinates updates to an object.

16
Online Evolution

• Internet-time implies constant change
• Need acceptable quality
• Meet target MTBF, low MTTR, no cascading failures
• Three approaches to managing upgrades
• Fast reboot Cluster at a time
• Minimize yield impact
• Rolling upgrade Node at a time
• Versions must be compatible
• Big flip Half the cluster at a time
• Reserved for complex changes
• Either way use staging area, be prepared to
  revert

17
Challenges

• The preference list (replica group) has
  membership that dynamically changes
• One cause of a changing membership is failure
• Failed node may return to the group
• How do you make sure that the routing works
  (dont want to send to something that is down)?
• How do you make sure that each replica has the
  same version of an object?

18
Challenges

• Changes occur frequently!
• In the OSDI paper on Map Reduce (Google), authors
  comment that during one experiment that involved
  2000 nodes, sets of 80 kept dropping out.
• What are the monitoring mechanisms needed to
  ensure that node changes are reported in a timely
  fashion?
• What should the availability be? How fast should
  it be? All of this has an implication on resource
  usage.

19
Challenges

• Essentially there are lots issues to be dealt
  with including
• Data persistence, load balancing, membership,
  failure detection, failure recovery, replica
  synchronization, overload handling, request
  marshalling, request routing, system monitoring
  and alarming, etc