Sun Clusters

1
Sun Clusters

• Glenn Barney
• COMS E6998.002
• Columbia University

2
Sun Cluster What is it?

• Sun cluster 3.1 Its software part of the Java
  Availability Suite
• Highly available service-level management
  platform for single clusters
• Deploys applications as services with high
  availability, performance, and scalability
• For Solaris, is tightly integrated with Solaris,
  and is on SPARC, x86 (new with 3.1) and X64
  processors.
• Primary and associated secondary clusters for
  data replication
• Along with the Solaris OS, Sun server, storage
  and net connectivity products, makes up the
  SunPlex environment
• Its NOT Sun Grid Engine, although the latest SGE
  can use clusters
• Like all sorts of Sun Software its free!
• TSUBAME Grid Cluster is 7 on top500.org
  supercomputer list.

3
Availability Metrics

• Sun has come up with a measurable and repeatable
  metric system for availability that is
  benchmarkable
• Focusus on the 3 Rs
• Rate defines the number of fault and maintenance
  events that occur on a system over a given period
  of time.
• Robustness is a systems ability to detect and
  handle fault, maintenance, and system-external
  events,
• Recovery follows an outage and measures how
  quickly a system is able to return to service
• Sun has developed benchmarks to measure these
  three and used them to improve availability in
  its Cluster platform.
• Sun uses white box and black box testing with
  user data as feedback
• SCATE Sun Cluster Automated Test Environment

4
Requirements for Sun Clusters

• Platform support With Solaris x86 and x64
  platforms
• Sun Fire V20 and V40z servers
• Sun StorEdge 3310 RAID, or 3310/3320 (JB)
• arrays, 3510, 6130, 6920, 9970/9980/9990
• on V40z and Galaxy servers
• Sun Dual Gigabit Ethernet and Dual SCSI/P
• adapters
• Sun Quad Gigabit Ethernet adapter
• Sun Ultra 320 Dual PCI SCSI host adapter
• card
• FC HBAs
• With Solaris SPARC platforms
• Servers
• Netra 120, 240, 440, t1120/1125, t1400/1405,
  T1 AC200/DC200, 1280, 20
• Sun Enterprise 220R, 250, 420R, 450, 3500,
  4500, 5500, 6500, 10K
• Sun Fire V120, V210, V240, V250, V440, V490,
  280R, 480, V880, V890, V1280, 3800, 4800/4810,
  6800, 12K, 15K
• Sun Fire E2900, E4900, E6900, E20k, E25k
• T1000 and T2000
• Sun storage

• Supported SAN switches
• Sun 1-Gb 8- and 16-port switches
• Sun 2-Gb 8-, 16-, and 64-port switches
• Brocade Silkworm 2800, 3200 etc switches
• McData 4300, 4500, 6064, 6140 switches
• Supported Interconnects
• On-board 100 Mb/sec port, Sun Quad
• FastEthernet, SunFastEthernet,
• and SunSwift adapters
• Sun Gigabit Ethernet adapter
• Peripheral Component Interconnect
• (PCI) Scalable Coherent Interface (SCI)
• Infiniband
• Supported public networks
• Asynchronous Transfer Mode (ATM)
• On-board 100 Mb/sec port, Sun Quad
• FastEthernet, SunFastEthernet, and
• SunSwift adapters
• Sun Gigabit Ethernet adapter

5
Disaster recovery

• Backup And Recovery include scripts or software
  to archived data
• Log Shipping uses the logs produced by the
  database to recover a standby database at a
  remote site.
• Availability Suite backs up the data, Instant
  image allows you to restore it
• With database replication, be careful when you
  replicate in case of bad data replication. Many
  people delay replication after a time gap has
  elapsed to ensure valid data.
• Global Network Service provides one IP for
  cluster, and global I/O does similar naming for
  I/O devices

6
Classically Three Main Clustering Tiers

• Local Clustering
• The classical sense of redundant basic
  clustering in a computer lab
• Offers restart, failover, and scalable services,
  capacity for modular growth, and a low entry
  price compared to fault tolerant systems.
• provides a solid level of continuous service
  availability in the event of application,
  operating system, or hardware failure in a single
  data center.
• Campus Clustering
• Components can be up to 40 km apart
• Use to prevent data loss or availability if a
  room floods or building burns down
• Metro Clustering
• cluster nodes can be separated by up to 400
  kilometers
• Uses dense wave division multiplexing (DWDM)
  technology
• But what about disasters that strike a whole
  region, such as the East Coast blackout of 2003?
  Answer Geographic Partnership (4th type)

7
Local Cluster Topology

• Three main topologies
• Clustered Pair up to eight servers, from pairs
  with dual-host storage, no extra I/O slots are
  needed, just use host bus adapters
• N1 A single backup node dual hosts storage.
  You get a single insurance node so you save
  costs.
• PairM Storage facilities attach to M nodes.
  Minimize I/O slot demands but all I/O requests
  have to go through the node hosting the storage.

8
Campus Cluster

• Brought about by advent of Fibre Channel, but
  still not as wide as WDMs.
• Robust cluster would consist of four nodes and
  two-way, host-based mirrors across sites.
• Typical setup Fibre Channel HBAs attach to
  storage subsystems. They use Gigabit interface
  converters to interface with other media.
• Scalable Coherent Interconnect (SCI) A
  specialized cluster interconnect for Sun Cluster.
  Data rate of 70 MBps with latency reduced to 10
  microseconds
• And then Sun Fire Link

9
Sun Cluster Geographic Edition

• Establishes a partnership between two clusters
  regardless of location, and no two clusters can
  be in more then one partnership with eachother
• Exchange heartbeats over SSL and IPsec, but can
  also use e-mail, HTTP, satellite, and microwave
  towers.
• Set to active-active or active-standby
• Also establishes Protections Groups
• Backbone of disaster management by creating
  resource groups and properties for data
  replication.
• Clusters can be defined in more then one
  protection group, and to keep track the
  configuration is identical on partner clusters

10
Geographic Configurations Examples
11
Sun Cluster Geographic Ed Software Architecture

• Components in a Sun Cluster Geographic Edition
  cluster
• SolarisTM 8, 9, or 10 software
• Sun Cluster software
• Sun Cluster Geographic Edition software
• Application Data Service Agents
• Data replication software
• Either Solaris Volume Manager or Veritas Volume
  Manager
• Cluster Configuration Repository keeps track of
  nodes in cluster

12
Sun Fire Link

• Used for adjacent machines, supports throughput
  of 4.8 GB/sec and maxes out at more then a
  teraflop
• Really gets 2.4 GB max sustained.
• Adds about 5 cost to the system
• Supports level-two and level-four striping
• Uses Remote Shared Memory (RSM) API

• RSM inter-node messaging that is based on
  direct access to memory residing on remote nodes.

13
Cluster Management

• Sun Cluster GUI Geographic Edition single point
  of management
• Configures, controls, monitors partnerships,
  protection groups, heartbeats, and each
  individual cluster
• Usses SSL and has a role based access control
  schema
• Optional command line interface for scripting
• Notifies admins of failure through GUI, email, or
  script execution
• Two ways to migrate between sites
• Managed Switch over for planned maintenance, the
  primary and secondary clusters are connected,
  both are used during switch
• Managed Takeover allows admins to start backup
  app services with replicated data on the
  secondary site through the GUI while the two
  clusters are not connected
• Both ways there is a backup DNS switch to the
  second site, the clients connect to the same URL
  and get a new DNS entry

14
Dynamic Sub-Clustering using Quorum

• If the cluster must be adjusted, Sun Cluster uses
  a quourm to decide which nodes are supposed to
  form the new subcluster.
• Insures data integrity, only a set of nodes that
  owns the majority of votes can form a new
  subcluster, others are shutdown or isolated.
• Nodes have one vote and quorum disks have as many
  votes as the sum of the nodess votes attached
  minus one.
• This is all done by the Cluster Membership
  Monitor.

15
Data Replication

• Data is continually copied from the primary
  cluster to the secondary cluster
• Uses Sun StorEdge Availability Suite software
  and Sun StorEdge 9900 TrueCopy software, which
  support synchronous and asynchronous replication
  modes
• Includes point-in-time and copy and remote
  mirroring.
• Host based storage can access any storage devices
  supported by Solaris and can use any Sun network
  adapter supporting TCP/IP.
• Maintains write order consistency using volume
  grouping.
• Maintains per-device bitmap volumes that are
  marked to indicate changed blocks for
  resynchronization during an outing.
• Both remote mirror primary and secondary hosts
  can failover to another host in their cluster to
  ensure continuity of the data replication process

16
Cluster File System

• CFS is a highly available, distributed,
  cache-coherent file system that allows UFS or
  HSFS file systems to be concurrently accessed on
  multiple cluster nodes.
• I/O requests can be broken up to increase
  throughput
• Tokens manage valid cache across machines.
• Logging mode provides UFS backup in case of
  failure

• Every active file has extra kernel memory
  structures to manage coherent caching of their
  attributes and data across the cluster.
• I/O takes places over a primary active and
  secondary inactive channel (as long as the
  cluster is set up that way), the secondary used
  for cache information and error conditions.
• Theres also the (old) Sun Parallel File system,
  which focuses on speed if, while the CFS on
  reliability.

17
Comparison to Compaq
18
References

• Sun Cluster Software Quality by Design for
  Advanced Availability www.sun.com/software/cluster
  /wp-advancedavail/wp-advancedavail.pdf
• Disaster Recover with Sun Java Availability Suite
  www.sun.com/software/whitepapers/javaenterprisesys
  tem/avail_disaster_recovery.pdf
• Sun Cluster Geographic Edition docs.sun.com/app/do
  cs/doc/819-4243/6n6djkdn5?aview
• Suns overal cluster website http//www.sun.com/so
  ftware/cluster/index.xml
• Campus Clusters Based on Sun Cluster 3.0 Software
  http//www.sun.com/software/cluster/wp-campusclust
  er/wp-campuscluster.pdf
• http//h18002.www1.hp.com/alphaserver/download/Gig
  a_IdeaByte_TruCluster_vs_Sun_Cluster.pdf
• http//www.sun.com/software/whitepapers/wp-globalf
  ileservices/wp-globalfileservices.pdf
• http//www.sun.com/software/cluster/dhb-JES-techtr
  end.pdf
• Sun Fire Link System Overview http//192.18.109.11
  /816-0697-10/816-0697-10.pdf
• Sun Cluster 3.0 Software Cluster File System
  (CFS) Making the Most of the GlobalFile Service
  http//www.sun.com/software/whitepapers/wp-globalf
  ileservices/wp-globalfileservices.pdf
• Designing Enterprise Solutions with Sun Cluster
  3.0 By Richard Elling, Tim Read
• http//docs.sun.com/app/docs/doc/819-0579/6n30dc0n
  1?aview