BP1500: Whats New with Fathom Clusters

1
BP1500Whats New with Fathom Clusters

• Kirsten Werne, Senior Software Engineer
• Kapil Jain, Software Engineer

2
Clustering Overview

• Clusters take advantage of redundant hardware for
  high availability
• Applications view a cluster as a single system
• All clustering implementations are designed for
  failover ours is!
• Fathom Clusters makes failover support easy!

3
Fail Over

• Cluster failover
• Application failover
• Fail back

biw
wdog
Failover set
aiw
broker
servers
executables
apw
IP address
.dn
clients
ai
files
bi
lg
4
The Home Grown Clustering Solution

• Hand-crafted cluster scripts
• Intimate knowledge of EACH OS cluster software
  manager
• Much tinkering to setup
• Easily out of date with addition of extents,
  pagewriters, or other changes made by a DBA
• Our goal is to make fail-over support easy!

5
Fathom Clusters solution

• We use vendors own API/script facilities to call
  OS cluster services directly
• Scripts built from working knowledge of your DB
  resources
• Consistent commands across all OS platforms
• Seamless integration of your DB with clusters
  software for DB structure changes

6
Logical view of Fathom Clusters
Primary Node
Secondary Node
(Standby)
(Working)
Fail-over
Fathom Clusters
Fathom Clusters
OpenEdge RDBMS
OpenEdge RDBMS
OS/Cluster Manager
OS/Cluster Manager
Shared Disk
Cluster Environment
7
Planning Progress Requirements

• Ya gotta have an OS cluster
• Progress 91d or higher
• Fathom Clusters and Progress RDBMS must be
  installed on each cluster nodes private disk
• Progress RDBMS and Fathom Clusters installation
  should be under the identical directory structure
  on each node in the cluster

8
Planning OS requirements

• Supported OS Clusters

Clusters SunCluster HACMP TruCluster ServiceGuard
Adv Server
OS Solaris AIX Tru64 HP-UX Win2000
API/Script Script Script Script Script API
Cluster monitoring tool scstat clfindres caa_stat
cmviewcl GUI cluster Admin.
9
Planning Network considerations

• Heartbeat network
• Cluster ip versus node ip

Cluster IP 3
Node B
Node A
Heartbeat
IP 1
IP 2
Lan1
Lan2
10
Planning Database considerations

• 200 databases? Ow!
• Pick the key databases for your application
• Based on available cluster resources
• Partition shared disk drives
• Network adaptors
• Number of nodes in cluster network
• CPU capability of each node

11
Planning Database location

• Shared disk partition which can be owned
  exclusively by one node at a time
• NOT on NFS mounted disk because you can not have
  exclusive ownership
• NOT on a nodes private disk

Node 1
Node 2
Progress DB
Private Disk
12
Planning extent locations

f
Foo.db
Node1
Node2
Sports.db
Sports.a1
13
Installation

• Progress 9.1D enterprise installation to be
  associated with the cluster product
• Installation is done as with other Progress
  products by using Progress supplied Cluster
  control codes
• Short installation time
• Environment variable, PSC_CLUSTER_PATH
  must point to Fathom Cluster
  top level

14
Installation contd

• Install cluster product at the identical location
  on all nodes
• Target directory tree is created and files copied

15
Whats installed?
PSC_CLUSTER_PATH/ ...bin lib templates b
in l
Executable DLL (Win)/ Shared Libraries
(UNIX) Templates scripts (script based cluster
manager) License (clust.cfg)
16
Logical view of Fathom Clusters
Primary Node
Secondary Node
(Standby)
(Working)
Fail-over
Fathom Clusters
Fathom Clusters
OpenEdge RDBMS
OpenEdge RDBMS
OS/Cluster Manager
OS/Cluster Manager
Shared Disk
Cluster Environment
17
Setup
Environment variables to set
PSC_CLUSTER_PATH DLC SHARED LIBRARY
PATH (LIBPATH/SHLIB_PATH) PATH
Top Level Directory of Fathom Clusters Top Level
Directory of Progress RDBMS PSC_CLUSTER_PATH/lib
PSC_CLUSTER_PATH/bin PSC_CLUSTER_PATH/bin
18
Using PROCLUSTER, the command line tool
Procluster ltdbnamegt ltqualifiergt
Enable Disable Start Stop Isalive
Registers database as cluster resource Unregister
s the database as a cluster resource Brings
Database Online, informs Cluster Brings Database
Offline, Informs cluster Status of Database
19
Register (enable) a database as a cluster resource

• Parameter startup file (.pf) needed in the
  database directory having same name as database
• procluster lt database_namegt enable
  AI/BI/APWn/WDOG
• Examples
• Procluster /usr1/progress/sports enable AI BI
• Procluster /usr1/progress/sports enable AI APW4
  WDOG
• Procluster /usr1/progress/sports enable

20
What happens during registration?
procluster ltdbnamegt enable
21
Sample registration file

• ResourceTypeDatabase
• ResourceName/usr1/progress/sports.db
• EnvironmentStringJDKHOME/tools/solaris/java/jdk
  1.3.0
• ApplicationExePath/usr3/dlc
• StartCommand/usr3/dlc/bin/_mprosrv
• StopCommand/usr3/dlc/bin/proshut
• StopOptions-by
• DatabaseName/usr1/ progress /testdb.db
• DatabaseDependency /usr1/progress/sports.db
• DatabaseDependency/usr1/progress/sports.b1
• DatabaseDependency/usr1/progress/sports.d1
• DatabaseDependency/usr1/progress/sports.d2

22
Starting a database
procluster ltdbnamegt start

Database

• Example
• procluster /usr1/progress/sports start

23
Stopping a database

procluster ltdbnamegt stop
Database
1
24
Stopping a database

procluster ltdbnamegt stop
Database
2
25
Stopping a database

procluster ltdbnamegt stop
3
26
Stopping a database

procluster ltdbnamegt stop
4
27
Stopping a database

procluster ltdbnamegt stop

• Example
• procluster /usr1/progress/sports stop

28
Un-register (disable) a database

• procluster lt database_namegt disable
• Un-registers any optionally supplied helper
  processes
• Un-registers database
• Un-registers cluster resources part of failover
  set
• Example
• procluster /usr1/progress/sports disable
• Psston win2k we remove cluster info from the
  registry

29
Clusters and the Progress DBA
Add Extent register extent Remove Extent
Unregister extent Unregister database Delete
database Shut-down database Inform Cluster
prostrct add prostrct remove prodel proshut
30
Troubleshooting

• Problem Database starts up and then fails
• Solution On Unix Parameter file missing
• Problem AIW starts and then fails
• Solution AI enabled as cluster resource but not
  enabled with the database
• Problem Database starts but is not persisted
  through failure
• Solution Database is started directly using
  proserve and not procluster
• Problem Database keeps trying to restart
• Solution Check if the scripts were modified
  causing a circular dependency which cant be
  resolved

31
Troubleshooting contd ...

• Problem Database does not start on any node
  Solution Restart attempts and nodes on which to
  run exhausted after failures
• Problem Cluster enablement of Database fails
• Solution Incorrect full path to the database
• Solution Shared disk partition not available on
  node used for registration

32
Example of database registration on HACMP for AIX

• su (you should be super user)
• Select unused volume group say share2 on
  shared disk
• Varyonvg share2 (if not already varied on)
• Select, say /usr3 (file system) part of Volume
  group share2 to create database
• Mount /usr3 (if not already mounted)
• Create database say sports
• Create ltdatabase_namegt.pf parameter file
• Setup environment

33
Example contd

• Procluster /usr3/sports enable (full path) (HACMP
  takes a few min for registration)
• Procluster /usr3/sports start (brings database
  online, mounts fs, varies on vg)
• Procluster /usr3/sports stop (bring database
  offline, unmount fs, varies off vg)
• To disable mount /usr3, varyonvg share2
• Procluster /usr3/sports disable

34
Technical Details - Solaris

• Cluster Manager Sun Cluster
• scstat for status of cluster resources
• Resource type PRGS.clstr
• Scrgadm/scswitch - to register/enable a resource
  respectively
• scha_control to relocate the resource group

35
Technical Details - AIX

• Cluster Manager - HACMP
• clfindres for status of cluster resources
• Enabling (registering) a resource takes a few
  minutes
• Changes made to the Resources should be
  synchronized
• Resource Groups are CASCADING

36
Technical Details Tru64

• Cluster Manager TruCluster Server
• caa_stat - for status of cluster resources
• caa_relocate - to relocate the application to a
  different node
• Placement policy is BALANCED by default
• caa_register - to update/register resource

37
Technical Details - HPUX

• Cluster Manager MC/Serviceguard
• cmviewcl - for status of cluster resource
• cmmodpkg - Enables/Disables switching attributes
  of High Availability package
• Placement policy is Min_Package_Node
• cmapplyconf - Verify and apply cluster/ package
  configuration files

38
Technical Details - NT

• Cluster Manager - Advanced server
• Cluster Administrator GUI
• The registry is correctly updated upon
  registration
• Unregistering the database removes its
  information from the registry

39
In Conclusion

• Fathom Clusters keeps your application cluster
  ready by giving you
• One identified resource set (which could include
  disks, ip addresses, extents)
• Grouped resources and dependencies into one fail
  over logical set
• Automatic script generation for cluster software
• Integration with Database utilities to keep
  structure modifications synchronized
• Platform independent deployment

40
Cheat Sheet - Recipe to take home

• Verify that 91d Progress RDBMS or higher is
  installed in same location on all nodes
• Install Fathom Clusters on all nodes in exact
  same directory location
• Set Fathom Cluster required environment vars
• procluster /disk1/prog/sports_db enable
• procluster /disk1/prog/sports_db start

41
Questions???
42
Appendix - OS References

• Compaq Tru64
• http//www.tru64unix.compaq.com/docs/cluster_doc/c
  luster_51A/TCR51A_DOC.HTM
• HP
• http//docs.hp.com/hpux/ha/index.htmlServiceGuard
  
• IBM
• http//publib.boulder.ibm.com/pseries/hacmp/hacmp_
  index.html
• Microsoft
• http//www.microsoft.com/windows2000/technologies/
  clustering/default.as
• Sun
• http//docs.sun.com/ab2/coll.572.9/CLUSTAPIPG/_at_Ab2
  TocView?Ab2LangCAb2Enciso-8859-1

43
What you dont get with Clusters version 1.0

• Support for
• Fathom Management Console
• Fathom Management Agent
• APPServer
• Webspeed
• What is your priority for the next cluster
  release?