SURENDER SARA

1
11i and RAC on Linux - 101
SURENDER SARA Verities Solutions8084 RFD Long
Grove, IL 60047Phone/Fax [phone]Email
info_at_veritiesllc.com
2
(No Transcript)
3
(No Transcript)
4
Two Node Architecture, Unprotected
5
Two Node Architecture, Protected Apps tier and
unprotected DB tier
6
Two Node Architecture, Protected Apps tier and DB
tier
7
FAILOVER CLUSTER OFFERINGS

• Veritas cluster server
• HP Service Guard
• Microsoft Cluster Service with Oracle Failsafe
• RedHat Linux Advanced Server 2.1
• Sun Cluster Oracle Agent
• Compaq, now HP, Segregated Cluster
• HACMP

8
RAC
9
Scalable RAC
10
Real Application Cluster

• Many instances of Oracle running on many nodes
• Multiple instances share a single physical
  database
• All instances have common data, control, and
  initialization files
• Each instances has individual, shared log files
  and rollback segments or undo tablespaces
• All instances can simultaneously execute
  transactions against the single database
• Caches are synchronized using Oracles Global
  Cache Management technology (Cache Fusion)

11
RAC Building Blocks

• Instance and Database files
• Shared storage with OCFS, CFS or raw devices
• Redundant HBA cards per HOST
• Redundant NIC cards per HOST, one for cluster
  interconnect and one for LAN connectivity
• Local RAID protected drives for ORACLE_HOMES (
  OCFS does not support ORACLE_HOME install)

12
CLUSTERINTER CONNECT

• FUNCTION
• - Monitoring Health, status and message
  synchronization
• - Transporting Distributed Lock manager messages
• - Accessing remote File system
• - Moving application specific traffic
• - providing cluster alias routing
• Interconnect Requirements
• - Low latency for short messages
• - High speed and sustained data rates for large
  messages
• - LOW Host CPU utilization
• - Flow Control, Error Control and heart beat
  continuity
• monitoring
• - switched network that scale well

13
INTERCONNECT PRODUCTS

• Memory Channel
• SMP Bus
• Myrinet
• Sun SCI
• Gigabit Ethernet
• Infiband Interconnect

14
INTERCONNECT PROTOCOL

• TCP/IP
• UDP
• VIA
• RDG
• HMP

15
IO CHANNEL HBA Products

• Adaptec
• DPT
• LSI Logic
• Interphase
• Qlogic
• Emulex
• JNI

16
FACRIC SWITCHES

• mcDATA
• EMC
• QLOGIC
• BROCADE

17
CLUSTER NODES

• NUMA
• SMP
• - shared system bus and IO
• - expensive and scalability problems
• - Adding more CPU can result into upgrading
  architecture components
• - DELL and HP-Compaq
• BLADE Servers
• - BladeFram system from egenera
• - egenera - 24 2 way and 4 way SMP processing
  resources
• - egenera - redundant central controllers
  ,redundant high-speed interconnects,
• PAN manager
• - egenera - PAN manager handles external storage
  mapping and virtualization
• - egenera - PAN manager handles , IO and network
  traffic to and from individual
• servers

18
CLUSTER TYPES

• Scalable High Performance Clusters - HPCC (high
  performance computing clusters
• HPCC
• parallel Applicaitons and Parallel databases
• MIDDLEware - parallel programming libraries
• OS -linux, WIN2k, UNIX
• Failover Clusters - HA clusters - STANDBY NODE -
  SHARED STORAGE- Failover in Minutes
• HPCC - fast ethernet/gigabit - infiband
  architecture -myrunet -Memory Channel
• HPCC - Nodes Multi CPU SMP servers ( variety of
  processors )
• HPCC - parallel database clusters - RAC, IBM
  Sysplex and Beowulf computers
• Application clusters - weblogic server,
  webshpere, oracle 9iAS
• Application clusters - weblogic server -
  servlets, JSP.EJB,RMI, JMS,JDBC are all clustered
• Appplication clusters - j2EE clustering - as in
  9iAS - client tier, web tier EJB tier, EIS tier
• NetAPP Filer Clusters - Access to same set of
  fiber channel disk
• NetAPP Filer Clusters - active - active
  configuration
• NetAPP Filer Clusters -own disk array and data
  access - on failover data acees via other node
• Network Load Balancer Cluster - HA and
  scalability for TCP/IP based services
• Network Load Balancer Cluster - redhat IP load
  Balancing Cluster ( Piranha)
• Failover Clusters - Failover Process
• Detecting failure by monitoring the heartbeat and
  checking status of resoruces

19
Oracles High Availability (HA) Solution
Stack
Real Application ClustersContinuous Availability
for all Applications
System Failure
Data GuardZero Data Loss
Data Failure Disaster
UnplannedDowntime
Flashback QueryEnable Users to Correct their
Mistakes
Human Error
Dynamic ReconfigurationCapacity on Demand
without Interruption
SystemMaintenance
PlannedDowntime
Online RedefinitionAdapt to Change Online
Data Maintenance
20
Shared Storage Options

• NFS Mounted storage ( Netapp )
• SCSI shared storage with OCFS, OFS, Raw devices
• Fiber channel Storage with fabric Architecture

21
11i Steps - 1

• Install RED HAT As 2.1 on all nodes
• Install 11i as single node install on Apps Tier
• Attached shared storage and install drivers for
  HBA

22
11i Steps -2 ( install OS Patches)

• rpm -Uv tar-1.13.25-9.i386.rpm
• this provides an updated version of tar
• allows a user to tar files from a running
  database on OCFS
• examples
• tar --o_direct -cvf /tmp/backup.tar

23
11i Steps -2 ( install OS Patches)

• rpm -Uv fileutils-4.1-4.2.i386.rpm
• this provides an updated version of cp and dd
• allows a user to copy files from a running
  database on OCFS
• examples
• cp --o_direct /ocfs/quorum.dbf /tmp/backup/quorum.
  dbf
• dd o_directyes if/ocfs/quorum.dbf
  of/tmp/backup/quorum.dbf

24
11i Steps -3 Install oracle provided RPMs

• /RAC_STAGE/ocfs/core/ocfs-support-1.0.9-11.i686.rp
  m
• /RAC_STAGE/ocfs/core/ocfs-tools-1.0.9-11.i686.rpm
• /RAC_STAGE/java/j2sdk-1_3_1_09-linux-i586.rpm.bin
• /ORA_ORA/unzip-5.50-30.i386.rpm
• /ORA_ORA/rhn-packages/zip-2.3-10.i386.rpm
• /ORA_ORA/rhn-packages/wu-ftpd-2.6.1-21.i386.rpm
• /ORA_ORA/PATCHES/hangcheck-timer-2.4.9-e.10-0.4.0-
  2.i686.rpm
• /ORA_ORA/PATCHES/hangcheck-timer-2.4.9-e.10-enterp
  rise-0.4.0-2.i686.rpm

25
11i steps -3 ( interconnect)

• ifconfig eth00 192.168.2.100
• route add -host 192.168.2.100 dev eth00
• Do this on each node
• Create watchdog file (oracle installer checks for
  this to install cluster option)
• touch /dev/watchdog
• Setup hangcheck-timer module
• vi /etc/modules.conf
• options hangcheck-timer hangcheck_tick30
  hangcheck_margin180
• modprobe hangcheck-timer

26
11i steps -5 OCFS.conf 5

• ocfstool ( from x windows)
• ocfs config
• Ensure this file exists in /etc
• node_name linux3.home.com
• node_number
• ip_address 192.168.1.100
• ip_port 7000
• comm_voting 1
• guid 9D3B77AF2FF26E92E25D00E04CA44B58

27
11i Steps -6 install OCFS

• mkfs.ocfs -F -b 128 -L /s01 -m /s01 -u 500 -g 500
  0755 /dev/sda1
• srvconfig_loc/s01/oragsd-config

28
11i steps -7 OCM

• ls
• If cmcfg.ora exists
• cp cmcfg.ora cmcfg.ora.original
• If cmcfg.ora does not exist
• cp cmcfg.ora.tmp cmcfg.ora
• echo HostNamedc1node3inter gtgt cmcfg.ora
• vi cmcfg.ora
• comment out WatchdogSafetyMargin and
  WatchdogTimerMargin
• PrivateNodeNameslinux22 linux33
• PublicNodeNameslinux2 linux3
• MissCount210
• KernelModuleNamehangcheck-timer
• CmDiskFile/u02/oracm-qourum
• vi ocmargs.ora
• comment out first line, which contains the word
  watchdogd
• cd ../bin
• cp ocmstart.sh ocmstart.sh.original
• vi ocmstart.sh
• remove words watchdog and from line containing
  Sample startup script...

29
11i steps -4 ( cp/dd - DB files to shared storage
)

• cp --o_direct /d03/oracle/proddata/
  /s01/oracle/proddata/
• Recreate the controlfile

30
11i steps 8 init.ora / spfile

• Create UNDO TBS for each instance
• Enable and disable thread for instance 2 from
  instance 1 and vice versa

31
11i steps 9 instance 1

• RAC-specific Parameters
• cluster_database true
• cluster_database_instances2
• thread 1
• instance_number 1
• instance_name PRODi1
• service_names PROD
• local_listener PRODi1
• remote_listener PRODi2

32
11i steps 10 instance 2

• cluster_database true
• cluster_database_instances2
• thread 2
• instance_number 2
• instance_name PRODi2
• service_names PROD
• local_listener PRODi2
• remote_listener PRODi1

33
11i Apps tier 806/iAS tnsnames.ora

• PROD (DESCRIPTION
• (ADDRESS_LIST
• (ADDRESS(PROTOCOLtcp)(HOSTlinu
  x2)(PORT1521))
• (ADDRESS(PROTOCOLtcp)(HOSTlinu
  x2)(PORT1521))
• )
• (CONNECT_DATA(SERVICE_NAMEPROD)(SERVERD
  EDICATED))
• )
• PRODi2 (DESCRIPTION
• (ADDRESS(PROTOCOLtcp)(HOSTlinux
  2)(PORT1521))
• (CONNECT_DATA(INSTANCE_NAMEPRODi
  2)(SERVICE_NAMEPROD))
• )
• PRODi1 (DESCRIPTION
• (ADDRESS(PROTOCOLtcp)(HOSTlinux
  2)(PORT1521))
• (CONNECT_DATA(INSTANCE_NAMEPRODi
  1)(SERVICE_NAMEPROD))
• )

34
Modify DBC file for Failover

• APPS_JDBC_DRIVER_TYPETHIN
• APPL_SERVER_IDC814C1FC253D6E6CE030010A32015B1A267
  27139351347429425425239777212
• TWO_TASKPROD
• GUEST_USER_PWDGUEST/ORACLE
• DB_HOSTlinux2.home.net
• FNDNAMAPPS
• GWYUIDAPPLSYSPUB/PUB
• DB_PORT1521
• FND_MAX_JDBC_CONNECTIONS100
• APPS_JDBC_URLjdbcoraclethin_at_(DESCRIPTION(ADDR
  ESS_LIST(LOAD_BALANCEON)(ADDRESS(PROTOCOLTCP)(
  HOSTlinux2)(PORT1521))(ADDRESS(PROTOCOLTCP)(HO
  STlinux2)(PORT1521)))(CONNECT_DATA(SERVICE_NAME
  prod)))

35
WHAT can cannot failover

• SQL PLUS will failover using TAF
• JDBC will failover ( modules like
• Forms run time connections will not, users will
  have to reconnect

36
QA

• Contact surender.sara_at_veritiesllc.com