3D and Database Developments

1
3D and Database Developments
Gordon D. Brown Database Services
Manager e-Science, CCLRC
15th GridPP Collaboration Meeting ProgrammeRAL,
CCLRC11th - 12th January 2006
2
Overview

• Database Services
• 3D Background
• Oracle Streams Architecture
• 3D Schedule
• 3D Status
• Plans

Gordon D. Brown e-Science, RAL
2
30 October 2009
3
Database Services Structure

• Part of the Data Management Group in the
  e-Science Centre
• Structure
• Database Services Manager
• 4 DBA Staff Resource
• 1 System Administration Staff Resources
• More resources with Tier-1 Group collaboration

Gordon D. Brown e-Science, RAL
3
30 October 2009
4
Database Services Set-up

• Running Oracle 9i Enterprise Server with Real
  Application Clusters and Oracle 10g
• Using RedHat Linux (RHEL 3.0 AS) with Oracle
  Cluster File System (OCFS)
• Oracle Data Guard
• RMAN
• Oracle Enterprise Manager (OEM)

Gordon D. Brown e-Science, RAL
4
30 October 2009
5
Database Services Hardware Structure
6
3D Background

• Distributed Deployment of Databases
• LCG project, initiated by LHC experiments to
  co-ordinate the set-up of database services and
  facilities for relational data transfers as part
  of the LCG infrastructure
• The project goal is to provide a consistent way
  of accessing database services at tier-0 and
  collaborating LCG tier sites to achieve a more
  scalable and available access to non-event data

Gordon D. Brown e-Science, RAL
6
30 October 2009
7
Why a LCG Database Deployment Project?

• LCG provides an infrastructure for distributed
  access to file based data and file replication
• Physics applications (and grid services) require
  a similar services for data stored in relational
  databases
• Several applications and services already use
  RDBMS
• Several sites have already experience in
  providing RDBMS services

Gordon D. Brown e-Science, RAL
7
30 October 2009
8
Why a LCG Database Deployment Project?

• Increase the availability and scalability of LCG
  and experiment components
• Allow applications to access data in a consistent
  and location independent way
• Connect existing database services via data
  replication mechanisms
• Simplify a shared deployment and administration
  of this infrastructure during 24 x 7 operation

Gordon D. Brown e-Science, RAL
8
30 October 2009
9
3D is not

• Store all database data
• Experiments are free to deploy databases and
  distribute data under their responsibility
• Setup a single monolithic distributed database
  system
• Given constraints like WAN connections one can
  not assume that a single synchronously updated
  database will provide sufficient availability

Gordon D. Brown e-Science, RAL
9
30 October 2009
10
3D is not

• Setup a single vendor system
• Technology independence and a multi-vendor
  implementation will be required to minimize the
  long term risks and to adapt to the different
  requirements/constraints on different tiers
• Impose a CERN centric infrastructure to
  participating sites

Gordon D. Brown e-Science, RAL
10
30 October 2009
11
LCG 3D Service Architecture
Gordon D. Brown e-Science, RAL
11
30 October 2009
12
Oracle Streams Architecture
Gordon D. Brown e-Science, RAL
12
30 October 2009
13
3D Testbed Configuration
create table emp ( id number, name varchar2, .)
insert into emp values ( 03, Manuel, .)
14
3D Oracle Enterprise Manager
Gordon D. Brown e-Science, RAL
14
30 October 2009
15
Experiment Deployment Plans

• ATLAS
• Databases online / Tier-0 / Tier-1 / T2 (mysql)
• Oracle streams for conditions replication (COOL)
• LHCb
• Databases online / Tier-0 / Tier-1
• Oracle streams for conditions replication (COOL)
• Both above interested in FroNtier cache for
  conditions data in COOL

Gordon D. Brown e-Science, RAL
15
30 October 2009
16
Experiment Deployment Plans

• CMS
• Databases online / Tier-0 - DB cache Tier-1 and
  higher
• FroNtier/POOL for conditions data distribution
  (cache at T1/T2)
• Oracle streams as fallback
• ATLAS, LHCB CMS Oracle streams for online to
  offline replication
• ALICE
• Databases online / Tier-0 - Files Tier-1 and
  higher
• Alice s/w for copy/transformation, no 3D service
  request

Gordon D. Brown e-Science, RAL
16
30 October 2009
17
Database Clusters

• Several sites are testing/deploying Oracle
  clusters
• CERN, CNAF, BNL, FNAL, GridKA, IN2P3, RAL
• Several experiments foresee Oracle clusters for
  online systems
• Focus on database clusters as main building block
  also for Tier-1

Gordon D. Brown e-Science, RAL
17
30 October 2009
18
Database Clusters

• 3D will organize DBA level discussions on
  database cluster setup
• Involve Tier-1 and Online database admin teams
• Share test plans and expertise among LCG sites
  and experiments
• Cluster setup and existing test results
• Storage configuration and performance tests

Gordon D. Brown e-Science, RAL
18
30 October 2009
19
Proposed Setup Schedule

• Nov 2005 h/w setup defined and plan to PEB/GDB
• Jan 2006 h/w acceptance tests, RAC setup
• Begin Feb Tier-1 DB readiness workshop
• February Apps and streams setup at Tier-0
• March Tier-1 service starts
• End May Service review -gt h/w defined for full
  production
• September Full LCG database service in place

Gordon D. Brown e-Science, RAL
19
30 October 2009
20
Proposed Tier-1 3D Service Setup

• Propose to setup for first six months
• 2/3 dual-cpu database nodes with 2GB or more
• Setup as RAC cluster (preferably) per experiment
• Shared storage (eg FibreChannel) proposed to
  allow for clustering
• 100GB storage per node
• Need to clarify service responsibility (database
  or admin team?)
• Target s/w release Oracle 10gR2
• RedHat Enterprise Server to insure Oracle support

Gordon D. Brown e-Science, RAL
20
30 October 2009
21
RAL 3D Status

• Implementation early last year
• Still in test phase, using test machine
• 2 x 2.8GHz xeon processors
• 2GB RAM
• 120GB HDD
• RedHat Enterprise Linux 3

Gordon D. Brown e-Science, RAL
21
30 October 2009
22
RAL 3D Status

• Oracle 10.1.0.4 (single instance) installed
• Back-ups being configured for RMAN
• Successful set-up of
• Oracle Streams (3D)
• File Transfer Service (FTS)
• Oracle Enterprise Manager (OEM)
• COOL

Gordon D. Brown e-Science, RAL
22
30 October 2009
23
RAL 3D Plans

• Next hardware arriving Feb 2006
• 2 x 2 node Oracle cluster
• Dual Opteron CPU, 4GB Memory
• Shared storage of 4TB
• Oracle installation and pre-production testing
  March 2006
• Production in autumn 2006

Gordon D. Brown e-Science, RAL
23
30 October 2009
24
Summary

• Database applications s/w and distribution models
  firming up - driving application Conditions
  database
• ATLAS, CMS and LHCb require access to database
  data also outside CERN, ALICE only at CERN
• Online -gt offline replication based on streams
  for all three experiments
• LCG 3D project moving to pre-production phase
  inline with SC4 (March 06)
• Definition of concrete conditions data models in
  experiment s/w required
• Definition of full LHC setup after 6 month of
  deployment experience

Gordon D. Brown e-Science, RAL
24
30 October 2009
25
Questions (hopefully) Answers
Gordon D. Brown e-Science, RAL
25
30 October 2009