IBM PureData System for Operational Analytics

1
IBM PureData System for Operational Analytics

• James Cho
• Chief Architect PureData Transaction System
• And Operational Analytics Solutions
• Session Code

2
Agenda

• Introduction Pure Systems
• PureData for Operational Analytics
• Smart Analytics 5600
• Questions/Discussion

3
A new family of expert integrated systems
Systems with integrated expertise and built for
cloud
Built-in Expertise Capturing and automating what
experts do from the infrastructure patterns to
the application patterns
Integration by Design Deeply integrating and
tuning hardware and software in a ready-to-go
workload optimized system
Simplified ExperienceMaking every part of the IT
lifecycle easier - with integrated management of
the entire system and a broad open ecosystem of
optimized solutions
4
IBM PureSystems Family How much flexibility,
integration and workload optimization do you want
out of the box?
Data Platform
Infrastructure
Application Platform
New
Integrated and optimized application platform
Built on IBM middleware to accelerate
deployment of your choice of applications
Integrated and optimized data platform
Delivers high performance data services to
transactional and analytics applications
Integrated and optimized infrastructure with
flexibility Runs your choice of applications
and middleware

• New PureSystem with models optimized exclusively
  for data workloads

• Delivering application platform services

• Delivering IT infrastructure services

5
IBM PureData System

• Pattern based database deploymentin minutes, not
  hours1
• Handles more than 100 databases on 1 system2

System for Transactions

• 10-100x faster than traditional custom systems4
• 20x greater concurrency and throughput for
  tactical queries than previous Netezza
  technology5

System for Analytics
powered by Netezza technology

• Continuous ingest of operation data
• Handles 1000 concurrent operational queries3
• Up to 10x storage savings with adaptive
  compression6

System for Operational Analytics
1. Based on IBM internal tests and system design
for normal operation under expected typical
workload. Individual results may vary. 2.
Based on one large configuration 3. Based on IBM
internal tests of prior generation system, and on
system design for normal operations under
expected typical workload. Individual results may
vary. 4. Based on IBM customers' reported
results. "Traditional custom systems" refers to
systems that are not professionally
pre-built, pre-tested and optimized. Individual
results may vary. 5. Based on IBM internal
performance benchmarking 6. Based on client
testing is the DB2 10 Early Access Program
6
Operational AnalyticsExtreme concurrent query
volumes on real time information
Business Users, Call Centers, Online Queries, etc
100s to 1,000 Read and Update Queries
Business Analysts
SALES
2010 2009 2008 2007 2006 2005
Multiple, Concurrent Analytic Queries
BI Reports and Analytics
Data Warehouse
7
Performance and Mixed Workloads

• Mixed Workloads
• Data Mining
• Call Center
• Small, large and extra large queries
• High concurrency
• Example

8
Availability and more frequent Ingest

9
Proven Scalability

• Value to clients
• Can grow system as needed by adding data nodes
• Typical customers grow data by 30 per year

Extra Small (Min Config)
Medium
Small
0 to 18 Data Expansion Racks
1/3 Rack
2/3 Rack
Full Rack
Up to 133TB
Up to 266 TB
400 TB
10
Pure Data for Operational Analytics New Features

• Embedded 7710 Single box solution scales down
  as well as up
• The starting building block for 7700 scalable
• Consolidated GUI Console significantly improved
  operational experience
• Single pane of glass
• Alerts for all HW/SW components
• Maintenance for all firmware/software cluster
  wide
• Integrated OPM application management
• Enhanced availability fewer components, reduced
  outage time
• Integrated Backup leveraging 900 GB HDD
• Roving Standby, Hot swap SSD
• 56x fewer managed resources 1/3rd fewer
  relationships
• Enhanced next generation SSD and enclosure
• Dense (1U) Dual Controller SSD enclosure
• RAID-10 at high IOPS and bandwidth
• Higher Density simpler manufacturing, shipment,
  service
• One HA group per rack no FC cables cross racks
• P730 2U server ½ the rack space
• Dense SSD enclosure, Double Dense IO cards
• Higher I/O Bandwidth high ingest capability,
  high scaling

11
HA Simplification (small cluster)
PureData for Operational Analytics
Smart Analytics 7700
12
HA tools in PureData for Operational Analytics

• hareset
• Soft reset will bring the domain to a base state
• Clear failed, stuck, pending states - remove
  resource group requests across entire
  configuration
• Rebuild HA configuration completely
• drop the domain, create domain, create all
  resources/groups/relationships/equivs etc.
• hachkconfig
• Detect and repair resource model problems
• Detect OS configuration issues that impact HA and
  alert
• Can be run automatically or manually

13
A1791 Component Building Blocks
Capacity Add-on Data Racks

XL Foundation Rack
Data Rack (s)
Data Module 3
Data Module 2
Foundation Modules
Data Module 1
Roving Standby Module
Figures are not to scale
14
Config Summary
MTM XS - 8279-A01 (0.5D) S - 8279-A02 (1.5D) M
- 8279-A03 (2.5D) L - 8279-A04 (3.5D) Scalable to
9.5 D XL - 8279-A05 (3.5D) Scalable to 18.5 D
Data Expansion Add-on 8279-AD1 (1D) 8279-AD2
(2D) 8279-AD3 (3D)
Upgrade path
Capacity Expansion Add-On
Upgrade path
Add-On up to 5 x 8279-AD3
Add-On up to 2 x 8279-AD3
15
MES Upgrade / Add-on Options
Model Upgrade A01 A04
Data nodes add-on ( lesser than 9D)
Model Upgrade AD1 AD3
Data nodes add-on (10-18D)
16
A1791 Configuration Sizes
Size System Part Number Modules Modules Primary Data (TB) Backup Cold Data(TB) Comments
Size System Part Number Foundation Data Primary Data (TB) Backup Cold Data(TB) Comments
XS A1791-01 8279-A01 1 0 10.8 21.6 0.5D
S A1791-02 8279-A02 1 1 32.4 64.8 1.5D
M A1791-03 8279-A03 1 2 54.0 108.0 2.5D
L A1791-04 8279-A04 1 3 75.6 151.2 3.5D to 9.5D (add up to 2 full add-on racks AD3)
XL A1791-X4 8279-A05 1switches 3 75.6 151.2 3.5D to 18.5D (add up to 5 full add-on racks AD3)
Add-on data 1 A1791-E1 8279-AD1 n/a 1 21.6 43.2 1D
Add-on data 2 A1791-E2d 8279-AD2 n/a 2 43.2 86.4 2D
Add-on data 3 A1791-E3 8279-AD3 n/a 3 64.8 129.6 3D

• Capacity per data module (foundation module has
  half)
• 21.6 TB Primary Hot Active data
• 43.2 TB free space (for backups, cold data, etc.)

17
PureData System for Operational Analytics System
Console
System Console

• Provide Easy to Use, Common interfaces for
  management of Pure System Family

18
Simplified maintenance with pre-integrated fixes

• Single point of contact for support
• Automated updates for faster maintenance
• All hardware firmware and OS software patches
  integrated and tested together at the factory

19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
The Overview Dashboard At a Glance view
Details on OS-level system CPU and paging
utilization, and a break down of time within DB2
23
The Overview Dashboard At a Glance view
Throughput within DB2 (statements, transactions,
rows and connections) and average response time
24
The Overview Dashboard At a Glance view
Top 3 SQLs by elapsed time, CPU time, rows
read, or lock wait time
25
The Overview Dashboard At a Glance view
Performance focus choose a tab to show core
information on locking, I/O, SQL or
pureScale-specific metrics.

• Two key pureScale metrics shown here
• GBP hit ratio
• Average XI time

26
Performance Overview Report

• Provides good top-level view of overall system
  performance
• Choose metrics and sort orders that are most
  relevant for the database being analyzed
• Report is exportable to pdf/xls/ppt formats
• Useful for upline reporting
• Drill-down available via other reports

27
Drill down to alert in Database Performance
Monitor
28
Enhanced Information Center

• Information is organized according to topic area
  and operational lifecycle
• Overview, Planning, Getting Started, Operational
  Tasks, Advanced Tasks, Troubleshooting, Reference
• Each section has further detail, and links to
  relevant information
• Easily searchable within the Information Center
• Easily searchable from any search engine
• Ability for users to enter comments on topics and
  participate in discussions

29
Information Center Sample comment
30
ISAS 5600 R3 Announced April 9th GA June 2013
31
5600 R3 Highlights

• Optim Performance Manager 5.2
• New Software Stack
• SLES 11 DB2 10.1
• Enhanced HA
• Roving Standby
• Higher Availability with simpler HA management
• More stable cluster based file system (replacing
  NFS)
• fewer managed resources
• fewer relationships
• New ATK Deployment
• Simpler deployment model with new ATK
• Simplified software upgrades via master image

• Server Refresh
• 16 core, Intel Xeon E5-2600 series processors
• 2 cores per partition
• 192GB Memory
• 24 GB memory per partition
• Increased Storage Capacity
• 900 GB SAS-2 Disk Drives Standard
• Excess space intended for cold data and/or local
  backup
• SSD Standard
• Hot swap and RAID protected
• 10 Gb Database Interconnect Standard

32
Management Module

• New management module
• Incorporates Application module (separate in 5600
  R1 and R2)
• Provides rootfs images for core modules (xCAT)
• HA for applications ISW and xCAT
• Management Module hosts the following
  applications
• InfoSphere Warehouse (ISW) 10.1
• Optim Performance Manager (OPM) 5.2
• IBM Systems Director (ISD) 6.3
• xCAT 2.7.5

Management module
Management standby node
Management node
IBM x3650 M48 core E5-2680 64GB memory
IBM x3650 M48 core E5-2680 64GB memory
DS35246 disks
33
IBM System x3650 M4

• ISAS R3 PCI Adapters
• 3 x Dual Port HBAs
• 4 ports to disk
• 2 ports to LAN-free backup
• 1 x 10 Gb Ethernet NIC

34
PCI placement Core Warehouse Modules

• Dual Socket Configuration
• Slot 1 (Riser1/Slot 1) full height full length
• (empty)
• Slot 2 (Riser1/Slot 2) full height half length
• 2nd QLogic 8Gb FC Dual-port HBA
• Slot 3 (Riser1/Slot 3) full height half length
• 1st QLogic 8Gb FC Dual-port HBA
• Suggested slot priority 3, 2, 6, 1, 5, 4

• Slot 4 (Riser2/Slot 1) full height full length
• (empty)
• Slot 5 (Riser2/Slot 2) full height full length
• 3rd QLogic 8Gb FC Dual-port HBA (LAN-free
  backup)
• Slot 6 (Riser2/Slot 3) full height half length
• Emulex Dual Port 10GbE SFP VFA III (95Y3762)

(empty)
(empty)
QLogic 8Gb FC HBA
QLogic 8Gb FC HBA
QLogic 8Gb FC HBA
Emulex 10GbE
35
GPFS Design
MGMT
MGMT-STDBY
ADMIN
DATA1
DATA2
DATA3
DATA4
STDBY1
GPFS cluster 1/db2home, /dwhome, /db2fs,
/db2plog, /db2mlog, /stage
GPFS cluster 1/db2home, /dwhome, /stage
quorum-managernsd server
quorum-managernsd server
HA GROUP 1
DATA5
DATA6
DATA7
DATA8
DATA9
STDBY2
GPFS cluster 2/db2fs, /db2plog, /db2mlogRemote
from GPFS cluster 1/db2home, /dwhome, /stage
quorum-managernsd server
quorum-managernsd server
HA GROUP 2
36
What is a Terabyte Defining Terms
Usable Space
Available Space
Sys Overhead
Temp
Aggregates, Derived Tables
Indexes
Spinning Disk
Backup / Cold Data
RAID 6 13

Aggregates, Derived Tables

User Space with compression
Raw Hot Dataafter compression
Raw Hot Data
Backup Space
User Spaceno compression

• Spinning disk The total amount of storage
  available to the system, ie. Total drives x Drive
  capacity
• Available space The total amount of storage that
  is available to the database software after RAID
• Usable space The total amount of storage that is
  available for hot active data including
  supporting objects like indexes, and temps
• User space the amount of base data input to the
  system (Raw hot data) - number of records x
  record size plus any aggregates or derived
  tables assumed to be 55 of available usable
  space
• Backup Space / Cold Data The total amount of
  storage that is available for database backups
  and cold data, which is infrequently accessed
  historical data

37
Smart Analytics System 5600 R3 data sizing
Capacity Sizes Smart Analytics System 5600 Smart Analytics System 5600
Capacity Sizes 5600 R3 (900GB) 5600 R2 (300GB)
Data Modules 1 1
Spinning Disk (TB) 42.2 14.1
Available Database Space after RAID formatting (/db2fs) (TB) 26.2 8.7
Active Space _at_ 33 (TB) 8.7 8.7
User Space _at_ 55 (TB) 4.8 4.8
User Space Compressed Assuming 2.5x compression (TB) 12 12
Space available for index, temp, logs (TB) 3.9 3.9
Backup Space / Cold Data _at_ 66 (TB) 17.5 -
Peak User Space Compressed with Cold Data Compressed (TB) 55.7 -
Solid State Device for temp (TB) 1.2 0.6
Cold Data Compressed infrequently accessed
historical data w/o temps or logs 5600 R2 with
SSD option
38
ISAS 5600 Data Module Comparisons
39
James ChoIBMjamescho_at_us.ibm.com
Evaluate my session online www.idug.org/na2013/ev
al

• Session
• IBM PureData System for Operational