Current State - and Replacement

1
Current State - and Replacement

• Oct. 2008
• Kay Kasemir
• kasemirk_at_ornl.gov

2
Current State

• Has been in use for several years
• Works OK, but maintenance increasingly difficult
• Installation, dependence on older technology
• Maintenance of installed instances

3
Replacement

• Transparent to end users
• Except maybe better than before
• Easier to maintain in the long run
• Still interface to previous setups
• Allow import of previous configurations
• GUI tools access old state and replacement

4
(No Transcript)
5
Previous Setup
IOC
ASCII Config.
Channel Access
Archive Engine
config.xml
Binary Data Files
Data Server
XML-RPC

• History
• 2000Started byBob Dalesio
• 2003Index Tools,Data Server
• 2007CSS Client

CSS-based OPI
6
Details

• Older Java archive viewer also available.
• Archive Engines running on sampling computer,
  data server on serving machine.
• Scripts to restart engines, copy data, update
  indices.
• Headaches with data maintenance
• Index time grows with data
• Stuck when index files reach 2GB
• SNS Users faced with 80 sub-archives
• No clue what needs fixing after network/power
  problems
• No idea who contributes how many samples
• No way to remove selected channels or time ranges
• Improving on this means implementing an RDB

7
New Setup
IOC
Channel Access
ArchiveEngine
Samples
Config.
config.xml
RDB(Oracle/MySQL)
EngineConfig -Import
Other tools for config samples
CSS-based OPI
8
RDB Setup

• Currently supported MySQL or Oracle
• Main table difference Oracle TIMESTAMP has
  nanosecs, MySQL has separate nanosecs col.
• All tools auto-configure based on JDBC URL
• Table Schema
• ERD, MySQL DBD(in org.csstudio.archive.rdb/dbd)
• Setup for Oracle (table types, partitioning)not
  final

9
Configure Sample Engines

• Configuration is in RDB
• Directly use SQL
• EngineConfigImport for legacy config files
• View/Edit via web
• Hierarchical
• Sampling engine (name, where to run, )
• Groups
• Channels
• No more duplicate channels!

10
Web Interface to Engine Config in RDB

• Tomcat/JSP/Servlets to view and edit

11
Web Config View Channel Stats
12
Web Config View Sample Stats
??
OK
13
Run Archive Engine

• Command-line tool similar to previous engine
• More settings (RDB, Logging, JCA/CAJ, )
• Compiled-in defaults from preferences.ini and
  plugin_customization.ini
• Can specify overrides via
• -pluginCustomization my_plugin_customization.ini
• Mandatory command-line options for instance
• engine_name to locate config in RDB
• port for consistency, must match settings in RDB
• Built-in web server for status, restart, stop
• http//hostport/ Empty page (used to be main)
• ./main Main page
• ./reset Reset statistics
• ./restart Stop, re-read configuration, start
  again
• ./stop Stop

14
View Data

• CSS Data Browser
• As before, just different URL jdbcoracle
• Web Config View shows some stats
• Custom reports based on RDB data

15
Report Example, SNS Beam Instr. Group

• May be SNS-specific, but shows usefulness of data
  in RDB

16
Hurdles

• Months Get new Oracle server configured
• Interface cards for storage array, fiber switches
• Firewall holes for office access, backup, admin
• Changes in 10g
• SELECT MIN(stamp), MAX(stamp) -gt NULL, NULL
• No range partitioning on Index-Organized-Tables
• Configuration issues
• ORA-01654 unable to extend
• ORA-00257 archiver error
• What used to be impossible is now easy,but may
  still expensive
• DELETE FROM SAMPLES WHERE

17
Stats

• 37 sample engines, 78000 channels
• Host that runs sampling engine
• CPU load 45, zero disk I/O wait, very responsive
• Oracle performance
• 8000 samples/sec peak
• Sustained maybe ¼ of that
• Details ofOracle server load,data
  storage,backups,network access to data
• No longer have to care

18
Other Ideas

• Jlabs MyA
• Operational, but
• Wrapper code around MySQL to create cluster
• Handles less meta info (units, limits, )
• Only monitor mode, no scan
• Viewer is one-of C/TCL/Tk
• Gabriele Carcassi mentioned RDDTool
• Toolset for logging data with data-aging
• Command-line RPN tools, web viewer
• May be faster than general-purpose RDB
• but lacks advantage of gpp RDB

19
Message to take home

• All the pieces have been running for a few weeks
• Looks more sustainable than previous setup
• .. but not operational, yet
• Thanks go to
• Oracle Jeff Patton
• JSP Katia Danilova
• Beam Instr. Reports Mariano Padilla, Jeff,
  Douglas Edwardson
• IT Greg Lawson

20
Stuff
21
SNS Setup for EngineConfigImport Tool

• Imports existing XML config files into RDB
• On srv02 in /imported_config
• .xml files copied from archive1, includessome
  editing to remove duplicate channels
• Review .log files
• 'chmk/chmkhebt/PPS_HEBTPLC_BChmk100AlarmContactA
  already in 'pps/pps_hebt
• Maybe edit your .xml file, refer to import.sh
  to re-import

22
Getting Sources, Building binaries

• Get sources from DESY CVS repository
• Maybe binaries real soon from ChannelArchiver
  web page
• Build binaries
• Find somebody who understands Eclipse RCP, how
  to use
• org.csstudio.archive.rdb/EngineConfigImport.produc
  t
• org.csstudio.archive.engine/ArchiveEngine.product
• org.csstudio.sns.product/SNS_CSS.product (or DESY
  CSS.product)
• Settings
• Compile-time defaults usually in
  preferences.ini and plugin_customization.ini
  files.
• Either change, or prepare file for
  -pluginCustomization command-line option