Channel Archiver Overview

1
Channel ArchiverOverview

• Jan. 2003

2
Channel Archiver

• Channel Access client
• Stores samples in disk files
• Design target handle 10000 values/sec
• Documentation, SnapshotsLANL EPICS Page.From
  http//www.aps.anl.gov/epics,see Other Sites,
  LANL, Tools, Archiver.

3
Operation

• Config. file lists channels to archive Example
  ltchannelgt ltperiod sgtchannel_X 1channel_Y 10
  Monitor might change every 0.2 secondchannel_Z
  0.2 Monitor
• Scanned
• Periodically store most recent value
• Monitor
• Store all incoming values up to buffer limit
• Original time stamps are stored!

4
1s Period? Data _at_ 9.96, 10.98, 11.76 s ?
5
Components
Storage
Retrieval
Native
CA
Online (also possible via scripting)
6
ArchiveEngine

• Syntax ArchiveEngine options config
  dir
• Options-d ltdesciptiongt-p ltportgt change
  HTTPD from 4812-l ltloggt write log file

7
Engines HTTPD

• URL of engines HTTPD http//ltmachinegtltportgt
• Status Config., not data!
• Client Pull Updates on reload
• Changes (added groups/channels) written to cfg
  subdirectory, original config. files unchanged

8
Directory File

• File name, not a directory name
• Specified when running archive engine
• Default freq_directory. Better dir.
• Used by retrieval tools
• Details of directory and data files are given
  in documentation
• Note Keep directory data files together, dont
  modify them.

9
ArchiveExport

• Syntax ArchiveExport options dir
  channel names
• Options-start mm/dd/yyyy hhmmss-end
  mm/dd/yyyy hhmmss-fill-interpolate
  ltsecondsgt-gnuplot-Matlab

10
Export Options

• Original Time Stampsill-suited for Spreadsheets
• Fill missing values by repetition
• Linear Interpolationfor given period
• (initial N/A until all channels have valid
  value)

11
Fill, Interpol., ...
12
MATLAB

• ExportToolsgenerate MATLAB command file
• ASCII, portable
• No MATLAB binaries required
• Full value info,time status
• Big slow

13
WinBrowser

• Win. Only

14
CGI Export

• Data Access from any Web browser
• Download formats Spreadsheet, Matlab

15
CGI Export Online Plots
16
Scripting

• Allows writing perl/tcl/python programs for
• Automated generation of daily beam statistics for
  Web page
• Specialized scripts to answer questions
  likeHow often was XX below 10.0 and for how
  long?

17
ArchiveEngine Details

• More options, see manual
• !write_period ltsecondsgt
• Groups for
• getting organized
• conditional archiving
• !group ltother filegt
• ltchannelgt ltperiodgt Disable

18
Retrieval f. Multiple Archives

• Run several ArchiveEngines, creating several
  small enough data sets
• Look at them all at once
• master_version1 List sub-archives, most
  recent first/archives/2001/july/dir/archives/200
  1/june/dir/archives/2001/may/dir Then
  check Fred's "xyz" archive /home/fred/xyzarchive/d
  ir
• Sub-archives should not overlap

19
Suggested Setup

• Long Term ArchiveSample 10min, kept forever
• Middle TermSample every 60s, restarted after 15
  days, switching between two directories
• Short TermUp to Monitor, toggling two
  directories every shift
• High-grained data for last shift, good data for
  last 2 weeks, some log of everything kept forever

20
CAManager (Thomas Birke, BESSY)

• ArchiveEngine control GUI

Status of (multiple) ArchiveEngines
Start/Stop
Periodic re-start in different directory,
updating a MultiArchive File
21
Data Management

• Do not separate Directory Data files!
• Periodically restart Engine in new directory to
  keep data sets manageable (CAManager helps)
• ArchiveManager program can copy selected channels
  time range, performing basic repairs and
  compression while doing this

22
Summary

• ChannelArchiver is a Toolset for archiving any
  ChannelAccess data
• Some generic retrieval options, scripting and
  Matlab allow further analysis
• Future Plans
• Port to R3.14 and RH8
• CORBA interface for data retrieval(common w/
  JLab archiver, replace tcl/perl/python binding,
  allow Java access to data)