JLab RDB-based Archiver - PowerPoint PPT Presentation

1 / 8
About This Presentation
Title:

JLab RDB-based Archiver

Description:

Requirement to support monitoring of 1M channels. Manageability ... Dual quad-core CPUs. 16 GB memory ~$8K in 2006 ($4K for the memory) EonStore RAID box ... – PowerPoint PPT presentation

Number of Views:28
Avg rating:3.0/5.0
Slides: 9
Provided by: bick3
Category:
Tags: rdb | archiver | based | jlab

less

Transcript and Presenter's Notes

Title: JLab RDB-based Archiver


1
JLab RDB-based Archiver
  • Matt Bickley
  • (work by Anthony Bavuso and Chris Slominski)

2
What? Another Archiver?
  • ScalabilityRequirement to support monitoring of
    1M channels
  • ManageabilityRequiring no interaction by
    administrators
  • Design issues
  • Data latency (10-20 minutes from Czar)
  • Independent deadbanding (not using ADEL)
  • Data loss

3
Data loss from Czar
  • Users saw transients in StripTool that were not
    recorded by Czar
  • IOC CPU loading less than 30
  • Czar system loading less than 20
  • Confirmed data loss with a test setup

Archive
Analyzer
Czar
Test IOC
Database
CA Server
FFFF FFFE FFFC . .
FFFF FFFE FFFD FFFC . .
FFFF FFFE FFFC . .
4
MYA Design
User Commands
Watchdog Thread
Copycat Thread
Main Thread
MySQL Database
EPICS Thread
Control System
5
MYA realtime thread scheduling
  • Prioritization, highest to lowest
  • Channel Access communication
  • Database scribe threads
  • Client communication, watchdog, etc.
  • No loss of data
  • If work queue gt 2M events then database
    insertions are prioritized
  • Peak queue length 160,000 events
  • Average queue length 5,000 events

6
System Hardware
  • Dell PowerEdge 2850
  • Dual quad-core CPUs
  • 16 GB memory
  • 8K in 2006 (4K for the memory)
  • EonStore RAID box
  • 16 300GB SCSI disks (2 TB of storage)
  • RAID 01
  • 21K in 2006 (17K for the disks)

7
Performance
  • 50,000 channels monitored
  • 25,000 updates/second
  • 4,000 database insertions/second
  • History data retrieval rate of 160,000/second(Get
    one day of a 1 HZ channels data in 0.5 seconds)
  • Store at least 1 year of data online

8
Whats Next
  • Cut over to MYA for operations
  • Sandbox archiver
  • Users specify archive requests
  • Limited data retention (up to 1 month)
  • Limited archive request lifetime
  • OR
  • Archive everything
  • 300,000 channels
  • Keep the data for 1 week (?)
  • Needs more memory, much less disk space
Write a Comment
User Comments (0)
About PowerShow.com