Development of the TANGO Alarm System

1
Development of the TANGO Alarm System
Lorenzo Pivetta ICALEPCS 2005
Sincrotrone Trieste - ELETTRA
2
Outline
Overview
Requirements System characterization
Description of the Alarm System
Alarm visualization
Summary
Conclusions
2
Development of the TANGO Alarm System
Lorenzo Pivetta ICALEPCS 2005
3
Overview
Alarm asynchronous notification that some event
has happened or that a given state has been
reached
Alarm system a complex that allows for creating,
configuring and managing alarms
The Alarm Collector an alarm system developed
for the TANGO control system framework
3
Development of the TANGO Alarm System
Lorenzo Pivetta ICALEPCS 2005
4
Requirements (1)
The Alarm System should fulfil some requirements
to be successfully and profitably integrated into
the control system
Easily configurable at runtime tools to add or
remove alarms without restarting the system
Support for complex alarm rules based on formulas
is desirable
Flexible enough to carry some specific additional
information, such as a specific message for the
machine operator
Consistent between multiple clients they must
share and show the same information
4
Development of the TANGO Alarm System
Lorenzo Pivetta ICALEPCS 2005
5
Requirements (2)
Alarm rules based on input values from multiple
computers of the control system are required
...
Alarm client
Alarm client
Workstation
Alarm client
Centralized alarm system
Server
Equipment controller
Equipment controller
Low-level computer
Equipment controller
...
A centralized system satisfies this
requirement in a simple and effective manner
Ion pump
C.C. PS
Corrector PS
Controlled device
ID
Penning
5
Development of the TANGO Alarm System
Lorenzo Pivetta ICALEPCS 2005
6
Alarm rules
In order to define each alarm and specify the
associated condition an alarm rule should be
written
Each alarm rule is made by three distinct fields
a unique self-explaining label the alarm name
the condition to be evaluated the alarm formula
an optional text message
The alarm formula is made by identifiers,
numbers, operators and simple mathematical
functions
6
Development of the TANGO Alarm System
Lorenzo Pivetta ICALEPCS 2005
7
Alarm formula
Identifiers are the TANGO attributes that
specify the input values
Binary and logical operators are available
, , , , , , -, (), ltlt, gtgt, lt, gt, lt, gt,
!, , , !
...that allow for
comparing a value with a predefined mask, a
reference or another value
complement, negate, shift...
combining several logical conditions
Round brackets can be used to define the priority
of combined conditions
Simple mathematical functions like abs, fabs etc.
can be used
7
Development of the TANGO Alarm System
Lorenzo Pivetta ICALEPCS 2005
8
Alarm rule examples
Basic alarm rule check power supply status
sr/pscid/s1.1/off (sr/pscid/s1.1/stat
0x40) C.C. PS off
Combining the status of two devices correction
coil power supply is off and insertion device
feed-forward orbit correction loop is enabled
sr/pscid/s1.1/off ((sr/pscid/s1.1/stat 0x40)
(sr/carid/s1.1/stat 0x100)) Correction
Coil Power Supply OFF
Mixing digital and analog values corrector
status and supplied current
sr/psch/s10.1/highthr ((sr/psch/s10.1/stat
0x80) (sr/psch/s10.1/curr gt
15.0)) Corrector PS supplied current high
threshold
8
Development of the TANGO Alarm System
Lorenzo Pivetta ICALEPCS 2005
9
Alarm message
As a result of the formula evaluation the status
of a given alarm could assume two values ALARM
or NORMAL
An acknowledge flag is also associated to each
alarm (ACK, NOT_ACK)
The alarm system composes a formatted text string
called alarm message containing all the
relevant information
alarm status
time stamp
acknowledge flag status
alarm name (identifier)
optional text message
Ex
Mon Sep 19 105055 2005 sr/rfamp/s3.1/off
ALARM NOT_ACK RF amplifier off
9
Development of the TANGO Alarm System
Lorenzo Pivetta ICALEPCS 2005
10
The Alarm Collector
The core of the alarm system is a special TANGO
device server based on a double client/server
architecture
as a client gathers input values from TANGO
devices and as a server provides alarm
notifications
Relies on the TANGO event system to collect input
values as well as to provide alarm notifications
no provision to work without events
increased efficiency, responsiveness and lower
network load
Manages the alarm system current status in a
centralized manner
configuration of the alarm system in the database
information consistency between multiple clients
methods for adding and removing alarms keep
system and database coherent
10
Development of the TANGO Alarm System
Lorenzo Pivetta ICALEPCS 2005
11
How it works?
Load alarm rules from database
...
alarm logger
alarm display
Subscribe to attribute events specified in the
alarm rules
4 - push alarm event
subscribe to
Wait for inputs, i.e. events
alarm event
TANGO
server
Got event evaluate formulas containing the
corresponding attribute
DB
Alarm Collector
1 - load alarm
client
rules from DB
2 - subscribe to events
specified in alarm rules
3 - push events
Not all formulas!
...
Build fresh alarm message list
server
server
server
Push alarm event to interested clients
device
device
device
11
Development of the TANGO Alarm System
Lorenzo Pivetta ICALEPCS 2005
12
Scanner and parser
The alarm formula evaluation is made at runtime
by means of a lexical scanner and a parser
GNU Flex and Bison used to write the scanner and
the parser
The scanner is able to split each alarm formula
into its basic tokens operators, numbers and
attributes
Each token is assigned a unique value to be
returned to the parser while scanning the formula
The parser can apply to the operands the rules
specified in its grammar and finally evaluate the
whole formula
12
Development of the TANGO Alarm System
Lorenzo Pivetta ICALEPCS 2005
13
Alarm visualization
Acknowledge push buttons
Yellow ALARM and ACK status
Green NORMAL and NOT_ACK status
Red ALARM status
Acknowledge flag
Message
Time stamp
Alarm name
Alarm status
Courtesy V.Forchi'
13
Development of the TANGO Alarm System
Lorenzo Pivetta ICALEPCS 2005
14
Summary
Proven to be flexible and quite easily
configurable
Support of digital and analog input values
Straightforward use of complex alarm formula
Collection of input values transparent from any
TANGO device server
Centralized alarm management consistent between
multiple alarm clients
More than one Alarm Collector instance feasible,
for instance to separate two logical domains
14
Development of the TANGO Alarm System
Lorenzo Pivetta ICALEPCS 2005
15
Conclusions
An alarm system integrated into the TANGO control
system framework has been proposed and developed
Successfully tested at ELETTRA
Plans to be adopted in the new booster control
system
But... still young, thus available for
improvements
wider TANGO events support
support alarm hierarchy for simpler alarm
dependency management
...
15
Development of the TANGO Alarm System
Lorenzo Pivetta ICALEPCS 2005
16
Thank you for your attention
Development of the TANGO Alarm System
Lorenzo Pivetta ICALEPCS 2005