XDAQ evaluation

1
Evaluation of the XDAQ framework
J. Adamczewski, H.G. Essel, S. Linev EE/GSI
2
Outline

• Requirements of DAQ framework for CBM
• The CMS XDAQ framework
• XDAQ on the CBM IB-Cluster
• Summary and further evaluation

3
DAQ for CBM

• Triggerless data transport until filter farm
• B-net 1000 x 1000 nodes, high speed connections
• Linux may run on all DAQ nodes (even FPGAs)
• Test cluster with InfiniBand small
  demonstrator set-up within next year

4
Data Acquisition framework requirements

• Configuration of multiple nodes (connections
  and algorithms, database?)
• Controls and monitoring
• Data transport (format, protocol, hardware
  drivers?)
• Message logging
• Error handling and failure recovery (ARMOR?)
• Modular architecture, extensability!
  (sub-detector tests)
• User interface?

5
The CMS XDAQ framework

• Standard DAQ framework for LHC CMS experiment
  (Orsini, Gutleber) http//xdaqwiki.cern.ch
• C libraries on Linux, modular packages
• Each node context, xdaq process with embedded
  xdaq applications
• Configuration XML
• Data transport I2O protocol (Intelligent IO)
• Communication http, cgi SOAP messages

6
The CMS XDAQ framework

• XDAQ features continued... http//xdaqwiki.cern.c
  h
• State machines (sync/async FSM)
• Message logger, error handler
• Monitoring tool
• Hardware access library (HAL)
• Front End Driver (FED kit, for CMS!)
• Job Control (task handler for node control)
• others
• exceptions
• threads
• infospace
• data (de)serializers , ...

7
XDAQ messaging and peer transport
8
XDAQ I2O messaging and peer transport
9
XDAQ web server
hyperdaq web interface
10
XDAQ executive with applications

• Common XML configuration file
• Each node knows all applications on all
  nodes!
• Unique addressing by context (url) and
  application id

11
XDAQ SOAP messaging

• example commandsConfigure, Enable, Halt,
  Reset,..
• any new commands may be defined
• web interface not suited as real control
  system UI!
• XDAQ applications may exchange SOAP messages
  with other UI (Labview?)

xrelay send SOAP control messages
12
XDAQ on the CBM InfiniBand cluster

• Installation problems
• XDAQ distribution not 64 bit save!
• Code adjustments required
• kernel modules for CERN linux excluded!
• drivers for CMS hardware (FED) excluded
• coretools done, worksuite partially,
• ? Basic evaluation possible!

13
XDAQ tests on CBM InfiniBand cluster

• Tested features on IB cluster
  http//wiki.gsi.de/cgi-bin/view/Daq4FAIR/DaqClus
  terSoftwareXdaqStatus
• hyperdaq webserver (main test controls ui)
• xml configuration setup
• control variable access and export (infospace)
• SOAP messaging (test controller application)
• state machines (sync./async. with web/SOAP ui)
• monitoring application (very raw ui!)
• multithreading (workloop framework)
• exceptions, data wrappers, message logging,
  toolbox,
• i2o data transport (tcp roundtrip example)

14
I2O roundtrip on CBM InfiniBand cluster

• Standard XDAQ roundtrip benchmark
• sender node posts I2O frame to receiver node
• receiver callback sends frame back to sender
• sender callback sends frame back again, etc
• perfmeter class records bandwidth, latency,
• web display of results on sender node (java!)
• Code independent of transport
  implementation! XML configuration file does set
  up (tcp ethernet, tcp IB, fifo)

15
I2O roundtrip on CBM InfiniBand cluster
XDAQ executive
sender
receiver
EnableAction()
token()
token()
tcp over IB
default()
16
XDAQ pseudo roundtrip with udapl rdma

• Developed udapl roundtrip in xdaq application
• Use of C wrapper for udapl (S. Linev)
• XDAQ roundtrip example, but without I2O
• no package return in receiver callback, but
  IB-RDMA write-read from sender
• measurements with XDAQ perfmeter tool
• ? XDAQ compatible with UDAPL library
  (multithreading!)
• Overhead compared with plain UDAPL!

17
XDAQ pseudo roundtrip with udapl rdma
XDAQ executive
sender
receiver
ConfigureAction()
ConfigureAction()
EnableAction()
uDAPL/RDMA
Benchmark()
allocated mem
default()
18
I2O roundtrip on CBM InfiniBand cluster
penalty of XDAQ framework? measurement? procedure?
minimum latency
19
Summary and Outlook

• CBM DAQ system needs software framework
• XDAQ offers many required features
• Tests on CBM IB cluster
• SOAP controls, web server, monitor, i2o over tcp
• XDAQ works together with uDAPL library (IB)
• To do
• I2O implementation of uDAPL performance?
• Configuration and control system?
• XDAQ scalability? Job Control?
• Implementations for DAQ demonstrator hardware?