LDAQ the New Lujan Center Data Acquisition Application

1
LDAQ the New Lujan Center Data Acquisition
Application

• Frans Trouw, Gary Cooper and Paul Lewis
• Lujan Center, Los Alamos National Laboratory

2
Outline

• Brief Overview of Lujan Center
• Description of hardware
• Existing data acquisition application (LOS)
• Issues with LOS (reminder, see Pauls talk)
• Guiding principles for new application (LDAQ)
• Overview of existing new LDAQ application
• Experiences with LDAQ (good!)
• Summary

3
Data Acquisition at the Lujan Center - hardware

• Detectors are all 3He gas tubes or area detectors
• Pre-amps at detectors generate a differential
  signal
• Custom Time-of-Flight boards take differential
  input and convert into events in a buffer
• VME (VXI) crate with single board computer
  collects events from TOF units

4
Overview of Hardware Configuration
5
Crate contains SBC, ROC module, and 1-11 TOF units
ROC module
TOF unit
SBC
To Detectors
6
LOS Development Methodology
7
xtUML is in principle attractive

• High level description generates consistent code
  and less errors
• Extending or changing model generates new code
  that takes into account interdependencies
• Less hand-coding gt less manpower
• Quality is consistent and assured

8
xtUML has Significant Drawbacks

• Requires the desire to think at a very high level
  of abstraction requires expert
• Not all of the code can be generated
• Highly dependent on message passing
• The community using xtUML is very small (lt 100 at
  SMUG) - early adopters
• Only available for Windows
• Debugging next to impossible!

9
xtUML communication risky?

• Message passing (MP) is critical to xtUML
• MP is done using Windows message queues or
  sockets
• A single communication failure kills the whole
  system
• VxWorks 5 network stack obsolete
• Windows network stack documentation limited
  (being generous here)

10
LOS move to Linux MP problems!

• As Windows is so opaque and a single user
  environment, xtUML system libraries were ported
  to Linux
• Communication was implemented using a typical
  client server model (TCP/IP sockets)
• Linux and VxWorks 5 network stacks are
  incompatible

11
LDAQ design principles

• VXI crate LDAQ processes should be independent
  entities fault tolerant
• No host computer controlling task
• Minimal code base
• Homogenous and state of the art operating system
  environment
• Open source preferred transparency, quality
  cost

12
What is LDAQ?

• Threw out all of LOS except hand coded pieces
  that do VME communication
• Moved VXI CPUs from VxWorks to Linux
• Host computer is Linux
• Wrote a minimal application that integrates the
  VME calls about 1 month of effort
• LDAQ runs independently as a user application on
  each VXI crate

13
LDAQ visual description
Local Network Computer
20 Hz
LANL
Start Stop
LDAQ
MySQL (on host)
ROC module
Ready
Data
Trigger
TOF units
14
LDAQ benefits versus LOS

• Code can be changed and recompiled in seconds
  using Eclipse (gt 1/2h for LOS)
• LDAQ failure on any VXI crate does not affect
  others can be restarted
• Remote cross-debugging using gdb successful
• Linux is a true multi-user system (security)
• Open source no licensing costs

15
Single Event and Histogram

• LDAQ does single event out of the box
  temporary NFS limitations
• Single event -gt large amount of data, use only
  when useful
• LOS histograms implemented in LDAQ
• Live data transfer call implemented
• Defined time-focusing histogram e.g. live
  GSAS coming very soon!

16
Experience with LDAQ

• Pharos has run LDAQ for most of this year only
  failures were power cuts!
• HIPD and more recently HIPPO are successfully
  using LDAQ with legacy scripting
• LDAQ is a user application changing SBCs is
  straightforward (e.g. PPC -gt Intel) cost,
  effort
• Restart takes seconds
• Infrequent/development VME bus hangs now
  limiting reliability

17
Why is LDAQ successful?

• Each VXI crate is an independent agent does not
  know about others
• No controlling task
• Minimal features, implement bells and whistles
  externally (e.g. run timing)
• Linux is robust and transparent
• Keep it simple!

18
LDAQ development - observations

• Exclusively using Linux - economical (up front
  and manpower) cultural challenges
• Does require significant Linux expertise
• MySQL straightforward, robust free
• Agile development, develop features as needed
  requires IS collaboration
• Close collaboration between physical and computer
  scientists

19
LDAQ - future

• LDAQ development is essentially done!
• Documentation
• Needs external setup control scripts
• Scripting leave to IS, provide, or port
  existing control scripts?
• Integration with slow controls EPICS
• Visualization toolkit
• Real time analysis