Panoramix

1
Panoramix

• A visualization environment for LHCb

2
Goals

• Handle event display and statistical
  visualization in a consistent way.
• LHCb coarse graining organization in term of
  projects

Panoramixvis
Brunel rec
DaVinci phys
Gauss sim
LHCb detector desc event model
Geant4
Gaudi/SEAL/POOL
OpenScientist

• Panoramix is on top of everything (that
  explains the below eXtreme debugger).
• Panoramix subjected to a software agreement
  between LAL and LHCb.

3
It works !!!

• From the collection of Thomas Ruf (done on
  Windows)

4
Engineering choices

• Plugged directly to the data framework (Gaudi).
  No client / server model.
• Since the data framework is in C (and most of
  native vis and GUI tools are in C / C) use C
  for all.
• Target the three laptops of the moment Linuxes,
  Windows, MacOSX by using their native
  environments.
• Used one powerful scene manager, Inventor, for
  all 3D or 2D data representation.
• Try to put in reusable elsewhere packages
  things not fully dependant of the experiment.
  Here the reusable elsewhere packages being the
  ones put in the OpenScientist distribution.
• See OpenScientist slides for more on Engineering
  choices.

5
Vis and GUI products

• Rendering layer more than ever OpenGL.
• Scene graph manager OpenInventor. By using the
  Coin implementation of System In Motion (Norway).
• Do all graphic with the uppers, including
  histogram and function plotting.
• GUI see OpenScientist slides for the
  problematic of a choice. Describe the GUI in XML
  and use the OnX package to create the GUI by
  using native toolkit of desktop providers. This
  is simple and give the best performances.
• Scripted callback
• mainly dld/c (string front end to
  dlopen/dlsym)
• ltwidget classPushButtongt
• ltactivate execCgtOnX ui_echo
  hellolt/activategt
• lt/widgetgt
• and Python (with LHGDict wrapping)
• ltactivate execPythongtfrom import
  OnX ui.echo(hello)lt/activategt

6
OnX
Xt, Win, Qt, gtk
XML
OnX
Dld, Python, CINT
GL,Inventor
7
GUI paradigm

• A la PowerPoint a compact GUI panel organized
  around a document area.

This is 100 in opposition to a terminal prompt
spawning viewers around the screen
8
Panoramix on a MacOSX by using native Cocoa
(NextStep) / OpenGL (not released yet since all
below layers not yet released on OSX).
9
Connection to the data framework

• data framework should be understood as the
  software permitting to manipulate the event
  model and permitting to connect the event
  model (the data) to facilities like storage,
  graphic, GUI, scripting. The data framework is
  here Gaudi. Gaudi does not do any kind of graphic
  (no GaudiDataObjectdraw method around and
  this is correct). Gaudi manipulates services that
  permits someone to operate the transient store
  containing, in memory, instances of the event
  model.
• OnX is here the interactivity framework. It
  does the connection between GUI (from an XML
  description), viewers, scene manager (Inventor),
  renderer (OpenGL) and scripting.
• The connection data framework interactivity
  framework is done through the Gaudi OnX service,
  put in a standalone package Vis/OnXSvc.
• Various elements of the event model have a
  representation (a datarep). A datarep code is
  in general a Gaudi converter for the Inventor
  technology (an SoConverter). An SoConverter, for
  example SoMCParticle, build from a data instance
  (here an MCParticle) an Inventor scene graph.
  When built, the scene graph is send to OnX to be
  displayed, in general, in the current viewer.
• A visualization request starts from a scripted
  GUI callback the OnXSvc is searched, then a
  data convertion for Inventor request is
  activated for various pieces of data.
• Note that, due to a massive usage of abstract
  interfaces, things are nicely decoupled (for
  example an SoConverter is not a StorageConverter.
  A direct consequence is that someone can get rid
  of a technology and migrate to a new one, without
  touching to everything.

10
The Vis/So packages

• Physicists enter into action by programming some
  SoConverter put in some Vis/So package.
• SoDet (G.Barrand, S.Ponce) representation of
  the detector. This is very generic and is now
  sparely touched. Coupled to the LHCb XML det
  description, it offers a very flexible way to
  enter and view a geometry. Note that there are
  various interactive goodies that permits to
  expand or contract volumes (from J.Boudreau
  SoDetectorTreeKit). The GUI can view the detector
  and event tree in a tree widget.
• SoEvent historical. Used to gather first
  datareps like SoMCParticle, SoMCHit (P.Mato) and
  representations for the trackers (J.Van.Tilburg
  NIKHEF).
• SoCalo (I.Belyaev ITEP) dedicated package to
  gather reps for calorimeters.
• SoRich (C.Jones Cambridge) reps for the RICH.
• SoStat (G.Barrand LAL) to represent histograms
  of the Gaudi transient store. It uses the HEPVis
  SoPlotter nodekit.
• SoHepMC to visualize HepMCs. Used in
  conjunction of the Geant4 simulation (Gauss).
• People did not seem to suffer of Inventor
  programming.

.
11
The applications

• Two packages remain Vis/Panoramix and
  Vis/LaJoconde.
• Panoramix it contains the main Panoramix.exe
  program which is a standard Gaudi main. This
  package contains also the Panoramix.onx and other
  .onx XML files describing the GUI.
• It contains also C callbacks and various
  Python scripts sufficiently generals to be put
  in the common pot. It is in scripts that people
  can setup their views (number of regions with
  their size and position, background colors, etc)
  and decide of what they put in their scenes.
  This could be done in Python or directly in C
  (in case some Python wrapping is lacking, too
  tricky C is needed or intensive speed is
  needed) by using the OnX dld/C callback
  system.
• LaJoconde historically introduced to connect to
  The DaVinci code (where the secret of LHCb
  physic is hidden). It had been introduced for
  coarse graining organization but will probably
  be transferred back to Panoramix in future so
  that physicists manipulate only one front end
  program. This will depend of the capabilities to
  be able to build coarse graining Job options
  files (by saying easily I want or I do not want
  to use DaVinci) and be able to access
  corresponding packages and libs at run time.

12
Speed

• Due to basic choices (stick to desktop providers
  and be plugged directly to the data-framework) it
  is fast and the author wait anybody that claims
  to have something faster.
• BUT, we must always have an eye on Inventor. It
  is a powerful tool and then can be dangerous. A
  scene graph may become too fat and slow down
  the creation of a data representation. But there
  are numerous ways to optimize things (share
  color, line styles nodes, create dedicated nodes
  to handle huge collections of data, etc).
• Experience shows that on local machines scene
  graph with 10 000 up to 100 000 nodes are ok. For
  more orders of magnitude, you have to think at
  optimizing.
• BUT SOMEONE MUST BE AWARE THAT A CROWDY
  SCENE GRAPH OFTENLY MEANS A CROWDY PICTURE ON THE
  SCREEN.
• Then the secret of being fluent is not in being
  able to handle big scene graph, it is in being
  fluent in selection of data (selection of what
  you really want to see).
• Up to now, at least for what is done in LHCb,
  speed is sufficient. (I am more worried by what
  is (not) done in more bigger experiments)

13
Problems (1)

• All this works and had been put in place very
  early.
• Problem remains around visualization of boolean
  operations over volumes. It is the case of
  coplanar edges. Dixit E.Chernyaev, the author of
  the algorithm (the BooleanProcessor of Geant4)
  there are cases where things cannot be decided
  (easily) and the best way is to avoid to have
  edge to edge situations. Fine, ok. But it is
  hard to explain to the detector people that they
  have to add or remove some matter here or there
  in order to pass a visualization algorithm
  !!!!!
• But a non-algorithmic solution perhaps remains
  use the stencil capabilities of OpenGL to do
  that. Code had been found in Australia but not
  yet put in an Inventor node and integrated /
  tested in the system. A side effect it probably
  kill the vector PS production and then only
  pixmap PS production would remain for these
  views.
• If someone has ideas ? Other algorithms to try,
  etc

14
Problems (2)

• Not yet fluent in fully stand alone installation
  on a (remote) local machine. We remember that a
  display is on top of everything and that the
  three common desktop are targeted. Then be fluent
  in installing the display passes by being fluent
  in installing all the below
• Today, binaries (Windows, Linux) are regularly
  reconstructed on the /afs/cern.ch areas (lot of
  thanks to Florence Ranjard) and users run them on
  their local machine. (Doing remote X11 on lxplus
  is definitely not recommended). Then we
• Use CERN as a central integration / binary
  construction / distribution place THIS IS GOOD.
  (I strongly believe that this is the number one
  role of CERN).
• DO NOT USE CERN at run time but use local
  machines seeing CERN with afs. But CERN anyway
  around at run time as a STRONG afs server
  provider.
• For more, we strongly rely on efforts done around
  CMT (and PACMAN ?) to provide at some time
  straight forward source or binary local
  installation of everything. (See Christian
  Arnault poster about ATLAS software distribution).

15
Problems (3)

• LCG where are we going here ?
• Can we build bosonic software with fermionic
  people ?
• Panoramix on top of much more things not really
  deeply though to work together.
• MacOSX port had been a pure nightmare (And the
  story is not finished here. The only presence of
  ROOT induces to run in bind_at_launch mode that
  slows down the startup of everything. There are
  mysteries around dynamic loading and execution of
  static object constructors done with Gaudi and
  SEAL / PluginManager).

root/th
cint
root/io
pool
pi
gaudi/seal
python
lcgdict
16
A problem ?

• In the software agreement, was mentioned that a
  physicist should be involved (at partial time) to
  do the sync ltengineeringgt ltphysicgt during the
  whole story.
• This never happened.
• For example, despite that Panoramix received code
  contributions from numerous people and that it is
  used by various people, no physicist strongly
  visualization driven spontaneously emerged to
  survey that all element of the event model
  received a representation. The engineer is NOT
  paid to have all the LHCb detector in head along
  with all the event model and physic algorithms
  and to decide how things should be represented.
• It would be welcome to avoid a last minute rush
  on that last point.

17
Problems (4)

• http//www.lal.in2p3.fr/Panoramix.
• html pages and paper doc produced with Doxygen
  from dedicated .h files (and not the ones of the
  code). Some of the files loadable from the
  program to form a help pull down menu and
  contextual helps.
• But the content is clearly insufficient.
• Nice user page of Thomas Ruf
• Basement for a wiki ?
• But for programmers (writing SoConverter), doc of
  Gaudi and Inventor is online.
• But once more, is it to the engineer to document
  everything and in particular all what can be
  represented with various options, etc?

18
Problems (5)

• How to fit 48 hours in one day ?
• (Transplantation of a superconducting
  positronic brain ? ())
• Need manpower ? Sure, the needed army that LCG
  will not put in place for doing graphic

Due to the international context, the US Robots
refuses to do that on a French for the moment.
19
Next

• Next main issue is going to be the integration of
  stat tools to do analysis. For LHCb it will
  consist to operate The DaVinci code from the
  interactive tools. Here the way to deal with
  events is a little bit different
• Display pick one event, view things in it, pass
  to another event, etc
• Analysis pick and configure one or a bunch of
  algorithms (that will book histos and tuples),
  run over thousands of events, look histograms.
  Repeat again by modifying algorithms.
• Thats right that what is done BY THE PHYSICIST
  is different but things to put in place BY THE
  ENGINEER ARE EXACTLY THE SAME IN BOTH SITUATIONS
  a GUI, operate on the event model, produce
  graphical representations from data.
• The only significant differences are around the
  data that must be visualized. In one case it is
  things in the event model, in the other case it
  is common things like histograms and tuples.
• In the second case, then a key issue is to
  INTEGRATE A GOOD PLOTTER, that is to say a good
  piece of software able to create graphical
  representation of histograms.

20
HEPVis / SoPlotter, AIDA

• Due to the choice of Inventor as scene manager,
  the author targets a plotter done with Inventor
  here the HEPVis / SoPlotter one developed mainly
  be the author. SoPlotter covers now a lot of
  common cases (see OpenPAW poster for pictures).
• The Vis / SoStat package does the connection
  Gaudi-histos / SoPlotter. This is in fact
  available for long but not yet heavily used due
  to the fact that connection to DaVinci is a
  little bit young. (and probably because people
  had taken some habits with other tools).
• Then, with SoPlotter, the same Panoramix
  environment can then be used also by working on
  DaVinci. (It opens also the way to do nice things
  like putting histo within events).
• For scripting, someone can already get from
  Python the OnXSvc and do a vis request over an
  histo managed by Gaudi. Someone can also
  customize a lot of the SoPlotter.
• What have to be provided now is to standardize
  the way to do that. The plan is to used the AIDA
  API and interfaces, mainly because it had been
  though and discussed by various parties and that,
  being pure abstract interfaces, it does not
  compel us to tied to another system having
  orthogonal backend GUI and graphic.

It is crucial to keep in sync the LHCb software
with AIDA
21
Integration of other tools (1)

• Had been demonstrated than from a python shell,
  someone can spawn a DaVinci task with
• Panoramix (to display an event or plot an histo)
• Hippodraw
• ROOT
• Then physicists will have tools.
• But, shall software engineers be happy with that
  ? NO. All these tools have incompatible backend
  technologies (rendering, GUI) and there is no
  hope to provide one tool within a common GUI that
  integrate all that in a consistent interactive
  way.
• A physicist will be let with a desktop crowded
  with standalone GUI windows, each having its own
  logic. An already announced pain.
• Then, some job (low priority) is done to try to
  integrate some of these tools within Panoramix

22
Integration of other stat tools (2)

• Hippodraw the design of it is so that someone
  can easily get rid of the default GUI and
  rendering (all in Qt), and comes with new ones.
  Had been done an OpenGL driver to hippodraw. From
  that it is quite easy to integrate a hippo window
  within Panoramix. An embryo of reimplementation
  of the famous hippo inspector done with OnX
  (then mainly in XML) exists now (OpenScientist /
  Hippo package).
• ROOT it will go through an implementation over
  Inventor of the painful TVirtualPad (some kind of
  strange mixture of GUI, rendering and scene
  manager virtual methods). A prototype exists
  (OpenScientist / Mangrove package). Someone can
  visualize a TGeo with OnX and Inventor and also
  the bins of a TH. But more work have to be done
  to have the full THistPainter over Inventor. But
  this is feasible. (Any volunteer ? No one, as
  usual).
• Java / JAIDA / jas hard to see how we can have
  jas drawing in an Inventor viewer or an OpenGL
  window not managed by the awt / swing. Moreover
  it appeared difficult to have Inventor drawing in
  an awt / swing window. Strong expertise required
  to go on.
• If no help from the developers of these tools is
  coming, the author will continue to put the
  effort on the SoPlotter (and OpenPAW) and to its
  AIDA front end the AIDAIPlotter.

23
Must be mentioned Agora

• Agora is a prototype of visualization environment
  for ATLAS using the same technologies than
  Panoramix (OpenGL, Inventor, OnX, AIDA).
• It makes sense since the data-framework is the
  same than LHCb (Gaudi).
• Proof of feasibility done.
• In fact here, it is a shame that LHCb and
  ATLAS had not been able to converge on XML for
  the det desc.
• Anyone interested ?

24
Conclusions

• Panoramix is regularly released and used and had
  already given services (for the XML det desc
  debugging, reconstruction debugging).
• Even if not yet an overflow, views appear in
  meetings and workshops and some are very very
  nice.
• Engineering choices are done (let us say up to
  the startup). But the overall architecture will
  permit to migrate parts if needed.
• More work around the release building and
  distribution are needed if the three common user
  laptops are targeted (Linux, Windows, Mac).
• At the engineering level, efforts will be put now
  on the final (!) integration of statistical tools
  AIDA / OpenPAW for sure, hippodraw probably,
  ROOT with a gun in the back. Rendez vous at next
  CHEP for that.
• Are physicists happy ? (is a physicist ever happy
  ?). It is not to the author to give an answer but
  there are not too much screams, shouts, laptop
  thrown on the wall for the moment