Analysis Tools: Exploiting Component Architecture through AIDA, HepRep, JAS and WIRED

1
Analysis ToolsExploiting Component Architecture
through AIDA, HepRep, JAS and WIRED

• Max Turri
• Mark Dönszelmann, Tony Johnson,
• Joseph Perl, Victor Serbo
• SLAC

2
Outline

• The Component Architecture
• FreeHEP Java library
• Data Analysis
• AIDA
• JAS3
• Event Display
• HepRep
• WIRED
• References

3
The Component Architecture

• Interfaces first!

4
The Component Architecture

• A three step procedure
• Identify the irreducible blocks
• Design the interfaces
• well defined behavior
• flexible
• language neutral
• Implement the interfaces
• discover pitfalls in the interfaces

5
The Component Architecture

• Advantages
• Software with modular structure
• Easy to update/re-write individual blocks
• Collaboration between tools providers to improve
  interfaces
• Overall better tools
• Learn only the interfaces
• Experiments or collaborations choose the
  interfaces
• No ties to a specific implementation
• Users choose the desktop tool
• or parts from different tools

6
The Component Architecture
BaBar, GLAST,LCD, Geant4
Data Analysis Visualization
Event Display
HepRep
AIDA
Abstract Interfaces
IceCube, CLEO
WIRED
FRED
7
FreeHEP Java Library

• Components and tools in Java for HEP and beyond.
• The library on which our tools are based.
• http//java.freehep.org

8
FreeHEP Java Library

• Introduction
• HEP Specific Components
• General Components
• AID

9
FreeHEP Java Library - Introduction

• Started as a convergence of
• WIRED
• Java Analysis Studio (JAS)
• A common base library
• minimize unnecessary duplication of work
• maximize code reuse
• Library of HEP-wide Java software
• Some very HEP specific
• Some general purpose components and tools

10
FreeHEP Java Library - Introduction

• Open source
• All code and documentation in CVS, web browseable
• All code licensed under LGPL to make it freely
  reusable
• Minimal interdependence between packages
• Take what you like, leave what you dont.
• Pure Java where possible
• We include some C code for wrappers to make
  components available to C
• Now have contributions from
• Atlas, Babar, IceCube, US Linear Collider
  Detector (LCD)
• Others welcome to contribute

11
FreeHEP Java Library - HEP Specific Components

• Physics package
• 3-vector, 4-vectors and utilities
• Jet Finding, Event Shape routines
• Diagnostic Event Generator
• HEP IO
• Access to a number of common HEP formats
• StdHEP
• HBook
• Root
• HEP3D
• Extensions to Java3D specific for Geant4
• YaPPI Yet another Particle Property Interface
• XML Particle Property Database

• AIDA - Abstract Interfaces for Data Analysis
• Interfaces and JAIDA - Java reference
  implementation
• AIDAJNI C wrapper around AIDA
• HepRep - HEP Representables for Event Display
• Interfaces and JHepRep - Java reference
  implementation
• HepRepJNI C wrapper around HepRep
• Future
• Plot Widget (JAS)

12
FreeHEP Java Library - General Components

• Application Framework
• General framework for Java applications using
  plugin and service technology.
• AID Abstract Interface Definition
• Compiler to make C headers and Java Interfaces
  from aid files.
• Graphics3D
• Non-Java3D implementation of 3D graphics, for
  WIRED.
• FreeHEP IO
• Extensions to the JavaIO system to handle
  specific encodings, ASCII85,
• PSViewer
• PostScript Viewer to embed viewing of EPS in
  applications, or for previews.

• Swing Extensions
• FreeHEP Tools
• JNeeds java package dependency
• Ant extensions to the build system
• VectorGraphics
• Low level Input and Output of binary vector
  graphics formats (CGM, EMF, PDF, SVG)
• Vector output files, extension to Java2D
• XML
• XMLIO classes to store/restore state of an
  application
• RecordLoop/EventLoop (IceCube)

13
FreeHEP Java Library - AID

• Abstract Interfaces Definition
• Hyper-language use to write interfaces
• very much like java with and
• Automatic generation of C header files and java
  interfaces
• More languages can be added
• Minimize inconsistencies
• HepRep and AIDA are generated with AID

14
Data Analysis

• Interface AIDA
• Tool JAS

15
AIDA

• Abstract Interfaces for Data Analysis
• http//aida.freehep.org

16
AIDA

• Mission Statement
• Features
• Advantages
• Organization and Developers
• Status
• Interfaces
• Tools and Users
• Future

17
AIDA Mission Statement
The goal of the AIDA project is to define
abstract interfaces for common physics analysis
objects, such as histograms, ntuples, fitters.
The adoption of these interfaces makes it easier
for developers and users to select and use
different tools without having to learn new
interfaces or change their code. In addition it
is possible to exchange data (objects) between
AIDA compliant applications through a standard
XML format.
18
AIDA - Features

• Interfaces start with I
• Define only pure virtual methods
• AIDA defines behavior, analysis tool provides
  implementation
• Object factories
• independence of user code from implementation
• easy to switch implementations
• IAnalysisFactory af IAnalysisFactory.create()

19
AIDA - Advantages

• Use same code with any AIDA-compliant analysis
  tool
• Used for experiments code, e.g. reconstruction
  or individual users analysis
• User needs to learn only one set of interfaces
• Interoperability of analysis tools
• Under way

20
AIDA - Organization and Developers

• Born from the desire of tools providers to
  converge to a common vision improving each
  others work
• Open source project
• Currently a CERN-LAL-SLAC effort
• Interfaces are designed by discussion and
  (eventual) consensus
• Takes some time, but result is well though out
  and robust
• Past and present contributors
• Guy Barrand, Pavel Binko, Grzegorz Chwajol, Mark
  Dönszelmann, Wolfgang Hoschek, Tony Johnson,
  Emmanuel Medernach, Dino Ferrero Merlino, Lorenzo
  Moneta, Jakub Moscicki, Ioannis Papadopoulos,
  Andreas Pfeiffer, Max Sang, Victor Serbo, Max
  Turri

21
AIDA - Status

• AIDA Version 3.0 (Oct 2002) second end-user
  release.
• AIDA 3.2 coming out soon
• AIDA 2.2 first end-user release
• Interfaces available in C and Java
• automatically generated with AID
  (http//java.freehep.org)
• easy to add more languages

22
AIDA - Interfaces

• IHistogram
• ICloud
• IProfile
• IDataPointSet
• ITuple
• IEvaluator
• IFilter
• IPlotter
• IFitter
• IFunction
• ITree

23
AIDA - Tools and Users

• Four implementations are available within
• Anaphe/Lizard (C)
• http//anaphe.web.cern.ch/anaphe
• Open Scientist (C)
• http//www.lal.in2p3.fr/OpenScientist
• JAIDA/JAS (Java) AIDA-JNI 3.0 (C)
• http//java.freehep.org/lib/freehep/doc/aida
• PAIDA (Python)
• http//paida.sourceforge.net/
• Users
• GEANT4 Some examples
• BaBar online (via JAS)
• GAUDI/ATHENA users
• Linear Collider users
• LCG has considered AIDA in their Architectural
  Blueprint
• Physicist Interfaces (PI) group is working to
  create an AIDA implementation based on ROOT and
  SEAL classes

24
AIDA - Future

• Achieve component level interoperability
• e.g. use ANAPHE histograms, fit with JAS3 fitter
  and plot with OpenScientist
• define developer interfaces
• first prototype due in October
• Remote Access to AIDA
• Access to remote AIDA objects (through CORBA,
  RMI, SOAP, ) from another AIDA implementation.
• Access to AIDA objects through WEB Browser (with
  limited functionality)
• AIDA 4.0 end of 2003

25
JAS

• Java Analysis Studio
• http//jas.freehep.org

26
JAS

• Introduction
• JAS3 a new JAS
• JAS3 Architecture
• Tour of major components of JAS3
• AIDA
• Root and PAW file readers
• Tuple Explorer
• Java compiler and loader
• Scripting Languages
• Pnuts
• Record loop
• Spreadsheet
• Status

27
JAS - Introduction

• JAS is a
• general purpose
• open-source
• almost fully in java
• data analysis tool with the following
  features
• Plotting of 1d, 2d and 3d histograms, XY plots,
  scatterplots etc.
• High quality print output.
• Fitting (binned or unbinned) using an extensible
  set of optimizers including Minuit.
• Export of plots in a variety of formats including
  PS, EPS, PDF, SVG, GIF, PNG.
• Easy to learn GUI for performing common analysis
  tasks (plotting, ntuple-analysis, fitting etc.).

28
JAS - Introduction

• More complex analysis can be performed using a
  variety of scripting languages (pnuts, jython,
  etc.), or by writing Java analysis modules.
• Able to read data in a variety of formats
  including aida, hbook, root, SQL databases,
  text-files.
• AIDA compliant analysis system.
• Built-in editor and compiler.
• Simple spreadsheet capabilities.
• Logbook for recording analysis tasks, and
  optionally publishing results to the web.
• Highly modular structure
• allows addition, replacement or removal of
  modules to customize application for a particular
  problem domain.

29
JAS3 - a new JAS

• First version of JAS2 released in 2000
• Has worked well, but some limitations
• Too heavily based on event loop
• Not all analysis tasks involve looping over
  events
• No support for scripting
• Limited fitting capabilities
• Limited N-Tuple analysis capabilities
• Often used for online monitoring, often used
  outside HEP
• Not really designed for this use

30
JAS3 - a new JAS

• Opportunities
• Realized a lot of overlap with WIRED project
• Created FreeHEP Java library to
• Pull out (re-factor) common parts of JAS, WIRED
• Implement new common requirements for JAS and
  WIRED
• Vector Graphics, Application Framework, Utilities
  
• Formed collaboration with developers of similar
  tools to create AIDA package

31
JAS Architecture

• Design based on Application Shell, into which
  many (optional) modules can be plugged
• The provided services are implemented by one (or
  more) plugin modules
• Loose coupling between modules allows for modules
  to be removed without breaking remainder of
  program
• Extra modules (experiment specific) can be easily
  added
• Highly customizable for different application
  domains
• HEP/Astrophysics/Other
• DST analysis/Online Monitoring/GRID analysis
• Experiment/User specific modules
• Modules can be updated independently of shell
• Possible to release bug fixes fast

32
JAS3 Components

• Tour designed to give you an overview of the
  capabilities of JAS3, you can try them out for
  yourself with the built-in Tutorial and/or
  examples

Welcome Page, gives initial info and links to
example scripts and programs
Memory monitor
33
JAS Opening
Drag from explorer
Use file menu
34
JAS Graphical Interface to AIDA
Histograms, Clouds, Tuples all presented in AIDA
tree
.aida files, .hbook files, .root files all
presented as AIDA objects
Drag items onto page, or use (popup) menus
35
JAS Printing
Or copy/paste into Word, PowerPoint etc.
Can send individual plots or full page direct to
printer
Or save as PS, EPS, PDF, SWF, SVG, PNG, GIF
36
JAS Java Editor, Compiler and Loader
JAS3 allows any Java program to be loaded. This
example main routine is taken directly from the
AIDA manual
Built-in editor for writing analysis code
Tree shows loaded programs
Built-in Java compiler
37
JAS Scripting
Can also write and run scripts
Console allows direct interaction with scripting
language
38
JAS Record Sources
Opening record (or event) based files causes the
run control toolbar to appear
Job controls. Also supports random access and
tagged data sets (mainly for event displays)
39
JAS Tuple Explorer - Plots
Works with any tuple, read from file or
dynamically created
Histogram
Profile
ScatterPlot
XY Data (More appropriate for smaller data sets)
40
JAS Tuple Explorer Define Columns
41
JAS Tuple Explorer - Cuts
42
JAS Tuple Explorer - Tabulate
43
JAS Spreadsheet

• Simple spreadsheet plugin
• for
• Displaying results
• Calculations
• Simple Plots
• Supports reading/writing
• .csv files
• Excel files
• Cut/Paste with Excel etc
• Coming Soon
• Scripting interface
• GUI for building plots
• User defined functions
• Java, scripting

44
JAS Miscellaneous Features
Save/Restore configuration
User Preferences
Plugin Manager
45
JAS Status

• Currently released JAS3 version 0.7.2
• AIDA functionality is quite solid
• Documentation is still quite limited
• Talks, tutorials, built-in example scripts and
  programs accessible from simple built-in
  web-browser
• If you are used to JAS2 you will find some
  functionality not yet ported to JAS3
• Remote (client/server) access to data (this is
  being worked on)
• 3D Lego/Surface plots

46
JAS - Help us make it better

• We have recently purchased and are now heavily
  using a commercial issues tracking system.
• Used not only for bugs, but also for tracking
  tasks, ideas etc.
• Please report any problems you find, and make
  suggestions for changes or improvements you would
  like to see.
• http//bugs.freehep.org

47
Event Display

• Interface HepRep
• Tool WIRED

48
HepRep

• A Generic Interface Definition for HEP Event
  Display Representables
• http//heprep.freehep.org

49
HepRep

• Introduction
• Representable Example
• Current Use
• Future Plan

50
HepRep - Introduction

• Interface for Representables
• Essential Spatial Information of a physics object
• track, calorimeter hit, etc
• Physics Attributes
• momentum, energy, etc
• Bridge between server and client event display
• Spatial information is assembled and shipped as
  HepReps
• complicated details of the objects are lefts on
  the server side
• rendering decisions are deferred to the client
• Language independent
• AID generated in Java and C
• Protocol independent
• Corba, RMI, XML, JNI
• Compliant Tools
• WIRED (Java)

51
HepRep Representable Example

• A fitted track is shipped as
• A set of points with Attributes
• helix parameters
• descriptive information
• track number, particle id, etc
• On the server side it is represented as
• a dotted line,
• or as set of individual step momentum vectors,
• or as a set of helix segments.

52
HepRep Current Use

• Different experiments/collaborations
• have specific HepRep implementations
• use different protocols
• Not all of them can use FRED

53
HepRep Future Plan
RMI
LCD(java)
Java Shared HepRep Factory
HepRep2
Java
RMI
Corba
IceCube(java)
Java
WIRED 3 (Java)
XML
HepRep2
Corba
XML
BaBar(c)
Java
C Shared HepRep Factory
Corba
HepRep2
Geant4(c)
XML
Corba
FRED(C/Ruby)
HepRep2
GLAST(c)
C
XML

• Through an abstract HepRep factory (from FreeHEP)
• Instantiate a concrete HepRep implementation
• Choose the protocol
• Choose the tool

54
WIRED

• WWW Interactive Remote Event Display
• http//wired.freehep.org

55
WIRED

• Introduction
• Overview of WIRED 3
• Toggling visibility
• Orientation Actions
• Parallel Projection
• Fish-Eye Projection
• ?-Z Projection
• Mouse Function
• Rotation, Scaling, Translation
• Picking
• Plans for WIRED 4

56
WIRED - Introduction

• Features
• Experiment independent Framework for Event
  Displays in Java
• 3D Wire Frames
• Interactivity (Rotate, Zoom, Translate)
• Picking
• Special projections
• Linear 3D projections
• Non-Linear, Fish-Eye Blow Up projections
• Layering of Graphics Primitives
• Allows hits to be on top of tracks
• HepRep compliant (not for LCD yet)
• Client-Server model
• Publish quality output
• PDF, PS, EMF, SVG
• GIF, JPEG, PNG, PPM
• Based on FreeHEP library
• Experiments and Users
• BaBar, LCD and GLAST

57
WIRED Overview of WIRED3 Toggling visibility
58
WIRED Overview of WIRED3 Orientation Actions
59
WIRED Overview of WIRED3 Parallel Projection
60
WIRED Overview of WIRED3 Fish-Eye Projection

• Blows up Center, compresses Outwards

61
WIRED Overview of WIRED3 ?-Z Projection

• Cuts detector and folds up two halves

62
WIRED Overview of WIRED3 Mouse Function
63
WIRED Overview of WIRED3 Rotation, Scaling,
Translation
64
WIRED Overview of WIRED3 Picking
65
WIRED - Plans for WIRED4

• WIRED4 just as a JAS3 plugin
• better integration with JAS3
• Use HepRep2 as its backbone
• Better memory usage
• Physics information on Picking
• Programmatic access for filtering
• Interactive cuts on attributes

66
References

• FreeHEP Java Library - http//java.freehep.org
• AIDA http//aida.freehep.org
• JAS http//jas.freehep.org
• JAIDA http//java.freehep.org/jaida/
• HepRep - http//heprep.freehep.org
• WIRED http//wired.freehep.org