Writing software or writing scientific articles?

1
Writing software or writing scientific articles?

• Maria Grazia Pia
• INFN Genova, Italy
• T. Basaglia (CERN), Z. Bell (ORNL),
• P. Dressendorfer (IEEE), A. Larkin (IEEE), other
  authors
• Please let me know if you wish to be in the
  authors list
• IEEE Nuclear Science Symposium 2007
• Honolulu, HI, USA

2
Physics Today, March 2004, 61-62
Do software-oriented physicists follow similar
publication patterns as their hardware-oriented
colleagues?
Are there any different habits of
software-oriented publication in HEP and other
radiation physics disciplines?
No scientometric study on this topic yet
3
Background
1997
1987
2007
4
Data analysis

• Main source of data
• ISI Web of Science (covers year gt1990)
• Google Scholar (HEP experiments year lt 1990)
• Publisher web site and search engines (Elsevier
  Science Direct, IEEExplore etc.)
• Internal editorial data IEEE TNS (thank you!)
• Detailed analyses cover years 2002-2006
• Citation searches 1990-today (ISI Web of Science
  coverage)
• Automated searches
• But manual inspection of a partial sample at
  least avoid blind analysis!
• Introduction of noise background evaluation to
  be refined
• Manual scan for paper classification
• In many cases no other way to evaluate the
  pertinence of papers
• Some degree of subjective evaluation (1-10)
• Conservative bias assign to software in case of
  sw/hw ambiguity
• Cross checks with other databases (INSPEC, CDS
  etc.)
• For a few samples

5
HEP experiments
How does software-oriented HEP literature
production compares to hardware-oriented one?

• A set of reference HEP experiments
• LEP, LHC, Tevatron, PEP-II, HERA, fixed target,
  astroparticle
• Apologies to those not included in the
  statistics no judgment of merit!
• Publications on technical journals only
• Exclude papers on physics results
• Hardware
• Software
• Trigger/DAQ
• More hardware-oriented in the early days (LEP
  era)
• More software-oriented nowadays (LHC era)
• Manual scan ( 300 papers/experiment at most)

6
Hardware vs software papers in HEP
LEP full experimental life-cycle ALEPH, DELPHI,
L3, OPAL
LHC the new generation ALICE, ATLAS, CMS, LHCb
In between CDF, ZEUS, BaBar
Labs CERN, DESY, FNAL, LNGS, SLAC
Fixed target NA48
Astroparticle LNGS, GLAST
7
HEP technical publications Most popular journals
8
Grid computing

• The big hype in HEP nowadays
• Not only in HEP
• Large investments (funds, manpower)
• Large literary production (2002-2006)
• Grid/distributed computing journals 4572 papers
• NIM A IEEE TNS 10386 papers

What are the publication trends in this active
computing domain?
Where does HEP stand in the picture?
9
Source ISI Web of Science 2002-2006
All types of publications journals, proceedings
10
Computing journals
Different publication habits US/EU academic
environment Asian univ.
Institutes
Countries
11
Where is HEP?
Grid computing plays a major role in LHC
experiments HEP labs/institutes play leading
roles in grid development
Countries
IEEE TNS makes the difference! No regular paper
on grid-computing in NIM (only in NIM-proceedings)
12
Simulation - Monte Carlo
One of the main areas of software contribution to
experimental physics research Event
generators Particle transport
Which domains for simulation papers ?
13
Monte Carlo codes
Statistics in ISI Web of Science, 2002-2006
Mixed sample
Geant4 citations Others word search
Includes GEANT-FLUKA (11)
Beware often Geant4 is mentioned as GEANT in
published papers
14
Journals where mentioned
A large fraction of Monte Carlo literature is
published in medical physics and radiation
protection journals
General Medical Radiation Protection Nuclear
Journal categories
HEP Monte Carlo papers represent only a fraction
of NIM Monte Carlo papers (all classified as HEP)
15
Monte Carlo / Simulation
Distribution of articles across experimental
topics ISI Web of Science, 2002-2006
Other disciplines publish more papers on Monte
Carlo / Simulation than HEP
16
Computing - Software

• Generic keyword search too noisy
• Restrict search to a subset of technical journals
• Computing software algorithm Monte Carlo
  simulation
• Still some noise introduced in the sample
• Some software papers not retained by the
  selection
• Comp. Phys. Comm. 62 sample retained
• Fraction of CPC missed mostly theoretical,
  non-radiation physics
• Tests with other keyword searches do not modify
  the conclusions substantiall
• Better check needed for TNS on noise introduced
• Sample selected mostly detector application
  papers

17
Software - Computing

• Keyword search in ISI Web software computing
  algorithm
• Top 10 Nuclear Technology journals
• Periods gt 1990 and 2002-2006

18
Citation statistics

• Not necessarily the best metric of scientific
  relevance
• but widely used (journal impact factor)
• Most cited papers in HEP labs/institutes
• CERN, INFN, other labs
• Most cited papers in selected technology journals
• NIM A, TNS, Med. Phys., Phys. Med. Biol., Rad.
  Prot. Dos.
• Most cited papers in top 10 Nuclear Technology
  journals

81269 papers in total
Where do software papers stand?
19
Most cited papers - CERN

1. Sjostrand THigh-energy-physics event generation
   with Pythia-5.7 and Jetset-7.4 Comp. Phys. Comm.
   82 (1) 74-89 Aug 1994 Times cited 1835
2. Antoniadis IA possible new dimension at a few
   TeVPhys. Lett. B 246 (3-4) 377-384 Aug 30 1990
   Times Cited 981
3. Amaldi U, Deboer W, Furstenau HComparison of
   grand unified theories with electroweak and
   strong coupling-constants measured at LEP 
    Phys. Lett. B 260 (3-4) 447-455 May 16
   1991Times cited 801
4. Agostinelli S, et al.GEANT4 - a simulation
   toolkitNIM A 506 (3) 250-303 Jul 1 2003 Times
   cited 657

20
Most cited papers - INFN

1. Gammaitoni L et al.Stochastic resonance Rev.
   Mod. Phys. 70 (1) 223-287 Jan 1998Times cited
   1574
2. Marchesini G et al.HERWIG 5.1 - A Monte-Carlo
   event generator for simulating hadron emission
   reactions with interfering gluons Comp. Phys.
   Comm. 67 (3) 465-508 Jan 1992Times cited 999
3. Abe F et al.Observation Of top-quark production
   in (p)over-bar-p collisions with the Collider
   Detector at Fermilab Phys. Rev. Lett. 74 (14)
   2626-2631 Apr 3 1995Times cited 739
4. Agostinelli S et al.GEANT4-a simulation
   toolkit NIM A 506 (3) 250-303 Jul 1 2003 Times
   cited 657

HEP paradox? Few software publications but
software articles are most cited (much more than
hardware ones!)
21
How does it compare to other labs?

• FNAL
• No software papers among the 100 most cited ones
• DESY
• Software paper in 4th rank of DESY most cited
  ones
• Lonnblad LARIADNE Version 4 - a program for
  simulation of QCD cascades implementing the color
  dipole model Comp. Phys. Comm. 71 (1-2) 15-31
  AUG 1992 Times Cited 427
• LLNL
• Most cited software paper 88th
• Prestridge DSSignal scan - a computer-program
  that scans DNA-sequences for eukaryotic
  transcriptional elements Computer Applications
  in the Biosciences 7 (2) 203-206 APR 1991 Times
  Cited 325

22
Most cited papers NIM A

1. Agostinelli S et al.GEANT4-a simulation
   toolkit NIM A 506 (3) 250-303 Jul 1 2003 Times
   Cited 663
2. Radford DCESCL8R and LEVIT8R - Software for
   interactive graphical analysis of HPGe
   coincidence data sets NIM A 361 (1-2) 297-305
   Jul 1 1995 Times Cited 491
3. Kubota Y et al.The CLEO-II detector NIM A 320
   (1-2) 66-113 Aug 15 1992 Times Cited 453
4. Adeva B, et al.The construction of the L3
   experiment NIM A 289 (1-2) 35-102 Apr 1 1990
   Times Cited 450
5. Ahmet KThe OPAL detector at LEP NIM A 305 (2)
   275-319 Jul 20 1991 Times Cited 442

Top two software!
Large-scale HEP detectors
23
Most cited papers IEEE TNS

• Cherry SR et al.MicroPET A high resolution PET
  scanner for imaging small animals IEEE Trans.
  Nucl. Sci. 44 (3) 1161-1166 Part 2 Jun 1997
  Times Cited 234
• Melcher CL, Schweitzer JSCerium-doped lutetium
  oxyorthosilicate - a fast, efficient new
  scintillator IEEE Trans. Nucl. Sci. 39 (4)
  502-505 Aug 1992 Times Cited 189
• Strother SC, Casey ME, Hoffman EJMeasuring pet
  scanner sensitivity - relating countrates to
  image signal-to-noise ratios using noise
  equivalent counts IEEE Trans. Nucl. Sci. 37 (2)
  783-788 Part 1 Apr 1990 Times Cited 167
• Summers GP et al.Damage correlations in
  semiconductors exposed to gamma-radiation,
  electron-radiation and proton-radiation IEEE
  Trans. Nucl. Sci. 40 (6) 1372-1379 Part 1 Dec
  1993 Times Cited 160
• Hoffman EJ et al.3-D phantom to simulate
  cerebral blood-flow and metabolic images for PET
• IEEE Trans. Nucl. Sci. 37 (2) 616-620 Part 1 Apr
  1990 Times Cited 134

24
Most cited papers Med. Phys. Phys. Med. Biol.

1. Nath R,et al.Dosimetry Of Interstitial
   Brachytherapy Sources - Recommendations Of The
   AAPM Radiation-Therapy Committee Task Group No
   43 Med. Phys. 22 (2) 209-234 Feb 1995 Times
   Cited 610
2. Rogers DWO et al.Beam - A Monte-Carlo Code To
   Simulate Radiotherapy Treatment Units Med. Phys.
   22 (5) 503-524 May 1995 Times Cited 391
3. Studholme C, Hill DLG, Hawkes DJAutomated
   Three-Dimensional Registration Of Magnetic
   Resonance And Positron Emission Tomography Brain
   Images By Multiresolution Optimization Of Voxel
   Similarity Measures Med. Phys. 24 (1) 25-35 Jan
   1997 Times Cited 305
4. Farrell Tj, Patterson MS, Wilson BA
   Diffusion-Theory Model Of Spatially Resolved,
   Steady-State Diffuse Reflectance For The
   Noninvasive Determination Of Tissue
   Optical-Properties Invivo Med. Phys.19 (4)
   879-888 Jul-Aug 1992 Times Cited 300
5. Gabriel S, Lau RW, Gabriel CThe dielectric
   properties of biological tissues .2. Measurements
   in the frequency range 10 Hz to 20 GHz Phys.
   Med. Biol. 41 (11) 2251-2269 Nov 1996 Times
   Cited 263

25
Top 10 Nuclear Technology journals

1. Agostinelli S et al.GEANT4-a simulation
   toolkit NIM A 506 (3) 250-303 Jul 1 2003 Times
   Cited 663
2. Ahlbom A et al.Guidelines for limiting exposure
   to time-varying electric, magnetic, and
   electromagnetic fields (up to 300 GHz) Health
   Phys 74 (4) 494-522 Apr 1998 Times Cited 547
3. Murray AS, Wintle AGLuminescence dating of
   quartz using an improved single-aliquot
   regenerative-dose protocol Radiat. Meas. 32 (1)
   57-73 Feb 2000 Times Cited 499
4. Radford DCESCL8R and LEVIT8R - Software for
   interactive graphical analysis of HPGe
   coincidence data sets NIM A 361 (1-2) 297-305
   Jul 1 1995 Times Cited 491
5. Kubota Y et al.The CLEO-II detector NIM A 320
   (1-2) 66-113 Aug 15 1992 Times Cited 453

657 ? 663 Grown while preparing the slides
26
Who cites Geant4?
72 total citations
HEP physics 33 of top 10
Medical physics 14 of top 10
Technology journals 46 of top 10
Nuclear physics 5 of top 10
27
Who does not cite Geant4?(but mentions it in
the paper)
Scientific software is not commonly perceived as
academic research deserving to be cited
28
Meditations
29
and action

• Computing Software is the largest track (
  abstracts) at this conference
• It was the largest last year too, but few
  software papers presented at the conference were
  followed by journal submission
• Proceedings are not the same as publication in a
  refereed journal!
• IEEE TNS
• Highest impact factor in its category
• Welcomes software-related papers
• our hardware-oriented colleagues give us a good
  example!

Manuscript type for software papers
Instrumentation