Run I Data Access Comments From Friendly Users

1
Run I Data Access(Comments From Friendly Users)

• Greg Landsberg
• (DØ Collaboration Meeting 1/13/00)

2
Contents

• History
• Run I is Dead?
• Steps to the Disaster
• When CDF Sets a Good Example
• Ways to Fix Things
• Discussion

3
History

• Run I 1992-1996 (A, B, and C)
• Death to VMS! 1997
• Decision to keep both sets of mDST online 1999
• First Run I data access meeting Spring 1999
• Massive disk exodus 1999
• Grand finale (total collapse) December 1999
• Open discussion - Today

4
Spring 1999 Meeting Highlights

• FNALDØ/DØTNG will be reduced to a few boot nodes
  that will run indefinitely
  
• Hence users will have possibility to run limited
  MC, TRIGSIM, and NOISY
  
• PICK_EVENTS will run through the Run II data
  access model (SAM/ENSTORE) by the end of Summer
  1999
  
• PIXIE will be ported to Unix (SGI)
• Data will be moved to Unix disks (SGI)

5
Doom Reality

• PICK_EVENTS was working intermittently due to
  continuous disk failures and data migration
  
• No work on porting the PICK_EVENTS to the Run II
  framework has been done
  
• PIXIE has not yet been ported to Unix
• FNALDØ/DØTNG clusters were reduced to the very
  minimum we were told that the disk system will
  become read-only
  
• Data is disappearing from the disks w/o any
  notice we are not notified about DØFS disk
  failures and the failed disks with the data are
  merely removed from the system
• That resulted in 20 data loss as of mid-1999,
  which NO ONE WAS AWARE OF!

6
400 Physicists Failed to Login

• By actively destroying Run I environment, DØ
  management has de facto discouraged anyone to do
  Run I analyses, even though Run I physics groups
  are the only physics groups within the
  collaboration, and Run I physics is the only
  physics we will be publishing for the next 2-2.5
  years
• By jeopardizing Run I physics we make our
  experiment increasingly unattractive to new
  students and postdocs
  
• Moreover, we will soon be loosing existing
  students and postdocs by depriving them from the
  data access
  
• We are creating an artificial gap in our
  publications which is bad for everyone, including
  the people who are not currently involved in the
  analyses, or people who are 100 Run II
  taskforce
• If we find something interesting in the first
  couple hundreds pb-1 of Run II data, we wont
  have an ability to go back and look at Run I data
  as an independent cross-check

7
In the Meantime, Across the Ring

• CDF managed to preserve Run I data access
  capabilities virtually intact
  
• To the best of my knowledge all major pieces
  (including MC, data access, event display, etc.)
  work fine
  
• CDF has redirected a number of Run II graduate
  students to do Run I analyses for their theses
  
• CDF has recently shown a dozen of new analyses in
  progress at the major conferences some of them
  on the very hot topics, such as search for extra
  dimensions

8
Missing Data

• The full Run I mDST data set available on disk
  was far from being complete for the entire last
  year
  
• Between 10 and 20 of the data was missing at
  any given time
  
• Over the last half-a-year I recovered all missing
  data except for 20 partitions which are still
  being recovered (many thanks to David Fagan,
  Bruce Knuteson, Lee Lueking, and Rich Partridge)
• The last 10 of missing data is currently sitting
  on a temporary disk, but is ready to be moved to
  the DØDATADST area
  
• I was told that about 10 of the fixed mDST are
  still missing

9
Vital Tools

• For anyone who has done a Run I analysis it
  should be obvious that the following tools are
  essential
  
• Full minimal set of data on-line ? mDST
• An ability to reprocess the entire data set fast
  ? DØDAD
  
• An ability to pick events in maximal format ?
  PICK_EVENTS
  
• An event display program ? PIXIE
• Utility for integrated luminosity calculation ?
  PDB
  
• Minimal set of MC possibilities ? GEANT
• Not so vital, but essential tools
• Data integrity verification tool ? FATMEN
• Trigger simulation ? TRIGSIM
• Noise overlap ? NOISY
• Absence of the vital tools makes a usually
  friendly user very cranky!

10
How to Fix Things

• The ultimate very short term steps
• Fix the mDST access problems - days
• Revive the DØDAD infrastructure (users can open
  existing DØDAD files) days
  
• Revive the PICK_EVENTS utility on DØFS days
• Longer terms necessary steps
• Rebuild the DØDAD databases to reflect the
  current data sets 1 month
  
• Port PICK_EVENTS into Run II framework few
  months
  
• Port PDB or make it running on VMS indefinitely
  1 month
  
• Poll universities to maintain a small MC farm for
  Run I MC (service job?) few months

11
How to Fix Things (contd)

• Finish porting PIXIE to Unix few months
• Ensure completeness of the fixed mDST data set
  1 month
  
• The most important thing is to realize that the
  uninterrupted Run I data access is the only thing
  that distinguish us from a software company or an
  electronics sweatshop
• We ARE physicists, and physics must be our
  highest priority if we cant maintain physics
  today, we already lost the game for tomorrow,
  before even joining it
• The management has to find necessary resources to
  make sure that we have uninterrupted access to
  the Run I data for the next two years