Readout Misbehaviors

1
Readout Misbehaviors

• Hot/Dead Regions
• most frequently occuring in crate x50
• generally occur in at least pairs of AFE boards
• when more than one pair at the same time, they
  are generally from the same sequencer
• data appears compromised during episode
• pulse height spectra are not reasonable when hot
• links look OK during episode
• address via run pause, minimal download
• FEBs
• crate 0x50 remains rather unpleasant 100 FEB
• most frequently occurring in crate x50
• address via run pause, minimal download and
  reinit VME
• Avoid SCL init as technique to recover from
  FEBs of crates x50 through x53
• some evidence that hot and/or dead regions can be
  caused by such an attempt at recovery

2
Attacking Readout Misbehaviors

• CFT_GUI modifications to localize fix
• Online Monitoring
• examines
• FEB_UTIL
• Upgrades to SDAQ code
• Data Integrity Checks
• online software
• offline analysis (dsdump and svxrawdump)
• Firmware Review
• CTS studies
• Sequencer Logic Analyzer
• Readout Clock Duty Cycle (and 53MHz readout)
• VTM Glink synch check
• VRB firmware updates to flag error conditions
• C0C0 normal end of HDI string
• CCCC Glink synch error
• CECE Timeout (gap gt 300 nsec in data stream)
• CFCF Input Buffer Overflow

3
CFTPS Weeks in Review

• Hot/Dead regions and FEBs significantly reduced
• Six AFEs swapped to address weak channels
• Virtual backplane swap
• calibration database issue identified
• returned grey cables to standard location
• 1B0/1B1 calibration problems addressed
• grey cable reseated at AFE backplane
• PDAQ examine enhancements
• most data integrity tests in place with results
  histogrammed
• dump event feature added
• AFE occupancy monitor development in progress
• VRB crate x52 and x53 firmware update installed
• CPS
• identified CPS examine map problem
• adjust bias voltages
• prep to reduce gains for CPS channels
• FPS
• pulser issue under study
• swapped 8 AFES to study dead regions

4
CFTPS Weeks in Review

• Monitoring missing events
• uMon replacing l3Xmon
• Spying on Vertex Examine
• L3 tracking
• results sensitive to both CFT and SMT performance
• trigger too
• East cold block temperature issues
• AFE slot 3B7 remains troublesome
• cause of most FEB encountered in CFTPS system
• Power supply for AFE crate 2B
• Discriminator threshold test
• run at nominal 2pe

5
Studies Period

• Expect beam studies through Wednesday afternoon
• update VRB firmware for crates x50 and x51
• update SCL receiver firmware in crate x53 VRBC?
• Calibrations (for additional discriminator
  studies)
• reduce CPS gains
• Anticipate a 3 hour access opportunity early
  tomorrow morning (if we are ready to test
  implementation of 1553 fix)
• new proms in sequencers for VRB crate x51
• document transformer status on SEQ an SEQC
• install updated firmware?
• Check on AFE crate 2B power supplies
• AFE board swapping
• 3B7
• mono and 254 challenged boards
• dead channels (SVX and discriminators)
• VSVX firmware update?
• FPS pulser testing