ATLAS Lehman Review, Silicon ROD

1
ATLAS Lehman Review, Silicon ROD
May 21 to May 23, 2003
Douglas Ferguson, Richard Jared, John Joseph and
Lukas Tomasek Wisconsin
2
ATLAS Lehman Review, Silicon ROD
Material Covered ROD Schedule and High Level
Summary Scope Major Events ROD Overview
Current Status
3
ATLAS Lehman Review, Silicon ROD
ROD Schedule and High Level Summary
4
ATLAS Lehman Review, Silicon ROD
ROD Schedule and High Level Summary
5
ATLAS Lehman Review, Silicon ROD
High Level Summary
9 of SCT
production RODs complete
Authorized Funds
Paid to Date/Committed
W Totals() MS() Labor() Travel() Totals()
MS() Labor() Travel( 1.1.3RODs 1905.5 1486.0
407.5 12.0 167.7 51.5 112.2 4.0 Milestones

ETC02 Actual Projected 1.1.3.8.1.6
Pixel ROD PRR 6/03
9/03 1.1.3.8.1.8 SCT ROD
PRR 4/03
6/03 1.1.3.8.2 SCT Prod Complete
10/03
11/03 1.1.3.8.3 Pixel Prod Complete
3/04 3/04
1.1.3.8.4 Pur ROD SCT Crates
3/03 9/02 Two
crates have been ordered 1.1.3.9.1
Installation complete 2/05
2/05
6
ATLAS Lehman Review, Silicon ROD
ROD Comparison of Costs Small call on
contingency The ETC03 had a 48K difference
from ETC02 (pixel included). This is in the
noise. The pixel ROD have been added to the
baseline.
7
ATLAS Lehman Review, Silicon ROD
Scope Baseline Scope ATLAS ROD Pixel
test beam support hardware Design to
production models SCT and pixel RODs
Development of ROD test stand Fabrication of
SCT RODs Fabrication of pixel RODs
Purchase of SCT and pixel ROD crates
Management Contingency allocated
8
ATLAS Lehman Review, Silicon ROD
Major Events 1. Fabrication and debugging of 9
RevC production model RODs has been performed. 2.
Successful testing of data flow through the RevC
production model ROD at Cambridge. 3. The test
stand software has been upgraded to support pixel
user evaluation. 4. DSP software has been
coded to support user evaluation of SCT and pixel
user evaluation. 5. The ROD passed the US ATLAS
FDR. 6. SCT user evaluation has had much effort
in the UK. They have configured modules, readout
events and histogrammed the the data. 7. The
ATLAS PRR was held on May 13, 2003. The chairman
Philippe Farthouat has said that we passed
(report on May 23). 8. Pixel user evaluation has
started. Modules have been configured. Testing of
events readout is currently being done . 9. The
pixels will require a revised ROD (no changes to
VHDL of C code. FPGAs will be changed to provide
more derandomizing space and the DSPs will be
changed to provide more memory space.
9
ATLAS Lehman Review, Silicon ROD
ROD Overview
10
ATLAS Lehman Review, Silicon ROD
11
ROD Implementation FPGA Front End / DSP Back End
FPGA Formatter and EFB
Xon/Xoff
Router FPGA S-Link DSP DMA
Router Halt Output
Data RCVR
Slink on BOC
96
96
Data Valid
S-link Data
43
S-link Data
Serial Input Link
Control Status for Block Xfer/ DMA / S-Link
...
DMA Cont.
DMA 1
DMA 4
R/W Bus
R/W Bus
Input Memory XCVR Control
Token and Dynamic Mask
Busy
DSP Event Trapping, Histograms (Four DSP
Chips)
Host Port Interface
Back of Crate Card / Front End Electronics
Control, Status, Mask
Event Data/Tirg Type
Dec. Trig Count
R/W Bus
Boot FPGAs
Program Reset Manager
Data Bus
BOC Setup Bus
VME Bus
32
VME Slave Interface
ROD Controller Operation, Command, Trigger
Cont / Addr
Serial Output Link
ROD Busy
Timing Interface Module
Trig Data
12
ATLAS Lehman Review, Silicon ROD
13
ATLAS Lehman Review, Silicon ROD
Current Status (ROD hardware) The data and
control path in the ROD have been coded and
tested. The SCT VHDL is in maintenance mode
(changes implemented as needs are found). C code
is mostly written for module calibration. EFB
FPGA VHDL in
maintenance mode Router FPGA VHDL
in maintenance mode Formatter FPGA
SCT VHDL in maintenance mode The formatter
for the pixel 40MHz links has been written. The
formatter VHDL for the pixel 80Mhz links has to
be written. Controller FPGA
VHDL in maintenance mode Program Reset Manager
VHDL in maintenance mode
14
ATLAS Lehman Review, Silicon ROD
Current Status (ROD software ) The infrastructure
software is written and tested. Many primitives
have been written and tested. The code needed
for module calibration has been written. The
code is being upgraded as the user evaluation
needs are defined from use. The histogramming
code will need (for speed) to be executed in
machine language once the full functionality is
tested by the users. Queuing of events in the
FIFOs will also be needed to improve the
speed. Future needs histogram
fitting, occupancy measurement, error
detection, counting processing.
15
ATLAS Lehman Review, Silicon ROD
Current Status (Test Stand Software ) The test
stand software is complete and is in maintenance
mode. Updates are made as needed.
Concerns The upgrade to the pixel ROD has
some risk because the FPGA and DSP pins will
change and the printed circuit card will be
rerouted. Three boards will be initially
fabricated to reduce risk. The time to calibrate
the SCT modules is too long. The C code will
have to be changed to improve speed.
16
ATLAS Lehman Review, Silicon ROD
US ATLAS FDR (short version) Review Committee
Report for the SCT Pixel ROD US-ATLAS FDR of
20-Aug-2002 Introduction A Final Design Review
of the Read Out Driver (ROD) intended for use by
the ATLAS SCT and Pixel subsystems was held at
LBNL on 20-Aug-2002. The primary intent of this
review was to evaluate the present status of the
ROD design for use by the SCT and Pixels with a
goal of approving the build of 9 pre-production
RODs so that further user evaluation can be
obtained prior to a Production Readiness Review
(PRR) as early as possible. While the goal is
for the ROD design to be compatible for both SCT
and Pixels, it is recognized that there has been
as yet no user evaluation of the design with
Pixel readout. Given the schedule constraints of
the SCT and the overall cost constraints of the
project, the plan is to evaluate the
appropriateness of the present ROD design for
both SCT and Pixels and decide on the
pre-production build knowing that further design
modifications may be needed for Pixels after
appropriate user evaluation is completed with
that readout. The review committee included
Christopher Bebek LBNL Kevin Einsweiler
LBNL Alex Grillo UCSC Henrik von der Lippe
LBNL Abe Seiden - UCSC
17
ATLAS Lehman Review, Silicon ROD

• US ATLAS FDR (continued)
• Recommendations
• The committee was impressed with the amount of
  work accomplished since the last review. Basic
  functionality of the SCT ROD has been shown along
  with the first examples of on-board
  histogramming. While more user evaluation is
  still needed to be sure that the ROD meets all
  requirements for SCT and Pixels, this is at least
  a good start. The major obstacle to such
  evaluation is software for a DAQ system and
  focused attention of users running the ROD
  through all its necessary operations. User
  evaluation of the Pixel version is awaiting a
  Pixel module to readout.
• The committee makes the following recommendations
  to the ROD design group
• It is clear that the ROD development is ready to
  proceed to implement all identified design
  changes into a new PCB layout. Also, more ROD
  units will enhance the chances to obtain more,
  much needed, user evaluation. Therefore, the
  committee strongly recommends that the design
  group proceed as quickly as possible to complete
  layout of the next version of the ROD PCB and
  fabricate sufficient boards to eventually load 9
  boards. Given the changes to be made to the
  layout, we recommend that initially only two
  boards be loaded and tested. The committee at
  first thought that only one should initially be
  loaded for testing but we have accepted the
  advice of the ROD design team that the initial
  debug of two boards facilitates diagnosis of
  problems by having two samples to compare. Once
  basic functionality is demonstrated, the
  remaining 7 boards can be loaded.
• Note 3 then 7 boards have been
  fabricated. These boards have been used in user
  evaluation.
• The alternate footprint (BGA456 in addition to
  BGA676) for the Formatter on pixel boards should
  not be executed. We believe that this creates
  too big a risk in the layout for an option that
  may not be needed or used. If the planned
  Formatter FPGA proves to have insufficient
  performance for the Pixel readout, one of the
  higher performing FPGAs with the same footprint
  can be utilized (405/EM or 600/E). The added
  cost in that case is a worthwhile risk instead of
  the risk to the layout now.
• Note The change was not executed.

18
ATLAS Lehman Review, Silicon ROD

• US ATLAS FDR (continued)
• The allowable power supply margins of the FPGAs
  and DSPs should be analyzed vs. the regulation
  spec of the DC-DC converter over the temperature
  and supply range expected for the board. If it
  is not convincing that a single supply will work
  reliably for long-term use, the power plane
  should be split and two DC-DC converters should
  be employed. While splitting the power plane
  does imply some risk, care at this time to avoid
  errors may be a better risk than possible
  reliability problems over the lifetime of the
  ROD.
• Note An analysis was performed and no
  change was made.
• Attempt to optimize the FPGA VHDL code as planned
  to reduce the utilization. At the time of the
  PRR, the utilization must be reviewed again. If
  it is still uncomfortably high to accommodate
  possible needed enhancements during
  commissioning, a decision will have to be made
  then to upgrade to the high cost FPGA.
• Note FPGA utilization is now Formater
  70, EFB 56 Router 67, Controller 70 and PRM
  50.
• A discussion should be held with the person now
  re-designing the S-link board for ATLAS reviewing
  the RODs S-link interface vs. those of the
  S-link to confirm that there are no compatibility
  issues.
• Note The S-link was reviewed with
  CERN and has been tested as part of the user
  evaluation.
• There seem to be legitimate concerns about how
  the detailed phase of the data streams from the
  on-detector electronics will be monitored and
  timed relative to the TIM-generated crossing
  clock. Since the BOC is the module responsible
  for this synchronization, it is suggested that
  this is the correct place to implement monitoring
  of the timing of the raw data streams. The
  SCT/Pixel off-detector design group should
  consider how this may best be implemented in the
  final system. However, we suggest that the
  pre-production RODs should simply provide a
  reasonable number of header-based test point,
  close to the FPGAs, for logic-analyzer analysis,
  but should not attempt to provide multiplexed
  front-panel access to the full set of input data
  streams.
• Note The BOC supports passing the raw
  data to the ROD. A multiplexer has been added to
  the Formater FPGA to allow selection of one
  stream at a time for monitoring on a front panel
  connector. Module timing is performed by varying
  the BOC timing and observing the results with the
  ROD.

19
ATLAS Lehman Review, Silicon ROD

• US ATLAS FDR (continued)
• Since the current ROD design does not meet the
  requirement to readout events via VME at 1kHz
  with a full 16 RODs/crate and it would require a
  major redesign to try to accomplish this, but it
  appears that one ROD/crate would be sufficient
  for any planned ROD use, the committee recommends
  that this requirement be changed to VME readout
  at 1kHz for 1 ROD/crate.
• Note The requirement was changed to
  100 Hz. Current test indicate that the ROD can be
  read out at full luminosity at 300 Hz.
• The SCT community is strongly urged to devote
  more resources to write the necessary DAQ
  software so that more detailed user evaluation
  can be performed. To expedite this work, it is
  recommended that one of the existing RODs be sent
  immediately to Oxford to allow that group to
  begin learning to use the ROD.
• Note The SCT community now has a large
  effort devoted to user evaluation and preparation
  for the ROD use at the macro assembly sites.
•  
• The committee wishes to thank the ROD design
  group for their time preparing for this review
  and presenting a clear status of the project.
  Those members of committee who are members of the
  ATLAS collaboration would also like to thank
  Christopher Bebek and Henrik von der Lippe for
  their contribution to ATLAS by spending the time
  to review this complex development.
•