Task T CMS at LHC

1
Task T - CMS at LHC

• Personnel
• Professors D. Carlsmith, S. Dasu, M. Herndon, D.
  Reeder, W. Smith
• Senior Scientist R. Loveless (? New)
  (? Retiring)
• Senior Electronics Engineer T. Gorski
• Associate Scientist P. Chumney, Assistant
  Scientists M. Grothe, A. Lanaro
• Postdoc Y. Baek
• Engineers D. Bradley, A. Golyash, M. Jaworski,
  A. Mohapatra, S. Rader
• Graduate Students M. Anderson, K. Grogg, J.
  Klukas, K. Lazaridis, J. Leonard, M. Weinberg
• Technician R. Fobes
• PSL Engineers F. Feyzi, J. Hoffman, P. Robl, D.
  Wahl, D. Wenman
• PSL Draft/Tech B. Dana, G. Gregerson, D. Grim,
  J. Johnson, D. Northacker, J. Pippin, R. Smith
• Outline of Talks Activities
• Overview Trigger Activity -- this talk
• Computing Software Physics Activity --
  Sridhara Dasu
• Common Muon Tracking Activity -- Dick
  Loveless
• Endcap Muon Alignment Activity -- Duncan Carlsmith

2
Wisconsin Senior Personnel Official
Responsibilities

• D. Reeder
• CMS Muon Institutional Board Chair
• Founding US CMS Collaboration Board Chair
• W. Smith
• CMS Trigger Project Manager
• CMS Steering Committee Management Board
• CMS Electronics Board, Steering Committee
  Review Team
• US CMS Trigger Level 2 Manager
• US CMS Steering Committee, Project Management
  Group
• S. Dasu
• CMS PRS (Physics Reconstruction Software)
  Online Selection Group Co-Convenor
• US CMS Calorimeter Trigger Level 3 Manager
• US CMS Advisory Software Computing Board Chair
• R. Loveless
• US CMS Muon Level 2 Construction Project Manager
• US CMS Common Projects Level 2 Manager
• CMS Endcap Muon System Tech. Coord., CMS
  Technical Board
• US CMS Steering Committee, Project Management
  Group
• CMS Tracker Integration Team

3
Calorimeter Trigger Personnelexperienced team

• Physicists (DOE university program support)
• Faculty Wesley Smith (CMS Trigger Project
  Manager)Sridhara Dasu (US CMS Cal. Trigger L3
  Manager)
• Associate Scientist Pamela Chumney
• 6 yrs on CMS Cal Trig, former HERMES Trig. Coor.
• Assistant Scientist Monika Grothe
• 3 yrs on CMS Cal Trig, 2 years ATLAS higher level
  trig., 3.5 yrs Babar,SVT Commissioner/Run-Coordin
  ator
• Engineers (US CMS project support)
• Tom Gorski -- Lead Engineer Designer
• 3.5 yrs on CMS Cal Trig, worked at PSL on SDC
  Zeus Cal. Triggers
• Replaced J. Lackey, who retired but is available
  for consulting
• Matt Jaworski -- Design support, Testing
• Worked on Zeus Calorimeter Trigger
• Phil Robl -- Copper Link Test/Development
• Lead PSL Engineer, Worked on Zeus Trigger, SDC,
  CMS EMU
• Technician (project support) Robert Fobes --
  Repair, assembly, parts, orders, inventory
• Worked on Zeus Trigger
• Students (work as TAs for now)
• Mike Anderson, Kira Grogg, Kristos Lazaridis,
  Jessica Leonard, Marc Weinberg
• Trigger emulator simulations, Trigger Rate
  Calculations

4
Endcap Muon Personnel

• Physicists (DOE univ. program support)
• Dick Loveless (EMU level 2 manager)
• Armando Lanaro (L3 manager for installation)
• Yongwook Baek (cabling, low voltage,
  installation)
• A. Lanaro Y. Baek based at CERN responsible
  for day-to-day work on integration and
  installation
• Duncan Carlsmith (Alignment) - CDF Task
• Jim Bellinger, CDF Scientist
• Jeff Klukas, Grad Student
• Engineers(US CMS project support)
• Farshid Feyzi, Dan Wenman (UW Physical Sciences
  Lab)
• Techs(project support -- paid as needed)
• Bart Dana, Mauro Giardoni, Jay Johnson, Dave
  Northacker, Ron Smith (UW -PSL)
• Tracker Common Projects (Endc. Disks)
• Dick Loveless
• PSL Personnel paid by US CMS Project

L to R F. Feyzi, D. Wenman, A. Lanaro J.
Johnson, M. Giardoni, Regis, Y. Baek
5
Computing Software Personnel

• Physicists (DOE university program)
• Sridhara Dasu
• Wisconsin Tier-2 Manager - DISUN
• Technical Coordinator, Grid Laboratory of
  Wisconsin (GLOW)
• Chair, USCMS Software Advisory Computing Board
• System Manager (DOE university program)
• Steve Rader
• Software Engineers (NSF-ITR US CMS Computing
  Project)
• Dan Bradley
• Ajit Mohapatra Will Maier
• Collaboration with U. Wisconsin Computer Sciences
  Dept. Condor Group
• Prof. Miron Livny
• Director of Condor Project
• Leadership role in Grid Projects PPDG, GriPhyN,
  iVDGL
• PI, GLOW

6
CMS Trigger DAQ Systems

• Level-1 Trigger Requirements
• Input 109 events/sec at 40 MHz at full L 1034
• Output 100 kHz (50 kHz for initial running)
• Latency 3 ?sec for collection, decision,
  propagation
• Output direct to computer filter farm (no
  physical level-2)

7
CMS Level-1 TriggerW. Smith, Project Manager

• Wisconsinfullyresponsible
• US CMS Trigger
• US CMSfully responsible
• US CMS partially responsible
• UNIQUEAll system present trigger objects
  w/location, ET, quality -- topological triggers
  possible

8
Wisconsin CMS Task Support(over past decade)

• UW 95-96 Support of Personnel
• J. Lackey, S. Dasu, PSL Engineering
• UW Equipment Support
• Teleconference Equipment
• Video Room Remodeling
• CAD Printer
• UW CAD support
• Purchase of Visula PCB Suite, incl. high speed
  router
• New version of software used for Zeus trigger
• UW Computing Support
• gt 700K
• 5 New Servers, 2 TB Disk, 1 GB/s network, Grad.
  Student RA
• 70 CPU system for simulation with Condor
  network upgrade
• Two new Computer Rooms with power AC
• Matching for GLOW shared computing facility

9
Calorimeter Trigger Crate

• Data from calorimeter FE on Cu links_at_ 1.2 Gbaud
• Into 126 rearReceiverCards

• 160 MHz point to point backplane
• 18 ClockControl ,126 Electron ID Receiver
  Cards -- all validated18 Jet/Summary Cards --
  all cards operate _at_ 160 MHz
• Use 5 Custom Gate-Array 160 MHz GaAs Vitesse
  Digital ASICs
• Phase, Adder, Boundary Scan, Electron Isolation,
  Sort

10
Regional Cal. Trigger Complete- U. Wisconsin
Rest waiting here
Five full crates at CERN

• Receiver Card Electron Isolation
  Clock Jet/Summary

fraction tested (needed)
Front
EISO
Adder
154/154 Tested (126)
mezz link cards
Front
DC-DC
158/164 tested (126)
23/25 tested (18)
Oscillator
Back
Phase ASIC
Bar Code
Clock
23/25 tested (18)
Back
PHASE ASICs
Clock Input
(27/28 Custom Backpl Tested) (18)
Clock delay adjust
BSCAN ASICs
BSCAN ASICs
Sort ASICs
MLUs
DC-DC Converters
11
5 RCT Full Crates Operating at CERN- U. Wisconsin
Front of Full RCT crate with Jet Capture Card
that continuously samples output, checks on the
fly for errors and provides readout of 256
crossings

• Rear of Full RCT crate fully cabled to HCAL
  trigger primitive logic

Used for integration tests board checkout after
shipment
12
New Jet Capture Card-- Tom Gorski
Signals Out
FPGAs
Translators
JSC Inputs
CCC Signals In
Delays

• Receives 14 Reg. Sums, 8 HF ETs, 8 e/g, and t
  and ? bits on 6 SCSI type cables
• Captures output from Jet Summary Card going to
  Global Calorimeter Trigger
• Being used in final validation of RCT crates, all
  cards will go through an extended full crate test
  with the JCC
• Software developed by Pam Chumney Tom Gorski
• Provides calorimeter trigger for CMS Magnet Test
  Cosmic Challenge
• Provides temporary trigger to Global Trigger
  while Global Calorimeter Trigger is being prepared

13
Trigger Commissioning

• Operate fully functional trigger electronics at
  CERN
• Employed in myriad tests preparation activities
• Tests in Electronics Integration Center
• Labs row of racks for all electronicssubsystems
  
• Test interfaces integration as much as
  possible before move to USC55
• MTCC in SX5
• Test multiple triggercomponents with
  multipledetector components duringthe magnet
  test.
• Verify trigger functions interfaces w/detectors
  onsurface
• Installation in Underground CountingRoom (USC55)
• Started Mid-July 06 -- ready for crates
• Racks Infrastructure installed, cooling
  operational
• RCT crates, power, infrastructure fully installed
  (next slide)
• Passed CERN Safety Review
• Card installation next

USC55
UndergroundCounting Room
14
Regional Calorimeter Trigger

• Crates, Fans, Power Supplies, Monitoring,
  Controls and other infrastructure installed,
  validated and passed safety check in July
• Ready for card installation

15
Electronics work in 2006-7

• Finish Test Master Clock Crate
• Develop detailed timing test system
• Use 4-crate test-bed in Madison to produce exact
  procedures
• Validate Clock Fanout Cards
• Multiple crate tests
• Check for datasynchronization
• Repair Analysis ofBoard Failures
• Spare Parts Inventories
• Install,
• Commission
• Operate
• Final System
• in USC55

Techn. R. Fobes (l.) Engr. T. Gorski (r.)
16
Cal Trigger Tasks at CERN - P. Klabbers, M.
Grothe students

• Retest RCT Crates arriving at CERN
• Comprehensive long term tests with Jet Capture
  Card performed in RCT lab in electronics
  integration center
• Install RCT Crates in Underground Counting Room
• Connect test 1000 E,HCAL cables
• Connect test to Global Cal. Trigger
• Install Test Master Clock system
• Synchronize all within 4 ns to centraltiming
  system
• Diagnostic Online Software
• Downloading detailed test patterns
• Trigger Emulator
• Bit-level match to actual electronics
• Integration with Trigger Supervisor
• Online Monitoring Control
• Configuration of Memory Lookups
• Reprogramming
• Function Validation

Assoc. Scientist P. Klabbers (l.) Asst.
Scientist M. Grothe (r.)
(both resident at CERN)
17
Trigger Lab Setup at CERN

• Staging area - Prevessin 904
• Receive boards, crates, cables from Wisconsin
• Inspect and log in using barcodes and database
• Assemble crates for Integration/USC55
• Power up and run system tests
• Operate in water-cooled rack for extended tests
• Integration tests
• Racks with cooling on a raised floor nearby
• Storage and Repair
• Spare Crates and cards also available for use in
  testing and replacement of suspect cards
• Soldering station, scope, and tool storage
• Spare component storage
• Responsible P. Chumney M. Grothe
• When collisions start need added personnel so
  group members can actively pursue physics
  analysis (task B redirection).

18
Trigger Integration Progressin Electronics
Integration Center (Pr. 904)

• Large scale successful integration tests in
  central racks using common TTC infrastructure
  involving trigger primitives, regional global
  processing
• Calorimeter Trigger
• Muon Trigger
• Global Trigger

19
ECAL/HCAL/RCT Integration Tests Successful!

• HCAL HTR to RCT ?
• Used JCC to verify data
• 6 cables used - all in sync
• Latency measurements w/PMTas source of HCAL TPG
• ECAL TCC to RCT ?
• Valid link established
• Channel number sent as data
• 2 channels on 20 m cablessent in sync to RCT
• HCAL HTR ECAL TCC to RCT ?
• 28 cables to RCT from 9 TCC 18 HCAL HTR
  outputs
• Common clocking LTC, TTCci
• 28 channels in sync - links stable for gt10 hours
  error free running
• Hydra link cable checked as an option for two
  SLBs to one RCT RMC
• ECAL - HCAL Pattern tests
• RCT can lookup incoming data
• LUTs can be programmed to duplicate incoming data
• Have ORCA emulation to help predict outputs

20
RCT Integration Tests Successful!

• RCT to Local Timing Controller (for MTCC) Tests ?
• Receive 56 HCAL links (448 towers) in Green
  Barracks
• Send trigger out with RCT Jet Capture Card
• OR of towers in each half using Quality Bit
• Validated connection with LTC/GT
• GT ANDs top bottom - reduces rate
• RCT Jet Capture Card to GT Tests ?
• Backup Global Calorimeter Trigger
• JCC in spare VME slot of RCT Crate
• Currently sends up to 4 trigger bits
• Threshold ?ET, e/? candidate, MIP
• Tests using RCT pattern successful!
• ECAL TCC patterns yield JCC output
• Synchronized to Bunch crossing 0
• RCT- GCT Source Card
• First test with new GCT
• Completed successfully in July

21
Calorimeter Trigger inCosmic Challenge

• RCT
• Crate fully operational in Green Barracks at
  Point 5
• Cable connections made to HCAL
• Full modules above and below connected
• Allows for top-bottom coincidence

RCT output trigger
HCAL crate to RCT
RCT input from HCAL
22
Physicist Trigger MO Tasks(all from base
program)

• Change trigger Configuration
• Respond to changing beam/detector conditions
  physics priorities
• Study new trigger configurations
• Test runs, Monte Carlo studies, data studies
• Trigger Physics Analysis
• Understand detailed impact of trigger on physics
• Preparation for luminosity increases
• Monte Carlo studies of new conditions, validate
  with present data
• Respond to changing apparatus
• Changes in material, configuration, etc. ?
  changes in simulation
• Operations - 24x7 support during running
• Write, test maintain electronics test programs
• Maintain update bad channel list run daily
  checking programs
• Run Control maintenance
• Trigger data validation and calibration
• Online Offline analysis of rates efficiencies
• Monte Carlo data trigger simulation maintenance
• Continuous validation of trigger using simulation
  readout data
• Will need to augment UW personnel to perform
  tasks do physics

23
The Future - SLHC

• Luminosity upgrade x10 SLHC L 1035cm-2s-1
• Extends LHC mass reach by 20-30
• Modest changes to machine
• Detector upgrades needed -- especially the
  trigger tracker
• Time scale 2015
• Trigger higher luminosity tracking data
• Double operational frequency from 40 MHz to 80
  MHz
• Design for much higher rejection power for pileup
  to retain output rate of 100 kHz -- more
  sophistication
• Correlation of muon cal triggers with new
  tracking trigger
• Status of planning starting in FY04
• Workshops US CMS in Madison Feb 04, CMS at CERN
  Feb 04 Imperial College Jul 04, CERN, Jul 05
• CMS SLHC Level-1 Trigger and Tracker project
  started. W. Smith G. Hall (Imperial) chosen to
  lead effort.
• Workshop at Perugia April 06 proposes plan
• CMS management accepts plan and charges W. Smith,
  G. Hall, J. Nash with writing SLHC EOI for
  submission to CMS management in Sept. and to LHCC
  by end of the year.
• As project ramps up, needs additional personnel
• Use redirection of Task B effort

24
UW CMS Trigger Concerns

• Additional scientific staff to permit physics
  analysis
• Present staff will be saturated w/trigger
  maintenance operations
• Will need additional staff to permit physics
  analysis, use of Wisconsin computing resources.
• Propose to redirect Task B (ZEUS) staff to meet
  shortfall
• Project support of Lead Designer/ Expert
  Engineer T. Gorski
• Vital for commissioning and operations
• US CMS project support for MO supervised by base
  program
• Estimated technical support project needs for
  MO
• Based on Zeus Cal. Trigger MO 1992-2001
• Technician
• Operates, repairs, maintains test facility
• Repairs boards infrastructure under physicist
  guidance
• Total required 0.5 FTE
• Expert Engineer
• 5 trips/year for 2-3 weeks to make difficult
  repairs
• Complicated/Subtle problems, Modifications to
  trigger electronics
• Designer - available for consultation
• 2 trips/year for 2-3 weeks for review design
  issues

25
Trigger Summary

• Calorimeter Trigger
• Production Testing finished
• Setup of full testing facility/lab at CERN EIC
  done
• Successful Integration tests w/ECAL,HCAL, Global
  Cal Trigger
• Final Installation started
• Very good progress, but
• Major work from now on
• Production Shipping, Retesting, Installation of gt
  1500 Boards
• Software Development online DQM, controls,
  offline DQM, integration with Trigger Supervisor,
  Emulator Simulation
• More Integration Tests
• Installation and Commissioning
• Operations!
• Physics!
• SLHC RD

26
Task T Endcap Muon Activities(Talk by R.
Loveless)

• Project Management Dick Loveless
• Construction, test, integrate, install
  commission 468 CSCs, electronics infrastructure
  ( lower all this in 06/07!)
• Chamber Install., Cabling Test
• UW responsibility -- led by UW scientist A.
  Lanaro, assisted by UW postdoc Y. Baek
• Low Voltage System
• Wisconsin responsibility for all chambers
  crates -- R. Loveless UW Eng. A. Golyash
• Tested and ordered Wiener Maraton system
• Endcap Alignment
• Wisconsin responsibility -- Design procurement
  complete
• System installation -- A. Lanaro Y, Baek
• Alignment Reconstruction Corrections -- D.
  Carlsmith, assisted by J. Bellinger
• leverage CDF expertise

27
Task T Tracking Activities(Talk by R. Loveless)

• Started July 2005 with responsibility for design
  of infrastructure for new Tracking Integration
  Facility (integrate tracker operate 25 at any
  time)
• August 2006 TIF Infrastructure is delivered
  installation almost complete ?
• Additional Engineering
• Mechanics and installation of TrackerInterface
  Box (PP1) -- connection pointfor all pipes,
  cables, optical fibers insidevacuum cryostat
• Personnel
• Loveless, PSL engineers Dan Wenman Farshid
  Feyzi
• Tracking Reconstruction PhysicsAnalysis
• Implement silicon-only trackingalgorithms from
  CDF into CMS.
• Use calorimeter muon systems toidentify tracks
• Personnel
• Prof. Herndon, postdoc, 1-2 students

28
Task T Physics Activities(Talk by S. Dasu)

• Preparation for CMS Physics TDR (2006) early
  data analysis
• New energy regime ? rapid discoveries
• Must begin operations (late 2007) with well
  understood trigger, reconstruction and analysis
  software
• Wide-ranging physics interest and expertise in
  the group
• Analysis efforts coming to forefront as detector
  work finishes
• We studied simulated Higgs, SUSY, SM EW and QCD
  datasets
• Prof. Dasu Dr. Klabbers verified that trigger
  systems capture all the important physics
  efficiently while satisfying DAQ bandwidth
  requirements
• Ensure trigger systems operate at full
  performance at turn-on and beyond
• Prof. Dasu led PTDR.v2 online selection work (L1T
  and HLT)
• UW grad student Leonard worked on L1 rates and
  efficiencies
• Dr. Grothe wrote the diffractive physics sections
• Full simulation, trigger design and analysis in
  new CMS SW
• Substantial summer student work supervised by
  Dasu, Herndon Smith
• Anderson (H???), Grogg (H?W(?e?)W(?jj))
• Lazaridis (H?Z(?ee)Z(???)), Weinberg
  (pp?SUSY?Like sign leptons)
• Herndon (Z ???) for tracking reconstruction
  studies
• Creation of physics samples is computationally
  intensive ? Tier-2 effort

29
UW Computing Activities(Talk by S. Dasu)

• Rader DOE Univ. Bradley, Maeir, and Mohapatra
  NSF Support
• A CMS Tier-2 computer center from FY2005
• Data-Intensive Science University Network (DISUN)
  institute
• Leadership in Grid Community
• OSG - Open Science Grid (develop tools and
  provide cycles)
• GLOW - Grid Laboratory of Wisconsin (Lead role in
  development and implementation with CS Condor
  group to benefit entire UW campus IceCube, CS,
  Genomics, Chem. Eng., Med. Phys., etc.)
• Benefits both CMS and ATLAS Computing support
  for all HEP tasks
• Simulation production using local UW resources
• The single largest institute producing CMS events
  2002-2006
• Over 60M events produced, simulated,
  reconstructed analyzed with ORCA in 05-06
• Over 17 M events produced with new CMSSW since
  commissioning 2 weeks ago

Rader UW Tier-2 Systems Lead 547 kSI2000
CPU 428 batch slots 90 TB raw storage
10 Gbps bandwidth 24h/7d operation200 user
accouts
Mohapatra Production czarBradley Software
expert
30
UW Software Activities(Talk by S. Dasu)

• Trigger Controls Software
• Prof. Dasu Core VME software and framework
• Dr. Klabbers Testing and MTCC operations
  software
• Dr. Grothe Integration with CMS Trigger
  Supervisor
• Trigger Emulator Software
• UW Student Leonard RCT emulator in CMSSW
• Trigger Monitoring Software
• New effort needed now to develop under
  supervision of Klabbers
• Trigger Physics Studies
• New effort needed now to ensure good trigger for
  first physics
• Computing Resources Management Software
• SW Engineer Bradley New scheduler,
  Computing-On-Demand
• Contributions to Condor, OSG and ROOT projects

New Effort Postdoc redirection from Task B and
student support (Leonard full year, others from
summer 07)
31
Wisconsin Concerns

• DOE Core Program Manpower at Wisconsin
• Major effort on trigger software required
• Tasks board testing, monitoring/controls,
  diagnostics, configuration downloads
  documentation, modeling, physics simulation, etc.
• In US CMS Plan as core program manpower
• Major effort on testing, installation,
  integration tests commissioning
• Planned as activity of base program manpower
• For Trigger, Computing,, Muon Projects (and now
  Tracker)
• Project-supported personnel need core program
  supervision
• Wisconsin responsibilities are growing with time!
• Long term trigger MO Physics Analysis SLHC
• Increase in personnel needed
• Particularly so physicists can participate in the
  physics harvest
• Needs continued support eventual growth
• Provide growth from Task B redirection