SNS Project Status

1
SNS Project Status

• Ernest L. Williams Jr.
• 27 April 2005
• EPICS Collaboration Meeting _at_ SLAC

2
Outline

• Installation Status
• Global Systems
• Network
• Timing
• MPS
• CCR
• Accelerator Systems
• Cryogenic Controls
• PPS
• Target Controls
• Operational Improvements
• Issues
• Summary

3
The Spallation Neutron Source

• The SNS construction project will conclude in
  2006
• At 1.4 MW it will be 8x ISIS, the worlds
  leading pulsed spallation source
• An upgrade to 3MW has been approved for CD0
  planning.

4
Communication Infrastructure is nearly complete

• CD4 Fiber Backbone is complete
• Tubes installed for future addition of blown
  fiber
• Global cables nearing completion
• Ethernet
• Timing
• MPS
• All accelerator cable SROs entered
• CISCO Switch firmware bug found (and fixed)

Fiber Backbone
Cable Infrastructure
5
Timing System Installation is Keeping Up

• All IOCs (including NADS)
• Multiple Fixed Rep Rates for RF Systems
• CLO Test Timing Network
• Infrastructure for Diagnostics, LLRF, XFD, and
  Power supply groups
• Line Synch monitor
• Delivered from BNL
• Setting up test network for XFD Chopper testing
• General Time added to EPICS
• Provides best time from multiple sources
• Assures monotonicity
• Accepted into EPICS Core

• SNS Event Link Monitor
• Event statistics
• Actual Events vs. Scheduled Events

6
Number of MPS Inputs has nearly Doubled(170 ?
313)
170

• All linac MPS hardware installed includes 27
  new chassis
• Copper phased out, using fiber everywhere
• After CD4, corrector PS status will be added

7
Operations have moved to the CCR(Central Control
Room in CLO)

• Move took place in February as scheduled
• Two console arcs are complete (ahead of schedule)
• PPS and ODH systems moved (That was the worrisome
  part)
• Software Development Area in use
• All work planned for FY05 is complete
• Servers are moved to CER (Central
  Equipment Room)
• TPS also now in the CCR
• Prototype Console moved to Target Control Room
• Upgraded all multi-head OPIs
• Faster processor
• More RAM
• Faster graphics adapter (Matrox G450)
• X-windows and Java Apps have a much better
  performance

PPS Equipment
8
Linac Controls Installation isKeeping Up with
the Project Schedule

• We have kept up (barely!!) with RF testing, RF
  Conditioning, PS testing, cryomodule cool-down,
  etc.
• Medium-Beta SCL controls mostly complete
• Cryomodule MB1 controls testing remains
• Some warm section vacuum controls testing
  remains
• Some MPS hardware installation remains
• High-Beta SCL controls approaching completion
• RF, Cryo, Vacuum controls checkout are well
  along
• Magnet power supply interface checkout has been
  integrated with overall power supply checkout
• HEBT controls installation in progress
• Automated procedures have helped keep schedule

9
Linac LLRF Installation is Keeping Up(This is
the biggest task in terms of numbers and
complexity)

• Number of installed LLRF IOCs 46 (up from 12)
  for 80 systems (out of a total of 96
  systems)
• Configuration automated via scripts, avoiding
  inconsistencies between SCL systems
• From last ASAC "Add Adaptive feed forward to
  firmware if required"
• Was required for higher beam current
• Not foreseen in current FCM, so successfully
  added to IOC software
• recently refined to eliminate parameters that
  were hard to tune
• Automation of RF ramp-up, warmup, loop closure
• Allowed hands-off recovery from trips on the NC
  Linac
• Needs adaptation for SCL
• Waveform updates synchronized to timing system
• Allows physics apps to compare data from several
  systems for the same beam pulse
• Refining overview screens, giving better notice
  of trip reasons, active overrides, etc.

10
Cryogenic Control System nearly complete!!
tested

• All CHL Controls equipment installed and
• All IOC and PLC programming complete
• CHL Gas Management, Purifier and Warm Compressor
• CHL 2.1K and 4.5K Cold Boxes
• Medium and High Beta Cryomodules
• HEBT Service Building IOC and PLC
• 2K Cold Box Turn-on was automated
• Cryomodules will complete in April

Full RF Power
First Liquid
Cool Down
Warm Up
11
The Cryogenic Control System layout
2 Servers
4 OPIs
11 IOCs
17 PLCs
12
Ring Controls is ahead of scheduleBNL effort
completed in March

• Handover Plan - Complete and in use (cooling
  water and ramping software scope moved to ORNL)
• FSDs - Complete, in use and being updated as
  needed to keep them as-built
• Design - Essentially complete. All drawings and
  specs needed are in use for fabrication and
  installation and being updated as needed to keep
  them as-built
• Hardware Fabrication - WBS 1.9 work complete
  except for installing 1 PLC for cooling water
  skid. WBS 1.5 work complete except for 2 motion
  control racks, 2 PLCs for RF, and Vacuum racks
• Procurement - All hardware procured except 1 PLC
  for water skid and some miscellaneous parts
• Cabling - Schedule for Ring and RTBT cables
  depends on budget allocations
• HEBT Communications cables pulled, being
  terminated
• HEBT Process control cabling in process
• Software - Vac, BLM, Mag PSC/PSI, HPRF, Beam Dump
  TCs ready for handover. All software ready for
  field testing by Jun 05
• Test Plans - Drafted for all systems except RF -
  Will be updated as needed during installation

13
PPS System Installation is keeping up(in spite
of many configuration changes)

• 75 of the PPS controlled devices in the Linac
  have been connected
• The temporary Linac shielding has been removed
  and the Linac has been returned to one segment
  for phase 1.3
• PPS remote I/O racks for the HEBT are installed
• Conduit cable for PPS devices installed in HEBT
  tunnel
• Installation of conduit cable for PPS devices
  in Ring tunnel is 100 complete
• The PPS control room racks have been relocated to
  the CLO CCR as a part of phase 1.3 (SCL
  commissioning)

14
The Target Instrument PPS are alsothe
responsibility of the Controls Team

• Cable conduit design for target building
  complete, installation underway using AIMSI
  crafts (beamline 2 and transfer service bay) and
  fixed price subcontractor (high bay)
• Target PPS equipment design is 85 complete
• Target PPS equipment fabrication is 70 complete
• Equipment installation ramping up as electricians
  become available
• Instrument PPS design for beamline 2 underway.
  Prototype user panel being fabricated

15
The number of IOCs has risen dramatically

• 223 IOCs currently on Accelerator Network
• 109 VME / VXI --- vxWorks-based
• 2 Linux-based Soft IOC Servers
• Running 24 Soft IOCs (18 6)
• 90 Intel-Pentium --- Win32-based IOCs (the
  NADS)
• Wirescanners, Faraday Cups, BPMs, BCMs
• Upgraded control system to EPICS R3.14.7
• Improved reliability in channel access.
• Results in fewer white screens in the control
  room.
• Upgraded OS for VME/VXI-based IOCs
• vxWorks 5.5.1 (SNS CVS Release SNS07)
• Better network driver and network stack
  configuration
• Auto-retry on boot-up if boot server is
  unavailable.
• Better memory management

16
Power Supply Group is Routinely UsingAutomated
Testing and Calibration

• Method uses EPICS and so assures end-to-end
  testing including control system
• Resulted in large schedule reduction one test
  instead of two
• PS Calibration, current/field mapping and cycling
  profiles come from Oracle RDB
• 9 SCL IOCs installed, 8 operational
• 1 HEBT IOC installed and operational

17
Turn-on and Turn-off of HPRF Systemshas been
automated
State Diagram monitors the process for each HVCM
Overall summary is displayed
18
The New Archive Engine is stable and
amany-featured Java Archive Viewer is now in use

• Archives now well organized with central
  configuration file and scripts for
• generation of directory structure and skeleton
  files
• start/stop/restart/monitoring of sampling engines
• update of status web page
• email notification of engine status and disk
  usage
• Currently 11 distinct archives each running one
  or more sampling engines (total 25).
• (Conventional Facilities, Magnets, RF, Cryo, ...)
  
• Data rate kept to 2GB/day with occasional peaks
  at 4GB/day.
• Limiting performance factor is currently neither
  CPU nor network but disk I/O.
• New Java Viewer is stable and many features are
  being exploited
• Uses XAL Framework
• Web interface available
• Plots actual data in various formats
• Offers absolute and relative request times
• Performs computations, exports data to
  spreadsheets
• Investigating new uses via command line and web
• Scroll, zoom, legends, titles, multiple plot
  styles, arithmetic and boolean operations

19
The new Archive Viewer (Chevtsov)has many useful
features

• Relative start/end times "-5 d(ays)" ? "now"
• Multiple value axes
• Tool-tip info for individual data points
• XAL Framework

Support for calculations based on archived
data Export to spreadsheets, progress bar Green
squares in plot launch tool-tips for meta data,
in this case daily restart of sampling engine.
20
Database Collaboration with APS has grown(SNS,
APS, SLAC, TRIUMF, BESSY, FNAL, Diamond, PSI)

• IRMIS Integrated Relational Model for Installed
  Systems (v1.0 release in April)
• Agreed Core Schema describes control system
  with three hierarchies
• Signal
• Housing
• Power
• Uses the SNS Physics XAL Framework
• Pelaia et al
• RDB synchronized using crawlers
• 106 SNS IOCs successfully crawled
• EPICS versions 3.13.9, 3.14.6, 3.14.7
• 228,270 PVs found
• SCL 112141
• DTL      33607  
• CCL      27443  
• MEBT   10918  
• HEBT    8074    
• Crawlers in development for all EPICS clients,
  IOCs, edm, Archiver, Alarms, etc.

21
Issues

• Increased installation pace.
• RF Processing and installation checkout means
  Control Systems Group must provide operational
  support.
• Hardware/Software Configuration Management is
  highly correlated with reliability/availability.
  Work is in progress using a RDB, CVS, and good
  release control scheme.

22
Issues How to manage increasing number of IOCS?

• 223 IOCs currently on Accelerator Network
• 109 VME / VXI --- vxWorks-based
• 2 Linux-based Soft IOC Servers
• Running 24 Soft IOCs (18 6)
• 90 Intel-Pentium --- Win32-based IOCs (the
  NADS)
• Wirescanners, Faraday Cups, BPMs, BCMs
• Upgraded control system to EPICS R3.14.7
• Improved reliability in channel access.
• Results in fewer white screens in the control
  room.
• Upgraded OS for VME/VXI-based IOCs
• vxWorks 5.5.1 (SNS CVS Release SNS07)
• Better network driver and network stack
  configuration
• Auto-retry on boot-up if boot server is
  unavailable.
• Better memory management

23
Issues How to manage increasing number of IOCS?
24
Issues How to manage increasing number of IOCS?
25
Summary

• Installation is keeping pace
• Required use of more automated configuration and
  testing, database use
• PPS keeps pace notwithstanding changes
• Cryo controls nearing completion
• Will be ready for SCL commissioning
• Ready to pick up on Ring controls
• Operational Improvements as well
• Automated HPRF turn-on
• Automated 2K Cold Box turn-on
• Improved Archive viewing tools