Planning for the COV for the Facilities

1



2
Background

• Second COV Review of Facilities Division
• Present panel assesses operations of Divisions
  programs during FY 2004, 2005, 2006. Examines any
  files from this period.
• Division components for review
• - Synchrotron-Light and Electron-Beam Light
  Sources
• - Neutron Sources
• - Electron-Microscopy Sources
• - Nanoscale Science Research Centers
• - Accelerator and Detector Research
• At BES/DOE Germantown
• Reporting
• Presentation to BESAC at Summer meeting.
• Following report acceptance by full BESAC
  committee, COV report to be presented to Director
  of Office of Science.
•  

3
Recommendations - First COV

• Jacket format
• Timelines of the Review history for each Facility
  or Center. Each Timeline should take the form 
• ReviewgtRecommendationsgtResults (including
  written response to COV/BESAC)gtRe-Review and
  its Results, when necessary  
• History should be In front of jacket of most
  recent review for each facility and a brief
  overall review history for the facility.
  Cross-references to the full jacket for previous
  reviews are also useful.
• Several elements should be contained in the
  report and file of every BES facility review
  these were noted in the report.
• Review process
• Evaluation of the success of facilities should be
  done on the basis of quantifiable metrics these
  may vary with type of facility.

4
Recommendations -First COV (Contd)

• A revision of the review process, e.g. more
  executive sessions, more time to hear about
  related lab issues.
• Strongly recommended broad users input at all
  stages of construction of five centers, since
  they are designated as national user facilities.
• The committee felt strongly that it was crucial
  to have a clear and current definition of who
  exactly is a user.
• Consensus report may be best.
• Facilities Division operation
• Careful attention to coordination between the two
  major science program divisions and the
  Scientific User Facilities Division strongly
  recommend that science program managers
  participate in facility reviews.
• Careful thought to integration of nanocenter
  science across centers and with core programs.

5
DOE/BES Committee of Visitors (COV) Review Panel
X-rays/ Machines (4-5)  Science Tai-Chang
Chiang, UICC Gabrielle Long, Argonne Brent
Fultz, Caltech Z-X Shen, Stanford/LBL Machine
Persis Drell, SLAC Don Bilderback, Cornell
  Janos Kirz, LBL Sam Krinsky, BNL
  Microscopy (3)  John Silcox, Cornell Cev
Noyan, Columbia Miquel Salmeron,
LBL   Present BESAC Member, Former BESAC
Member
Overall Chair Richard Osgood, Columbia  BESAC
Chair John Hemminger attends 1st day of
review.   Neutrons (4)   Pat Gallagher, NIST
John Tranquada, BNL Sunny Sinha, San Diego James
Rhyne, LANL Nanoscience (4) Dave Litster, MIT
Miquel Salmeron, LBL Reginald Penner, UC -
Irvine Franz Himpsel, Wisconsin  
6
Charge to the Committee

•  Panel will consider and provide evaluation of
  following four major elements 
• 1. Assess efficacy and quality of processes used
  to
• (a) solicit, review, recommend, and document
  proposal actions
• (b) monitor active projects, programs and
  facilities. 
• 2. How has the award process affected
• (a) breadth and depth of portfolio elements
• (b) national and international standing of
  portfolio elements.
• Also
• Provide input for OMB evaluation of Basic Energy
  Sciences progress toward long-term goals. Each
  of the components of the Scientific User
  Facilities Division should be evaluated against
  each of the four-part long-term goals. If not
  applicable, please indicate so.
• Note OMB guideline ratings of (1) excellent, (2)
  good, (3) fair, (4) poor, (5) not applicable.
  Also, comment on observed strengths or
  deficiencies in any component or sub-component of
  the Divisions portfolio, and suggestions for
  improvement.

7
Report

• Introduction
• Charge 
• Committee Composition
• Response to Prior Review
• Review
• COV Review Process
• Facility Review Process
• Reports on Specific Classes of Facilities
  (Neutrons, X-rays, Electron Microscopy,
  Nanocenters, and Detectors and Accelerators)
• Response to Prior Review
• Documentation of Facility Review
• Users of Facilities- Definition of and Uniqueness
  for Each Facility Type
• Metrics
• Comments Facility Review Process
• Comments on Emerging Facilities
• Metrics
• General Comments
• Managing the SUFD Vision
• PART Evaluation
• Conclusions

8

• Summary
• COV concludes that the newly constituted
  Scientific User Facilities Division is well
  launched and is operating extremely well.
• Facility reviews are fair and even-handed and
  had significant and clear beneficial impact on
  several facilities, even though many of the
  facilities are just now reaching the point of
  operational review.
• COV finds that review process has served
  existing facilities well. In some cases reviews
  have promoted changes in management and
  operations and improved the scientific impact of
  these facilities. The reviews have added clarity
  and focus to wide spectrum of concerns from user
  community, facility personnel, and the BES.
• COV made specific recommendations for
  improvements and changes in review process, both
  in general and in terms of specific facility
  types.
•  
• The Committee was satisfied that the Division is
  operating well and expects further definition and
  refining of the review process as SUFD matures.
  
• The Committee continues to urge very careful
  attention to the coordination of the two major
  science-program divisions (Materials Sciences and
  Engineering and Chemical Sciences, Geosciences
  and Biosciences) with the Scientific User
  Facilities Division healthy growth of the BES
  organization will necessitate balance between
  these two organization units.
•  
• The committee gave PART ratings of 1) Materials
  Research - Excellent, 2) Chemistry - Excellent,
  3) Energy Research - N/A, and 4) Instrumentation
  - Excellent.
•  

9
COV-Review Recommendations
10
Facilities-Division-Review Recommendations

• Recommendation The overall basic review system
  works exceptionally well, do not change it.

But perhaps tweak it!

• Recommendation The Facilities Division staff
  should begin to plan for an improved strategy for
  the review process of the largest facilities.

• Recommendation To the extent possible, the
  results of the review should be provided in a
  timely fashion. In addition, the comments of the
  reviewers should be summarized separately from
  the letter containing requested actions by the
  SUF Division Director.

• Recommendation The Committee recommends that
  the planned increase by 5 in the SUFD staff
  proceed promptly it is needed for a well managed
  facilities program. In addition, allowance for
  increased travel, i.e. funds and time, to
  facilities to encourage informal evaluation of
  facilities should be made.

• Recommendation The Committee recommends that
  each SUFD review explicitly discuss
  collaborations between core-research programs and
  SUFD operations.

11
General Recommendations
12
Examples of Facility-Specific Comments
Nanocenters

• Users Definition and Uniqueness for Each
  Facility
• There may be categories of users who are not
  always physically present when they use the
  facility resources. With increased operating
  experience the NSRCs should develop methods to
  measure appropriately the services they provide
  to users. Especially in the NSRCs, a high-quality
  internal-research program is vital to ensure that
  the center offers stateof-the-art facilities.
  While this internal program can create some
  tension between the user demands and the internal
  program needs of the center, this tension should
  be treated as a part of the cost of operation and
  thus managed to optimize both goals.
• Metrics
• The distinctive character of the NSRCs makes
  careful consideration of their performance
  metrics essential.
• Response to Prior Review
• Centers created so far have all been developed
  with broad users input via user workshops, as
  was recommended by the previous COV.
• The integration of the NSRCs with the
  core-research programs of the DOE and of the host
  National Lab. . should form an element of
  reviews of the internal research at the centers.
• Documentation of Facility Review
• For the operational NSRC, the reviewers wrote
  detailed, constructive and frank reviews that
  were summarized very well by the program officer.

13
Other Facility-Specific Comments

• X-rays Include metric on remote users impressed
  by future-light-source planning
• Electron Microscopy Challenge is to convert from
  support to user center mode simulation and
  theory vital
• Accelerators and Detectors Vital need
• Neutrons Informal feedback to BES important
  accelerator program vital

Accelerator and detector program discussed in
Managing SUFD Vision.
14
Thanks to the COV members for generously giving
their time and thanks to Pat, Pedro, and the
staff for providing every assistance!