CSR: New Challenges and Opportunities

1
CSR New Challenges and Opportunities
Toni Scarpa
Center for Scientific Review National Institutes
of Health Department of Health and Human Services
PRAC, January 23, 2006
2
CSR Progress Report

• Changes that have Occurred
• Changes in Progress
• Changes under Discussion
• Challenges Present and Futures

3
Changes in CSR Operations 1

• Increased Communication and Transparency
• Within CSR
• With NIH and other Agencies
• With the Scientific Community

4
Changes in CSR Operations 2

• Increase uniformity
• Slate Nomination
• Summary Statements
• Posting all within one months of Study
  Section
• Posting Summary Statements of new
  investigators within one week
• More complete and structured resumes
• Unscoring
• Common practice
• Unscoring 50

5
Changes in CSR Operations 3

• Increase Efficiency
• Electronic Submission
• Text Fingerprinting, Artificial Intelligence
  Software

6
Potential of Knowledge Management Tools for Peer
Review

• Collexis Software or Others
• Knowledge management solutions
• Fingerprinting and text retrieving
• Disease coding
• Benefits for Peer Review
• Assigning applications to Integrated Review
  Groups or Study Sections
• Selecting reviewers (one application, multiple
  applications)
• Nine pilots are underway to begin to assess these
  benefits

7
Possible Changes in CSR Operations

• Facilitate work of IC program staff

8
Study Section Realignment

• Review of one IRG every month
• Total review every 2 years

9
Desirable Changes in CSR Review

• Shorten the review cycle

10
Shortening the NIH Review Cycle, Initial Steps

• For most research grants, we are posting Summary
  Statements within one month after the Study
  Section meeting, instead of two to three months
  after the meeting (effective Oct 05)
• We are conducting a pilot study to speed the
  review process for new investigators so they may
  revise and resubmit for the very next review
  cycle, 4 months earlier than before (effective
  Feb 06). Dr. Bradley, next talk.

11
Desirable Changes in CSR Review

• Shorten the review cycle
• Address concern that clinical research is not
  properly evaluated
• Improve the assessment of innovative, high-
  risk/high-reward research

12
The judging of grants has become a charade.
The American Society for Cell Biology
The judging of grants has become a charade. To
be funded, the experimental plan has become a
litany of experiments already accomplished so
that everything is feasible. When grants come
back with unfundable scores, new investigators
may not have sufficient resources to do the
experiments that show feasibility. Zena
Werb President, ASCB
Newsletter August 2005
13
Possible Changes in Current Systems

• Shorten the review cycle
• Address concern that clinical research is not
  properly evaluated
• Improve the assessment of innovative, high-
  risk/high-reward research
• Do more to recruit and retain more high-quality
  reviewers

14
Expanding Peer Reviews Platforms

• Electronic Reviews
• Telephone Enhanced Discussions
• Video Enhanced Discussions
• Asynchronous Electronic Discussions

Study Sections
Necessity ? Clinical
reviewers Preference ?
Physicists, computational biologists New
Opportunities ? Fogarty, International
Reviewers
15
Applications received for all of NIH and
applications referred for CSR review, FY 1998-2004
80,000
60,000
Number of applications
40,000
20,000
0
1998
2000
2002
2004
Fiscal year
Applications received for all of NIH
Applications assigned for review by CSR
16
Applications received for all of NIH FY 1998-2004
80,000
60,000
Number of applications
40,000
20,000
0
1998
2000
2002
2004
Fiscal year
17
Number of Research Grant Applications/Applican
t
18
Study Section Application/Reviewer
Ratio October Council Only
19
CENTER FOR SCIENTIFIC REVIEW FY 2004-2006
Non-Discretionary vs. Discretionary Spending
FY 2006
FY 2004
FY 2005
Non-Discretionary Discretionary
20
Challenges and Opportunities
Peer Review
21
Possible Short Term Approaches for Increasing
Efficiency for Reviewers and CSR

• Shifting Additional Grant Review to Institutes
• Replace Many SEPs with Smaller Parallel Study
  Sections
• Enlarge Study Section Membership and Decrease
  Frequency of Participation
• Pre Meeting Streamlining
• Various Review Platforms
• Hybrid Review Platforms
• Staggering Application Deadlines
• 2 instead of 3 reviews
• Shorten Applications
• More Structured Applications and Reviews

22
Peer Review An N.I.H. Conception

• Is the heart and soul of NIH
• Has produced an effective partnership between the
  federal government and research institutions
• Has created the best academic medical centers,
  the best biomedical/behavioral research and
  biotechnology
• Has made possible the best cures and the best
  prevention
• Has been admired and imitated here and abroad
• Has protected NIH against outside influence

23
This is CSR