The peer review process and your application

1
The peer review process and your application
William N. Elwood, Ph.D. Scientific Review
Officer Community-Level Health Promotion study
section
National Institutes of HealthU.S. Department of
Health and Human Services
2
NIH Peer Review
Summary Statement and Priority Score transmitted
to applicant and Institute
Researcher writes and submits application
Application assigned to NIH Institutes Study
Sections
Application assigned to 30-50 reviewers
3 Reviewers read and write critiques
3
Dual Review System for Grant Applications

• First Level of Review
• Scientific Review Group (SRG)
• Provides Initial Scientific Merit Review of Grant
  Applications
• Rates Applications and Makes Recommendations for
  Appropriate Level of Support and Duration of Award

• Second Level of Review
• Council
• Assesses Quality of SRG
• Review of Grant Applications
• Makes Recommendation to Institute Staff on
  Funding
• Evaluates Program Priorities and Relevance
• Advises on Policy

4
Center for Scientific Review

• Serves as central receipt point for most PHS
  grant applications
• Assigns applications to CSR Integrated Review
  Groups/Study Sections or Institute Scientific
  Review Groups
• Assigns applications to NIH Institute(s) as
  potential funding component(s)
• Conducts initial scientific merit review of most
  research applications submitted to the NIH in
  about 220 Study Sections and regularly recurring
  special emphasis panels

5
Types of Scientific Review GroupsWhere are
Applications Reviewed?
Groups
Applications Reviewed
Research Projects Academic Research Enhancement
Awards Postdoctoral Fellowships Small Business
Innovation Research Shared Instrumentation Program
Projects Centers Institutional Training
Grants Conference Grants Career Awards Small
Grants RFAs Contracts
CSR IRGs
Study Sections
Special Emphasis Panels
Institutes
Scientific Review Groups
Contract Review Committees
6
Peer Review of NIH Support Mechanisms
Review by CSR, the Institutes and CSR, or the
Institutes

• Requests for Applications (RFAs)
• Program Project Grant (P01)
• Development Awards (K01, K02, K05, K07, K08, K23,
  K24, etc.)
• Small Grants (R03)
• Center Grants (P30, P50, P60)
• Conference Grants (R13)
• Institutional Training Awards (T32, T35)
• MARC (F34, F36, T34)
• Minority Biomedical Support Grant (S06)
• Contracts

• Research Project Grant (R01)
• Small Business Grants (R41, R42, R43, R44)
• Individual Fellowships (F30, F31, F32, F33, etc.)
• Exploratory/Development Grants (R21, R33,
  R21/R33)
• Shared Instrumentation Grant (S10)
• Academic Research Enhancement Award (R15)
• Resource Grants (P40, P41, R24, R26, R28)

7
NIH Grant Receipt, Review, and Award Schedule
Jan-May May-Sept Sept-Jan
Receipt Dates
June-July Oct-Nov Feb-Mar
Review Dates
Sept-Oct Jan-Feb May-June
National Advisory Council Board Dates
Dec 1 Apr 1 July 1
Earliest Possible Beginning Date
8
CSR Peer Review -- 2007

• 76,000 applications received
• 54,000 applications reviewed
• 16,000 reviewers
• 240 Scientific Review Officers
• 1,800 review meetings

9
CSR 5 Review Divisions with 25 IRGs
Scientific Review Groups 48
Scientific Review Groups 50
Scientific Review Groups 43
Scientific Review Groups 44
Scientific Review Groups 55
10
Peer Review in CSR

• CSR Study Sections are managed by a Scientific
  Review Officer (SRO) who is a professional,
  usually at the MD, Ph.D. MD/PhD level, whose
  scientific background is close to the expertise
  of the study section
• Each CSR standing study section has 12-40 members
  who are primarily from academia
• As many as 60-100 applications are reviewed at
  each study section meeting

11
Scientific Review Officer

• Performs administrative and technical review of
  applications to ensure completeness and accuracy
• Selects reviewers based on broad input
• Manages study sections
• Prepares summary statements
• Provides requested information about study
  section recommendations to Institutes/Centers and
  National Advisory Councils/Boards

12
Confidentiality

• Review materials and proceedings of review
  meetings represent privileged information to be
  used only by consultants and NIH staff.
• At the conclusion of each meeting, consultants
  will be asked to destroy or return all
  review-related material.
• Consultants should not discuss review proceedings
  with anyone except the SRO.
• Questions concerning review proceedings should be
  referred to the SRO.

K185pp.46
13
Certification of Confidentiality and
Non-Disclosure
I fully understand the confidential nature of the
review process and agree (1) to destroy or
return all materials related to the evaluation
(2) not to disclose or discuss the materials
associated with the review, my evaluation, or the
review meeting outside of that meeting or with
any other individual except as authorized by the
Scientific Review Officer (SRO) or other NIH
designated official (3) not to disclose
procurement information prior to the award of a
contract and (4) to refer all inquiries
concerning the review to the SR0 or other
designated NIH official.
SIGNATURES
14
Review Criteria

• Significance Does the study address an important
  problem? How will scientific knowledge or
  clinical practice be advanced?
• Approach Are design and methods well-developed
  and appropriate? Are problem areas addressed?
• Innovation Are there novel concepts or
  approaches? Are the aims original and
  innovative?
• Investigator Is the investigator appropriately
  trained?
• Environment Does the scientific environment
  contribute to the probability of success? Are
  there unique features of the scientific
  environment or subject populations?

15
Research Involving Human Subjects

• Important Considerations
• Is the proposed study exempt from human subject
  review?
• Are there any apparent risks to the human
  subjects?
• Are the protections adequate?
• What are the potential benefits to the subjects
  and to mankind?
• Are the inclusions of minorities and both genders
  adequately addressed?

Risks include the possibility of physical,
psychological, or social injury resulting from
research.
16
Priority Scores/Percentile Rank

• For each study section, applications in the upper
  half generally are scored from 1.0-3.0, with 1.0
  the best score. Scores as low as 5.0 are
  possible.
• Individual scores are averaged and multiplied by
  100 to give the final priority score
• Unscored (lower half)
• Deferral

• plus past two meetings

17
Action

• Unscored
• Application is unanimously judged to be in the
  lower half of applications reviewed by the study
  section or scientific review group. No priority
  score is assigned. The summary statement
  provided to the applicant is a compilation of
  reviewers comments prepared prior to the
  meeting.

18
Summary Statement
Results are documented by SRO in a summary
statement and forwarded to the PI and the
assigned NIH Institute or Center, where a
funding decision is made. The Summary Statement
Contains

• Summary of Review Discussion
• Essentially Unedited Critiques
• Budget Recommendations
• Administrative Notes
• Priority Score and Percentile Ranking

19
NIH Peer Review Information on the Web

• National Institutes of Health (http//www.nih.gov)
  
• Office of Extramural Research (http//www.nih.gov/
  grants/oer.htm)
• Grants Policy (http//www.nih.gov/grants/policy/po
  licy.htm)
• Electronic Submission (http//era.nih.gov/Electron
  icReceipt)
• Center for Scientific Review (http//www.csr.nih.g
  ov)
• Resources for Applicants (http//www.csr.nih.gov/R
  esourcesforApplicants)
• CSR Study Section Rosters (http//www.csr.nih.gov/
  committees/rosterindex.asp)
• Review Group Meeting Dates (http//www.csr.nih
  .gov/Committees/meetings/ssmeet1.asp)
  

20
CSR Web Site http//www.csr.nih.gov

• About CSR
• News and Reports
• Peer Review Meetings
• Resources for Applicants

21
Helpful Handouts
Insiders Guide What Happens to
NIH Grant Application to Peer
Review Your Grant Application
Useful Web Links

http//cms.csr.nih.gov/publications/