Peer Review at the Center for Scientific Review

1
Peer Review at the Center for Scientific Review
Joanna M. Watson, PhD Scientific Review
Administrator Molecular Oncogenesis Study
Section Oncological Sciences IRG Center for
Scientific Review National Institutes of Health
2
NIH Process for a Research Grant
National Institutes of Health
University/ Research Center
Research Grant Application
Center for Scientific Review
Submits Application

Initiates Research Idea
Study Section
Institute
Advisory Councils / Boards
Allocates Funds
Conducts Research
Institute Director
3
What does CSR do?

• Referral
• Central Receipt Point
• Institute Assignment
• Referral to Integrated Review Groups (IRGs) and
  Study Sections
• Scientific Review
• Review of most research and research training
  applications for scientific merit

4
What Types of Mechanisms are Reviewed Where?

• Center for Scientific Review
• Study Sections and Special Emphasis Panels
• Research Projects (R01)
• Pilot Studies (R21)
• Small Grants (R03)
• Fellowships (F32/33)
• SBIR (R43/44)
• STTR (R41/42)
• Program Projects for some I/Cs

• Institute Review Offices
• Scientific Review Groups and Contract Review
  Committees
• Training Grants (T32)
• Career Awards (Ks)
• Program Projects (P01)
• Centers (P20/30/50)
• Cooperative Agreements
• Multi-institutional clinical trials
• Applications for RFAs and complex PAs
• Contract Proposals for RFPs

5
Central CSR Receipt of ApplicationsYesteryear
6
Central Receipt of ApplicationsToday
7
The CSR Referral Path for a Competing Grant
Application
Receipt office
Division of Receipt Referral
Breakout
Institute
Referral Officer
IRG Chief or Deputy Chief
Integrated Review Group (IRG)
SRA, IRG Study Section
8
Applications are Assigned to

• Institutes on the basis of
• Overall mission
• Specific programmatic mandates and interests
• Initial IRG and Study Sections on the basis of
• Specific review guidelines
• http//cms.csr.nih.gov/PeerReviewMeetings/CSRIRGDe
  scription/ONCIRG/

9
Assignment to Study Sections

• Within an IRG, applications are assigned for
  review to
• Standing Study Sections when the subject matter
  of the application falls within the purview of
  the referral guidelines for the study section
• Ad Hoc Special Emphasis Panels (SEPs) when the
  subject matter does not fit into any study
  section, or when assignment of an application to
  the most appropriate study section would create a
  conflict of interest. Also used for special
  mechanisms (e.g., fellowships, SBIRs, AREAS), and
  for newly created study sections that are not yet
  chartered.

10
CSR Review Divisions and IRGs
Division of Biologic Basis of Disease
Division of Molecular and Cellular
Mechanisms Donald Schneider, Ph.D.
Division of Physiology and Pathology Michael
Martin, Ph.D.
AIDS and Related Research (AARR) (9) Ranga V.
Srinivas, Ph.D.
Bioengineering Sciences and Technologies (BST)
(7) George Chacko, Ph.D.
Cardiovascular Sciences (CVS) (11) Joyce Gibson,
D.Sc.
Biology of Development and and Aging (BDA)
(6) Sherry Dupere, Ph.D.
Digestive Sciences (DIG) (7) Mushtaq Khan, Ph.D.
Biological Chemistry and Macromolecular
Biophysics (BCMB) (12) John Bowers, Ph.D.
Hematology (HEME) (5) Joyce Gibson, D.Sc.
Immunology (IMM) (11) Calbert Laing, Ph.D.
Integrative, Functional and Cognitive
Neuroscience (IFCN) (14) Christine Melchior, Ph.D.
Cell Biology (CB) (8) Noni Byrnes, Ph.D.
Genes, Genomes and Genetics (GGG) (11) Richard
Panniers, Ph.D
Musculoskeletal, Oral, and Skin Sciences (MOSS)
(12) Daniel McDonald, Ph.D.
Molecular, Cellular and Developmental Neuroscienc
e (MDCN) (15) Carole Jelsema, Ph.D.
Renal and Urological Sciences (RUS) (4) Daniel
McDonald, Ph.D.
Respiratory Sciences (RES) (4) Mushtaq Khan, Ph.D.
11
Referral in the ONC IRG
Dr. Elaine Sierra-Rivera
Fellowship, SBIR/STTR
Translational/Therapeutic
Basic
CE (Victor Fung) CG (Zhiqiang Zou) MONC (Joanna
Watson) CAMP (Elaine Sierra-Rivera) TCB (Angela
Ng) TME (Eun Ah Cho) TPM (Manzoor Zarger)
CDP (Sally Mulhern) CBSS (Mary Bell) RTB (Bo
Hong) CII (Steven Scholnick) DMP (Syed Quadri) DT
(Sharon Gubanich) BMCT (Suzanne
Forry-Schaudies) CONC (John Meyer)
F09 (Lambratu Rahman) ONC-L 10 (Hungyi
Shau) ONC-L 12 (Hungyi Shau) ONC-R (Bo Hong)
12
What does the Scientific Review Administrator do?

• Performs administrative and technical review of
  applications
• Recruits reviewers
• Must meet particular requirements
• Makes assignments
• Manages study section meeting
• Prepares summary statements

13
What Happens In A Study Section?

• Orientation
• Streamlining
• Discussion Scoring
• REVIEW CRITERIA Significance Approach Innovat
  ion Investigator Environment Human Subjects
  Vertebrate Animals
• Budget
• Model Organism Sharing
• Data Monitoring Plan

Scoreable Issues
Non-Scoreable Issues
14
Study Section Meeting Procedures

• Reviewers receive ALL applications
• Assigned reviewers present a summary of their
  critiques
• Open discussion among ALL reviewers
• Range of scores set
• All reviewers vote and score each application
• Discussion of budget amount and time, and other
  issues

Reviewers in conflict or potential conflict are
NOT allowed in the room during discussion and do
not score or see the applications.
15
Study Sections Actions and Outcomes

• Unscore (streamline)
• Initially identified on the basis of the average
  preliminary scores
• Consensus must be reached among entire study
  section
• Score (Merit Rating or Priority Score assigned)
• 1.0 (best) to 5.0 (worst)
• R01s percentiled against study section or CSR all
• Percentiles derived from previous two rounds and
  current round
• For some institutes, R21s are percentiled using
  the R01 percentile base for the study section,
  but the R21s are not included in the percentile
  base
• Not Recommended for Further Consideration (NRFC)
• Application has serious ethical problems in Human
  Subject or Animal use or significant scientific
  flaw
• Defer
• Review Committee needs more information to decide
  on the scientific merit of the application or
  problem identified during the review

16
Summary Statement

• The summary statement is the official document of
  the review of the application. At CSR, all
  summary statements released within 30 days of the
  meeting.
• A summary statement is prepared for all
  applications and includes
• Face page administrative information
• Investigator, Program Director contact
  information, review group, council round,
• Priority Score and Percentile Ranking (if scored)
• Human subjects, animal welfare codes
• Requested budget
• Resume Summary of Discussion (if scored)
• Description of the proposed studies (if scored)
• Reviewers Critiques
• Assessment of Human Subjects, Inclusion of
  Gender, Minority, and Children
• Assessment of Vertebrate Animal Welfare
• Budget Recommendations (if scored)
• Administrative Notes
• i.e., Budget overlap, Model organism sharing plan
  missing
• Meeting Roster

17
Overall Timeframe from Application Submission to
Award

• Three overlapping cycles occur per year

JAN FEB MAR APR MAY JUN JUL
AUG SEP OCT NOV DEC JAN FEB
MAR APR MAY JUN JUL