Peer Review Process

1
Peer Review Process
Scientific Review Branch Division of Extramural
Research NIEHS/NIH Leroy Worth, Jr.
2
NIH EXTRAMURAL TEAM
3
Application Review Process

CIERD APRIL 5, 2005 Dr. Teresa Nesbitt Dr.
Linda Bass NIEHS, NIH
4
Review Process for a Research Grant
National Institutes of Health
Research Grant Application
School or Other Research Center
Center for Scientific Review
Assigns to IRG/Study Section IC
Study Section
Initiates Research Idea
Submits Application
Evaluates for Scientific Merit
Institute
Evaluates for Program Relevance
Advisory Councils and Boards
Allocates Funds
Conducts Research
Recommends Action
Institute Director
Takes final action for NIH Director
5
Overall Timeframe from Submission to Award

• There are three overlapping cycles per year

JAN FEB MAR APR MAY JUN
JUL AUG SEP OCT NOV DEC
JAN FEB MAR APR MAY JUN
JUL
Cycle 1
Review
Council
Receipt
Award
Referral
6
Receipt Dates
Depend on the Type of Application

• Jan, May, Sept 10 Institutional Training Grant
  Applications
• Jan, May, Sept 25 Academic Research Enhancement
  Award
• Feb, June, Oct 1 New Research Applications
• Mar, Jul, Nov 1 Revised, Competing
  Continuations, and
• April, Aug, Dec 1 Small Business Technology
  Transfer
• April, Aug, Dec 5 Individual NRSA Applications
• May, Sept, Jan 1 AIDS Applications

Applications
Supplemental Applications
and Small Business Innovation Research
Applications
7
Center for Scientific Review (CSR)
8
Center for Scientific Review

• Serves as central receipt point for most PHS
  grant applications
• Assigns applications to CSR Integrated Review
  Groups/Study Sections or Institute Scientific
  Review Groups
• Assigns applications to NIH Institute(s) as
  potential funding component(s)
• Conducts initial scientific merit review of most
  research applications submitted to the NIH in
  more than 100 Study Sections

9
Scientific Review Administrator
Designated Federal official with overall
responsibility for the review process, including

• Performing administrative and technical review of
  applications to ensure completeness and accuracy
• Selecting reviewers based on broad input
• Managing study section meetings
• Preparing summary statements
• Providing any requested information about study
  section recommendations to Institutes and
  National Advisory Councils/Boards

10
NIH Referral and Review SystemRegular Research
Grant Applications
Program Policy Considerations
NIGMS
NINDS
NCRR
NIA
NIAAA
NIAID
NEI
Referral
NIDCR
NIAMS
NINR
CSR
NIEHS
NIMH
NIDCD
NHLBI
NLM
NHGRI
Scientific Review
NCCAM
NICHD
FIC
NIDDK
NIBIB
NIDA
NCMHD
Funding Decisions
Scientific Management
11
CSR Review Divisions
Division of Molecular and Cellular
Mechanisms Donald Schneider, Ph.D.
Division of Physiology and Pathology Michael
Martin, Ph.D.
Biochemical Sciences IRG (BCS) Zakir Bengali,
Ph.D.
Cardiovascular Sciences IRG (CVS) Joyce Gibson,
D.Sc.
Bioengineering Sciences and Technologies IRG
(BST) Sally Amero, Ph.D.
Digestive Sciences IRG (DIG) Mushtaq Khan, Ph.D.
Biology of Development and and Aging
(BDA) Sherry Dupere, Ph.D.
Hematology IRG (HEME) Joyce Gibson, D.Sc.
Biophysical and Chemical Sciences IRG (BPC) John
Bowers, Ph.D.
Integrative, Functional and Cognitive
Neuroscience IRG (IFCN) Christine Melchior, Ph.D.
Cell Development and Function IRG (CDF/CB) Marcia
Steinberg, Ph.D.
Musculoskeletal, Oral, and Skin Sciences IRG
(MOSS) Daniel McDonald, Ph.D.
Genetic Sciences IRG (GNS-GGG) Richard Panniers,
Ph.D.
Renal and Urological Sciences IRG (RUS) Daniel
McDonald, Ph.D.
Molecular, Cellular and Developmental Neuroscienc
e IRG (MDCN) Carole Jelsema, Ph.D.
Respiratory Sciences IRG (RES) Mushtaq Khan, Ph.D.
12
Peer Review of NIH Support Mechanisms
CSR
Institutes

• Research Project Grant (R01) Program Project
  Grant (P01)
• Postdoctoral Fellowship (F32) Center Grant
  (P30, P50, P60)
• Senior Fellowship (F32) Institutional
  Fellowship (T32)
• Fogarty International Center Academic Career
  Award (K07)
• Fellowship (F33) Mentored Clinical Scientist
• Short-Term Training (T35) Development Award
  (K08)
• Small Business Grants (R41, R42 Conference
  Grant (R13)
• R43, R44) MARC Fellowships (F34, F36,
  T34)
• Academic Research Enhancement Minority
  Biomedical Support
• Award (R15) Grant (S06)
• Biomedical Research Support Resource Grant
  (P40, P41, R24,
• Shared Instrumentation R26, R28)
• Grant (S10) Contract

Reviewed by CSR upon request
13
Grant Application Receipt and Assignment
14
Applications Submitted to NIH

• Over 60,000 grant applications are submitted to
  NIH each year, of which 25-30 are funded
• Competing grant applications are received for
  three review cycles per year

15
I need to make a correction on page 32 can you
find my application?
16
Applications are Assigned by

• Referral Officers
• Professional scientists, most of whom also serve
  as scientific review administrators of CSR study
  sections

17
Review Process for a Research Grant Application
18
Initial Review in CSR
19
Criteria for Selection of Peer Reviewers
20
Criteria For Selection of Peer Reviewers

• Demonstrated Scientific Expertise
• Doctoral Degree or Equivalent
• Mature Judgment
• Work Effectively in a Group Context
• Breadth of Perspective
• Impartiality
• Interest in Serving
• Adequate Representation of Women and Minority
  Scientists

21
CSR Study Sections

• Each CSR standing study section has 12-24 members
  who are primarily from academia
• CSR standing study sections convene face-to-face
  meetings
• As many as 60-100 applications are reviewed by
  each study section

22
Review of Research Grants

• REVIEW CRITERIA
• Significance
• Approach
• Innovation
• Investigator
• Environment
• Overall Evaluation Score Reflects Impact on
  Field

23
Review Criteria (continued)

• Significance Does the study address an important
  problem? How will scientific knowledge be
  advanced?
• Approach Are design and methods well-developed
  and appropriate? Are problem areas addressed?
• Innovation Are there novel concepts or
  approaches? Are the aims original and
  innovative?
• Investigator Is the investigator appropriately
  trained?
• Environment Does the scientific environment
  contribute to the probability of success? Are
  there unique features of the scientific
  environment?

24
BOTTOM LINE

• WHAT are you proposing to do?
• WHY is this important?
• Can YOU do it?

25
Scientific Review Group or Study Section Actions
(Summary Statement)

• Scored, Scientific Merit Rating (priority scores
  and percentiles)
• Unscored (lower half)
• Deferral

26
What Determines Which Awards Are Made?

• Scientific merit
• Program considerations
• Availability of funds

27
Common Problems in Applications

• Lack of new or original ideas
• Absence of an acceptable scientific rationale
• Lack of experience in the essential methodology
• Questionable reasoning in experimental approach
• Uncritical approach
• Diffuse, superficial, or unfocused research plan
• Lack of sufficient experimental detail
• Lack of knowledge of published relevant work
• Unrealistically large amount of work
• Uncertainty concerning future directions

28
Information on the World Wide WebSelected Sites
of Interest

• National Institutes of Health (http//www.nih.gov)
  
• Office of Extramural Research (http//www.nih.gov/
  grants/oer.htm)
• Grants Policy (http//www.nih.gov/grants/policy/po
  licy.htm)
• Center for Scientific Review (http//www.csr.nih.g
  ov)
• Referral and Review (http//www.csr.nih.gov/refrev
  .htm)
• CSR Study Section Rosters (http//www.csr.nih.gov/
  committees/rosterindex.asp)
• Review Group Meeting Dates
  (http//www. csr.nih.gov/committe
  es/meetings/ssmeet1.asp)
• CSR Reorganization News (http//www.csr.nih.gov/r
  eview/reorgact.asp)

29
A question well stated is a question half
answered

• Dewey

30
Take Home Message!!