Title: Peer Review of Research Grant Applications
1Peer Review of Research Grant Applications
Sources for this talk NIH WEB-SITES Prof. Joey
Granger Prof. Michael Flessner
2There is no grantsmanship that will turn a bad
idea into a good one, but..There are many
ways to disguise a good one.
Former NIH Director
3Topics
- Overview of NIH Grant Review/Selection process
- What you need to do before sending in the grant
- Carefully select the agency examine success
rates - Tailor title, selling points to the review group
- Recommendations on schedule for preparation of
application - Details of the selection process
- Action Items after the letter of evaluation
- Respond as if your life depended on it
- Resubmission
- Was the evaluation fatal or did they give you
recommendations
4National Institutes of Health
- Much of the biomedical research in the United
States is supported by the Federal Government,
primarily the National Institutes of Health (NIH)
5U. S. Department of Health and Human Services
6NIH Funding in FY 2004 By MechanismTotal 27B
7National Institutes of Health
Office of the Director
National Institute on Alcohol Abuse and Alcoholism
National Institute of Arthritis
and Musculoskeletal and Skin Diseases
National Cancer Institute
National Institute on Aging
National Institute of Child Health and
Human Development
National Institute of Allergy and Infectious
Diseases
National Institute of Diabetes and Digestive
and Kidney Diseases
National Institute of Dental and Craniofacial Rese
arch
National Institute on Drug Abuse
National Institute of Environmental Health
Sciences
National Institute on Deafness and
Other Communication Disorders
National Eye Institute
National Human Genome Research Institute
National Heart, Lung, and Blood Institute
National Institute of Mental Health
National Institute of Neurological Disorders
and Stroke
National Institute of General Medical Sciences
National Institute of Nursing Research
National Library of Medicine
National Center for Complementary and
Alternative Medicine
National Center on Minority Health and Health
Disparities
National Institute of Biomedical Imaging and
Bioengineering
Fogarty International Center
National Center for Research Resources
Center for Information Technology
Center for Scientific Review
Clinical Center
8A Typical Institute/Center
Office of the IC Director
National Advisory Council
Board of Scientific Counselors
Extramural
Intramural
Scientific Programs
Laboratory Studies
Clinical Studies
Grants
Contracts
9National Institutes of Health
Office of the Director
National Institute on Alcohol Abuse and Alcoholism
National Institute of Arthritis
and Musculoskeletal and Skin Diseases
National Cancer Institute
National Institute on Aging
National Institute of Child Health and
Human Development
National Institute of Allergy and Infectious
Diseases
National Institute of Diabetes and Digestive
and Kidney Diseases
National Institute of Dental and Craniofacial Rese
arch
National Institute on Drug Abuse
National Institute of Environmental Health
Sciences
National Institute on Deafness and
Other Communication Disorders
National Eye Institute
National Human Genome Research Institute
National Heart, Lung, and Blood Institute
National Institute of Mental Health
National Institute of Neurological Disorders
and Stroke
National Institute of General Medical Sciences
National Institute of Nursing Research
National Library of Medicine
National Center for Complementary and
Alternative Medicine
National Center on Minority Health and Health
Disparities
National Institute of Biomedical Imaging and
Bioengineering
Fogarty International Center
National Center for Research Resources
Center for Information Technology
Center for Scientific Review
Clinical Center
10Center for Scientific Review
- Serves as central receipt point for most PHS
grant applications - Assigns applications to CSR Integrated Review
Groups/Study Sections or Institute Scientific
Review Groups - Assigns applications to NIH Institute(s) as
potential funding component(s) - Conducts initial scientific merit review of most
research applications submitted to the NIH in
about 220 Study Sections and regularly recurring
special emphasis panels
11Review Process for a Research Grant Application
12Topics
- Overview of NIH, AHA Grant Review/Selection
process - What you need to do before sending in the grant
- Carefully select the agency examine success
rates - Tailor title, selling points to the review group
- Recommendations on schedule for preparation of
application - Details of the selection process
- Action Items after the letter of evaluation
- Respond as if your life depended on it
- Resubmission
- Was the evaluation fatal or did they give you
recommendations
13(No Transcript)
14Tips on how to direct your NIH application to a
specific study section
- Determine which study section has the most
appropriate expertise for your grant. DETERMINE
YOUR POTENTIAL REVIEWERS BEFORE WRITING YOUR
GRANT - Application cover letter should indicate which
study section has the most appropriate expertise
for your proposal - Select an application title that is consistent
with study section expertise - Use key words in first few sentences of abstract
and specific aims page that are consistent with
study section expertise
15Title and Abstract
- Examine the topics funded by the study group
- Call the CSR-SRA for advice
- Title should be descriptive despite space limit
- Most important Abstract It is what sells your
proposal to the major reviewers and to the entire
study section - Include hypothesis/purpose/goals
- Give some idea of the approach
- Include elements of anticipated results and their
significance - The abstract sells the proposal! Make it
interesting!
16Review of Research Grants
-
- REVIEW CRITERIA
- Significance
- Approach
- Innovation
- Investigator
- Environment
- Overall Evaluation Score Reflects Impact on
Field
17Review Criteria (continued)
- Significance Does the study address an important
problem? How will scientific knowledge be
advanced? - Approach Are design and methods well-developed
and appropriate? Are problem areas addressed? - Innovation Are there novel concepts or
approaches? Are the aims original and
innovative? - Investigator Is the investigator appropriately
trained? - Environment Does the scientific environment
contribute to the probability of success? Has
the PI asssembled an appropriate team of
investigators to address problem?
Provide information on significance, approach,
and innovation on specific aims page
18Specific Aims
- Different approaches
- Statement of hypothesis
- Questions
- Straightforward goals
- Make them clear and concise
- Include a 1 sentence overview of the approach
with each Specific Aim - Do not be overly ambitious narrowly-defined
goals are the best
19Central Hypothesis as a Diagram
Source J. Granger, PhD
20Background and Significance
- Background
- Include enough detail from a literature survey so
that you establish yourself as an expert in the
area. - Show the holes in the literature as a preface to
your approach to answering unanswered questions. - Significance
- Use whatever literature exists to demonstrate how
the solution which you seek will impact science
and medicine - Higher priority health problems will receive
preferential treatment.
21Preliminary Results
- Use this section to set up your approach to
solving your specific aims - Give pertinent details on published and
unpublished work which demonstrates your
competence to pursue the proposed specific aims. - Even for an R21, you will be more successful with
preliminary data - Use this section to demonstrate competency of
others on the team
22Approach
- If your approach is complex (such as a stepwise
proposal), start with an overview of the steps in
the proposed steps toward addressing the specific
aims. Diagrams? - Restate each specific aim. Demonstrate how you
will specifically address the question to be
answered. Include enough detail to confirm your
competence in the proposed techniques. Refer to
your previously published work or the cited
preliminary studies.
23Approach (contd)
- Discuss how the data will address the specific
aim. Discuss alternatives, if the outcome does
not fit your hypothesis. Discuss potential
problems. Specify the time and manpower required
to complete the specific aim. - Clinical Studies
- Previous experience in the team
- IRB approval
- Power calculation estimate of number of subjects
- Biostatistical support, Behavioral researcher
- Specify and justify population
24Approach (contd)
- Basic Research
- specify the number of animals, cell or molecular
resources - computational resources which you will need to
complete the proposal - Insure that a member of the team has performed
the proposed technique and published results with
it
25Common Problems in Applications
- Lack of a clearly stated hypothesis or question
(bold or italics and provide a diagram)
- A question well stated is a
- question half answered
- Dewey
26Highlight Central Hypothesis and Specific Aims
widespread dysfunction of the maternal vascular
endothelium by mechanisms that remain to be
determined. The central hypothesis to be tested
in this proposal is that a reduction in
uteroplacental perfusion pressure causes
hypertension by reducing renal-pressure
natriuresis. The reduction in pressure
natriuresis occurs as a result of placental
and/or leukocyte-derived cytokines causing
endothelial cell activation that leads to
enhanced formation of reactive oxygen species via
endothelin, angiotensin II, and
cytokine-dependent mechanisms (Figure1). These
abnormalities reduce renal plasma flow and
glomerular filtration rate thereby decreasing
renal sodium excretory function. To test this
hypothesis, arterial pressure, renal, hormonal,
and endothelial function will be examined in a
conscious, chronically instrumented rat model of
chronic PIH produced by long-term reductions in
uterine perfusion pressure (RUPP). Data obtained
in this model from the previous grant period
indicate that the hypertension produced by
decreased perfusion pressure to the
uteroplacental unit is associated with
proteinuria
27Common Problems in Applications
- Lack of a clearly stated hypothesis or question
(bold or italics) - Lack of productivity you must have published in
the area of research - Lack of experience in the essential methodology
role of the consultant - Lack of key preliminary data do not propose
something unless you have shown the possibility
of its success
28Common Problems in Applications-contd
- Diffuse, descriptive, or unfocused research plan
(incoporate mechanistic and hypothesis-driven
experiments) - Lack of sufficient experimental detail
(especially with new techniques) - Lack of knowledge of published relevant work
(esp. reviewers!) - Unrealistically large amount of work (This is not
a concern if productive) - Uncertainty concerning future directions
- (Expected results, interpretation, and future
directions, and experimental limitations)
29Timeline in Proposal Preparation
- 2 years ahead of proposal Plan your experiments
to obtain the results necessary for your next
proposal - 1 year ahead take a couple of weekends off and
plan your approach. Where do you have holes in
the approach? Can you get help? - 6 months ahead begin writing with the goal of a
smooth final draft at least 4 weeks ahead of the
deadline - 1 month ahead submit your draft to at least 2-3
senior people for their review. - Be humble and listen to their criticisms.
Rewrite where appropriate. - Make sure all of the boilerplate is written
appropriately.
30Topics
- Overview of NIH, AHA Grant Review/Selection
process - What you need to do before sending in the grant
- Carefully select the agency examine success
rates - Tailor title, selling points to the review group
- Recommendations on schedule for preparation of
application - Details of the selection process
- Action Items after the letter of evaluation
- Respond as if your life depended on it
- Resubmission
- Was the evaluation fatal or did they give you
recommendations
31Dual Review System for Grant Applications
- First Level of Review
- Scientific Review Group (SRG)
- Provides Initial Scientific Merit
- Review of Grant Applications
- Rates Applications and Makes
Recommendations for Appropriate Level of Support
and Duration of Award
- Second Level of Review
- Council
- Assesses Quality of SRG
- Review of Grant Applications
- Makes Recommendation to
- Institute Staff on Funding
- Evaluates Program Priorities
- and Relevance
- Advises on Policy
32Review Process for a Research Grant
National Institutes of Health
Research Grant Application
School or Other Research Center
Center for Scientific Review
Assigns to IRG/Study Section IC
Study Section
Initiates Research Idea
Submits Application
Evaluates for Scientific Merit
Institute
Evaluates for Program Relevance
Advisory Councils and Boards
Allocates Funds
Conducts Research
Recommends Action
Institute Director
Takes final action for NIH Director
33Typical Timeline for a New Individual Research
Project Grant Application (R01)
- There are three overlapping cycles per year
- Submit in February (June, October)
- Review in June (October, February)
- Council in September (January, May)
- Earliest award in December (April, July)
- Cycle 1----
- Cycle 2----
- Cycle 3----
34Receipt Dates
Depend on the Type of Application
- Jan, May, Sept 10 Institutional Training Grant
Applications - Jan, May, Sept 25 Academic Research Enhancement
Award - Feb, June, Oct 1 New Research Applications
- Mar, Jul, Nov 1 Revised, Competing
Continuations, and - Apr, Aug, Dec 1 Small Business Technology
Transfer - Apr, Aug, Dec 5 Individual NRSA Applications
- May, Sept, Jan 1 AIDS Applications
Applications
Supplemental Applications
and Small Business Innovation Research
Applications
35Peer Review of NIH Support Mechanisms
CSR
Institutes
- Research Project Grant (R01) Program Project
Grant (P01) - Postdoctoral Fellowship (F32) Center Grant
(P30, P50, P60) - Senior Fellowship (F32) Institutional
Fellowship (T32) - Fogarty International Center Academic Career
Award (K07) - Fellowship (F33) Mentored Clinical Scientist
- Short-Term Training (T35) Development Award
(K08) - Small Business Grants (R41, R42 Conference
Grant (R13) - R43, R44) MARC Fellowships (F34, F36,
T34) - Academic Research Enhancement Minority
Biomedical Support - Award (R15) Grant (S06)
- Biomedical Research Support Resource Grant
(P40, P41, R24, - Shared Instrumentation R26, R28)
- Grant (S10) Contract
Reviewed by CSR upon request
36Grant Application Receipt and Assignment
37Applications Submitted to NIH
- Over 60,000 grant applications are submitted to
NIH each year, of which 25-30 are funded - NIH commons site will reduce paperwork
38Applications are Assigned by
- Referral Officers
- Professional scientists, most of whom also serve
as scientific review administrators of CSR study
sections
39Assignment to CSR Study Sections
- Applications are assigned to Integrated Review
Groups (IRGs) for review. IRGs are clusters of
scientifically-related study sections - The IRG assignment is based on specific referral
guidelines for each IRG - Each of the 20 IRGs within CSR has 5 - 8 standing
study sections
40CSR Review Divisions
Division of Biologic Basis of Disease Elliot
Postow, Ph.D.
Division of Molecular and Cellular
Mechanisms Donald Schneider, Ph.D.
Division of Physiology and Pathology Michael
Martin, Ph.D.
AIDS and Related Research IRG (AARR) Ranga V.
Srinivas, Ph.D.
Biochemical Sciences IRG (BCS) Zakir Bengali,
Ph.D.
Cardiovascular Sciences IRG (CVS) Joyce Gibson,
D.Sc.
Bioengineering Sciences and Technologies IRG
(BST) Sally Amero, Ph.D.
Digestive Sciences IRG (DIG) Mushtaq Khan, Ph.D.
Biology of Development and and Aging IRG
(BDA) Sherry Dupere, Ph.D.
Hematology IRG (HEME) Joyce Gibson, D.Sc.
Immunology IRG (IMM) Calbert Laing, Ph.D.
Biophysical and Chemical Sciences IRG (BPC) John
Bowers, Ph.D.
Integrative, Functional and Cognitive
Neuroscience IRG (IFCN) Christine Melchior, Ph.D.
Cell Development and Function IRG (CDF) Marcia
Steinberg, Ph.D.
Musculoskeletal, Oral, and Skin Sciences IRG
(MOSS) Daniel McDonald, Ph.D.
Genetic Sciences IRG (GNS) Richard Panniers,
Ph.D.
Renal and Urological Sciences IRG (RUS) Daniel
McDonald, Ph.D.
Molecular, Cellular and Developmental Neuroscienc
e IRG (MDCN) Carole Jelsema, Ph.D.
Respiratory Sciences IRG (RES) Mushtaq Khan, Ph.D.
41Assignment to CSR Study Sections (continued)
- Within an IRG, applications are assigned for
review to - Standing Study Sections when the subject matter
of the application matches the referral
guidelines for the study section - Ad Hoc Special Emphasis Panels (SEPs) when the
subject matter does not fit into any study
section, or when assignment of an application to
the most appropriate study section would create a
conflict of interest. Also used for special
mechanisms (e.g., fellowships, SBIRs, AREAS)
42Assignment Notification Letter
- Dear Dr. Smith
- Your grant application entitled CEREBRAL VESSEL
INNERVATION IN HYPERTENSION has been received by
the National Institutes of Health and assigned to
a Scientific Review Group (SRG) for scientific
merit evaluation and to an Institute/Center for
funding consideration. Specific information
about your assignment is given below. The
initial peer review should be completed by March,
1999, and a funding decision made shortly after
the appropriate National Advisory Group meets in
May, 1999. Questions about the assignment should
be directed to the Scientific Review
Administrator (SRA) or the Division of Receipt
and Referral, Center for Scientific Review
(formerly Division of Research Grants) at (301)
435-0715. Other questions prior to review should
be directed to the Scientific Review
Administrator and questions after the review to
the program staff in the Institute/Center.
43Assignment Notification Letter (continued)
- Assignment Number 2 R01 HL12345 - 12A1
- Dual Assignment NS
- Scientific Review Group
- EXPERIMENTAL CARDIOVASCULAR SCIS SS (ECS)
- Information about SRGs may be found on the CSR
Home page (http//www.csr.nih.gov) - Scientific Review Administrator
- DR. ANSHUMALI CHAUDHARI, SRA
- CTR FOR SCIENTIFIC REV (PREV DRG)
- 6701 ROCKLEDGE DR RM 4128 MSC7802
- BETHESDA MD 20892
- (301) 435 - 1210
44Initial Review in CSR
45Peer Review in CSR
- CSR Study Sections are managed by a Scientific
Review Administrator (SRA) who is a professional,
usually at the Ph.D. level, whose scientific
background is close to the expertise of the study
section - Each CSR standing study section has 12 - 24
members who are primarily from academia - As many as 60 - 100 applications are reviewed at
each study section meeting
46Scientific Review Administrator
- Performs administrative and technical review of
applications to ensure completeness and accuracy - Selects reviewers based on broad input
- Manages study sections
- Prepares summary statements
- Provides requested information about study
section recommendations to Institutes/Centers and
National Advisory Councils/Boards
47Selection of Peer Reviewers
Active and Productive Researchers
Research Capability
Non-Research
Non-Doctoral
Scientific Community
48Criteria for Selection of Peer Reviewers
49Study Section Meeting
50CSR Study Sections
- Each CSR standing study section has 12-24 members
who are primarily from academia - CSR standing study sections convene face-to-face
meetings - As many as 60-100 applications are reviewed by
each study section - It is important to note that each grant is
reviewed by only 2 or 3 reviewers (primary or
secondary reviewer). - Average grant review takes 15-20 mins
51Review of Research Grants
-
- REVIEW CRITERIA
- Significance
- Approach
- Innovation
- Investigator
- Environment
- Overall Evaluation Score Reflects Impact on
Field
52Review Criteria (continued)
- Significance Does the study address an important
problem? How will scientific knowledge be
advanced? - Approach Are design and methods well-developed
and appropriate? Are problem areas addressed? - Innovation Are there novel concepts or
approaches? Are the aims original and
innovative? - Investigator Is the investigator appropriately
trained? - Environment Does the scientific environment
contribute to the probability of success? Has
the PI asssembled an appropriate team of
investigators to address problem?
Provide information on significance, approach,
and innovation on specific aims page
53Scientific Review Group or Study Section Actions
- Scored, Scientific Merit Rating (priority scores
and percentiles) - Unscored (lower half)
- Deferral
54Action
- Scored -- Scientific Merit Rating 1.0 to
approximately 3.0 - Based on the relevant review criteria, the
application is judged to be in the upper half of
applications reviewed by the study section or
scientific review group. The recommendation can
be for the requested time and amount or for an
adjusted time and amount. A priority score is
provided, and a summary statement prepared that
incorporates the written critiques plus a resume
and summary of the discussion.
55Action
- Unscored
- Application is unanimously judged to be in the
lower half of applications reviewed by the study
section or scientific review group. No priority
score is assigned. The summary statement
provided to the applicant is a compilation of
reviewers comments prepared prior to the
meeting.
56Action
- Deferral
- The study section cannot make a recommendation
without additional information. This information
may be obtained by a project site visit or by
submission of additional material by the
applicant.
57Summary Statement
- Once applications are reviewed, the results are
documented by the SRA in a summary statement and
forwarded to the Institute or Center (and the PI)
where a funding decision is made. - The summary statement contains
- Overall Resume and Summary of Review Discussion
- Essentially Unedited Critiques
- Priority Score and Percentile Ranking
- Budget Recommendations
- Administrative Notes
58Priority Scores/Percentile Rank
- For each study section, applications in the upper
half generally are scored from 1.0-3.0, with 1.0
the best score. Scores as low as 5.0 are
possible. - Individual scores are averaged and multiplied by
100 to give the final priority score
- Percentile ranking is calculated based on
results of current
59Dual Review System for Grant Applications
- First Level of Review
- Scientific Review Group (SRG)
- Provides Initial Scientific Merit
- Review of Grant Applications
- Rates Applications and Makes
Recommendations for Appropriate Level of Support
and Duration of Award
- Second Level of Review
- Council
- Assesses Quality of SRG
- Review of Grant Applications
- Makes Recommendation to
- Institute Staff on Funding
- Evaluates Program Priorities
- and Relevance
- Advises on Policy
60What Determines Which Awards Are Made?
- Scientific merit
- Program considerations
- Availability of funds
61Topics
- Overview of NIH, AHA Grant Review/Selection
process - What you need to do before sending in the grant
- Carefully select the agency examine success
rates - Tailor title, selling points to the review group
- Recommendations on schedule for preparation of
application - Details of the selection process
- Action Items after the letter of evaluation
- Respond as if your life depended on it
- Resubmission
- Was the evaluation fatal or did they give you
recommendations
62On Receipt of Letter(s) from a Funding Agency
- Answer with 5 working days or whenever the
response is due - If you cannot meet the letters deadline, call
the SRA - Call the SRA if you do not understand the
letters comments or requests - If you were triaged, carefully examine the pink
sheets as soon as you receive them - Above all, do not give up!
63Resubmission
- Contact the SRA about your score
- Talk to your department chairman about continued
support - Carefully address each comment in the review
- Do not resubmit prematurely
- Do not delay too long the makeup of the study
section may change - Second Resubmission often necessary
64 Inside the NIH Grant Review Process Video
- CSR has developed a video of a mock study section
meeting to show how NIH grant applications are
reviewed.
http//www.csr.nih.gov/video/video.asp
65Important Components of an Application
- Title and abstract
- Use to direct application to specific study
section - Abstract should be an abbreviated specific aims
page - Specific aims page
- Most important page. Sell your ideas on this
page ! - Address significance, innovation, and approach
- Provide clear overall hypothesis and specific
aims - Provide a summary of key preliminary data to
support rationale and feasibility of proposed
experiments
66Important Components of an Application (cont.)
- Background and Rationale
- Use this section to show significance of work
- Reiterate central hypothesis
- State unanswered questions and indicate how each
- specific aim will attempt to answer
questions - Preliminary data
- Provide key preliminary for each specific aim
- Use a s many figures as possible
67Important Components of an Application (cont.)
- Experimental Design
- Link each experiment to a specific aim
- Provide rationale for each experiment
- Provide clear experimental protocol for each
question (italics) - Provide expected results, interpretation, and
future directions, - and experimental limitations section
68CSR Web Site http//www.csr.nih.gov
- News and Events
- Resources for Applicants
- Study Section Information
- Employment Opportunities
- Contact Information
69NIH Commons (commons.era.nih.gov)
- a virtual place to conduct business with NIH
- makes grant management much more efficient
- the place where you will send your grants
-
- see your scores within three days of the meetings
- view your summary statements on the day they are
released - receive your award notices
- site you log in to as a reviewer to post your
critiques
70Challenges for Funding
- Money pool is shrinking
- Large number of applicants
- Lower probability of success
- Recommendations
- Do not submit prematurely
- Write (and rewrite)10 your application
- Ask for help from senior faculty (do not give
them the proposal the night before it is due)
71FY2006 budget request will, for the first time in
24 years, not keep pace with inflation. Genomics
and Protenomics, 2005
72AAAS Analysis as quoted in Genomics and
Protenomics, 2005
73Source NIH agency budget justification, FY 2006
74(No Transcript)
75NIH Grant Application Tips
- Plan ahead by writing specific aims page well in
advance of - submission . Identify key preliminary data for
application - Identify potential study sections for review.
Know who your potential reviewers! Try to direct
your grant to appropriate SS. - Know the review criteria for grant application.
Clearly address each criteria in the application
76NIH Grant Application Tips contd
- Seek advice from UMC colleagues or external
experts - Have a positive attitude about writing grants. It
is an opportunity for you to organize your
thoughts and ideas in a logical fashion. It also
a 5 year blueprint for laboratory experiments - Do not be discouraged by rejection! Be persistent
and respond to criticism in positive manner
77You miss 100 of the shots you never take.
Remember!!