How to Be Successful in the NIH Peer Review Process for Grants

1
How to Be Successful in the NIH Peer Review
Process for Grants

• Olivia Bartlett, Ph.D.
• Chief, Research Programs Review Branch
• National Cancer Institute
• (301-496-7929, op2t_at_nih.gov)

2
(No Transcript)
3
(No Transcript)
4
Total NIH Budget
5
NIH Extramural Budget
6
NIH Extramural Award Mechanisms
7
NIH Solicitations for Applications

• Announcements in the NIH Guide for Grants and
  Contracts indicate new or ongoing interest of one
  or more NIH I/Cs in supporting research,
  training, resources in a field
• Program Announcement (PA)
• Addresses a relatively broad field/category of
  research
• Usually no set-aside I/C budget
• Usually submit on regular receipt dates
• Regular review criteria for type of applications
• Request for Applications (RFA)
• Addresses a well defined area of research
• Set-aside I/C budget for RFA applications
• Submit on special, one time only receipt dates
• Often special eligibility and/or review criteria
• Often special application format and/or
  submission instructions

8
Responsibilities of NIH Extramural Staff

• Scientific Review Administrator (SRA)
• In Center for Scientific Review and in each
  Institutes Scientific Review Office
• Organizes, manages, conducts, reports scientific
  peer review of grant applications and/or contract
  proposals
• Liaison between applicants and reviewers
• Program Officer/Director
• In NIH Institutes
• Manages a portfolio of awarded grants/contracts
• Monitors scientific progress made on
  grants/contracts
• Grants/Contracts Management Officer
• Fiscal stewardship of portfolio of awarded
  grants/contracts
• Negotiates fiscal aspects of awards
• Monitors financial progress made on
  grants/contracts

9
Review Process for a Grant Application
National Institutes of Health
Institution
Principal Investigator
Center for Scientific Review
Assign to IC IRG/Study Section
Initiates Research Idea
Submits Application

Study Section
Review for Scientific Merit
Institute
Evaluate for Relevance
Allocates Funds
Advisory Councils and Boards
Conducts Research
Recommend Action
Institute Director
Takes final action
10
Dual Review System for Grant Applications

• First Level of Review
• Scientific Review Group (SRG)
• Provides initial Scientific Merit
• Review of grant applications
• Rates applications and makes
  recommendations for appropriate level of support
  and duration of award

• Second Level of Review
• Advisory Council/Board
• Assesses quality of SRG
• Review of grant applications
• Makes recommendation to
• Institute staff on funding
• Evaluates program priorities
• and relevance
• Advises on policy

11
NIH Grant Receipt, Review, and Award Schedule

• Jan-May
• May-Sept Receipt Dates
• Sept-Jan
• June-July
• Oct-Nov Review Meetings
• Feb-Mar
• Sept-Oct
• Jan-Feb National Advisory Council Board Dates
• May-June
• Dec 1
• Apr 1 Earliest Possible Beginning Date
• July 1

12
Applications Submitted to NIH

• More than 73,000 grant applications were
  submitted to NIH in FY 2004
• All are received and processed by the NIH Center
  for Scientific Review (CSR) Application Receipt
  Room
• Make sure your application is complete and
  correct when submitted!

13
NIH Center for Scientific Review

• Serves as central receipt point for Grant
  Applications for NIH and other DHHS components
• Assigns applications to CSR Integrated Review
  Groups/Scientific Review Groups (Study
  Sections) or to Institute Scientific Review
  Groups
• Assigns applications to NIH Institute(s) as
  potential funding component(s)
• Manages more than 200 standing Study Sections and
  continuing Special Emphasis Panels for the
  initial scientific merit review of most research
  applications submitted to the NIH

14
Review Panels (Study Sections) in the NIH Center
for Scientific Review are Grouped by Topic
Division of Molecular and Cellular Mechanisms
Division of Physiological Systems
Division of Clinical and Population-based Studies
Social Sciences, Nursing, Epidemiology Methods
Immunological Sciences
Biochemical Sciences
Infectious Diseases and Microbiology
Integrative, Functional, and Cognitive
Neuroscience
Risk, Prevention and Health Behavior
Cell Development Function
AIDS and Related Research
Behavioral and Biobehavioral Processes
Genetic Sciences
Cardiovascular Sciences
Surgery, Radiology and Bioengineering
Oncological Sciences
Biophysical Chemical Sciences
Molecular, Cellular and Developmental Neuroscience
Brain Disorders and Clinical Neuroscience
Nutritional and Metabolic Sciences
Endocrinology Reproductive Sciences
15
How is Your Application Assigned within NIH?

• Based on specific written guidelines
• To Study Sections based on
• Topics addressed in the application
• Areas of expertise in the Study Section
• To awarding Institute/Center based on
• Overall mission of the Institute/Center
• Specific programmatic mandates and interests of
  the Institute

16
Who will Handle Review of Your Application within
NIH?

• Center for Scientific Review Unsolicited
  Applications
• Research Project grants (R01)
• Fellowships (F32/33)
• Pilot Studies (R21)
• Small Grants (R03)
• SBIR (R43/44)
• Program Projects for some I/Cs
• Applications for simple PAs

• Institute Review Offices Solicited and I/C
  Mission-targeted Applications
• Training Grants (T32)
• Career Awards (Ks)
• Program Projects (P01)
• Centers (P20/30/50)
• Cooperative Agreements
• Multi-institutional clinical trials
• Applications for RFAs and complex PAs
• Contract Proposals for RFPs

17
NIH Review Criteria for Research Project Grants

• Review Criteria
• 1. Significance
• 2. Approach
• 3. Innovation
• 4. Investigator
• 5. Environment
• Types of Applications
• Unsolicited research project grants (R01s),
  small grants (R03s), pilot project grants (R21s),
  program projects (P01s)
• Most R01s, R03s, R21s, P01s for RFAs

18
Updated Review Criteria for Research Project
Grants Now in Effect

• Significance Does this study address an
  important problem? If the aims of the application
  are achieved, how will scientific knowledge or
  clinical practice be advanced? What will be the
  effect of these studies on the concepts, methods,
  technologies, treatments, services or preventive
  interventions that drive this field?
• Approach Are the conceptual or clinical
  framework, design, methods, and analyses
  adequately developed, well-integrated,
  well-reasoned and appropriate to the aims of the
  project? Does the applicant acknowledge
  potential problem areas and consider alternative
  tactics?
• Innovation Is the project original and
  innovative? For example Does the project
  challenge existing paradigms or clinical
  practice, address an innovative hypothesis or
  critical barrier to progress in the field? Does
  the project develop or employ novel concepts,
  approaches, methodologies, tools or technologies
  for this area?

19
Updated Review Criteria for Research Project
Grants Now in Effect

• Investigator Is the investigator appropriately
  trained and well suited to carry out this work?
  Is the work proposed appropriate to the
  experience level of the principal investigator
  and other researchers? Does the investigative
  team bring complementary and integrated expertise
  to the project (if applicable)?
• Environment Does the scientific environment in
  which the work will be done contribute to the
  probability of success? Do the proposed studies
  benefit from unique features of the scientific
  environment or subject populations, or employ
  useful collaborative arrangements? Is there
  evidence of institutional support?

20
Review CriteriaOther Considerations

• Human Subjects Protection
• Data and Safety Monitoring Plan (for ALL clinical
  trials)
• Plans for Inclusion of Women, Minorities and
  Children in Clinical Research
• Animal Welfare Protection
• Any RFA-specific criteria, if applicable
• Appropriateness of the Budget

21
Review CriteriaPriority Score

• A single global score is assigned by each review
  committee member not in conflict for each scored
  application.
• The score is to reflect the overall impact that
  the project could have on the field.
• The emphasis on each review criterion may vary
  from one application to another, depending on the
  nature of the application and its relative
  strengths. An application does not need to be
  strong in all criteria to receive a high priority
  score.

22
Initial Review Group Actions

• Not Scored (UN)
• Application not in top half of all applications
• Not Recommended for Further Consideration (NRFC)
• Lacks significant and substantial merit or
  serious ethical problems in Human Subject or
  Animal use
• Deferred
• Review Committee needs more information to decide
  on the scientific merit of the application
• Scientific Merit Rating (Priority Score) Assigned
• 1.0 (best) to 5.0 (worst)
• Target a mean score of 3.0 for all applications

23
What Determines Which Applications are Awarded?

• Scientific merit, as indicated by priority score
  and/or percentile ranking
• Each NIH Institute/Center sets its own paylines
• Programmatic considerations of the awarding NIH
  Institute/Center
• Balance of models, geographic sites, approaches,
  etc in portfolio
• Availability of funds
• Funds for competing grant awards limited most
  of IC budget for non-competing continuation
  grants/programs
• Doubling of NIH budget 1998 2003
• Essentially flat budget in FY 2005 and beyond
  means tighter paylines for all ICs

24
Modular Research Grants

• For most grant applications requesting up to
  250,000
• Request budget in modules of 25,000 each
• Justify the requested level of support based on
  overall requirements, scientific aims, and scope
  of project
• No automatic cost of living increases in future
  years, but may justify increase in of modules
  requested
• Simplified budget justification/narrative
• Follow special instructions for Modular Budgets
  in PHS 398 instructions
• Reviewers may make adjustments in 25,000 modules
• Other Support and additional budget detail will
  be requested after review if an award is likely
• Awards issued without direct cost category
  breakdown

25
Career Development Awards

• K01- Mentored Research Scientist Development
  Award
• Usually for Ph.D.s, for basic research NCI
  Temin Award
• K02 - Independent Scientist Award
• Additional time/effort support for researcher
  with R01
• K05 - Senior Scientist Award
• K07 - Academic Career Award
• K08 - Mentored Clinical Scientist Development
  Award
• For clinicians to get basic/laboratory research
  training
• K12 - Mentored Clinical Scientist Program Award
• K22 - Patient-Oriented Research Transition Awards
• K23 - Mentored Clinical Scientist Development -
  POR
• K24 - Mid-Career Patient-Oriented Research Award
• K30 - Clinical Research Curriculum Development

26
Ongoing Changes in NIH Grants Programs

• Electronic Grant Receipt open to all modular
  research project applications Feb 1, 2005
• Flat budget for FY 06 paylines expected to
  tighten, more revised applications
• Coming soon
• Different application form 424 Form will be
  used for all Federal grant programs
• Multiple Principal Investigators
• Increase in biodefense funding/initiatives
• NIH Roadmap Initiatives

27
NIH Roadmap RFA/PA Initiatives

• Purpose To identify major opportunities and
  gaps in biomedical research that no single
  Institute at NIH could tackle alone but that the
  NIH as a whole must address, to make the biggest
  impact on the progress of medical research
• New Pathways To Discovery
• Building Blocks, Biological Pathways, and
  Networks
• Molecular Libraries Molecular Imaging
• Structural Biology
• Bioinformatics and Computational Biology
• Nanomedicine
• Research Teams Of The Future
• High-Risk Research
• Interdisciplinary Research
• Public-Private Partnerships
• Re-engineering The Clinical Research Enterprise
• Clinical Research Networks/NECTAR
• Clinical Research Policy Analysis and
  Coordination
• Clinical Research Workforce Training
• Dynamic Assessment of Patient-Reported Chronic
  Disease Outcomes
• Translational Research
• NIH Roadmap Website http//nihroadmap.nih.gov/

28
There is no amount of grantsmanship that will
turn a bad idea into a good one..But there
are many ways to disguise a good one.

• Dr. William Raub
• Past Deputy Director, NIH

29
Choosing Your Research Project

• What Makes a Research Project Outstanding,
  Singling it Out From All Others Under
  Consideration?
• Has the potential to lead to seminal observations
• Leads to new ways of thinking
• Lays the foundation for further research in the
  field
• Clearly defines the importance of the research
  problem
• Has only two to three interrelated specific aims
  and links all parts of the application
• Addresses a difficult problem in a way that seems
  simple in retrospect, leaving the reviewers to
  wonder why they didn't think of the idea first
• Is written at a level understandable by all
  reviewers

30
Preparing to Write a Grant Application

• Critically Assess Yourself
• Do you have the necessary expertise, resources,
  personnel and preliminary data to be competitive?
• Assess the Competition
• Who are the important contributors to the field?
  (remember, they might end up being your
  reviewers)
• What have competitors accomplished? What have you
  accomplished? How are you going to take what's
  been done a step further?
• Assess the Potential for Your Idea
• What's already been done/reported/funded in your
  area? What are the gaps?
• Search the literature and the database of funded
  grants in the field, e.g., NIH CRISP (Computer
  Retrieval of Information on Scientific Progress)
  system

31
Psychology of the Review Process

• Reviewers are
• Over committed, over worked and tired
• Inherently skeptical and critical
• Informed strangers
• A happy reviewer is likely to be a more positive
  one, so make their job easier
• Flow diagrams, charts, figures
• Well organized, clearly written application
• Avoid things that irritate reviewers
• Not following instructions ie, exceeding the
  page limits, font
  too small, putting information in the wrong
  section, omitting or mislabeling
  references/figures
• Spelling, grammar, and math errors, etc.

32
Advice for Preparing an Application

• Read instructions thoroughly and follow them
• Never assume the reviewers will know what you
  mean
• Refer to literature thoroughly and thoughtfully
• Explicitly state the rationale of the proposed
  studies
• Include well-designed, easy to follow tables and
  figures
• Include flow diagrams for overview, and for
  complex experiments and protocols
• Address priorities if patients, reagents or
  resources will be limited
• Include data analysis/interpretation plans and
  methods
• Involve the statistician EARLY in project design

33
Key Features of Successful Applications

• Hypothesis
• A meaningful hypothesis AND a means of testing it
• A sound rationale for the hypothesis
• Preliminary Data
• Shows proper training for the research proposed
  and the ability to interpret results
• Documents feasibility of the proposed project
• Include alternative interpretations of results
  and address limitations of methods
• Well Organized Research Plan
• Aims focused, not diffuse, and related to each
  other and the hypothesis
• Rationale for methods proposed, with alternatives
  addressed
• Research flow and priorities clearly indicated
• Sufficient experimental detail to show you
  understand methods
• Emphasize MECHANISM - avoid descriptive data
  gathering

34
In God We Trust.All Others Must Bring Data.
35
Key Features of Successful Applications, cont

• Biosketches
• Indicate your qualifications to carry out the
  work proposed
• Dont pad with lots of in preparation
  manuscripts
• Add a senior collaborator, if needed, to provide
  expertise you lack
• Literature Cited
• Be thorough, but critical, in citing previous
  work in the field
• Description
• Most read part of the application
• Basis for referral to study section and funding
  Institute/Center
• Write it last, after the Research Plan is
  finished
• State problem, specific aims, types of methods to
  be used
• Letters of Collaboration
• Should be strong and definitively state what will
  be provided

36
Most Common Reasons for Unscored or Not
Recommended for Further Consideration

• Lack of new or original ideas
• Diffuse, superficial or unfocused research plan
• Lack of appreciation of published relevant work
• Lack of experience in essential methods
• Questionable reasoning in experimental approach
• Lack of sound rationale for hypothesis or methods
• Aims dont address hypothesis
• Unrealistically large amount of work proposed
• Lack of sufficient experimental detail
• Uncertainty about future directions of work
• Serious concerns about risks to human subjects or
  use of animals

37
Tips for More Successful Grantsmanship

• Know your field -- network, collaborate, avoid
  isolation
• Know the NIH program officer(s) in your field
• Information about upcoming initiatives,
  opportunities, gap areas
• Information about potential collaborators, NIH
  resources
• Explain NIH policies, procedures, award
  mechanisms, eligibility requirements
• Advice in revising unfundable applications
• Know the Peer Review System and your SRA
• Review criteria and receipt/review schedules
• Explain NIH policies, procedures, award
  mechanisms, eligibility requirements
• Problems with referral or review
• Use the NIH and other websites to get latest
  information, forms, policies
• Prepare applications well before the deadline
• Follow instructions and proofread carefully
• Get honest feedback from seasoned reviewers, and
  revise accordingly

38
Selected NIH Web Sites of Interest

• National Institutes of Health (http//www.nih.gov)
  
• NIH Office of Extramural Research homepage, with
  links to the NIH Guide, application forms, grants
  policy information, and resources for new
  investigators http//grants1.nih.gov/grants/oer.
  htm
• Overview of NIH Extramural Research, with links
  to tools and FAQs http//grants1.nih.gov/grants/w
  elcome.htmintroduction
• Career Development Awards Information
  http//grants1.nih.gov/training/careerdevelopmenta
  wards.htm
• Links to all NIH Institute and Center homepages
  http//www.nih.gov/icd/
• NIH Center for Scientific Review
  (http//www.csr.nih.gov)
• Has links to Resources for Applicants, standing
  Study Section rosters, policy information, review
  procedures and review criteria, and advice for
  investigators submitting clinical research
  applications