AASCU

1
The Center for Scientific Review
March 3, 2007
Michael A. Steinmetz, PhD
Scientific Review Administrator and Referral
Officer Center for Scientific Review
2
Review is a Dual Process

• Scientific Evaluation
• Scientific Review Groups (SRGs) evaluate
  scientific merit and assign priority scores
• CSR reviews 70 of all applications
• Review divisions of funding institutes review the
  rest
• Advisory Councils at funding institutes
• May concur with priority score and recommend
  funding
• Provide special consideration of proposal that
  address high program priorities
• Consider investigator appeals

3

• CSR now receives most types of NIH grant
  applications electronically via Grants.Gov
• eRA Commons is a web-based system for secure
  information exchange with applicants and
  applicant organizations
• Applicants must establish personal commons
  accounts to track review progress and to retrieve
  scores and summary statements
• http//www.grants.gov/
• http//commons.era.nih.gov/

4
Receipt and Referral

• Referral officers assign applications to a
  Scientific Review Group for review
• Applications are also assigned to one or more NIH
  institutes for funding consideration
• A cover letter can help direct application toward
  appropriate SRG and institute assignments

5
Scientific Review Groups

• CSR has approximately 230 SRGs (or Study
  Sections) with specialized focus and appropriate
  expertise
• Each SRG consists of about 20 regular members
  plus ad-hoc members as needed
• Each SRG is run by a Scientific Review
  Administrator (SRA)
• Special Emphasis Panels are established as needed
• CSR web pages provide descriptions and rosters
  WWW.CSR.NIH.GOV

6
The Role of the SRA

• Nominates all regular members of the SRG
• Nominates chairman to moderate discussions
• Selects ad-hoc members as needed to insure
  sufficient expertise and adequate manpower
• Organizes SRG meetings
• Assigns three or more reviewers for each
  application
• Assures fairness in review process
• Writes resume of any discussion that takes place
• Generates final summary statements
• Represents review at institute council meetings

7
The Study Section

• Regular members are appointed to 4-year terms
• Chairperson serves a 2-year term
• Review 60-100 applications in each of the 3
  cycles per year
• Each member is assigned 8-12 applications to
  review
• Reviewers must have required expertise
• Members must have a scientific reputation and
  peer-reviewed support
• Panels require gender, minority, and geographical
  balance

8
Pre-meeting Review

• Administrative review of application
• Conflicts of interest are identified
• Same institution
• Mentor/Mentee relationships
• Recent co-author
• Financial gain
• Long-standing scientific disagreement
• Reviewer assignments are made by the SRA
• Protection of confidentiality

9
The Review Meeting

• Internet Assisted Review (eRA commons)
• Streamlining
• Applications unanimously judged to be below the
  40th percentile are not scored or discussed
• Investigators receive critiques provided by the
  assigned reviewers
• Preliminary scores (1.0 - 5.0) provided by the
  assigned reviewers
• Reviewers provide description and individual
  critiques
• Chairperson moderates open discussion
• Final scores from assigned reviewers establish
  scoring range
• Budget recommendations

10
Review Criteria

• Significance
• Approach
• Innovation
• Investigator
• Environment

11
Significance

• Does this study address an important problem? If
  the aims of the application are achieved, how
  will scientific knowledge or clinical practice be
  advanced? What will be the effect of these
  studies on the concepts, methods, technologies,
  treatments, services, or preventive interventions
  that drive this field?

12
Approach

• Are the conceptual or clinical framework, design,
  methods, and analyses adequately developed, well
  integrated, well reasoned, and appropriate to the
  aims of the project? Does the applicant
  acknowledge potential problem areas and consider
  alternative tactics?

13
Innovation

• Is the project original and innovative? For
  example, does the project challenge existing
  paradigms or clinical practice. Does it address
  an innovative hypothesis or critical barrier to
  progress in the field? Does the project develop
  or employ novel concepts, approaches,
  methodologies, tools, or technologies for this
  area?

14
Investigators

• Are the investigators appropriately trained and
  well suited to carry out this work? Is the work
  proposed appropriate to the experience level of
  the principal investigator and other researchers?
  Does the investigative team bring complementary
  and integrated expertise to the project (if
  applicable)?

15
Environment

• Does the scientific environment in which the work
  will be done contribute to the probability of
  success? Do the proposed studies benefit from
  unique features of the scientific environment, or
  subject populations, or employ useful
  collaborative arrangements? Is there evidence of
  institutional support?

16
Other Criteria

• Are there safety issues if studies involve human
  subjects?
• Are there specific plans to insure the
  participation of women, minorities, and children?
• Are there animal welfare issues?

17
Budgets

• Discussed after the application is scored
• SRG may recommend a shorter duration if request
  is not appropriate for the amount of work
  proposed
• SRG may recommend a smaller award if requested
  amount is not adequately justified
• Final award amount and duration is determined by
  program officials at the funding institutes

18
Post-meeting

• Scores from all panel members are multiplied by
  100 and averaged (100 500 range)
• Scores are converted to percentiles using score
  distributions from the current and two previous
  meetings
• Summary statements
• Description provided by applicant
• Resume of discussion written by SRA
• Written critiques from assigned reviewers
• Scores and critiques are made available to the
  investigators (eRA Commons) and to the assigned
  institutes for funding consideration (30 days)

19
Contact Information
Michael A. Steinmetz, PhD Center for Scientific
Review National Institutes of Health (301)
435-1247 Steinmem_at_csr.nih.gov WWW.CSR.NIH.GOV