Evaluation Report

1
Evaluation Report for Biomedical Technology
Research Resources Program May 22, 2007
Thomas F. Budinger M.D., Ph.D.
University of California, Berkeley Lawrence
Berkeley National Laboratory
2
Process of the Evaluation
APRIL 13, 2007
Introduction Barbara M. Alving, M.D. (NCRR
Director) DBT Program and Mission Michael
Marron, Ph.D. (NCRR) Portfolio Today Abraham
Levy, Ph.D. and Amy Swain, Ph.D.
(NCRR) Sustaining the Portfolio Douglas
Sheeley, Sc. D. (NCRR) Core Technology
Development Examples Mark Ellisman, Ph.D.,
University of California San Diego
Collaboration and Service Bruce Tromberg,
Ph.D., University of California
Irvine Training and Dissemination Joel Stiles,
M.D., Ph.D., Carnegie Mellon University Report
Outline and Assignments 3 hours
Committee APRIL 16, 2007 Science Writer Draft
to TFB APRIL 16 MAY 11, 2007 E-Mail
Modifications by the Committee MAY 14, 2007
Final Draft approved by the Committee
3
PANEL COMPOSITION
Thomas F. Budinger, M.D., Ph.D.
(chair) Professor, Univ. of California,
Berkeley Jerome Cox, Sc.D. Senior Professor of
Computer Science, Washington University Gary
Gottlieb, M.D., M.B.A. President, Brigham and
Women's Hospital Professor, Harvard medical
School Ian D. Jardine, Ph.D. Vice-President for
Global RD, Thermo Fisher Scientific Corp. David
T. Kingsbury, Ph.D. Chief Science Officer, Gordon
and Betty Moore Foundation Robert C. Liddington,
Ph.D. Professor, Burnham Institute for Medical
Research Bettie Sue Masters, Ph.D. Professor,
University of Texas Health Science Center-San
Antonio Franklyn G. Prendergast, M.D.,
Ph.D. Professor, Mayo Clinic College of
Medicine Michael W. Vannier, M.D. Professor,
University of Chicago
4
Relevance to NIH Program
5
50 BTRRs in 5 AreasBasic Science
Components that underpin the translational P-41s

• Optics Laser
• Microscopy
• Fluorescence spectroscopy
• In Vivo diagnosis

• Structural Biology
• Synchrotron x-ray technologies
• Molecular level microscopy
• Nuclear Magnetic resonance

• Systems Biology
• Mass spectrometry
• Proteomics
• Glycomics glyco-technology
• Flow cytometry

• Imaging
• Cellular to organism level microscopy
• MRI MRS
• Image-guided therapy

• Information
• Technology
• Genetics
• Systems models
• Molecular dynamics
• Visualization
• Imaging informatics

Current translational topics
6
Role of BTRR in NIH Programs

• The P-41 mechanism, including its five components
  has been a unique tool in the NIH mission.
• Only a few program projects in the past developed
  core technologies (e.g., emission tomography, NMR
  spectroscopy, electron microscopy) and these have
  all but disappeared.
• The P-41 mechanism provides a venue for
  integration of physicists, engineers, biologists,
  chemists and physicians.
• The model of P-41s is seen as that for evolving
  roadmap initiatives. But these and P50s can not
  replace P-41s.

7
Number of BTRR Awards
66
61
59
55
50
44
43
1975
2005
1980
1985
1990
1995
2000
8
BTRR Awards 1962-2006
Total N 188 Ave Length 11.7 yrs
Number of Awards
9
BTRR Program Funding
10
BTRR Collaborators and Users (2006)
5
HI 6
United States 6,171Foreign 793Total 6,964
11
Balance

• The need to provide technology that will enhance
  CTSAs should be met by BTRR of NCRR.
• This might change the balance but should not
  distract from the successful P-41s.
• To lose basic science P41s through attrition to
  enhance translation would harm the future
  evolution of new technologies.

12
New instrumentation measurements on samples and
patients
Massive data in genomics, protenomics and
metabolomics
Computational Informatics
Relations, cause/effect, hypotheses
Clinical Science Progress
Examples

• Obesity Proteonomics / Genomics
• Neurodegeneration Cellular Renewal

13
Does BTRR get the best PIs ?
The panel suspects the following

• Investigators do not understand the philosophy of
  BTRR and are turned away based on the fear of the
  service requirements.
• Investigators, expert in a core technology are
  too junior to launch a full scale P-41.
• Roadmap initiatives have attracted interest from
  would be BTRR investigators

Recommendations

• Verify the above suspicions.
• Investigate methods to better advertise the BTRR
  program through announcements of new directions.
• Institute preprograms or a two phase mechanism
  that is similar to the P20/P50 mechanism of NCI
• Put more emphasis on the R01 initiative

14
Biomedical Technology Award Types

• Biomedical Technology Resource Resources (P41)
• Research Project Grants (R01 R21)
• Resource-related Research Projects (U24-BIRN)
• Specialized Centers (U54-Roadmap Centers)
• SBIR (R43, R44) and STTR (R41, R42) Grants
• Shared Instrumentation Grants (S10)

15
METRICS
FOR QUALITY Whereas, the evaluation panel is
unanimous in their assessment of the outstanding
nature of the program overall, there needs to be
a concerted effort to provide metrics for quality
and significance of contributions beyond the
number of collaborators, publications and other
items required from P-41 directors. These
include ? Patents
? Technology transfers ? Spin-offs and
startups ? Papers and citations,
including uniqueness and citation impact
? Leveraging of institute and foundation
support. ? Success of trainees
? How the program supports clinical and
translational research ? How NCRR
programs align with evaluation of Roadmap
results.
Question NCRR is called on to answer Are
the innovations, trainee records, contributions
of collaborators, and new
technologies disseminated by the BTRR program of
major significance to the mission of NIH
and will they be essential for the future?
16
BTRR Publications
1,943
1,788
1,541
1,533
Number
2003
2004
2005
2006
Journal articles and books
17
External Advisory Committee
BTRR

• Assist Council and NCRR leadership in the
  evaluation of the balance and quality of
  individual P-41s as well as the integration of
  the BTRR program with clinical translational
  science activities.
• Help staff develop metrics for assessment of
  on-going quality.
• Develop methods to encourage BTRR collaborations
  with industry, foundations, federal and state
  agencies.

18
Acknowledgements

• Dr. Michael Marron
• Dr. Kathleen Brennan
• Dr. Francis Horne