National CTSA Consortium Evaluation:

1
National Center forResearch Resources
NATIONAL INSTITUTES OF HEALTH
A c c e l e r a t i n g a n d e n h a n c i
n g r e s e a r c h f r o m b a s i c d i
s c o v e r y t o i m p r o v e d p a t I e
n t c a r e
National CTSA Consortium Evaluation Progress
and Next Steps Lori Mulligan Director Science
Policy Public Liaison NCRR February 8, 2008
2
Evaluating the Consortium Multiple Inputs
CTSA Oversight Steering Committees (i.e.,
Informatics, Translation, Education, etc.)
External Advisors Stakeholders
National CTSA Consortium Evaluation
CTSA Evaluation Steering Committee
Professional Evaluators
3
Evaluating the CTSA Consortium

• Success toward fulfilling priorities, milestones,
  and goals set by the Consortium and RFA
• Effectiveness, adaptability, and responsiveness
• Effectiveness of methods used to foster
  collaboration
• Ability to develop, share, adopt and disseminate
  policies, practices, tools, curricula, etc.
  Ability to serve as a model for other programs.
• Demonstrate significance/impact of Consortium
  model to improve health (i.e., Return on
  Investment)

4
Proposed Design for National CTSA Evaluation Six
Sub-Studies
5
CTSA Evaluation Phases

• Phase I National CTSA Evaluation Feasibility
  Study
• Phase II National CTSA Consortium Evaluation
  Study - will refine the design and begin the
  process evaluation of the Consortium.
• Phase III National CTSA Consortium Outcomes
  Study

6
Discussion

• What do we expect to achieve by working together
  as a national consortium that couldnt be done
  alone?
• How do we best capture and demonstrate the
  progress, accomplishments, and uniqueness of the
  Consortium?

7

• The End

8
Evaluating the CTSA Program

• Institutional Local Evaluations
• Consortium National Evaluation

9
National Evaluation Goals of the CTSA Program

• Institution
• Scientist
• Consortium

• Assess the effectiveness of the CTSA to integrate
  clinical and translational science at the
  Institution
• Assess the recruitment, retention, and career
  advancement of clinical and translational science
  researchers
• Assess the effectiveness of an integrated
  national CTSA Consortium

10
National Evaluation Goals of the CTSA Program

• Consortium (continued)
• Science

• Assess the effectiveness of the National
  Institutes of Health as a partner with the CTSA
  Consortium
• Assess the acceleration of the translation of
  basic research into clinical studies
• Assess the acceleration of clinical studies into
  community practice

11
National Evaluation Goals of the CTSA Program

• Health Practice
• Community

• Assess the dissemination of new methods, tools,
  and resources into medical practice
• Assess the impact of the CTSA Consortium on
  improving public knowledge and involvement in
  clinical and translational research
• Assess the impact of the CTSA program on
  improving health

12
(No Transcript)
13
Institutional Creating an academic home for CT
Science

• Recruit, train and retain thru new curricula,
  advanced degrees, and mentoring
• Improve institutional data sharing
• Increase use and efficiency of cores/shared
  facilities
• Attract and engage new partners
• Increase community participation in clinical
  research
• Increase multidisciplinary C T research
  collaborations
• Increase number of CTS discoveries, patents,
  licenses, and publications

14
INPUTS
ACTIVITIES
OUTPUT INDICATORS
SHORT-TERM OUTCOMES (13 YEARS)
INTERMEDIATE OUTCOMES (47 YEARS)
LONG-TERM OUTCOMES (810 YEARS)
NCRR CTSA Award

• Institutional
• Recruit, Train, Retain the Next Generation of
  Translational Scientists
• Promote Trans-disciplinary Collaboration
• Build or Enhance the Research Infrastructure
• Attract and Engage New Partners
• Streamline internal processes
• Consortium
• Build the national Consortium
• Promote sharing of ideas, curricula, resources,
  data, products, and systems
• Create networked environment to expedite
  clinical and translational research
• Federal
• Support for and participation in National
  Consortium Activities
• Funding of Specialized Projects (Informatics,
  etc.)
• Changes in Policies and Procedures (Review
  Process, Administrative and Fiscal Policies,
  etc.)
• Attract and Engage New National Partners (FDA,
  Private Foundations, etc.)
• Trans-NIH Initiatives

• Changes in Trainees, Scholars, Mentors, and
  Investigators
• Changes in Amount of Transdisciplinary Research
  
• Changes to Curriculum Programs
• Changes in Organization and Sharing of Data
• Leverage funding from partners
• Increased Engagement of Community-Based
  Physicians in Clinical Studies
• Improved Engagement, Recruitment, and Retention
  of Human Subjects in CTSA Research Projects
• Identification of Barriers to Cooperation
  Collaboration in Adult Pediatric Communities
• Growth of New Public/Private Partnerships
• Development of Formal and Informal
  Collaborations
• Sharing of Practices Policies through the
  Consortium
• Evidence of IC-funded Projects Utilizing and
  Benefiting from CTSA Resources
• Increase in Number of Clinical Studies
  Developed from Basic Research
• Increase in the Number of Clinical Studies
  Conducted in Community Settings
• Creation of Mechanisms for Communicating
  Community Research Findings and Priorities to
  Basic Scientists
• Dissemination of Research Tools, Informatics,
  Findings, Guidelines to Targeted Groups
• Evidence of NIH Responsiveness to Emerging
  Needs of Consortium
• Effectiveness of NIH as a Partner in the CTSA
  National Consortium

Increase in the Number of Clinical and
Translational Scientists Conducting Clinical and
Translational Research in the US
Increase career advancement and retention of CTS
researchers and increase number of researchers
with advanced degrees in CTS

• Clinical and Translational Resources
• Pilot Collaborative Studies Mechanisms
• Biomedical Informatics
• Research Design, Biostatistics Ethics
• Regulatory Knowledge Support
• Patient Clinical Interactions Resources
• Community Relationships Attitudes
• Translational Technologies Resources
• Research, Education, Training, and Career
  Development

Increased Integration and Collaboration within
and across CTSA Institutions
Increased efficiency for developing and sharing
tools and research methodologies
Decreases in administrative, legal, regulatory
and ethical barriers and interruptions in CT
research
Perceptions of Various Institutional Stakeholder
Groups with Scope, Pace Extent of Change
Improved Public Knowledge of, Support for, and
Participation in Clinical and Translational
Research
Development of an Integrated National CTSA
Consortium
Accelerated Translation of Basic Research into
Clinical Studies
Improved Speed Efficiency of Translation of
Clinical Studies Into Community Practice Settings
Improved Translation of Feedback from Community
Research to Basic Research

• Existing University Infrastructure Resources

Accelerated Dissemination of New Methods, Tools,
Resources into Medical Practice
Establish data sharing networks
New Products, Drugs, and Medical Devices Brought
to Market
Societal Return in Investment from NIH Funding of
the CTSA Consortium
15
Draft Conceptual Framework of the

National CTSA Consortium
16
National CTSA Consortium Working Together to
Transform C T Science Across the Nation

• More effective and efficient clinical research
  management
• Decrease in administrative, legal, regulatory and
  ethical barriers
• Develop and share practices, policies, tools,
  curricula, etc., within and beyond the Consortium
  
• Interact with NIH-funded networks and multi-site
  programs
• Serve as model for non-CTSA institutions to adopt
  similar approach
• Demonstrate societal return on investment

17
Evaluating the ConsortiumProcess

• Analyze Consortium activities through
• Strategic Plan
• Interviews with PIs and Committee Chairs
• Workshop outcomes
• Meeting summaries
• CTSAweb.org and CTSA Wiki
• Aggregated information from annual progress
  reports
• Track External Advisory Board assessments of
  Consortium contributions, interactions and
  collaborations
• Interview internal and external stakeholders to
  ascertain perceptions of the Consortiums
  effectiveness as well as their involvement