Update on the CTSA Evaluation Steering Committee

1
Update on the CTSA Evaluation Steering Committee
Harold Pincus, Columbia Chair, CTSA Evaluation
Steering Committee
2
Purpose

• Coordinates institutional evaluation efforts with
  the National CTSA Consortium Evaluation.
• Provides a forum to exchange information about
  institutional evaluation approaches, challenges,
  and progress.

3
Institutional Evaluation WorkgroupsMet February
7 2008

• Definitions and Measurement (Chair B. Trochim,
  Cornell)
• Identify, prioritize, analyze potential
  definitional constructs (i.e., translational,
  disciplinarity, collaboration, etc)
• Synthesize definitions and measurement approaches
• Prepare evaluation briefs/whitepapers that
  summarize sound measurement approaches, tools,
  and methods
• Work with other workgroups, committees as
  appropriate
• Social Network Analysis (Chair J. Rainwater, UC
  Davis)
• Making relationships visible collaboration
  between informatics and evaluation
• S. Lurie (Rochester) presented SNA example of
  relationships among key functions
• S. Johnson (Columbia) showed SNA example of
  co-authorship /betweenness among obesity
  investigators

4
Institutional Evaluation WorkgroupsMet February
7 2008

• Shared Resources (Chair J. Segal, Johns
  Hopkins)
• Identify resources which can be shared
• Provide a mechanism to share
• Create a community of committed individuals
  willing to share instruments as well as processes
• IRB (Chair TBD)
• New workgroup to form as outcome from Feb 8
  meeting (see next slide)

5
Round Robin SessionsHeld February 8 2008

• IRB Issues and Evaluation (Facilitator P.
  Moberg, Wisconsin)
• Consider possibilities to harmonize across
  institutions
• Explore discussions with other CTSA Steering
  Committees (e.g., Alternate IRB models,
  Regulatory, Clinical Research Ethics)
• Identify ways to achieve greater transparency
  through
• Set of tools to educate IRBs about evaluation
• Example of Key Features that an IRB would review
  for exemption
• Convene a subset of CTSA evaluators to engage
  with American Evaluation Association

6
Round Robin SessionsHeld February 8 2008

• The Role of the Evaluator (Facilitator D. Rubio,
  U Pittsburgh)
• Discussed different evaluation approaches
  underway core performance v. program
  evaluation external v. internal consultants
• Identified challenges how best to capture real
  change/transformation how do you evaluate
  leadership and to whom and how are results
  reported
• Consider what role the evaluator has or should
  have with external advisory board

7
Round Robin SessionsHeld February 8 2008

• Evaluating Indirect Benefits of the CTSA
  (Facilitator A. Dozier, Rochester)
• Consider economic impact
• Ability to draw different skill sets into C T
  research
• Assess nature of new partnerships and ability to
  attract unexpected collaborators
• New ways to engage communities
• Assess cultural changes
• Generate examples of indirect effects and post on
  Wiki (i.e., Rochester economic impact)

8
Next Steps

• Convene an April web conference of full Eval
  Committee
• Cohort 2 to present evaluation plans
• Consider holding a meeting in conjunction with
  the American Evaluation Association Annual
  Meeting in November
• Encourage Workgroup chairs to continue momentum
  Set up IRB Workgroup
• Encourage members to suggest web conferences on
  specific topics
• Create Wiki space to provide updates on the CTSA
  National Evaluation