Commercial Influence and LearnerPerceived Bias in CME

1
Commercial Influence and Learner-Perceived Bias
in CME

• Michael Steinman MD
• Christy Boscardin PhD
• Leslie Aguayo CCMEP
• Robert B. Baron MD MS

Society for Academic Continuing Medical
Education April 2009
2
Disclosure

• No conflict of interest

3
Background

• Commercial support accounts for well over half of
  funding for CME in academic medical centers (AMC)
• AMCs and other accredited CME providers have
  adopted practices to limit the influence of
  commercial support on CME outcomes

4
Background

• Some have developed standardized instruments to
  predict risk in CME
• For example, Barnes, et al have developed a
  rating instrument with good inter-rater
  reliability
• No published data on relationship between CME
  activity characteristics measured prospectively
  and actual or perceived bias in real-world CME.

5
Background

• Modeled after Barnes et al and local expert
  opinion, UCSF developed a risk index for use in
  peer review of proposed CME activities.
• High and intermediate risk activities received
  more intensive review by the UCSF CME Governing
  Board.
• UCSF also began to evaluate all CME activities
  for learner-perceived bias.

6
Research Questions

• How common is learner-perceived bias in live
  directly-sponsored CME activities?
• Could a prospective risk index predict
  learner-perceived bias in CME activities?
• Do other CME activity characteristics predict
  learner-perceived bias in CME activities?

7
Methods Data Collection

• 228 directly sponsored live CME activities
  sponsored by UCSF, 2005-2007
• Each activity evaluated by CME staff using risk
  index, assessing activitys potential for
  commercial bias--rated high, intermediate, or low
  risk
• Activities rated by staff to have unacceptable
  risk of commercial influence excluded by staff
  review. Examples include activities with
• a single commercial sponsor
• no fees for learners i.e. free CME no
  UCSF faculty leadership in activity

8
UCSF Risk Score

• First time activity
• First time activity chair
• First time event planner
• Third party event planner
• Initial contact with UCSF CME made by non-UCSF
  faculty or staff
• Co-sponsored activity
• Activity chair with industry conflict of
  interest
• More than one non-UCSF faculty
• Commercial support greater than 20 of course
  budget
• Only two commercial supporters
• Registration fee below market value
  (lt25/credit)
• High honoraria or chairs fee
• Projected surplus greater than 20 of expenses

9
UCSF Risk Score

• Each item receives one point
• 0-2 points lowest risk
• 3-5 points intermediate risk
• 6 points highest risk
• Note jointly sponsored activities, all highest
  risk

10
Methods Data Collection

• Activities thought by staff to have unacceptable
  risk of commercial influence excluded by staff
  review. Examples include activities with
• a single commercial sponsor
• no fees for learners i.e. free CME no
  UCSF faculty chair
• Remaining applications submitted for peer review
  by faculty members of CME Governing Board.
• Activities then rejected, approved, approved with
  audit. (Approved with audit activities required
  additional measures to mitigate risk)

11
Methods Data Collection

• Evaluation
• was activity free of commercial bias
  (yes/no)
• overall rating of course (5-point Likert
  scale)
• Perceived bias data not available for 15
  activities
• these 15 had similar distribution of risk
  characteristics and overall quality scores
• but were smaller (75 participants v. 132
• p .02)
• Final analytic cohort 213 activities

12
Methods Data Analysis

• Outcome variables
• Per cent of respondents who perceived activity
  was free of commercial bias
• Overall course quality
• Predictor variables
• UCSF risk score (high, intermediate, low)
• Each activity characteristic used to create
  risk score

13
Methods Data Analysis

• Analysis
• Binomial regression (using binreg command in
  Stata 10.0)
• Weighted each activity in proportion to number
  of respondents
• Regression model of all predictor variables

14
Results Characteristics of 213 Live CME
Activities

• Characteristic ()
• Risk score
• Low 80
• Intermediate 17
• High 3
• Commercial support
• None 34
• 1-19 23
• 20-49 21
• gt50 22
• First time activity 10

15
Results Perceived Bias and Overall Quality of
213 Live CME Activities

• Characteristic ()
• Response rate to evaluation 56 (15)
• Free of Commercial bias 97
• gt95 72
• 91-95 23
• lt90 5
• Overall quality of activity 4.4 (0.2)
• Will you change practice? 82 (5)

16
Results Risk Score and Bias

• Association Yes/No p
• Risk score and bias No 0.63
• Commercial support
• and bias No 0.90
• 14 other individual
• risk factors and bias No NS
• First time chair Yes 0.04
• and bias
• Note Inclusion of all factors in a single model
  explained only 3 of the variance in bias

17
Results Risk Score and Overall Quality

• Association Yes/No p
• Risk score and quality No NS
• Commercial support
• and quality No NS
• 14 other individual
• risk factors and quality No NS
• First time chair Yes 0.01
• and quality (4.23 v. 4.4)
• Note Inclusion of all factors in to single
  model explained only 9 of the variance in bias

18
Summary

• In this study of 213 CME activities, organized by
  an academic provider, the vast majority were
  perceived to free of commercial bias.
• Extent of commercial support, a variety of event
  characteristics, and a summative risk index
  were not associated with the perceived bias or
  perceived overall quality.
• Rates of commercial bias were low regardless of
  the presence or absence of perceived risk factors
  for commercial bias.

19
Discussion

• Interpretation 1 CME activities in general are
  free of commercial bias.
• ACCME regulations require disclosure and
  resolution of conflicts unlikely 100 effective
• Experts in drug promotion have documented
  marketing strategies that rely heavily on
  clinician education to boost sales
• Other reports have documented that many MDs
  believe that industry supported CME is biased
• Many commentators have noted that it is
  unlikely that industry would spend 1.2B per year
  on CME if it did not help bottom line

20
Discussion

• Interpretation 2 The UCSF screening and planning
  process successfully rejected activities with
  commercial bias that might have been permitted by
  other providers.
• More rigorous than required by ACCME
• lt15 of speakers report a conflict of interest
• This screening may have weeded out activities at
  higher risk of commercial bias.
• Furthermore, higher risk activities that were
  allowed to proceed were subject to more intense
  oversight and mitigation of risk

21
Discussion

• Interpretation 3 low rates of perceived bias may
  be explained by the insensitivity of the simple
  yes/no question.
• Commercial bias is subtle and a binary
  question may fall to capture the full range of
  learner perceptions.
• Moreover, learner perceptions of bias may not
  correspond to actual bias in CME activities.
• Potential subtle forms of bias in CME
• Curricula design focused on drug/device
  therapies rather than non-drug therapies
• Faculty may devote attention to aspects of
  disease management for which a supporters drug
  (or class of drugs) may be used
• Clinical trial results themselves (and
  guidelines derived from them) may be subject to
  commercial influence

22
Policy Implications

• This study can be interpreted as either
  supporting or contradicting recommendations to
  eliminate commercial support for CME
• On the one hand one might conclude that
  safeguards offered by the UCSF CME program result
  in CME activities with little if any commercial
  bias
• One the other hand our results are consistent
  with the observation that commercial bias is
  difficult to detect and the only way to safeguard
  against industry influence is to eliminate
  industry involvement in CME
• More research, with larger sample sizes and
  more sophisticated measures of bias, is needed to
  systematically investigate the presence and
  impact of bias in CME

23
Limitations

• Single academic institution
• Modest number of activities
• Discrete set of criteria for vetting and
  managing activities
• Binary outcome variable for learner-perceived
  bias
• Only included activities that had passed an
  initial screening process to identify activities
  with unacceptable risk of commercial influence.
  Thus, unable to assess perceived bias in
  activities perceived to be at highest risk

24
Conclusions

• In this study of 213 CME activities, rates of
  learner- perceived bias were very low
• No differences in perceived bias between
  activities with and without potential risk
  factors for commercial bias.
• Rigorous planning and management can effectively
  screen out activities with explicit bias.
• Further research is needed to evaluate the
  presence of subtle forms of commercial bias and
  risk factors to predict these biases in CME.