Randomized Clinical Trials of Webbased Tobacco Cessation Interventions

1
Randomized Clinical Trials of Web-based Tobacco
Cessation Interventions
Workshop on WATIs Toronto - January 2004

• John Noell
• Ed Lichtenstein
• Garth McKay
• Shawn Boles
• Oregon Research Institute
• Lynne Swartz
• Oregon Center for Applied Science

2
Key Points

• RCTs are necessary for establishing the value of
  WATIs
• RCTs of web-based interventions pose special
  problems
• Some lessons learned from previous RCTs

3
Why RCTs???

• RCTs versus
• no evaluation at all
• How can you be sure you are not harming people?
• How can you tell if there is ANY value to what
  you are offering?
• other systematic evaluations
• Only RCTs cancel the noise (by distributing it
  between conditions)

4
R x E x A x I x M Public health value
(Gratuitous ad)

• The potential RE-AIM R is huge! (And, the
  A-I-M prospects are excellent but may wel
  depend on the E.)
• Reach
• Effectiveness
• Adoption
• Implementation
• Maintenance
• (Glasgow, Vogt, Boles 1999)

5
Point 1 Value of RCTs

• To promote the development of (the right) WATIs
  we need to know about their efficacy and
  effectiveness.
• RCTs are critically important for assessing
  efficacy and effectiveness relative to
  alternatives.
• RCTs are needed to discriminate between WATIs.
• (But they may not be the ONLY route to wisdom.)

6
Point 2 Special Challenges

• Recruitment
• Assignment
• Attrition
• Measurement
• Process (esp. exposure)
• Outcomes

7
Recruitment Balancing Internal and External
Validity

• Who can you reach via the net?
• Who do you want to reach?
• If you build it will they come?
• (If so, how quickly?)
• Can you recruit a sample that matches who will
  (or might?) use your site?
• (Can you accept that these might not be the
  people you wish would use it?)

8
A Recruitment Lesson?

• In the one RCT more than 25 of the subjects
  assigned to the treatment arm completed the
  baseline assessment (enough to collect a payment)
  and did not even complete the ensuing
  introduction to the program.
• Who were they? Smokers ready to quit or people
  looking for easy ?

9
Assignment

• Unique assignment issues
• Identity are they who they say??
• Fidelity to condition Do they stay assigned?
• Randomization factors (beyond the usual, what
  about)
• type of net access (esp. speed and always on
  status)
• type of browser
• computer location and configuration
• other?

10
(An assignment side note)

• Does assignment your intervention mean they
  cannot use anything else?
• In both typical clinical trials and the real
  world, people will use other aids.

11
Attrition

• Overall attrition
• Pure web access can be essentially anonymous.
  This may mean a lack of interpersonal commitment
  which could lead to greater attrition.
• E-mail addresses are unstable. Does asking for
  snail mail or phone contact info bias the sample?
  (External validity issue)
• Differential attrition
• Particularly when an outcome is socially
  undesirable (e.g. admitting failure in stopping
  smoking), greater attrition can result and
  anonymity may aggravate this. (Internal validity
  issue)

12
Sessions used (from NRT)

• Lastseen Left ()
• Intro 27.7 92.3 1 4.6 67.7 2 3.1 64.6
  6 1.5 63.1 8 1.5 61.5 9 9.2 52.3 (Quit day)
  10 6.2 46.2 11 3.1 43.1 12 1.5 41.5
  13 4.6 36.9 14 3.1 33.8 15 1.5 32.3
  16 3.1 29.2 21 29.2 ---

13
Process Measurement (Teasing apart medium from
message)

• Measuring exposure to the intervention
• Does time spent at a given page or in a given
  section mean anything?
• What does number of hits at a site or at a page
  really mean?
• Tracking views of elements in dynamically built
  pages poses special challenges versus flat page
  hit counts

14
Process Measures (Part 2)

• Measuring exposure to other websites or other
  materials - esp. for Control subjects (e.g.,
  tracking all Internet use) External validity
  issue
• Via tracking software (Completeness?)
• Via self-report? (Validity?)
• Other?

15
Outcomes Measurement

• Electronic data collection is very convenient (no
  hand entry, no re-coding, internal logic checks,
  etc.), but
• what is the validity of data collected via the
  web?
• Do we need biochemical verification? (Is
  face-to-face collection necessary?)
• Are we ready for component-level analyses?
  Yes!!!!! Uhh, but

16
Unique intervention delivery issues

• Is any consistency in intervention experience
  desirable? If so, how can we achieve it with
• Differing browsers?
• Differing connection speeds?
• Differing configurations?
• (Different choices?)
• Do we need to measure these differences if we (a)
  have large randomly distributed samples, and (b)
  assume that the Internet population will always
  be diverse on (most of) these factors?

17
Sample results from one pilot study and two RCTs

• ORI Quit Smoking Network (browsing-style)
• ORCAS 1-2-3- Smokefree
• (highly structured one-pass)
• ORCAS NRT
• (structured multiple session)

18
Overview of Results

• QSN (n370 all Tx, no Ctrl)
• 90 day quit rate
• Tx - 18 (32 of responders)
• 1-2-3 Smokefree (n351)
• 90 day quit rate
• Tx 12.3 (24.1) Ctrl - 5.0 (8.2)
• NRT (still recruiting)
• 30 day rate (n233) Tx 18.7 Ctrl - 6.6
• 120 day (Responders n149 of 267 56)
• Tx 14.8 (40) Ctrl - 6.7 (12.9)

19
ORI QSN

• Total of 606 subjects in six months
• 72 Female 28 Male
• 81 Caucasian
• At 90 days
• 56 completed assessment
• 15 of e-mail addresses bounced
• 29 No response
• 81 responded by web, 5.5 via e-mail, and 13.5
  by postal mail

20
ORCAS 1-2-3 Smokefree

• Nearly all recruitment via worksites (posters,
  intranet links, etc.)
• Total of 351 subjectsover 12 months
• 52 Female 48 Male83.5 CaucasianAt 90
  days 56 completed assessment 50.9 of Tx
  61.1 of Ctrl

21
ORCAS NRT (a multiple session
highly-structured intervention)

• Nearly all recruitment via worksites (posters,
  intranet links, etc.)
• First 304 cases to hit 30 day assessment
• Male 50.4 female 49.6
• 83.7 Caucasian (6.8 Black)
• At 120 Days
• 56 responded to survey
• 47 of Tx 64 of Control

22
Lessons learned

• The two biggest issues we have faced to date are
  recruitment retention.
• Recruitment via web is slow
• Recruitment via Worksites can work but requires a
  lot of effort,
• and can provide only specific types of
  populations
• THIS CAN LIMIT GENERALIZABILITY!!
• Attrition has tended to be very high.

23
Summary

• We need RCTs.
• It aint easy.
• But, its worth doing anyway.