Randomized Clinical Trials in Rheumatology: Reporting Safety and Outcome Measures

1
Randomized Clinical Trials in RheumatologyReport
ing Safety and Outcome Measures

• Yusuf Yazici, MD
• Assistant Professor of Medicine,
• NYU School of Medicine
• Director, Seligman Center for Advanced
  Therapeutics
• NYU Hospital for Joint Diseases

2
Overview

• RCT trials are the leading source of data in
  determining treatment choices
• Good source of efficacy data
• Completeness of safety reporting is inadequate
• Methodological concerns
• Reporting selective results
• Errors in reporting
• Focus on
• Time to event
• Standardized incidence ratios (SIR)
• Radiographic data reporting
• Adequate sample size
• False impression of safety
• Disease activity assessment measures

Ioannidis J, Lau J. JAMA 2001285437443.
3
Time to adverse event
4
Time to adverse event

• Problem in reporting AEs in RCT
• Time to event
• Clue to causality
• When to expect AEs in clinical care
• Instantaneous risk might be different for drugs,
  but the cumulative risk will be similar
• Average risk does not reflect the true risk
  faced by patients, especially when decisions are
  being made at the beginning of treatment

5
Risk of adverse effects related to drug exposure
Collet JP, Bolvin JF. In Pharmacoepidemiology,
3rd Edn, Strom BL ed. 2000.
6
TNF inhibitors and lymphomas

• An increased frequency of lymphomas, tuberculosis
  and demyelinating CNS disease has been associated
  TNF inhibitor use
• SEER database
• Based on annual incidence in the population,
  which is expected to be evenly distributed
• If there is some skewing as to time of
  malignancies in the treatment arm, this would
  make comparisons based on an annual incidence
  inaccurate

1. Brown SL, et al. Arthritis Rheum 200231513158.
2. Yazici Y, Yazici H. Arthritis Rheum 2004 50
   S302

7
TNF inhibitors and lymphomas

• TNF inhibitor use and lymphoma development, as
  reported to the Federal Drug Administration
  (FDA)1
• Twenty-six lymphomas
• Fourteen occurred within 2 months of use
• Non-Hodgkins lymphoma (NHL)
• Two-monthly incidence in normal population
  3/100,000
• Ten out of 16 occurred in 2 months2
• Two-monthly NHL occurrence in etanercept users
  10/100,000
• Nearly 3-4 times

1. Brown SL, et al. Arthritis Rheum
20024631513158 2. Yazici H. Arthritis Rheum
2003482389.
8
Demyelinating central nervous system disease

• Tumor necrosis factor-a antagonist use and
  demyelinating CNS disease
• Nine cases among 77,152 users of etanercept
  during a 19-month period
• They noted that this frequency was not different
  from that found in the general population
  46/100,000 per annum
• Half of the cases occurred within 4 months of
  etanercept use
• Four-monthly incidence risk of etanercept use of
  5.8/100,000
• Four-monthly risk in the general population on
  the other hand would be 2.0/100,000, if the upper
  limit, 6.0/100,000, quoted for the general
  population is used

Mohan N, et al. Arthritis Rheum 20014428622869.
9
Coccidioidomycosis and TNF inhibitors

• Risk of coccidioidomycosis in patients treated
  with TNF-a antagonists
• Thirteen cases of coccidioidomycosis associated
  with TNF-a therapy were reported
• Again seven out of 13 cases developed within 2
  months of starting TNF-a therapy

Bergstrom L, et al. Arthritis Rheum
20045019591966.
10
Summary of malignancies in randomized controlled
trials
Median 19 weeks
Bongartz T, et al. JAMA 200629522752285.
11
Time to neoplasia
14
12
10
8
Number of neoplasms
6
4
2
0
012
1324
2536
3748
4960
6172
72
Weeks
Based on data from Bongartz T, et al. JAMA
200629522752285, courtesy of Dr H Yazici
12
Infections
Curtis JR, et al. Arthritis Rheum
20075611251133.
13
Infections

• Hospital medical records with claims database
  reviewed
• Median follow up of 17 months
• Tumor necrosis factor-based treatment compared
  with MTX (or other DMARD-based) treatment
• Reviewed 187 out of 217 suspected bacterial
  infections
• TNF-a antagonist 65 in 2393 exposed persons
  (2.7)
• DMARD 58 in 2933 exposed persons (2.0)
• Number needed to harm 143
• Median time from exposure to medication to
  infection was lt30 days for both groups
• 93 occurred in 90 days
• Multivariable adjustment approximately two times
  in TNF-a versus MTX

Curtis JR, et al. Arthritis Rheum 2007
5611251133.
14
Increased early risk

• In the first 6 months
• Thirty-two infections in TNF-a cohort
• Nineteen infections in DMARD cohort
• Adjusted hazard ratio of infection in the first 6
  months after start of treatment
• TNF-a compared with MTX controls was 4.2 (95
  confidence intervals CI 2.0-88.8)
• Similar to JAMA paper

Curtis JR, et al. Arthritis Rheum 2007
5611251133.
15
Surveillance, epidemiology and end-results
database (SEER)

• If timing of a malignancy is not evenly
  distributed, we should not be using a yearly rate
  to compare incidence rates
• SEER 12 expected lymphomas in RA patients over
  12 months, 1 per month
• TNF-a 12 seen in RCT
• Conclusion is that there is no difference
• TNF-a lymphoma
• Eight in the first 4 months 2/month
• Four in the last 8 months 0.5/month
• In the case of lymphoma development
• Using annual incidence risk in the general
  population as a comparator is inappropriate
• Development of such lymphomas was within the few
  months after drug initiation in more than 50 of
  these patients1
• The expected random occurrence of lymphomas in
  the general population would make the real risk
  look smaller2

16
Tumor necrosis factor, time to AE

• As more than half of these cases occur during the
  first months of therapy, quarterly or monthly
  (rather than annual) incidence risks in the
  normal population should be used for comparison
• Increased incidence risks for TNF-a
  antagonist-related adverse events (AEs) would
  necessitate a reexamination of the screening
  protocols and patient selection criteria used for
  these therapies
• Does not carry the same message for the
  practicing physician
• Variable period comparison needs to be used
• 2-, 3-, 4-, 6-monthly rates
• Excluding events in the first 60180 days
• Not done with other AEs
• No strong evidence base for doing so

17
Time to adverse event reporting in randomized
controlled trials

• All RCT of cyclo-oxygenase-2 (COX-2) inhibitors
  and TNF-a inhibitors
• Industry-sponsored RCTs (91)
• One-third gave time to AE in the report either as
  a table, text or Kaplan-Meier curve
• No better reporting for serious adverse event
  (SAE)
• Eight out of 17 RATNF-a trials reporting
  malignancy used the Surveillance, Epidemiology
  and End Results (SEER) database

Yazici Y, Yazici H. Ann Rheum Dis 200766124127.
18
Time to adverse event and serious adverse event
COX-2 (26 studies) TNF-a (44 studies) Total (70 studies)
Number of patients enrolled Number of patients enrolled Number of patients enrolled Number of patients enrolled
Mean 1541 251 -
Median 602 112 -
Range 678076 201049 -
Duration of RCT Duration of RCT Duration of RCT Duration of RCT
Mean number of weeks 12 43 -
Median 6 24 -
Range 4156 252 -
n () n () n ()
AEs (n and reported) 19 (73) 36 (82) 55 (79)
Table given for AEs 21 (81) 34 (77) 55 (79)
Time to AEs reported (including SAEs) 6 (23) 17 (39) 23 (33)
Time to SAEs reported 3 (12) 16 (36) 19 (27)
Time reported for ?50 of SAEs 3 (12) 9 (20) 12 (17)
Time reported for gt50 of SAEs 0 0 0
Time reported for all SAEs 0 7 (16) 7 (10)
SIR based on annual SEER figures 0 8 (18) 8 (11)
Patient-years as time frame 2 (8) 4 (9) 6 (9)
Yazici Y, Yazici H. Ann Rheum Dis
200766124127. COX-2cyclo-oxygenase-2
TNF-atumor necrosis factor-a RCTrandomized
controlled trial AEadverse event
SAEserious/severe adverse event
SIRstandardized incidence ratio
SEERSurveillance, Epidemiology and End-Results.
19
Patient years
20
Patient years

• Commonly used in RCT patientyears to define
  the time frame of AE incidence
• What is the problem?
• Relatively rare idiosyncratic drug reactions
  usually occur early in the treatment course and
  in only a few individuals
• Apart from the few with AEs, remaining patients
  who are prescribed the drug will never get these
  reactions however long they use the drug

Paterson KR. BMJ 19953101470.
21
Patient years

• AE 20 chance in the first month, and not seen
  after
• 100 patients followed for one year
• 80 of patients after one year will have no AE
• 20/100 patient years
• One more year, no one gets AE
• 20/200 patient years 10/100 patient years
• Nothing has changed in the absolute risk when it
  is started in a new patient.

22
Patient years (2)

• Unduly inflate the denominator of the related
  incidence ratio potential of under-representing
  AE
• Late-onset AEs are also likely to be missed when
  we use patientyears
• In short, only when an event is (or is believed
  to be) likely to occur at any stage during
  continuous treatment with a drug then an event
  rate with a time component (rate per person year,
  etc) has a true meaning
• Not clear whether more than one event per patient
  goes into the numerator

Paterson KR. BMJ 19953101470.
23
Patient years (3)

• If more than one AE/patient is included in a
  numerator, statistics done with that incidence
  ratio will be erroneous
• An AE can repeat itself, like skin rashes in
  TNF-a antagonist use in any one patient
• This leads to over-representation of the said
  individual in tests of significance

Paterson KR. BMJ 19953101470.
24

• Statistics are like swimwear what they
  reveal is suggestive but what they conceal is
  vital.
• Ashish Mahajan, Lancet 2007

25
Reporting radiographic outcomes in RA RCTs
26
Conventional wisdom
Schett et al. Arthritis Rheum 2008
27
Conventional wisdom

• studies of anti-TNF therapy plus MTX, compared
  with the effect with MTX alone, have shown that
  although MTX is relatively effective at relieving
  clinical symptoms, it has little or no effect on
  underlying radiological progression.

Emery P, McInnes IB, van Vollenhoven R, Kraan MC.
Rheumatology (Oxford) 2008
28
Etanercept versus methotrexate
Klareskog L, et al. Lancet 2004363675681.
ACRAmerican College of Rheumatology
MTXmethotrexate.
29
Adalimumab versus methotrexate
Breedveld FC, et al. Arthritis Rheum
2006542637. plt0.001 vs ADA alone and p0.022
vs MTX alone plt0.001 vs ADA alone and p0.002
vs MTX alone p0.043 vs ADA alone plt0.001 vs
ADA alone and vs MTX alone ACRAmerican College
of Rheumatology ADAadalimumab MTXmethotrexate.
30
PREMIER X-ray
Breedveld FC, et al. Arthritis Rheum
2006542637. MTXmethotrexate.
31
Demographic and study characteristics of four
methotrexate-naïve randomized controlled trials
ERA TEMPO IFX early RA PREMIER
Patients, n 424 682 641 799
TNF arm, n 207 223 NA 274
MTX arm, n 217 228 282 257
Combination arm, n NA 231 359 268
Double blind duration 12 months 12 months 12 months 12 months
Mean TJC at baseline 31 33 33 31
Mean SJC at baseline 24 23 22 21
Primary efficacy outcome ACR-N (6 months) ACR-N (6 months) ACR-N (12 months) ACR 50 (12 months)
Primary radiographic outcome Sharp (12 months) Sharp (12 months) Sharp (12 months) Sharp (12 months)
Yazici Y, Yazici H. Clin Exp Rheumatol
200826449452.
32
12-month efficacy and radiographic outcomes in
four methotrexate-naïve randomized controlled
studies
Study ACR 20 ACR 50 ACR 70 ACR N Sharp NNT
ERA ETA 72 NR NR NR 1.00 15
ERA MTX 65 NR NR NR 1.59 -
TEMPO combination 85 69 43 18.3 0.54 12
TEMPO ETA 75 48 24 14.7 0.52 -
TEMPO MTX 76 43 19 12.2 2.8 -
IFX combination 62.4 45.6 32.5 38.9 0.4 13
IFX MTX 53.6 32.1 21.2 26.4 3.7 -
PREMIER combination 73 62 46 NR 1.3 10
PREMIER ADA 54 41 26 NR 3.0 -
PREMIER MTX 63 46 28 NR 5.7 -
Yazici Y, Yazici H. Clin Exp Rheumatol
200826449452. ACRAmerican College of
Rheumatology, NNTnumber needed to treat
ERAEarly RA ETAetanercept NRnot reported
NRvalues not given, only depicted in graphic
form MTXmethotrexate TNFtumor necrosis factor
inhibitor TEMPOTrial of Etanercept and MTX with
radiographic Patient Outcomes IFXinfliximab
ADAadalimumab.
33
Radiographic outcomes in four methotrexate-naïve
randomized controlled trials
5.7
6
5
3.7
4
3
Sharp scale
2.8
3
2
1.59
1.3
1
1
0.52
0.4
0.54
0
ERA ETA
ERA MTX
TEMPO
TEMPO
TEMPO
ASPIRE IFXMTX
ASPIRE MTX
PREMIER ADAMTX
PREMIER ADA
PREMIER MTX
ETAMTX
ETA
MTX
Sharp score
Yazici Y, Yazici H. Clin Exp Rheumatol
200826449452.
34
Radiographic outcomes in four methotrexate-naïve
randomized controlled trials
450
400
350
300
250
Sharp scale
200
150
100
50
5.7
3.7
3
2.8
1
1.59
1.3
0.54
0.52
0.4
0
ERA ETA
ERA MTX
TEMPO
TEMPO
TEMPO
ASPIRE IFXMTX
ASPIRE MTX
PREMIER ADAMTX
PREMIER ADA
PREMIER MTX
ETAMTX
ETA
MTX
Sharp score
Yazici Y, Yazici H. Clin Exp Rheumatol
200826449452.
35

• 1.3 (PREMIER combo) vs 5.7 (PREMIER MTX alone)
• 1.3/4480.003
• 5.7/4480.01
• 0.01-0.0030.007
• Similar to saying pain score of 49.8 (combo) vs
  49.1 (alone) is clinically significant and should
  lead to treatment change/preference

36
Abatacept limited inhibition of radiographic
progression mean change in total Sharp scores
(Genant modification)
4
Placebo MTX (n195)
3
Abatacept 10 mg/kg MTX (n391)
2.3
50inhibition
2
Mean changefrom baseline
1

1.2
0
-1
6
12
0
Months
Mean change from baseline at Month 12 Mean change from baseline at Month 12 Mean change from baseline at Month 12 Mean change from baseline at Month 12
Treatment Erosion score JSN score Total Sharp score
Placebo MTX 1.14 1.18 2.32
Abatacept MTX 0.63 0.58 1.21
Kremer J, et al. Ann Intern Med 2006144865876
Genant H, et al. Ann Rheum Dis 200564(Suppl
III)56. Abstract OP001. plt0.05 vs placebo
MTX plt0.01 vs placebo MTX. MTXmethotraxate
JSNjoint space narrowing.
37
A new way of assessing radiographic outcomes
cumulative probability plots
40
30
Increasing score
20
10
Change in total Sharp score
Unchanged
0
Decreasing score
10
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Cumulative probability
Landewe R, van der Heijde D. Arthritis Rheum
200450699706.
38
Probability curves
39
BeSt study design
Group 2Step-up Combo
Group 1Sequential Mono
Group 3Initial Combo
Group 4 IFX MTX
n126
n121
n133
n128
MTX 15 mg
MTX 15 mg
MTX 7.5 mg/wk SSA pred 60?7.5 mg/day
MTX 25 mg IFX 3 mg/kg
n120
MTX 25 mg SSA pred
MTX 25 mg
MTX 25 mg
SSA
MTX IFX 10 mg/kg
MTX SSA
MTX CsA pred
MTX SSA HCQ
Leflunomide
SSA
MTX IFX
n25
MTX SSA HCQ pred
Leflunomide
MTX IFX
Leflunomide
n49
MTX IFX
MTX CSA pred
Gold
Gold
n12
Gold
MTX CSA pred
MTX CsA pred
AZA pred
AZA pred
Leflunomide
AZA pred
Gold
Goekoop-Ruiterman YP, et al. Arthritis Rheum
200552338190.
40
BeSt study
Goekoop-Ruiterman YP, et al. Ann Intern Med
200714640615.
41
TICORA intensive versus routine control of
disease activity in rheumatoid arthritis

• Objective
• 18-month study to determine whether closely
  monitored step-up therapy with non-biologic
  DMARDs would result in significantly better
  outcomes than routine care
• Study population
• 110 patients aged 1875 years with RA of lt5
  years duration and active disease (Disease
  activity score DAS gt2.4)
• Outcomes
• Primary
• of patients achieving a DAS score of lt2.4
• Secondary
• of patients achieving remission (DAS score
  lt1.6)
• of patients achieving American College of
  Rheumatology20, 50 and 70 responses

Grigor C, et al. Lancet 2004364263269.
TICORATight Control of Rheumatoid Arthritis.
42
TICORA intensive versus routine control of
disease activity in rheumatoid arthritis (2)

• Intensive care group
• Monthly review of disease activity and
  measurement of DAS
• Intra-articular corticosteroid injection of
  swollen joints as required
• Intramuscular corticosteroid injection given as
  bridge therapy during first 3 months of a new
  DMARD
• Structured escalation of therapy if DAS gt2.4
  after 3 months of a new DMARD
• SSZ ? MTX SSZ HCQ ? ?MTX (up to 25
  mg/wk) ?SSZ (up to 5 g/d) ? ?prednisolone 7.5
  mg/d ? switch to MTX CSA ? switch to
  leflunomide
• Routine care group
• Review every 3 months (with no measure of DAS)
• Management at the discretion of attending
  rheumatologist (i.e. DMARD therapy
  intra-articular, intramuscular and/or oral
  corticosteroids)

Grigor C, et al. Lancet 2004364263269.
TICORATight Control of Rheumatoid Arthritis.
43
Tight control for rheumatoid arthritis TICORA
study
44

Tight control for rheumatoid arthritis TICORA
study (2)
Number of patients responding at 18-month assessment Number of patients responding at 18-month assessment Number of patients responding at 18-month assessment Number of patients responding at 18-month assessment Number of patients responding at 18-month assessment
Intensive group (n55) Routine group (n55) Odds ratio (95 CI) p
EULAR good response 45 (82) 24 (44) 5.8 (2.413.9) lt0.0001
EULAR remission 36 (65) 9 (16) 9.7 (3.923.9) lt0.0001
ACR 20 response 50 (91) 35 (64) 5.7 (1.916.7) lt0.0001
ACR 50 response 46 (84) 22 (40) 6.1 (2.514.9) lt0.0001
ACR 70 response 39 (71) 10 (18) 11 (4.527) lt0.0001
Grigor, et al. Lancet 2004364263269.
Mantel-Haenszel procedure used. EULAREuropean
Union League Against Rheumatism ACRAmerican
College of Rheumatology.
45
TICORA 2

• 96 RA patients, with lt1 yr of disease
• No previous DMARDs (except HCQ)
• DAS28 every month, treatment adjusted
• IA as needed
• Primary outcome was DAS28
• Blinded assessment
• 85 had erosions at baseline
• DAS28 6.9

Saunders SA et al. Arthritis Rheum 2008581310-17
46
TICORA 2
47
TICORA 2
48
TICORA 2
49
Efficacy versus safetyversus efficacy and
safety trials
50
Efficacy and safety randomized controlled trials

• Trend of increasing number of RCT called
  efficacy and safety trials
• All RCT of TNF-a inhibitors in RA, psoriatic
  arthritis (PsA) and ankylosing spondylitis (AS)
• Only original reports
• Efficacy, safety or efficacy and safety
• Power for safety and efficacy
• Type II error
• Power
• Alpha
• Beta
• Expected treatment effects in both arms

Yazici Y, et al. Rheumatology 20084710541057.
51
Powering for safety analysis

• Twenty-four out of 34 (71) so-called efficacy
  and safety trials
• Five called such in the title
• Of these 24 so-called efficacy and safety
  trials, only 1 gave safety as a primary or
  secondary end point
• Only one study, labeled safety gave it as an
  endpoint
• No power calculations in 22/24 to look at AE in
  efficacy and safety studies
• Only 3/22 had type II error discussion

52
Efficacy, safety and efficacy and safety
TNF-atumor necrosis factor inhibitor-a
RCTrandomized controlled trial.
53
Powering for safety analysis (2)

• 11 gave statistics for SAE
• 2/11 had power calculations
• 2/11 showed statistically significant differences
• Of the remaining 9, only 2 discussed type II
  error
• 24 articles gave partial statistics for AE
• 18/24 (75) no type II discussion
• 6 out of 34 (18) did not give any power or
  sample size calculation for efficacy
• 19/28 (68) with power information gave all 3
  components

54
Before and after 2004
55
Efficacy and safety

• Designating RCT as safety as well as efficacy
  has the potential to give false impressions that
  lack of AE increase is evidence of overall safety
  of the medication tested
• Only one of these trials was powered to look at
  safety
• In addition, RCT are highly selective for lack of
  major comorbidities
• So even if they were powered to look at safety,
  results need to be interpreted with caution
• Phase IV trials
• Randomized controlled trials report on AEs, but
  these efficacy trials cannot provide any
  conclusive evidence about safety these
  shortcomings should be reported and required by
  journals and reviewers

56
Disease activity measurement in RA
56
57

Pincus et al. Arthritis Rheum 2003
58
Patientreported outcomes
Strand et al, Rheumatology 2004
59
Conclusions

• Time to adverse effect
• Causation
• What to expect for the clinician
• Patient years
• Falsely give sense of experience
• Rare adverse effects hard to analyze
• SEER
• Radiographic data not as important as clinical
  response
• Randomized controlled trials provide information
  about efficacy and not safety. They report on AEs
  that happen during a defined time in a small,
  selected population

60

• We become confident in our educated guesswork to
  the point where it is easy to confuse personal
  opinion with evidence, or personal ignorance with
  scientific uncertainty
• David Naylor, MD, PhD (1954)

61
Esra Leyla