Protocol Development and Statistical Analysis Plans

1
Protocol Development and Statistical Analysis
Plans
Petra Rauchhaus TCTU Clinical Trials Statistician
2
(No Transcript)
3
Importance of the Protocol
Funders
Journals
Ethics
Trialists
Patients

• Provide rationale for the trial
• Define trial goals and processes
• Define methods of analysis/ reporting
• Enable scientific and ethical review
• Provide a Trial Roadmap

Policy makers
Systematic Reviewers
Ethics
Healthcare providers
Journals
4
Importance of the Protocol

• GCP Requirement
• Ethics Committee requires a protocol for
  submission
• Part of the EU Clinical Trials Register (EUDRACT)
• Ensures in Multi-Centre Trials that all centres
  perform the study in the same way
• Journals require a registered protocol for
  publication
• Not only for CTIMPS, Non-CTIMPS also benefit from
  a good protocol

5
What could go wrong?

• Missing details of basic trial design
  (uncontrolled/ controlled/ randomized)
• Imprecise or missing description of the primary
  outcome in the protocol
• Sample Size calculation not reported
• Limited methodological information
• Interventions not well defined
• Planned subgroup analyses missing
• Favourable reporting of positive outcomes
• Adverse events suppressed in reports

6
Lack of general information
Allocation Concealment
Blinding
Primary Outcome
Power Calc.
Adverse Events Reporting System
Chan AW et al, BMJ 2008 Al-Marzouki S et al,
Lancet 2008
7
Lack of statistical information
Handling deviations
Primary Outcome Analysis
Adjusted Analyses
Subgroup Analyses
Handling Missing Data
Chan AW et al, BMJ 2008 Al-Marzouki S et al,
Lancet 2008
8
Protocol standards

• There is a number of support documents
• ICH Guideline E6 defines the protocol structure
  (15 sections with several sub-points each)
• SPIRIT (Standard Protocol Items for Randomized
  Trials) initiative by statisticians, journal
  editors and PIs
• CONSORT guidelines to report trials
• EQUATOR Networkhttp//www.equator-network.org/
  
• TASC SOP 14 Writing a protocol
• Protocol Template on the TASC websitehttp//www.
  tasc-research.org.uk/_page.php?id208

9
Definition of a protocol

• Pre-Trial Document containing transparent
  description of
• Background and objectives
• Population and interventions
• Methods and statistical analysis
• Ethical and administrative aspects

10
Title

• A title uniquely identifies the project
• It should summarize the aim and methods of the
  trial
• Important information (e.g. randomized,
  double-blind, parallel group) should be included
  in the title
• Indexers on websites such as PubMed may not
  classify a report correctly if the authors do not
  explicitly report information in the title
• A Prospective Randomized Study of Medial
  Patellofemoral Ligament (MPFL) Reconstruction

11
Synopsis

• Brief overview over the study aims and conduct
• Should contain sufficient information about a
  trial to serve as an accurate record of its
  conduct
• Should accurately reflect what is included in the
  full protocol and should not include information
  that does not appear in the body

12
Background

• The Declaration of Helsinki states that
  biomedical research involving people should be
  based on a thorough knowledge of the scientific
  literature
• Thus, the need for a new trial should be
  justified in the introduction
• Explain the scientific background and rationale
  for the trial
• Report any evidence of the benefits and harms
• Ideally, it should include a reference to a
  systematic review of previous similar trials or a
  note of the absence of such trials

13
Objectives

• Objectives are statements what the researcher
  means to do
• Objectives can be seen as smaller problems in the
  larger research area
• E.g. Improving cancer care is a large research
  area which is too broad to be tested within a
  trial.Impact of physiotherapy on QOL of late
  stage lung cancer patients is testable within a
  trial.
• Ensure that objectives are specific, measurable
  and clinically important
• Changing objectives can sometimes make a trial
  better

14
Outcomes

• Is the measurable part of the objective
• Ensure that the outcome is appropriate to the
  objective it serves.
• Define clearly what the outcome is and how it
  will be measured
• If outcomes are measured several times, specify
  time point of interest
• If possible, use validated and measurable
  outcomes
• If there is more than one assessor, state how
  many there are and how discrepancies in
  measurement will be handled

15
Trial Design

• Define the type of trial, e.g. parallel group,
  cross-over or factorial
• Define the conceptual framework, e.g.
  superiority, non-inferiority, equivalence or
  other
• If a less common design is employed, authors are
  encouraged to explain their choice
• This is especially important because it might
  have implications on sample size or analysis
• Include allocation ratio if more than one group,
  and unit of allocation (patient, practice, lesion)

16
Eligibility Criteria

• Should be well defined and appropriate to the
  trial
• Define which patient groups are involved and how
  they relate to the objectives
• Eligibility criteria which are too narrow can
  jeopardize the study
• Eligibility criteria too wide can invalidate the
  outcomes
• E.g. Including Stage IV Cancer patients in a
  study examining the effectiveness of two
  different treatments might fail, as the diseases
  is too advanced already to make a difference

17
Sites and Locations

• Goes hand in hand with the eligibility criteria,
  as certain subjects need certain locations
• E.g. primary care, hospital wards, specialized
  units
• Healthcare institutions vary in their
  organisation, experience, and resources
• Social, economic, and cultural environment and
  the climate may also affect a studys validity
• Especially important in multicentre trials,
  particularly in international studies

18
Interventions

• Describe all interventions including controls in
  great detail
• It must be possible to be reproduced if necessary
• If you compare to usual practice describe what
  that means, do not assume everyone knows
• If interventions are variable, e.g. adaptation of
  radiation doses or drug regimes, define rules of
  application
• In dose-escalation studies, define stopping rules

19
Sample Size

• Sample size calculations are based on previous
  trials measuring the outcome
• Ensure that the patient population matches the
  trial population
• Where no previous trials are available, sample
  size is often based on assumptions
• Sample size is only as accurate as the
  assumptions
• Where more than one outcome is present, sample
  size is calculated for the primary outcome
  usually
• It is possible to use the largest sample size to
  get the best power

20
Interim Analysis

• Interim Analysis can diminish the trial power
• Error rates increase as the number of analysis
  increases
• E.g. doing 5 interim analysis requires a p-value
  of 0.01 rather than 0.05, and can give an error
  rate of 19 rather than 5
• Use only when necessary
• Some trials require interim analysis, e.g. for a
  DMC
• If possible, separate the DMC analysis from the
  main analysis

21
Randomisation

• Randomized trials are the gold standard
• Randomization requires a program to be written
• Sequence generation must be reproducible at any
  stage
• Define criteria for stratification and
  minimisation
• Try to avoid predictable block sizes
• If possible, blinding should be employed
• Blinded studies require an independent
  statistician
• Minimization is dynamic and therefore less
  predictable

22
Allocation

• Allocation concealment is not blinding
• Define how the allocation is applied to the
  subjects
• Define how to conceal allocation until the
  subject is included into the trial
• Ensure that the person doing the screening is not
  familiar with the allocation sequence
• Decide whether to include a subject into the
  trial before the allocation
• If possible, use a third party to allocate
  subjects

23
Statistical Analysis

• Statistical analysis must be described
• Descriptive statistics should be defined for an
  overview over the data
• Define the appropriate methods for the data
• Describe briefly missing or spurious data
• Keep the description of the statistical analysis
  short
• Mention checks of normality and independence
• Do not hesitate to involve a statistician with
  this part of the protocol
• A detailed statistical analysis plan (SAP) should
  be written during the course of the trial

24
Statistical Analysis Plan (SAP)
25
Statistical Analysis Plan (SAP)

• It is critical link between the conduct of
  the clinical trial and the clinical
  study report. 
• General statistical analysis is defined in the
  clinical protocol
• The SAP contains a more technical and detailed
  elaboration of the analysis
• Recommended by the CONSORT guidelines and ICH
  Guideline E9 (Statistical Principles for Clinical
  Trials)

26
Why write an SAP?
Implement the Trial as outlined in the Protocol
Establish Good StatisticalPractice
Study Design (Clinical Protocol)
Study Methods (Data Collection Trial conduct)
Study Analysis provides checks on the original
design
Study Analysis(SAP)
Analysis of the planned study design, adapted
tothe study methods
27
GCP requirements

• The statistical authorship of the SAP should be
  clear
• Version and date should be clearly defined
• The SAP should be reviewed/ updated immediately
  before the blinded code is broken or before
  analysis begins in an unblinded trial
• The SAP should be signed off by the PI/ CI and
  the Statistician (and other members of the study
  team where applicable)
• Changes in the SAP after study end should be
  justified and fully documented in the statistical
  report

28
When to write an SAP

• The SAP is written during the trial, after the
  clinical protocol is final
• It must be finalized and signed off before the
  end of the trial to avoid bias
• If the study is blinded, it must be finalized
  before the blind is broken
• The SAP should be reviewed and possibly updated
  as a result of the blind review of the data
• In adaptive trials, it must be finalized before
  the first interim analysis
• Regulatory factors, such as a special protocol
  assessment at the FDA, may affect the timing

29
Changes in Study Methods

• Protocol Amendments during the trial
• Change in the planned treatment (new developments
  in therapy or guidelines)
• Recruitment does not go as planned
• Early termination of the trial can change patient
  numbers
• Adding or removing a group
• Addition or removal of a planned test or
  procedure
• Changes in the outcomes or how they are measured

30
SAP Contents

• A brief description of the purpose
• The study rationale as laid out in the protocol
• Definition of analysis populations (usually ITT)
• How subject data will be summarized (descriptive
  statistics or counts/ percents)
• Which statistical tests will be used on which
  data
• The statistical methods to be used for the
  endpoints
• When and how to impute missing or partial data
• Mocks (or shells) of all unique TLF's
• Quality control of the analysis

31
Writing an SAP

• Refer to TASC SOP 05 (Statistical Analysis Plans
  for Clinical Trials of Investigational Medicinal
  Products)http//www.tasc-research.org.uk/_page.ph
  p?id266
• Follow the section headings laid out in the SOP
• Contact the study statistician if present
• If no study statistician is present, TASC
  statisticians can review the SAP
• Distribute the SAP to all members of the study
  team that can contribute
• Finalize the SAP before the study is finished

32
Benefits

• Clear Protocol and SAP show that a study was done
  according to GCP standards
• Avoid biased analysis by defining the study
  populations before study end
• Defined handling of missing data, outliers and
  data deviations make the analysis more
  transparent
• Clearly defined subgroup analysis ward off data
  dredging
• The study report and resulting papers will be
  more likely to be of high quality

33

• Any Questions?