Parts of the Research Study

1
Parts of the Research Study

• Title, Abstract, Methodology, Results, Discussion

2
Even before the Title...

• Where is study published?
• Respected journal?
• Is journal in same field as the research study?
• Is journal peer reviewed?
• Was paper revised?
• Is is published in a journal of like content?

3
Title

• First potential source of bias
• It should not state any conclusions
• It should reflect the actual content as clearly
  and concisely as possible
• It should be consistent with the abstract and
  summary

4
Authors

• Each author should have participated sufficiently
  in the work represented by the article to take
  public responsibility for the content.
• Conception or design, analysis or interpretation
  of data
• Drafting the article or revising it for
  critically important content, or in final
  approval
• Participation only in data collection doesnt
  qualify for authorship

5
Authors

• Persons who contributed intellectually, but whose
  contributions do not justify authorship may be
  named separately.
• Authors should list credentials for carrying out
  research.
• Conflicts of interest should be noted.
• 55 of articles today have multiple authors

6
About Authors...

• Reputable?
• Independent from drug company?
• Affiliated with research institutions?
• No conflicts of interest or bias?
• If funded by Drug Company, it should be declared
  as such
• Reader bias-- too much weight given for
  credentials, big names, etc.

7
Abstract

• Purpose provide a brief summary of the research
  to help the reader determine if the article is
  worth reading in its entirety.
• Some are structured abstracts and some are have
  restrictions about number of words used.
• You cannot form a critical opinion of the studys
  validity without reading the whole article.

8
Introduction

• Contains the specific problem which exists
• Rationale for the study with background material
  (review of literature)
• Identifies the purpose of study study
  objective- stated as a study hypothesis.
• Should not contain bias or any results.

9
Introduction Should Include

• Statement of the importance of anticipated
  results from the study
• Reasons for doing drug efficacy studies
• there is no other effective treatment for cond.
• This drug is potentially superior to other drugs
• Due to low SE, this would be a better choice
• Cost savings
• Pharmaceutical properties- tolerance, safety

10
Purpose of Study/Study Objective

• Should be explicitly stated--you shouldnt have
  to infer purpose
• Is/are objective(s) reasonable?
• Are there too many objectives to be answered in a
  single study?
• Will results measure the study objective, ie. Are
  there valid endpoint measurements?

11
Study Objective

• Describes anticipated relationships between
  factors to be studied
• Specific and reasonable enough to study
• Define clearly and exactly what the investigators
  are going to do
• Relevant to what the investigators would like to
  determine
• Stated as null or alternative hypothesis

12
Null Hypothesis

• This assumes that there is no relationship
  between the factors to be studied and the
  outcome.
• Is assumed to be true until proven otherwise.
• Stated as There is NO difference between
  products, or, Both products are equal

13
Alternative Hypothesis

• Assumes that there is a relationship between the
  factor to be studied and the outcomes.
• Two types of alternative hypothesis
• one tailed indicated the direction of the
  relationship between the factor to be studied and
  the outcome
• two tailed indicates there is a relationship
  between the factor and outcome but doesnt state
  the direction

14
Examples of Hypothesis

• Null Pravastatin is equivalent to Simvastatin in
  terms of lowering of cholesterol.
• One tailed Pravastatin is more effective than
  Simvastatin in lowering cholesterol.
• Two tailed Pravastatin and Simvastatin differ in
  their efficacy to lower cholesterol.

15
Methodology

• Written so study could be repeated from the
  investigators description.
• Includes design, patient selection criteria,
  sample size, inclusion/exclusion criteria,
  randomization, controls, blinding, etc.
• Determines internal validity of study

16
Study Design

• Study design guides evaluation methods
• RCT methods of treatment assignments, blinding
  and controls
• Longitudinal duration of the follow up
• Crossover study use and details of washout
  period
• Retrospective methods to avoid recall bias
  should be included

17
Validity

• Related to precision and accuracy
• Internal validity adequacy of the study as a
  whole.
• Relies on study design, bias, and random
  variation
• External validity can results be extrapolated to
  other settings
• Relies on inclusion/exclusion criteria

18
Internal Validity

• A study has internal validity if the following
  have been done properly
• Study design
• controls, blinding
• Methods of patient selection
• sample size, random sampling, inclusion/exclusion
  criteria, external validity
• Randomization
• Outcomes and endpoint measurements
• Statistical analysis

19
External Validity

• External validity is determined by
• inclusion criteria --
• Are the study participants like your patient
  population, ie. Elderly, diabetic, CHF, etc.
• exclusion criteria --
• Who is not included in study, ie. Diabetics,
  elderly, CHF, etc.
• both criteria are used to determine if results
  can be extrapolated to other settings.

20
Homogeneous Groups

• Study groups are closely related in terms of
  important clinical characteristics or disease
  attributes
• The more homogenous, the easier to identify and
  quantify the effects which a drug exerts
• Increases the internal validity of the study.

21
Heterogenous groups

• Patients differ in one or more identifiable
  clinical characteristics of the disease or
  condition being treated.
• Acceptable when there arent enough patients who
  meet some narrowly defined inclusion criteria.
• Acceptable when the results of the study wont be
  affected by the differences

22
Inclusion Criteria

• Characteristics patients must have to be eligible
  for participation in study
• Homogeneous groups preferred--easiest to identify
  and quantify effects and increases internal
  validity of study.
• Heterogeneous groups okay when results wont be
  affected by the differences.

23
Exclusion Criteria

• Characteristics which prohibit the patient from
  participating in the study
• Examples presence of other disease states,
  severity of disease, other medications/therapies
  affecting study results, patient safety, ethics,
  compliance.
• Exclusion criteria helps ensure the study sample
  is homogenous.

24
Patient Selection Criteria

• How many patients did they have in the study?
• Is this number appropriate for the study design?
• Does the study population represent the
  population from which it is drawn?
• Was random sampling truly done?

25
Sample Size

• Determined during initial planning stages of
  study
• Need enough subjects to allow for significant
  differences between treatment groups to be
  detected statistically.
• Need to balance statistical concerns with subject
  availability, cost, time constraints

26
Sample size considerations

• RCTs with small number of subjects may not be
  adequate to determine long term toxicity.
• Other study designs may be needed based on the
  study sample size.

27
Sample size factors

• Alpha or level of significance the probability
  of obtaining a false positive result -- indicated
  as the p-value.
• Beta probability of false negative
  result--indicated as the power.
• Delta amount of difference that one wants to
  detect between groups
• variance or standard deviation needed

28
Sample Size Factors

• Investigator sets 4 factor levels, goes to table
  (or program) and selects appropriate sample size.
• Very rough minimum, 30 patients needed for
  parallel study, 15 needed for crossover
• Increasing sample size beyond certain point can
  lead to wasteful time and money--law of
  diminishing returns

29
Random Sampling

• Selection of population into the study
• Each member of the population has the same
  opportunity to be selected into the study.
• Each is selected independently of anyone else.

30
Non-Random Sampling techniques to beware of

• Consecutive non-random sampling accept every
  patient who meets study criteria until a certain
  number is reached.
• Convenience non-random sampling select patients
  from a population which is easily or readily
  accessible.
• Systematic non-random sampling Every nth person
  is selected for study inclusion

31
Controls

• What are investigators comparing the study
  drug/test to?
• Active control
• Placebo control
• No control
• Historical control

32
Active Control

• Study drug is compared to another drug
• Tells only relative efficacy
• Is study drug more, less or of equal efficacy to
  comparison drug

33
Placebo Control

• An inactive medicine without pharmacological
  effect.
• Same dosage form and route
• It will contain small amount of sugar, lactose or
  other inert substance which has no therapeutic
  action.
• Can tell actual efficacy
• Minimizes bias, controls confounders

34
Ligation of Mammary Artery Trials

• 1940s, double blinded, placebo controlled trial
  (sham operation vs. actual operation)
• saved lives of many high risk patients from going
  through risky surgery which was not effective.

35
No Treatment Control

• Refers to a group of patients in a study who do
  not receive any study drug or placebo
• Tells actual efficacy
• Ethical concerns arise for placebo and no
  treatment control groups
• Salk polio vaccine trials in 1950s

36
Historical Control

• Utilizes a group of patients from who data have
  previously been collected.
• Uses effectiveness of surgical procedures, rare
  diseases, oncology studies
• Disadvantages inability to determine if control
  group was truly comparable, esp. when
  disease/condition can change over time.

37
Blinding

• Open label
• Both investigator and patient know treatment
• Single blind
• Investigator knows who is receiving which
  treatment, but patients dont know what they are
  receiving.
• Double blind
• Neither investigator and patient know treatment

38
Keeping the study blinded

• Make placebo look like active drug
• Double Dummy-- patients take 2 drugs each-- one
  placebo and one study drug.
• RPh often involved with studies-- we keep
  investigators and patients blinded.
• Unblinding can occur

39
Randomization

• Refers to
• assignment of patients to a treatment group in a
  parallel or time series design
• Assignment of the order of treatments in a
  crossover design
• Purpose of randomization in assignment to groups
• -reduces bias, keeps groups balanced

40
Simple Randomization

• Random numbers table
• Pulling names out of a hat

41
Systematic Randomization

• Selecting a treatment group in which every nth
  person is selected for a treatment group
• Acceptable if the starting point for selection
  (random sampling) is determined properly
  (randomly).

42
Block Randomization

• Useful when using small numbers of patients
• Ensures equal number of patients are randomized
  to each treatment group.

43
Cluster Randomization

• The population is divided into natural groupings
  (geographical locations) and a random sample is
  selected from each group.
• A multicenter study across the U.S. All lpatients
  from SE are divided into treatment and placebo
  groups, all from NE are divided, etc.

44
Stratified Randomization

• Patients are assigned to subgroups, called
  strata, based on important characteristics called
  confounding factors. Then a separate
  randomization schedule for each stratum is
  chosen.
• Useful when confounding factors will have large
  effect, and when small sample size

45
Non-Random Assignment

• Potential Bias increased
• May use hospital admission numbers, phone
  numbers, SS, days of the week patients joined
  the study, etc.
• Tendency to show larger treatment effects and
  increase the risk of false positive results
• Results are difficult to evaluate--need
  multivariate modeling in statistical analysis

46
Outcome Measurements

• Do measured endpoints match objective endpoints?
• Are they measured correctly?
• Is statistical analysis done?
• By independent investigator?

47
Results

• Clearly presented and accurately reflect the
  study hypothesis.
• Summary of study groups
• all patients should be accounted for
• reasons for missing data explained
• why drop-outs occurred
• Is length of study appropriate for study
  objective?

48
Patient/Subject Drop-out

• Drop-outs change balance of groups
• Reasons for drop-outs can impact results
• Non compliance with study protocol
• development of side effects
• lack of efficacy
• subject was found not to meet inclusion criteria
• developed another condition which interfered
• Unavailable for follow up

49
How to Handle Data from Drop-outs

• Intent-to-treat method all data from all
  patients are included in analysis, regardless of
  whether or not their treatment was modified in
  any way
• Exclusion of subjects method patients are
  excluded from analysis if their treatment was
  modified in any way.

50
Intent-To-Treat

• Advantage reflects normal or actual clinical
  practice for a drug, in which patients are often
  started on a drug and later have their therapy
  altered.
• Disadvantage If large numbers of patients drop
  out or have therapy altered, the true efficacy of
  the drug itself will be obscured

51
Intent to Treat measurements

• Intent to treat method takes drop out patients
  and measures their scores by
• A. Their last score or measurement at the time
  they dropped out
• B. The average for the entire group
• C. The worst score or measurement for the group.

52
Exclusion of Subjects

• Advantage The true efficacy of a drug in the
  regimen outlined in the study can be better
  determined.
• Disadvantage Patients can drop out of a study
  for reasons that can affect the usefulness of a
  drug in practice-- this method will not always
  reflect the actual clinical usefulness of a drug

53
Missing Data

• Patient completes study but one or more of their
  data measurements are missing
• The greater the number of variables to measure in
  a study, the greater chance that certain data
  points will be missing
• Missing data points are not significant if
• only a small percentage of data points are
  missing
• missing points occur by chance rather than by a
  single factor

54
Options for Handling Missing Data

• Dropping patients with incomplete data from study
• Submitting the mean of the other scores or
  mathematically estimating the value for the
  missing point
• Excluding the missing points for analysis

55
Types of Data

• Raw data actual measurements obtained
• Derived data measurements which have had some
  manipulations
• Summary data results which represent the combine
  data for all patients

56
Types of Data

• Derived data should be accompanied by the raw
  data it was prepared from to allow for
  interpretation.
• Summary data should be accompanied by the
  individual data to allow you to
• fully evaluate how well it represents all the
  patients
• determine if appropriate statistical analysis
  performed
• repeat calculation

57
Outcomes Reported

• Data presented must be
• complete
• clear
• missing data must be explained and accounted for
• Results section determine whether a study has
  fulfilled its objectives and proven or disproven
  its hypothesis

58
Tables and Graphs

• Clear
• Accurate
• Not misleading
• Simple

59
Things to watch for when analyzing the data

• Graphs with skewed vertical axis or no zero point
• misleading line graphs
• Truncated bar graphs
• Percentages
• Columns and rows not equaling 100
• Sample size inflated

60
Percentages

• Listed as
• percent cure rate
• percent response rate
• percent of patients achieving desired outcome
• Percent change can be misleading without knowing
  baseline value.
• Exact amount of change most valuable

61
Sample Size

• Artificially inflating sample size when repeated
  observations of a particular parameter are made
  and the author considers the total number of
  observations, and not the number of patients to
  be the sample size.

62
Discussion/Summary

• Form your own conclusions before reading the
  Discussion/Summary
• Watch out for persuasive language
• Watch out for downplay of conflicting evidence
• Study objectives have to be consistent with
  results

63
What to watch out for in the Discussion/Summary

• Author bias, reader bias
• Investigator interpretation of percentage change
  or degree of change relative to control.
• Biased citation or related publications
• Cause and effect relationship
• Errors in explaining a non-significant p-value
• Statistical significance vs. clinical
  significance
• Quality of life vs. death as endpoints
• Inappropriate conclusions