Clinical Research: Getting Started

1
Clinical ResearchGetting Started

• Edward A. Panacek, MD, MPH
• UC Davis Medical Center
• Department of Emergency Medicine
• (Syllabus)
• SAEM, May16, 2007, Chicago

2
Lecture Objectives

• Oriented to novice researchers !
• An understanding of the research process
• Knowledge of the mechanics of research
• Research questions, hypotheses, analysis
• How to get started..

3
What is research?

• Think about this for a moment

4
What is research?Common answers

• An answer to a question
• The development of new knowledge
• The organized collection and analysis of
  observations or data
  
• A way to see your name in print in the journals
• A vehicle for tenure at the university

5
What is research?

• The NIH definition
• A systematic investigation designed to develop
  or contribute to generalizable knowledge

6
What is research?

• In the purest, simplest sense, it is.
• A search for truth truth in the universe

7
Truth in the universe

• Can this ever really be known?
• Can we study the entire universe?
• Can our measurements be absolutely accurate?
• Can our data handling be error free?

8
Truth.or not truth
Research A process of drawing inferences
9
Why do research?

• A desire to know something
• Intellectual curiosity
• To contribute to the knowledge base of the
  specialty
  
• To change the world !
• Researchers can have an influence much greater
  than clinicians or educators

10
How is research performed?

• By using the
• The scientific method!
• And what is that?

11
The scientific method is the basicanatomy of
research
12
The relationship of the scientific method to the
truth
External validity
Internal validity
13
Research versus the truth

• Each step in the performance of research and the
  drawing of inferences can suffer from error
  
• There are two main types of error
• Systematic error AKA bias
• Random error occasional mistakes
• Sloppiness, etc.

14
Types of errorThe mirror image analogy

• Consider observing or measuring something by
  looking at its reflection in a mirror
  
• You cant see it directly
• Bends or distortions in the mirror (like a
  carnival mirror) are the equivalent of systematic
  error. They are always there
  
• Dirt on the mirror is like random error.
  Polishing the mirror (data cleanliness) can
  minimize the effects of random error

15
Goals of properly done research

• Minimize the effects of random error.
• Attention to detail
• 2. Eliminate sources of systematic error.
• Know sources of bias, to recognize them
• Eliminate or control them in the study design
• Both are designed to bring the investigator
  closer to the real truth
  

16
The research processStarts with a research
question
17
Sources of research questions

• Journal clubs
• Literature searches
• Presentations at meetings
• Limitations or future directions discussions
  in articles
  
• Daily clinical experiences in the ED

18
Projects always start with the all important
research question

• Take a moment.
• Consider a potential research question for your
  next project

19
All research starts with the question to be
addressed

• Great question
• potential for a great project
• Flawed question
• Confused or incomplete project
• Impractical question
• Waste of time performing the project
• No question ?
• No answers !

20
Where in a manuscript do you find the authors
research question?

• In the last paragraph of the introduction

21
Pitfalls in research questions I

• Problem Too many questions
• Cant focus on one specific question
• Desire to answer several questions with one
  project
  
• Solution
• Group into categories
• Prioritize in order of importance or feasibility
• Start with the most important one

22
Pitfalls in research questions II

• Problem Too broad a question
• Cant focus on details
• Results in no testable hypothesis
• Solution
• Divide and conquer
• Break down into sub-questions
• Then group and prioritize

23
Research projects and questions

• Each project must have one primary research
  question
  
• Answering this is the priority of the study
• It is the target which drives all other decisions
  in the study
  
• Can sometimes add in one or two sub-questions,
  but be careful !

24
Projects without questions

• Occasionally, some purely observational studies
  do not seem to have a focused research question
  
• e.g. simple descriptive summary of all the county
  EMS runs for the year
  
• However, even such studies should have some
  reason for doing them, some purpose, some question

25
A properly formatted research questionThe four
PICO elements

• P patients
• I intervention
• (independent variable)
• C comparison
• O outcome
• (dependent variable)

• S subjects
• E exposure
• C comparison
• O outcome

26
Using PICO to develop a question

• Includes each of the following
• Intended patients/subjects to study
• The independent variable of interest
• Intervention, exposure, pt characteristic
• The item of primary interest of the study
• Comparison group or procedure
• Outcome measure
• Combines them together as a question

27
Question example What is the HIV infection rate
in IV drug abusers?

• Issues to address
• Define rate (prevalence, incidence?)
• Define drug abuser
• Define study population sample source
• Define HIV infection (PCR or Western blot)
• Final result
• What is the rate of HIV zero positivity by PCR
  test in a participating sample of SFGH methadone
  clinic patients?
  

28
PICO in practice

• Essentially all proposed research questions
  should include each of the PICO elements
  

29
Research question comments

• It has been proposed that if your entire question
  can be stated in one breath, it is too short
  
• That is extreme. It need not include your entire
  inclusion and exclusion criteria
  
• The research question is analogous to the pt
  chief complaint. It is not the entire history and
  exam.

30
Refining the research questionMaking it a
Finer question

• F feasible
• I interesting
• N novel
• E ethical
• R relevant

31
Considering the FINER elements of a question

• Feasible can it be done with the resources that
  are available?
  
• Interesting do you have passion for it?
• Novel is it a new research idea?
• Ethical does it comply with research ethics and
  codes of conduct?
  
• Relevant does it pass the so what test?

32
Applying the FINER criteria

• Carefully review your proposed research question
• Research in progress meetings can be helpful for
  this
  
• Consider how it meets each of the FINER
  criteriaor not
  
• Then decide
• Do you still want to proceed with your proposed
  project? Is it feasible, etc.?

33
The FINER review

• This is where most good research questions die an
  early death
  
• (or should have died)

34
Other Research Project Terms

• Aims
• Purpose
• Objectives
• Questions
• Hypotheses
• All projects must have a research question. A
  study hypothesis should also be considered. Other
  requirements are determined by grant application
  instructions, etc.

35
Study goals

• Some grant applications request specific terms be
  used
  
• Objectives, aims or purpose generally define the
  general or overall goals of the project
  
• Sometimes written as the general focus and scope
  of the project
  
• Can involve multiple study questions

36
Grant proposal example

• The purpose of this study is to assess tissue
  oxygen delivery and utilization in trauma
  patients with shock
  
• The objective of this research is to develop
  appropriate endpoints for resuscitation in ED
  patients with trauma
  
• The study aims are to determine the effects of
  resuscitative efforts on measures of oxidative
  stress, e.g. glutathione and ketones
  
• From prior Chuck Cairns, MD, EMF grant
  application

37
In grant applications, study questions are often
listed among the goals

• The goal of this project is to answer the
  following questions
  
• In trauma pts, does aggressive resuscitation
  improve measurements of glutathione and ketones?
  
• In trauma pts, are improvements in glutathione
  and ketones associated with less morbidity and
  mortality?
  

38
Hypotheses

• For each project, with its proposed question you
  should also establish your hypotheses
  
• Null hypothesis (Ho)
• Alternative hypothesis (Ha)
• Study hypothesis

39
Hypotheses

• What are hypotheses and why have them?

40
Hypotheses

• A hypothesis is the investigators educated best
  guess of the answer to a question.
  
• A prediction of the real truth

41
What is the null hypothesis?

• This concept is remarkably poorly understood, but
  serves as the basis for classic statistical
  testing

42
What is the null hypothesis?

• That there is no difference between the study
  groups !
  
• (data set A) - (data set B) (0)
• empty set null set

43
The alternative hypothesis

• That there is a difference between the two study
  groups
  
• (Data A) - (Data B) (some ) ? 0 not
  null
  
• However, it does not define the direction of the
  difference
  
• Concept of two-tailed vs. one-tailed statistical
  tests

44
What is the study hypothesis?

• The investigators theorized answer to the
  research question
  
• It usually is the alternative hypothesis
• but with a specific direction of difference
• It could be the null hypothesis
• What are examples of this?

45
When Ho is the study hypothesis

• Pharma wants to show that their new 4th
  generation cephalosporin is as good as other
  existing drugs in that category
  
• When an investigator tries to show that a cheaper
  or safer therapy is as good as the
  expensive/toxic therapy

46
Why have a null hypothesis?

• Is it possible to test a question?
• Ho is boring and usually not our real interest,
  so do we really need it?
  
• Why not just test the alternative hypothesis (Ha)
  ?
  
• Concepts of Karl Popper

47
Karl Popper

• An intellectual leader of the scientific
  revolution
  
• Believed science should be based on the
  hypothetico-deducto process
  
• Propose a hypothesis and test it
• If evidence can be found to contradict the
  hypothesis, then it is fully rejected
  
• However, evidence in support is only true for the
  specific conditions of that experiment

48
Accepting and rejecting hypotheses

• When there are only two possible hypotheses and
  there is evidence that one is not true, by
  default, the other hypothesis is accepted as
  being true
• If reject Ho, means Ha is accepted

49
Hypothesis testing question

• Your project is finished and in your submitted
  abstract, youve written your results as
  
• treatment A was 20 better than Tx B in terms of
  outcome Y (p0.03). Therefore, treatment A should
  be the preferred Tx for this condition
  
• Your student research assistant asks you what
  p means, what does .03 mean, and why are you so
  confident in your conclusion?

What, exactly, would you say to the student ?
50
What you should say

• p probability
• 0.03 means if Ho were true, the probability of
  obtaining results with this, or a more extreme
  distribution difference, by chance alone is only
  .03 (3)
• Since that is a very small probability, I am
  accepting it as sufficient evidence that the Ho
  is not true and therefore I am accepting the Ha
  instead (i.e. that there is a real treatment
  difference).

51
Lets make sure we understand this principle

• Another study example

52
A study finds that a computerized reminder system
in the ED is associated with a 10 increase in
wound check f/u visits (p0.03)

• Which of these interpretations is correct?
• With this observed 10 difference between
  groups, there is only a 3 probability that the
  intervention had no effect.
  
• Given the assumption that the intervention had
  no effect, there is only a 3 probability of
  finding the observed 10 difference by chance
  alone.

53
Statistical analyses

• Statistical tests do not care about
• Truth in the universe
• Your research question
• Your hypothesis
• Stats tests are only a tool to quantitate the
  effects of chance on data distributions
  
• Investigators, not stats tests, accept or reject
  hypothesis and draw conclusions

54
Interpreting study results

• Your data is your data
• Your results are your results
• Sometimes the results are self-evident
• More often, statistical tests are needed to
  quantitate the effects of random chance on the
  data
  
• Do not confuse clinical significance with
  statistical significance

55
The next steps in a research projectFleshing
out the study protocol

• Defining all variables
• Identifying the sample population
• Inclusion/exclusion criteria
• Stipulating study interventions/measurements
• Sample size calculations
• Analysis plan
• Selecting an overall study design

56
Study designs

• Those that prove cause effect
• RCTs
• Those that argue cause effect
• Cohort studies
• Those that show associations
• Case-control
• Cross-sectional

57
Those topics are complex and are the subject of
other lectures or courses

• EMBRS (Emergency Medicine Basic Research Skills)
  Course
  
• 12 day workshop sponsored by EMF/ACEP
• Given once a year in Dallas
• Spends an entire day on
• Research study designs
• Developing study protocols
• Other courses and lectures
• Textbooks

58
Getting Started Summary

• Understand the basic anatomy of research
• Every project must start with a question
• Do not rush through this step
• Properly format the question using PICO
• Apply the FINER criteria to refine
• Be realistic. Do not proceed if not feasible
• Generate your study hypotheses
• Understand hypothesis testing

59
This was only getting started

• Well done studies are a lot of work
• Good luck with your next projects !

60
The End
Thank you to the SAEM for this opportunity to
participate in the meeting (The syllabus for this
lecture has been sent to SAEM and should be on
the web site soon)