FORMULATING THE QUESTION PRICNIPLES OF STUDY DESIGN - PowerPoint PPT Presentation

1 / 43

About This Presentation

Title:

FORMULATING THE QUESTION PRICNIPLES OF STUDY DESIGN

Description:

P population - peri-menopausal women. I intervention - hormone replacement patches ... provider behaviour (learning curves in surgery) ... – PowerPoint PPT presentation

Number of Views:58

Avg rating:3.0/5.0

Slides: 44

Provided by: abd1

Category:

more less

Transcript and Presenter's Notes

Title: FORMULATING THE QUESTION PRICNIPLES OF STUDY DESIGN

1
FORMULATING THE QUESTION PRICNIPLES OF STUDY
DESIGN

Marion Campbell

2
Purpose of Health Services Research

Health services research is the discipline
which seeks knowledge will lead to improvements
in the delivery of health care.
Crombie IL (1996)

3
Desirable Attributesof Study Design

Clear definition of the study question
Precise estimate of variables/outcomes of
interest
Reliable and valid estimation of likely effects
of alternative interventions without systematic
bias
Observed effects attributable to intervention
Observed effects generalisable to other contexts

DEFINING THE QUESTION

5
Typical scenario

A clinician/NHS manager will come to you with an
idea for a new intervention/service. They want
you to help them decide what the impact of this
new development might be.
You then need to change this vague concept into a
researchable question

6
The PICOT method

P - Patient population who is to receive the
intervention?
I Intervention - What treatment do you need to
test the effect of?
C Comparator - What treatment are you comparing
against (could be alternative treatment or no
treatment)?
O Outcome - How is the effect of the
intervention measured?
T- Target - What is the target of the study? ie.
are you asking whether the new intervention is
'superior' to the alternative, or 'non-inferior'?

7
Example

NHS manager wants to explore the impact of
hormone replacement therapy
P population - peri-menopausal women
I intervention - hormone replacement patches
C comparator - No treatment
O outcome - Effect on Bone density
T target superiority study

8
HSR question

Original
What is the impact of HRT?
Revised
Does the provision of HRT patches to
peri-menopausal women result in increased bone
density when compared with a policy of no
treatment?

9
The truth is out there!

Aim of research study is to identify the Truth
ie find the true answer to the question
However, research methods can be open to
Systematic error bias
Chance/random error imprecision
Need to ensure that these errors are minimised in
any study design

BIAS

11
Bias (1)

Any process at any stage of inference which
tends to produce results of conclusions that
differ systematically from the truth.

12
Bias (2)

Unbiased

Biased
13
Bias (3)

Many different types of bias over 100 sources
of bias identified in analytical research.
Principal sources of bias
?

14
Spot the potential for bias
15
Bias (4)

Many different types of bias over 100 sources
of bias identified in analytical research.
Principal sources of bias
selection
performance
attrition
detection

16
Selection bias

About
who gets into the study and
who gets which intervention

17
Who gets included in a study

Bias may occur if subjects selected are likely to
give systematically different answers to
population of interest
e.g. interview survey of attitudes to alcohol
consumption in which interviewer stands outside
pub.

18
Who gets which intervention

In observational/non-randomised studies, bias may
occur if there are systematic differences between
patients receiving different interventions which
influence outcome.
In randomised trials, both known and unknown
prognostic variables randomly distributed between
the treatment groups unlikely to have
systematic bias.
TC Chalmers, P Celano, HS Sacks, and H Smith NEJM
1983

19
Performance Bias

Bias (at the intervention stage) may occur due
to
non specific placebo effects of intervention
failure to blind patients to intervention group
failure to provide adequate control activity

20
Attrition Bias

In any research design attrition of sample may
introduce bias
More specifically
In an RCT withdrawal, drop out or cross-over of
individuals after assignment to treatment may
introduce bias

21
Detection bias

Failure to assess outcome blindly may introduce
bias
Research has shown that failure to allocate
blindly was associated with reporting of a
greater effect of intervention.
Schultz KF, Chalmers I, Grimes DA Altman DG
(1994) JAMA
Schultz KF, Chalmers I, Hayes RJ Altman DG
(1995) JAMA

22
Four main sources of bias
23
Minimising Bias

Researchers need to
be aware of potential biases when planning
research, and
develop methods to overcome most significant
biases

PRECISION

25
Precision - Background

Research tends to use information from samples
rather than involving the total population
Sampling can lead to imprecise results (random
variation around the truth)

26
PRECISION
Precise
Imprecise
27
PRECISION

Precision is a function of sample size the
larger the study, the more precise the
information
Researchers must, however, trade off sample size
and resources available to undertake a study
Statistical calculations are available to allow
researchers to determine the most effective
sample size to use (covered later in course)

28
Relationship between bias and precision
29
(No Transcript)
30

GENERALISABILITY

31
Generalisability(1)

The degree to which the results of a study hold
true for situations other than those pertaining
to the study, in particular for routine clinical
practice.
Deeks JJ, Glanville JM, Sheldon TA (1995).
Undertaking systematic reviews of
Research on effectiveness. NHS Centre for
Reviews and Dissemination

32
Generalisability(2)

Surveys often conducted in specific geographical
areas of specific groups of individuals
Important to consider how generalisable results
are to other geographical settings and groups

33
Generalisability(3)
www.socialresearchmethods.net/kb/sampterm.htm
34
Generalisability in RCTs

Often have tight inclusion criteria e.g. specific
populations may be excluded
Trials may be conducted in specific settings
(e.g. secondary care settings)
Important to consider whether study results are
generalisable to patient groups and settings you
are interested in

35
Generalisability(4)

Linked to a studys
external validity can we infer that any causal
links identified in the study can apply to
different types of persons, settings and times?

36
Estimating External Validity

Construct validity relates to whether
intervention designed or delivered in ways which
would be unlikely to be replicated in other
settings and which might modify effectiveness
External validity/applicability relates to
whether the study population was unrepresentative
of the population to which we wish to generalise
the findings

APPLICABILITY

38
Applicability (1)

Extension of generalisability from just exploring
whether characteristics of study population are
similar to those of population we wish to
generalise to
Increasing recognition that other factors may
influence the effect of an intervention known
as effect modifiers

39
Applicability(2) Potential effect modifiers

Effect modifying factors which might influence
applicability include
patient factors (ethnicity, baseline risk)
provider behaviour (learning curves in surgery)
cultural factors (cultural beliefs in health
promotion behaviour)

40
Applicability (3)

Increasing interest in methods to identify which
groups of patients are likely to benefit from
treatment rather than average effect of
treatment.
Development of methods are likely to have
significant impact upon design and conduct of
research to improve applicability of study
results.
Glasziou and Irwig (1995). BMJ

41
Example of differential effects
Hypothetical drug trial to reduce seizure in
epilepsy
Favours control
Favours treatment
42
Some useful papers

Schulz KF, Chalmers I, Hayes RJ, Altman DG.
Empirical evidence of bias dimensions of
methodological quality associated with estimates
of treatment effects in controlled trials. JAMA
1995 273408-12
Jadad AR, Moore RA, Carroll D, et al. Assessing
the quality of reports of randomized clinical
trials Is blinding necessary? Controlled Clin
Trials 1996 171-12.
Moher D, Jadad AR, Tugwell P. Assessing the
quality of randomized controlled trials current
issues and future directions. Int J Tech Assess
in Health Care 1996 12195-208.
Clancy MJ, Overview of research Designs. Emerg
Med J 20021946-549
Brennan P, Croft P. Interpreting the results
from observational research chance is not such a
fine thing BMJ 1994309727-730

43
More

Prevention of coronary heart disease with
pravastatin in men with hypercholesterolemia. J
Shepherd J, Cobbe SM, Ford I, Isles CG, Lorimer
AR, Macfarlane PW, McKillop JH and CJ Packard CJ,
for the West of Scotland Coronary Prevention
Study Group. N.Engl.J.Med. 1995333(20)1301-1307.
(has an example of a subgroup effect)
Sackett DL. Bias in analytic research. J Chron
Dis 1979325163. (hard to get a hold of this
one)