Title: FORMULATING THE QUESTION PRICNIPLES OF STUDY DESIGN
1FORMULATING THE QUESTION PRICNIPLES OF STUDY
DESIGN
2Purpose of Health Services Research
- Health services research is the discipline
which seeks knowledge will lead to improvements
in the delivery of health care. - Crombie IL (1996)
3Desirable Attributesof Study Design
- Clear definition of the study question
- Precise estimate of variables/outcomes of
interest - Reliable and valid estimation of likely effects
of alternative interventions without systematic
bias - Observed effects attributable to intervention
- Observed effects generalisable to other contexts
4 5Typical scenario
- A clinician/NHS manager will come to you with an
idea for a new intervention/service. They want
you to help them decide what the impact of this
new development might be. - You then need to change this vague concept into a
researchable question
6The PICOT method
- P - Patient population who is to receive the
intervention? - I Intervention - What treatment do you need to
test the effect of? - C Comparator - What treatment are you comparing
against (could be alternative treatment or no
treatment)? - O Outcome - How is the effect of the
intervention measured? - T- Target - What is the target of the study? ie.
are you asking whether the new intervention is
'superior' to the alternative, or 'non-inferior'?
7Example
- NHS manager wants to explore the impact of
hormone replacement therapy - P population - peri-menopausal women
- I intervention - hormone replacement patches
- C comparator - No treatment
- O outcome - Effect on Bone density
- T target superiority study
8HSR question
- Original
- What is the impact of HRT?
- Revised
- Does the provision of HRT patches to
peri-menopausal women result in increased bone
density when compared with a policy of no
treatment?
9The truth is out there!
- Aim of research study is to identify the Truth
ie find the true answer to the question - However, research methods can be open to
- Systematic error bias
- Chance/random error imprecision
- Need to ensure that these errors are minimised in
any study design
10 11Bias (1)
- Any process at any stage of inference which
tends to produce results of conclusions that
differ systematically from the truth.
12Bias (2)
Biased
13Bias (3)
- Many different types of bias over 100 sources
of bias identified in analytical research. - Principal sources of bias
- ?
14Spot the potential for bias
15Bias (4)
- Many different types of bias over 100 sources
of bias identified in analytical research. - Principal sources of bias
- selection
- performance
- attrition
- detection
16Selection bias
- About
- who gets into the study and
- who gets which intervention
17Who gets included in a study
- Bias may occur if subjects selected are likely to
give systematically different answers to
population of interest - e.g. interview survey of attitudes to alcohol
consumption in which interviewer stands outside
pub.
18Who gets which intervention
- In observational/non-randomised studies, bias may
occur if there are systematic differences between
patients receiving different interventions which
influence outcome. - In randomised trials, both known and unknown
prognostic variables randomly distributed between
the treatment groups unlikely to have
systematic bias. - TC Chalmers, P Celano, HS Sacks, and H Smith NEJM
1983
19Performance Bias
- Bias (at the intervention stage) may occur due
to - non specific placebo effects of intervention
- failure to blind patients to intervention group
- failure to provide adequate control activity
20Attrition Bias
- In any research design attrition of sample may
introduce bias - More specifically
- In an RCT withdrawal, drop out or cross-over of
individuals after assignment to treatment may
introduce bias
21Detection bias
- Failure to assess outcome blindly may introduce
bias - Research has shown that failure to allocate
blindly was associated with reporting of a
greater effect of intervention. - Schultz KF, Chalmers I, Grimes DA Altman DG
(1994) JAMA - Schultz KF, Chalmers I, Hayes RJ Altman DG
(1995) JAMA
22Four main sources of bias
23Minimising Bias
- Researchers need to
- be aware of potential biases when planning
research, and - develop methods to overcome most significant
biases
24 25Precision - Background
- Research tends to use information from samples
rather than involving the total population - Sampling can lead to imprecise results (random
variation around the truth)
26PRECISION
Precise
Imprecise
27PRECISION
- Precision is a function of sample size the
larger the study, the more precise the
information - Researchers must, however, trade off sample size
and resources available to undertake a study - Statistical calculations are available to allow
researchers to determine the most effective
sample size to use (covered later in course)
28Relationship between bias and precision
29(No Transcript)
30 31Generalisability(1)
- The degree to which the results of a study hold
true for situations other than those pertaining
to the study, in particular for routine clinical
practice. - Deeks JJ, Glanville JM, Sheldon TA (1995).
Undertaking systematic reviews of - Research on effectiveness. NHS Centre for
Reviews and Dissemination
32Generalisability(2)
- Surveys often conducted in specific geographical
areas of specific groups of individuals - Important to consider how generalisable results
are to other geographical settings and groups
33Generalisability(3)
www.socialresearchmethods.net/kb/sampterm.htm
34Generalisability in RCTs
- Often have tight inclusion criteria e.g. specific
populations may be excluded - Trials may be conducted in specific settings
(e.g. secondary care settings) - Important to consider whether study results are
generalisable to patient groups and settings you
are interested in
35Generalisability(4)
- Linked to a studys
- external validity can we infer that any causal
links identified in the study can apply to
different types of persons, settings and times?
36Estimating External Validity
- Construct validity relates to whether
intervention designed or delivered in ways which
would be unlikely to be replicated in other
settings and which might modify effectiveness - External validity/applicability relates to
whether the study population was unrepresentative
of the population to which we wish to generalise
the findings
37 38Applicability (1)
- Extension of generalisability from just exploring
whether characteristics of study population are
similar to those of population we wish to
generalise to - Increasing recognition that other factors may
influence the effect of an intervention known
as effect modifiers
39Applicability(2) Potential effect modifiers
- Effect modifying factors which might influence
applicability include - patient factors (ethnicity, baseline risk)
- provider behaviour (learning curves in surgery)
- cultural factors (cultural beliefs in health
promotion behaviour)
40Applicability (3)
- Increasing interest in methods to identify which
groups of patients are likely to benefit from
treatment rather than average effect of
treatment. - Development of methods are likely to have
significant impact upon design and conduct of
research to improve applicability of study
results. - Glasziou and Irwig (1995). BMJ
41Example of differential effects
Hypothetical drug trial to reduce seizure in
epilepsy
Favours control
Favours treatment
42Some useful papers
- Schulz KF, Chalmers I, Hayes RJ, Altman DG.
Empirical evidence of bias dimensions of
methodological quality associated with estimates
of treatment effects in controlled trials. JAMA
1995 273408-12 - Jadad AR, Moore RA, Carroll D, et al. Assessing
the quality of reports of randomized clinical
trials Is blinding necessary? Controlled Clin
Trials 1996 171-12. - Moher D, Jadad AR, Tugwell P. Assessing the
quality of randomized controlled trials current
issues and future directions. Int J Tech Assess
in Health Care 1996 12195-208. - Clancy MJ, Overview of research Designs. Emerg
Med J 20021946-549 - Brennan P, Croft P. Interpreting the results
from observational research chance is not such a
fine thing BMJ 1994309727-730
43More
- Prevention of coronary heart disease with
pravastatin in men with hypercholesterolemia. J
Shepherd J, Cobbe SM, Ford I, Isles CG, Lorimer
AR, Macfarlane PW, McKillop JH and CJ Packard CJ,
for the West of Scotland Coronary Prevention
Study Group. N.Engl.J.Med. 1995333(20)1301-1307.
(has an example of a subgroup effect) - Sackett DL. Bias in analytic research. J Chron
Dis 1979325163. (hard to get a hold of this
one)