Udo Buchholz,

1
Operational research methods and examples

• Udo Buchholz,
• WHO/Stop TB/TME

2
What is operational research? (OR)

• Definitions found on the internet
• "Mathematical common sense"
• "Systematic study, by observation and experiment,
  of the working of a system, e.g. health services,
  with a view to improvement"
• "Using scientific methods to attack a complex
  problem or system"

3
In the beginning there was ... a question
NTP manager in the morning
4
Description of defaulters in Russia1

• Profession unemployed 26, labourers 21,
  students of vocational schools 19, disabled 7
• Education incomplete secondary education 70
• Residence homeless 5, gt5km away from treatment
  site 26
• Behavioural risk factors alcoholism 44

1Data are from W. Jakubowiak, Russia
5
Are these variables risk factors for default?
use of patient cohort for cohort study
6
Social support system

• Examples from different oblasts
• Food incentives
• Hygienic kits
• Free transportation
• Psychological support
• ....

7
Adherence with social support
8
More examples

• "Defaulting from anti-tuberculous treatment in a
  teaching hospital in Rio de Janeiro, Brazil"
  (IJTLD 2004)
• "A concurrent comparison of home and sanatorium
  treatment of PTB in South India" (BWHO 1959)
• " 'Lost' smear positive PTB cases where are they
  and why did we lose them?" (IJTLD 2005)

9
Determinants of a study

• Problem or question
• Data available
• Funding and staff available
• Political or hierarchical support
• ? Type of study

10
Which scientific methods can we use? - Type of
studies

• Descriptive studies
• Analysis of surveillance data
• Ecological study (correlational)
• Cross-sectional survey
• Analytical studies
• Observational (case-control study, cohort study)
• Experimental
• Other
• E.g. capture-recapture study

11
Example Surveillance data reveal large
provincial differences of ss TB/all PTB
12
No. of slides/patient is correlated with
proportion of ss/PTB
13
Ecological comparison (correlational)

• Correlation of aggregated or group data
• Association on the individual level is unknown
  and may be different
• Many relationships on global level are strictly
  speaking of ecological nature

14
Example of an "ecological" comparison The
prevalence of HIV in TB patients (y-axis) against
the prevalence of HIV in adults (x-axis).
15
Cross-sectional survey

• Collection of representative data
• Based on sampling size calculations, sampling
  frame and sampling scheme
• Simple random sample
• Systematic sampling
• Cluster sample (design effect!)

16
Surveys are frequently used in TB epidemiology

• Sampling universe is the population
• Prevalence surveys
• Tuberculin skin test surveys
• Sampling universe is "all TB patients"
• Proportion of diagnosed new TB patients with HIV
  test
• Sampling universe is the number of culture
  positive TB patients
• Drug resistance surveys

17
Analytical studies

• Are used to identify risk factors or other forms
  of "exposure" and their association with an
  outcome, e.g. death, default, etc.
• Make use of a comparison group
• Hypotheses are tested
• Null hypothesis "There is no association of
  exposure and outcome" or "Exposure and outcome
  are independent"
• ? We then calculate the probability that this is
  true based on the data

18
Case control study

• Starts with a group of cases, i.e. with a certain
  outcome, that is consistent with a case
  definition
• The case definition must be specific in regards
  to time, place and person
• E.g. "a person with smear positive TB diagnosed
  in Geneva city in 2004"
• Then select a group of persons without the
  outcome from the same population, here for
  example the general population
• From the case definition it follows "a person
  without TB living in Geneva in 2004"

19
Case control study ascertainment of exposure
status

• After identification of cases and controls the
  exposure status preceding the outcome is
  investigated
• E.g. income (high versus low)
• Thus, the directionality is usually retrospective

20
Selection of controls

• Imagine the cohort from which the cases would
  have arisen
• Or Would the control have been a case if he/she
  had had the outcome in question?
• Example cases of rare kidney disease in the Mayo
  clinic

21
Typical control options

• Friend controls
• Neighbourhood controls
• Physician controls
• Hospital controls
• Population-based controls
• Consider
• Selection bias
• Feasibility

22
2 x 2 table (CCS (1))

• 50/1000 ss TB cases (5) were poor, but only 5
  of 2000 (0.25) among the non-TB persons
• ? Ss TB patients were 20 times more likely than
  the general population to be poor, however ...

23
2 x 2 table (CCS (2))

• The chances of ssTB patients to be poor is
  expressed as the odds probability of poverty /
  prob of rich 50/1000 / 950/1000 0.053
• The odds of non TB persons for poverty is
  therefore5/2000 / 1995/2000 0.00251
• The ratio of the two odds (the odds ratio (OR))
  is 0.053/0.00251 21

24
Use of case control studies

• When type of outcome is rare
• We can examine gt1 exposure
• Usually relatively quick and inexpensive
• Disadvantages
• Not useful for rare exposures
• Because exposure is in the past watch out for
  recall bias
• Selection of cases and controls often not
  straightforward (selection bias)

25
Cohort study

• Starts with a group of people or a population
  that can be divided in two groups based on a
  defined exposure which some have and some don't
• The groups are then followed-up and an outcome is
  counted
• A case definition is still important
• The directionality is usually forward, but can
  also be backwards (retrospective cohort study)

26
2 x 2 table (cohort study)

• We follow 100 low income TB patients and 200 high
  income TB patients up for adverse outcomes
• It turns out that 20 of 100 (20) poor have a bad
  outcome versus 10 of 200 (5) of the rich.
• Thus, the poor are 4 times more likely to have an
  adverse treatment outcome.
• Measure of association is the risk ratio (RR)
  0.2/0.05 4

27
Use of cohort studies

• When exposure is rare
• We can examine gt1 outcome
• The outcome measure for the strata is an
  incidence rate or (cumulative) risk and the
  overall point estimate the rate ratio or risk
  ratio (RR)
• Disadvantages
• Not suitable for rare outcomes
• Not ideal for outcomes in the far future (unless
  you have much time or lots of scientific
  altruism)
• Watch out for loss to follow-up (they may
  represent a certain category of patients)

28
The TB quarterly "cohort"

• Pro- or retrospective cohort study
• (Nested) case-control study

Default?/Cure?
Default?/Cure?
Retro-
Pro-
Cases Controls Default Cured
Information may be available from start of
treatment
29
2 x 2 table
ill
Not ill
exposed
exposed (unprotected)
exposed (unprotected)
HEALTHY
DISEASED
not exposed
not exposed (protected)
not exposed (protected)
HEALTHY
DISEASED
30
Cohort study
healthy
ill
exposed
not exposed
31
Case control study
ill
healthy
exposed
not exposed
32
Analytical study experimental / intervention
study

• Prospective
• Use of a cohort
• Exposure is usually an intervention, a drug or
  vaccine
• Patients are ideally randomized which guarantees
  minimisation of bias
• Example IPT intervention study in South African
  gold miners recruitment in random sequence
  comparison before / after IPT phase

33
Steps for a OR protocol (1)

• Starts with a problem or question e.g. "Why is
  there no decline in urban TB in Japan?"
• Gathering of information
• Analyse exhaustively routinely collected
  (surveillance) data and disaggregate also by
  province etc
• Talk with stakeholders
• Investigation of the literature
• Contact other countries
• Develop a hypothesis
• Depending on money and staff available generate
  a protocol but this can also be used to generate
  money and staff

34
Steps for a OR protocol (2)

• Writing of the protocol
• You can structure it similar to a scientific
  paper
• Introduction/rationale
• Objective
• Methods (study type, sample size, case
  definitions used, inclusion/exclusion criteria,
  training, data collection, data entry (double
  entry?, data validation), quality control, lab
  methods, method of analysis)
• Ethical considerations
• Results shell tables, expected figures
• Timeline
• Budget
• Appendices (questionnaire, maps, consent form...)
• Good idea to do a pilot feasibility, cost, first
  crude data ?verify sample size assumptions

35
Now it is up to you