Use of Microbial Risk Assessment in Decision-Making

1
Use of Microbial Risk Assessment in
Decision-Making
Slide show on www.risk-modelling.com/firstmicrob
ial.htm

• David Vose Consultancy
• 24400 Les Lèches
• Dordogne
• France
• www.risk-modelling.com
• Email
• David Vose's secretary David Vose

2
Introduction

• Applying CODEX guidelines in reality
• Difficulties
• Other ways of thinking
• Experience with microbial modelling
• Some survey results
• The Dutch experience
• Some US experience
• Modelling challenges
• Comparison of some complete models
• Reviewing a model in context

3
Codex Alimentarius CommissionFAO/WHO (1995)

• Microbial risk assessment is a scientifically-base
  d process consisting of four steps
• Hazard Identification The identification of known
  or potential health affects associated with a
  particular agent
• Exposure Assessment The qualitative and/or
  quantitative evaluation of the degree of intake
  likely to occur
• Hazard Characterization The quantitative and/or
  qualitative evaluation of the nature of the
  adverse effects associated with biological,
  chemical and physical agents that may be present
  in food For biological agents a dose-response
  assessment should be performed if the data is
  available
• Risk Characterization Integration of Hazard
  Identification, Hazard Characterization and
  Exposure Assessment into an estimation of the
  adverse effects likely to occur in a given
  population, including attendant uncertainties.

4
OIE experience

• OIE produced guidelines for animal import risk
  assessments (for the management of disease
  spread)
• Now in its second edition
• Guidelines were offered as a way to help member
  (including developing) countries understand how
  to perform a r.a.
• First Ed. guidelines were used too literally,
  both by analysts and lawyers, and found to be
  often impractical or irrelevant to the risk
  question
• Lesson keep guidelines non-specific, encourage
  understanding rather than prescribing a formulaic
  approach
• Popular interpretation of CODEX guidelines suffer
  similarly

5
(1) Risk analysis uses observations about what
we know to make predictions about what we dont
know. Risk analysis is a fundamentally
science-based process that strives to reflect the
realities of Nature in order to provide useful
information for decisions about managing risks.
Risk analysis seeks to inform, not to dictate,
the complex and difficult choices among possible
measures to mitigate risks...
Society for Risk Analysis Principles for Risk
Analysis
6
(2) Risk analysis seeks to integrate knowledge
about the fundamental physical, biological,
social, cultural, and economic processes that
determine human, environmental, and technological
responses to a diverse set of circumstances.
Because decisions about risks are usually needed
when knowledge is incomplete, risk analysts rely
on informed judgment and on models reflecting
plausible interpretations of the realities of
Nature. We do this with a commitment to assess
and disclose the basis of our judgments and the
uncertainties in our knowledge.
Society for Risk Analysis Principles for Risk
Analysis
7
Current modelling

• Microbial QRA is a developing science
• Were making a lot of progress, but it is still
  in infancy
• Mostly producing farm-to-fork
• Models the whole system but very poorly
• Not designed to model any decision question well
• Often relies on poor data, surrogates, and
  guesses
• Almost never is a decision question posed
  beforehand
• Assessors have probably over-sold QRAs
  usefulness
• Managers have expected too much

8
F2F Achilles Heels

• Very little data available, system being modelled
  is hugely complex!
• Uncertainty, variability, inter-individual
  variablility
• Take too long to complete, too easy to make
  mistakes
• F2F considers only pathogen on the food source
• E.g. not E.coli produced during life of animal,
  appearing in water, vegetables, farmers exposure
• Predictive microbiology still unreliable
• Broth data doesnt translate well to food
  (usually overestimate, but some data Tamplin,
  USDA shows lag period can be shorter, e.g.
  E.coli in ground beef, Listeria in processed
  hams)
• Models often not based on physical/biological
  ideas, so we dont learn
• Attenuation may not be death, and ignores
  reactivation of bacteria
• D-R models inadequate
• Dont describe variability observed
• P(illdose, infected) P(illinfected)?
• Feeding trial data dont match epi data can
  hugely underestimate the risk
• Little cost-benefit analysis effort made
• Including actions affecting several risk issues
• Requires enormous resources impractical for
  many countries

9
Dutch observations on past QRAHavelaar, Jansen
(2002)

• The lessons learnt from risk analysis
  experiences
• Risk management has not always been an integral
  part of risk analysis so far
• Risk managers should be trained to understand
  risk assessment, and risk assessors should be
  trained to explain their work
• Available data are often of limited use for risk
  assessment and communication of data needs
  between risk assessors, food scientists and risk
  managers is a critical issue
• The risk manager questions usually require rapid
  results, whereas (farm-to-fork) risk assessment
  projects require several years to complete.
  Solving this conflict requires open
  communication
• Uncertainty is often large.

10
Our surveyInternet based, voluntary
participation, 39 valid responses
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
Completion times of some farm-to-fork QRAs
Final report
Final report
Draft report
Being revised
Draft report
Final report
15
Salmonella dose-responseEpi and feeding trial
comparison
Review by Amir Fazil in FAO/WHO (2001) D-R
mathematical models review by Haas (2002)
16
USDA-FSIS-FDA Salmonella Enteritidis

• Although the goal was to make the model
  comprehensive, it has some important limitations.
  It is a static model and does not incorporate
  possible changes in SE over time as either host,
  environment or agent factor change. For many
  variables, data were limited or nonexistent. Some
  obvious sources of contamination, such as food
  handlers, restaurant environment, or other
  possible sites of contamination on or in the egg
  (such as the yolk), were not included. And, as
  complex as the model is, it still represents a
  simplistic view of the entire farm-to-table
  continuum. Finally, the model does not yet
  separate our uncertainty from the inherent
  variability of the system. Much more work is
  needed to address this, and all other,
  limitations.

17
USDA-FSIS-FDA Salmonella Enteritidis

• Original model impetus was to evaluate effect of
  refrigeration temp from laying to retail on food
  safety
• Empirically must have little affect since it only
  deals with a few days in the life of an egg
• No cost-benefit attached
• Now being redone to focus on level of performance
  required for shell, and liquid egg pasteurisation
• i.e. much more decision focused

18
FDA Listeria risk assessment

• No specific decision questions attached
• Attempted to look at relative importance of a
  large list of Listeria-carrying foods
• Given the data available, perhaps the only method
  possible to estimate which food types contribute
  the greatest risk
• So a good QRA application

19
Remedies focusing on decisions

• Consider what is known about the risk problem,
  and data available immediately or within
  acceptable time frame
• Use epi data as much as possible
• Collect more epi data (e.g. Japan, Denmark)
• Consider what analysis could be done with this
  knowledge
• i.e. a risk-based reasoned argument for
  evaluating particular actions
• Estimate the possible magnitude of benefit for a
  risk action
• Note that it may not be possible to evaluate all
  actions
• Perform a cost-benefit analysis on these actions
• Perform a Value of Information analysis
• Determines whether it is worth collecting more
  data before making a decision
• Consider strategy to validate whether predicted
  improvement occurs
• Train data producers to supply maximally useful
  data
• E.g. microbiologists taken more than one cfu from
  a plate
• More inter-agency unity
• E.g. Farm (APHIS) Þ Slaughter (FSIS) Þ Retail
  (FDA)

20
Make it as simple as possible example

• Risk Human illness from SE in eggs
• A shell-egg selection system proposed that will
  reduce by 30 the number of contaminated eggs
  going to market
• Currently, 30,000 people a year suffer from SE
  from eggs
• What will be the reduction in cases if the new
  system is implemented?

Reduction in cases 3030,000 9,000
people/year
No need for models of D-R, bacterial growth,
handling, etc. Vulnerability to assumptions
smaller than from using F2F model
21
An example of the way forwardHavelaar, Jansen
2002

• Campylobacter Risk Management and Assessment
• Dutch proposal
• The main objectives of the project are to advise
  on the effectiveness and efficiency of measures
  aimed at reducing campylobacteriosis in the Dutch
  population. The two key questions are
• What are the most important routes
  (quantifiable?)?
• Which (sets of) measures can be taken to reduce
  the exposure to Campylobacter, what is their
  expected efficiency and societal support?

22
The way forward cont.

• The target of the assessment is not limited to
  estimating the possible reduction in disease
  incidence but to evaluate both costs and benefits
  of possible interventions and to access their
  acceptance by stakeholders.
• Interventions with low social support will
  require more effort to uphold, which increases
  their costs and reduces their efficacy.

23
Danish Vet Service Salmonella QRAA Bayesian
Approach to Quantify the Contribution of
Animal-food Sources to Human Salmonellosis -
Hald, Vose, Koupeev (2002)
Estimated number of cases of human salmonellosis
in Denmark in 1999 according to source Model
ranks food sources by risk. Easily updateable
with each years data. Bayesian update improves
estimate and checks validity of assumptions.
24
Fluoroquinolone-resistant Campylobacter risk
assessment
Model Contaminated carcasses after slaughter
plant probability affected people
25
Broiler
house



Transport

Slaughterhouse
model


Slaughter house

Hanging

Example of Farm-to-Fork model Campylobacter
in poultry Draft report 2001 Institute of Food
Safety and Toxicology Division of Microbiological
Safety Danish Veterinay and Food Administration
Scalding

Defeathering

Evisceration

Washing

Chilling

Export

Chicken parts
Whole
chickens



Chilled
Frozen


Further

Import

Processing

Retail

Catering

Cross

contamination

Consumer
model


Heat

treatment

Consumer



Cross
contamination
Behaves the same way as CVM model if prevalence
is reduced


Heat
treatment
Dose response

Risk estimation

26
AHI model Fluoroquinolone resistant Campylobacter
in poultry AHI / Cox 2000 A dynamic simulation
model (i.e. follow the path of a random
chicken) Used same data as CVM where
available. Strong potential because it reduces
model complexity (at the expense of simulation
time)and easy to follow. Difficulty is
availability of data to make model parameters
meaningful. Most model parameters in current
version represent nothing physical (factors),
so dont enlighten us as to what actions to
take. D-R and consumer handling difficulties
remain.
27
Reviewing a risk assessment

• Risk assessment should be decision focused
• It is not appropriate to review a risk assessment
  independently from the question(s) the assessment
  is addressing
• Eg because a point is moot if the decision is
  insensitive to the argument
• It uses science but is not itself scientific
  research
• So we have to go with the best weve got

28
Finally - risk assessors should gain hands-on
experience to ensure their models reflect the
real world
29
References

• Haas, C.N., (2002), Conditional Dose-Response
  Relationships for Micro-organisms Development
  and Applications. Risk Analysis 22 (3) 455-464.
• Havelaar, H. and J. Jansen, (2002), Practical
  Experience in the Netherlands with quantitative
  microbiological risk assessment and its use in
  food safety policy. Draft paper, RIVM, Bilthoven,
  The Netherlands.
• Hope, B.K., et al. , (2002), An overview of the
  Salmonella Enteritidis Risk Assessment for Shell
  Eggs and Egg Products. Risk Analysis 22 (3)
  455-464.
• Joint FAO/WHO Expert Consultation on the
  Application of Risk Analysis to Food Standards
  Issues (Joint FAO/WHO, 1995).
• Joint FAO/WHO Expert Consultation on Risk
  Assessment of Microbiological Hazards in Foods
  Risk characterization of Salmonella spp. in eggs
  and broiler chickens and Listeria monocytogenes
  in ready-to-eat foods. (2001), FAO headquarters,
  Rome.
• Teunis, P.F.M. and A.H.Havelaar, (2001), The
  Beta-Poisson Dose-Response Model Is Not a
  Single-Hit Model. Risk Analysis 20 (4) 513-520.