Microbial Risk Assessment: lessons learned and future directions

1
Microbial Risk Assessment lessons learned and
future directions

• Greg Paoli
• Decisionalysis Risk Consultants, Inc.
• Ottawa Canada

2
Or, A Risk Assessment of Microbial Risk
Assessment
3
One View of the Processing Stages in Risk
Assessment

• Problem Selection
• Model DMs Values
• Outcome Selection
• Scope Selection
• Tool Selection
• Evidence Acquisition
• Evid. Characterization
• Model Development

• Diagnosis and Experimentation
• Validation-Seeking
• Auditing
• Documentation
• Peer Review
• Communication with Risk Managers
• Dissemination

4
Hazard Identification

• Managerial Hazards
• Scope Hazards
• Evidence Hazards
• Computational Hazards
• Characterization Hazards
• Communication Hazards

5
Managerial Hazards

• Silence regarding Values
• Valuation of Knowledge Gains
• Linear Processes
• Limited Tool Development

6
Scope Hazards

• Keep it simple!
• This is preposterously simple. You must include
  the complexity or it can have no credibility!
• Scope decisions are qualitative risk assessments
• Under-valuation of multi-use mitigations
• Model resolution balance
• Risk-risk tradeoff considerations

7
Evidence Hazards

• Numerical Hazards (Mean Log Example)
• Evidence regarding Process Deviations
• Regional and Temporal Issues
• Human Factors
• Cross-contamination
• Model Uncertainty

8
Mean Log Example
2
5
2
1
2
1
2
1
2
2
9
Mean Log Example
100
100000
100
10
100
10
100
10
400
100030
Risks Differ by a Multiple of 250
10
Mean Log Example
4
5
3
1
4
1
3
1
3.5
2
Would this be considered an increase in risk?
11
Mean Log Example
10000
100000
1000
10
10000
10
1000
10
22000
100030
Risks Still Differ by a Multiple of 5
12
Characterizing the Extremes

• Illnesses result from combinations of rare events
• We can expect to characterize most everyday
  processes reasonably well.
• The devil is in the tails
• no data on the magnitude and probability of
  deviations.
• Need to prove that this as a common and dominant
  phenomenon

13
Cross-Contamination The Final Frontier

• Is it unmodellable for the population?
• How big and black can a black box be?
• How do we incorporate and work with model
  elements for which our state of knowledge can be
  best described as ignorance?

14
Uncertainty Characterization

• If we express our uncertainty for most, but not
  all, variables or model assumptions, can we still
  say we have captured uncertainty.
• How much uncertainty is enough?
• Simple Answer all of it
• Whats the Real-Life Answer?

15
Computational Hazards

• Risk Estimate Stability (Rare Events)
• Auditability and Error-Proneness
• Transparency (Strict vs. Real)
• Time consumption (e.g. 2-D Models)
• Inability to Provide Real-Time Decision Support
  or Managerial Learning
• Lack of Diversity in Approach

16
Characterization Hazards

• Choice of Measures
• Risk to Susceptibles
• Population Risk
• Risk per Serving/Preparation/Kilogram/Batch
• How Much Uncertainty has to be Included to Pass
  the Uncertainty Test
• Oversimplified Sensitivity Analysis

17
Communication Hazards

• Dissemination of Models for Review
• Expressing the Magnitude of Uncertainty
• How many audiences can we serve?
• Can stakeholders be meaningfully engaged in
  complex risk assessments?

18
Solution Sets

• Methodological Research
• Tool Diversification
• Modular System Characterization
• Linkages with Epidemiology
• Food Safety Objectives

19
Tool Diversification

• When all you have is a hammer
• Process Risk Models
• Analytical Models
• Bayesian Network Models
• Expert Systems
• Model Learning
• Causal Models
• Qualitative Risk Assessment
• Module Librairies

20
Tool Diversification

• Within PRM Approaches
• Diversify software, and/or
• Perform Needs Assessment
• Performance Comparisons
• Good Modelling Practice

21
Qualitative Risk Assessment

• Prone to Inferential Sloppiness
• A literature search with conclusions?
• It is possible to impose structure, but its not
  always welcome
• Absence of an Inferential Trail
• We should be cautious about conferring the label
  Risk Assessment to anything that uses the right
  terminology.

22
Risk Modules not Risk Assessments

• Need to build microbial risk assessment
  infrastructure
• Development away from the bright lights
• Carefully documented and computationally sound
• Examples of appropriate implementations
• Documented with limitations and caveats
• Shared library and shared experiences
• With this infrastructure, risk assessment can
  only get better, easier, more reliable

23
Linkages with Epidemiology

• Attributable Risk Problem
• Integration of Case-Control findings for sporadic
  cases
• Approaches for model validation
• Epidemiology re-thinking causality criteria
• Epi needs biological plausibility
• QRA needs epidemiological evidence

24
FSOs, Risk Measures

• Surrogate Variables
• Performance Indices
• Concrete Linkages to Objectives
• Qualitative Measures of Risk
• Farm-Level Measures

25
Process Characterization

• Distribution of Concentration across Units
• Prevalence of Contamination
• Indicator Levels
• Competitive Flora
• Homogeneity
• Distribution of Strains Present

• Information Provided with Food to Affect Later
  Handling
• Downstream Processing
• Extent of Pooling
• Growth Inhibitors
• Packaging and Insulation
• Information to Affect Traceback and Recall
• Target Consumer

26
A Tiered Approach to Methodology Development

• Tier 1
• systems modelling
• Multi-pathogen, indicators, sampling results,
  responses to deviations
• Modular abstraction to simpler forms
• Modular integration tools

27
Tiered Approach (Tier 2)

• risk assessment
• farm to fork
• plant to fork
• farm to plant
• Supplier to purchaser

28
Tiered Approach (Tier 3)

• risk-based expertise capture
• A Risk-Based Process Inspector
• Formal, structured qualitative analysis
• Advice replicates quantitative findings through
  querying the problem in qualitative terms as well
  as some quantitative data.

29
In the Defense of Risk Assessment

• Bending over backwards to meet the demands for
• the best and up-to-date data
• the best model and modelling technique
• the best software implementation
• good documentation of model
• high-quality report
• technical appendices for peer review
• non-technical summaries
• peer-reviewed publications
• all at once, without a safety net

30
In the Defense of Risk Assessment Are we
Shooting the Messenger?

• Systems are too complex for human reasoning
• Currently, Microbial Risk Assessors are
  Methodological Researchers
• Carefully formalizing some of the reasons why we
  have not been successful in the past
• We dont acknowledge the complexity
• Weve never really understand the systems well
  enough to control them reliably.
• Value of Information analysis sorely needed.

31
Conclusions

• Methodological Research is Key to the Future of
  Microbiological Risk Assessment
• There is no time to think!
• Diversify the portfolio of approaches
• Promote competition
• Manage expectations
• Assess knowledge gains
• Fairly evaluate alternative approaches