Science and Decisions: Advancing Risk Assessment

1
Science and Decisions Advancing Risk
Assessment

• Risk Assessment Specialty Section (RASS) Monthly
  Telecon
• Joseph Rodricks, Environ
• Jonathan Levy, Harvard School of Public Health
• May 13, 2009

2
Study motivation

• Risk assessment is at a crossroads, and its
  credibility is being challenged.
• Science is increasingly complex.
• Risk assessment is being extended to address
  broader environmental questions, such as
  life-cycle analysis and issues of costs,
  benefits, and risk-risk tradeoffs.
• Stakeholders are often disengaged from the
  risk-assessment process at a time when risk
  assessment is increasingly intertwined with
  societal concerns.
• Disconnects between the available scientific data
  and the information needs of decision-makers
  hinder the use of risk assessment as a
  decision-making tool.

3
Committees charge (I)

• An NRC committee will develop scientific and
  technical recommendations for improving the risk
  analysis approaches used by the U.S.
  Environmental Protection Agency (EPA)The
  committee will consider analyses applied to
  contaminants in all environmental media (water,
  air, food, soil) and all routes of exposure
  (ingestion, inhalation, and dermal absorption).
  The committee will focus primarily on human
  health risk analysis and will comment on the
  broad implications of its findings and
  recommendations to ecological risk analysis. In
  making recommendations, the committee will
  indicate practical improvements that can be made
  in the near term (2-5 years) and improvements
  that would be made over a longer term (10-20
  years).

4
Committees charge (II)

• Increased role for probabilistic analysis in risk
  analysis, including the potential expanded role
  for expert elicitation
• Scientific bases for and alternatives to default
  assumption choices made in areas of uncertainty
• Quantitative characterization of uncertainty
  resulting from all steps in the risk analysis
• Approaches for assessing cumulative risk
  resulting from multiple exposures to contaminant
  mixtures, involving multiple sources, pathways,
  routes
• Variability in receptor populations, especially
  sensitive subpopulations and critical life stages

5
Committees charge (III)

• Biologically relevant modes of action for
  estimating dose-response relationships, and
  quantitative implications of different modes
• Improvements in environmental transport and fate
  models, exposure models, physiologically based
  pharmacokinetic (PBPK) models, and dose-response
  models
• How the concepts and practices of ecological risk
  analysis can help inform and improve the concepts
  and practices of human health risk analysis, and
  vice versa
• Scientific basis for derivation of uncertainty
  factors
• Use of value-of-information analyses and other
  techniques to identify priorities and approaches
  for research to obtain relevant data to increase
  the utility of risk analyses

6
Committee membership

• Thomas Burke (Chair), Johns Hopkins Bloomberg
  School of Public Health
• A. John Bailer, Miami University
• John M. Balbus, Environmental Defense
• Joshua T. Cohen, Tufts Medical Center
• Adam M. Finkel, University of Medicine and
  Dentistry of New Jersey
• Gary Ginsberg, Connecticut Department of Public
  Health
• Bruce K. Hope, Oregon Department of Environmental
  Health
• Jonathan I. Levy, Harvard School of Public Health
• Thomas E. McKone, University of California
• Gregory M. Paoli, Risk Sciences International
• Charles Poole, University of North Carolina
  School of Public Health
• Joseph V. Rodricks, ENVIRON International
  Corporation
• Bailus Walker Jr., Howard University Medical
  Center
• Terry F. Yosie, World Environment Center
• Lauren Zeise, California Environmental Protection
  Agency

7
Evaluation strategy

• Committee concluded early on that risk assessment
  can be improved in two different ways
• Improving technical analysis (the development and
  use of scientific knowledge and information to
  improve characterizations of risk)
• Improving utility (making risk assessment more
  relevant and useful to risk management decisions)

8
Structure of report conclusions

• Design of risk assessment
• Uncertainty and variability
• Selection and use of defaults
• Unified approach to dose-response assessment
• Cumulative risk assessment
• Improving the utility of risk assessment
• Stakeholder involvement
• Capacity-building

9
Design of risk assessment

• Design The process of planning a risk assessment
  and ensuring that it has the attributes desired
  by the decision maker given various system
  constraints
• Analogy to product design What is the right
  car to buy?
• From decision-support perspective, there are
  multiple desirable attributes which may at times
  conflict with one another
• Use of best science and methods
• Inclusiveness of scope
• Inclusiveness of process
• Transparency
• Timeliness

10
Key conclusions

• Increased attention needed to the design of risk
  assessment at its formative stages
• Planning and scoping and problem formulation (as
  in EPA ecological and cumulative risk guidance)
  should be formalized and implemented.

11
Key design steps

• Planning and scoping Discussion among
  decision-makers, assessors, and stakeholders to
  establish issue to be assessed and goals,
  breadth, depth, and focus of assessment
• Problem formulation Technical implications of
  planning and scoping, with a conceptual model and
  an analysis plan
• Early identification of decision-making options

12
Related concept Value of information (VOI)

• Decision makers face the tension between acting
  now or delaying decisions while research is
  conducted
• VOI analysis offers a framework for systematic
  examination of this issue
• Part of committees charge, topic of interest to
  EPA
• Key point of emphasis
• VOI analysis is not possible in the absence of a
  structured analysis that includes information
  about risk management options and detailed
  uncertainty characterization

13
Conclusions about VOI

• VOI analysis is technically challenging for any
  realistic problem structure, may place
  unrealistic demands on science
• Needs knowledge of decision rules, well-defined
  uncertainty characterization, assumes primacy of
  risk assessment outputs
• Formal VOI may be impractical in most settings,
  but the essential reasoning behind VOI can be
  adopted
• Understanding the causal link between a specific
  source of information, how a decision-maker would
  change behaviors given this information, and how
  this could improve decisions
• Similar concepts can be applied to consider the
  value of methods rather than the value of
  information

14
Uncertainty

• A huge, cross-cutting topic
• There have been substantial differences among
  EPAsapproaches to and guidance for addressing
  uncertainty in exposure and dose-response
  assessment.
• The level of detail for characterizing
  uncertainty isappropriate only to the extent
  that it is needed to inform specific
  risk-management decisions appropriately.
• Inconsistency in the treatment of uncertainty
  among components of a risk assessment can make
  the communication of uncertainty difficult and
  sometimes misleading.
• Example How do you interpret Monte Carlo
  analysis with emissions assumed to be known,
  exposure model with limited parametric
  uncertainty, and epidemiology with uncertainty
  only related to studys statistical power?

15
Tiers of uncertainty analysisExample WHO, 2007

• Tier 0 Default assumptions, single value for
  result
• Tier 1 Qualitative but systematic identification
  and characterization of uncertainties
• Tier 2 Quantitative evaluation of uncertainty
  making use of bounding values, interval analysis,
  sensitivity analyses
• Tier 3 Probabilistic assessments with single or
  multiple outcome distributions reflecting
  uncertainty and variability

16
Variability

• Variability in human susceptibility has not
  received sufficient or consistent attention in
  many EPA health risk assessments
• Committee encourages EPA to move toward the
  long-term goal of quantifying population
  variability more explicitly in exposure
  assessment and dose-response relationships.

17
General conclusions re uncertainty and variability

• EPA should encourage risk assessments to
  characterize and communicate uncertainty and
  variability in all key computational steps.
• Uncertainty and variability analysis should be
  planned and managed to reflect the needs for
  comparative evaluation of risk management
  options.
• In the short term, EPA should adopt a tiered
  approach for selecting the level of detail in
  uncertainty and variability assessments
• This should be made explicit in the planning
  stage.
• EPA should develop guidance on the appropriate
  level of detail needed in uncertainty and
  variability analyses
• Provide clear definitions and methods for
  identifying and addressing different sources of
  uncertainty and variability.

18
Related topic Defaults

• Also called inference options, default
  options, science policy, risk assessment
  policy
• The best choice for parameters/models on the
  basis of risk assessment policy in the absence of
  data to the contrary
• By definition, cannot be proven
  correct/incorrect, but often has some scientific
  underpinning
• First formalized in the Red Book

19
Selection and use of defaults

• Established defaults need to be maintained for
  risk assessment steps that require inferences
• EPA, for the most part, has not yet published
  clear, general guidance on what level of
  evidence is needed tojustify use of
  agent-specific data instead of a default.
• Clear criteria should be available for judging
  whether, in specific cases, data are adequate for
  direct use or to support an inference in place of
  a default.
• There are a number of defaults (missing or
  implicit defaults) that are engrained in EPA
  risk-assessment practice but are absent from its
  risk-assessment guidelines.
• EPA does not quantify uncertainty when default
  assumptions are used.

20
Recommendations re defaults

• EPA should
• continue and expand use of the best, most current
  science to support and revise default
  assumptions.
• develop clear, general standards for the level of
  evidence needed to justify the use of alternative
  assumptions in place of defaults.
• work toward the development of explicitly stated
  defaults to take the place of implicit defaults.

21
Evidentiary standard for replacing default

• The committee recommends that EPA adopt an
  alternative assumption in place of a default when
  it determines that the alternative is clearly
  superior, that its plausibility clearly exceeds
  the plausibility of the default.

22
Showing uncertainty when using defaults

• To the extent feasible, EPA should move beyond
  qualitative description of uncertainty when
  default assumptions are used
• Long term improved probabilistic description of
  uncertainty commensurate with risk management
  needs
• Short term criteria for listing alternative
  values
• Goal Provide sensitivity analysis to illustrate
  impact of alternative assumptions and hence
  characterize robustness of risk estimates
• Limit attention to assumptions with plausibility
  comparable to the default
• Goal is not an exhaustive presentation of
  plausible estimates

23
Unification approach to dose-response assessment

• Historically, dose-response assessments at EPA
  conducted differently for cancer and noncancer
  effects
• Methods have been criticized for not providing
  the most useful results.
• A consistent approach to risk assessment for
  cancer and noncancer effects is scientifically
  feasible and needs to be implemented.

24
Current approach
25
Thoughts re current approach

• EPA has taken important steps to harmonize cancer
  and noncancer approaches, but with many
  scientific and operational limitations
• Noncancer effects do not necessarily have
  threshold or low-dose nonlinearity
• The mode of action of carcinogens varies.
• Background exposures and underlying disease
  processes contribute to population background
  risk, which can lead to linearity at the
  population doses of concern.
• RfDs and RfCs do not quantify risk for different
  magnitudes of exposure but rather provide a
  bright line with limited use in risk-management
  decision-making
• Cancer risk assessments usually do not account
  for human differences in cancer susceptibility
  (other than possible differences in early-life).

26
Dose-response relationship is dependent on
heterogeneity in background exposure (endogenous
and xenobiotic), biological susceptibility
27
Unification recommendations

• A consistent, unified approach for dose-response
  modeling that includes formal, systematic
  assessment of
• background disease processes and exposures
• possible vulnerable populations
• modes of action that may affect a chemicals
  dose-response in humans.
• Redefine the RfD or RfC as a risk-specific dose
• provides information on the percentage of the
  population expected above or below a defined
  acceptable risk (with specific degree of
  confidence).
• Formal introduction of variability into cancer
  dose-response modeling
• Will require implementation and development
• As new chemicals are assessed or old chemicals
  are reassessed
• Of test cases to demonstrate proof of concept.

28

29
Diagnostic questions to aid dose-response
assessment

• What is known or suspected to be the chemicals
  MOA?
• What underlying degenerative or disease processes
  might the toxicant effect?
• What are the background incidences and population
  distributions of these processes?
• Are there identified sensitive populations?
• What environmental contaminants in air, drinking
  water, food or in consumer products (e.g., in
  cosmetics) or endogenous chemicals (e.g., natural
  hormones) are similar to the chemical?
• Could they potentially operate by MOAs similar to
  that of the chemical in question?
• What chemicals might operate by a different MOA
  but have the potential to affect the same toxic
  process as the chemical under study?
• Can subgroups with particularly high exposures be
  identified?
• more

30
Conceptual dose-response models Based on MOA,
background exposure and disease processes
31
General structures of the three models

• Model 1 Estimate BMD, determine human POD with
  uncertainty, extrapolate linearly
• Like current cancer framework, but applied to
  non-cancer
• Model 2 Estimate BMD, determine human POD with
  uncertainty, determine risk-specific RfD given
  human heterogeneity
• Like proposals in the literature by Hattis, Evans
• Model 3 Estimate BMD, incorporate
  interindividual variability factor, extrapolate
  linearly while retaining variability
• Like current cancer framework with variability
  factor

32
Cumulative risk assessment

• EPA is increasingly asked to address broad
  public-health and environmental-health issues
  that stakeholder groups often consider
  inadequately captured by current risk assessments
• multiple exposures
• complex mixtures
• vulnerability of exposed populations
• There is a need for cumulative risk assessments
  as defined by EPA that include
• combined risks posed by exposure to multiple
  agents or stressors
• aggregate exposure to a given agent or stressor
• all routes, pathways, and sources of exposure
• Chemical, biologic, radiologic, physical, and
  psychologic stressors are considered.

33
Conclusions on cumulative risk

• Committee applauds the agencys move toward the
  broader definition, making risk assessment more
  informative and relevant to decisions and
  stakeholders.
• However, in practice, EPA risk assessments often
  fall short of what is possible and supported by
  agency guidelines.
• Little consideration of nonchemical stressors,
  vulnerability, and background risk factors.
• Because of the complexity of considering so many
  factors simultaneously, there is a need for
• Simplified risk assessment tools
• Orientation around pertinent risk management
  options to limit the number of stressors under
  formal consideration

34
Step 1 Develop a conceptual model for the stressors of primary interest for the analysis (stressors that would be significantly influenced by any of the risk-management options under study). MOA assessment, assessment of background exposures to chemical and nonchemical stressors that may affect the same health outcome, vulnerability assessment that takes into account underlying disease processes in the population Identify the receptors and end points affected by these stressors. Review the conceptual model and stressors, receptors, and end points of interest with stakeholders in initial planning and scoping.
Step 2 Use epidemiologic and toxicologic evidence and screening-level benefit calculations to provide an initial evaluation of which stressors should be included in the cumulative risk assessment. Gather stakeholder feedback and review and re-evaluate planning and scoping for the analysis. Focus the assessment only on stressors that contribute to end points of interest for risk-management options and are either differentially affected by different control strategies or influence the benefits of stressors that are differentially affected.
Step 3 Evaluate benefits of different risk-management options with appropriate characterization of uncertainty, including quantification of the effects of individual stressors and bounding calculations of any possible interaction effects.
Step 4 If Step 3 is sufficient to discriminate among risk-management options given other economic, social, and political factors, conclude the analysis otherwise, sequentially refine the analysis as needed, taking into account potential interactions.
35
Recommendations

• Draw on other approaches to incorporate
  interactions between chemical and non-chemical
  stressors in assessments, including those from
  ecologic risk assessment and social epidemiology
• Develop guidelines and methods for simpler
  analytical tools
• to support cumulative risk assessment
• to provide for greater involvement of
  stakeholders.
• In short-term, develop databases and default
  approaches to allow for incorporation of key
  non-chemical stressors in the absence of
  population-specific data, considering
• exposure patterns
• contributions to relevant background processes
• interactions with chemical stressors.
• In long-term, invest in research programs related
  to interactions between chemical and non-chemical
  stressors, including epidemiologic investigations
  and physiologically-based pharmacokinetic
  modeling.

36
Improving the utility of risk assessment

• Committee proposes a framework for risk-based
  decision-making
• At its core are the 4 steps as defined in the
  Red Book
• Key difference from the Red Book in the initial
  and final steps
• Framework asks implicitly
• What options are there to reduce the hazards or
  exposures that have been identified, and
• How can risk assessment be used to evaluate the
  merits of the various options?
• Risk assessment as a means to an end

37
Improving the utility (II)

• Under this framework, the questions posed arise
  from
• early and careful planning of the types of
  assessments (including risks, costs, and
  technical feasibility) and
• the required level of scientific depth needed to
  evaluate the relative merits of the options being
  considered.
• Risk management involves choosing among the
  options after the appropriate assessments have
  been undertaken and evaluated.

38
(No Transcript)
39
Phase I Problem Formulation and Scoping

• What is the problem to be investigated, and what
  is its source?
• What are the possible opportunities for managing
  risks associated with the problem? Has a full
  array of possible options been considered,
  including legislative requirements?
• What types of risk assessments and other
  technical and cost assessments are necessary to
  evaluate existing conditions and how the various
  risk-management options alter the conditions?
• What impacts other than health and ecosystem
  threats will be considered?
• How can the assessments be used to support
  decisions?
• What is the required timeframe for completion of
  assessments?
• What resources are needed to undertake the
  assessments?

40
Phase IIPlanning and conduct of risk assessment

• Stage 1 Planning
• For the given decision-context, what are the
  attributes of assessments necessary to
  characterize risks of existing conditions and the
  effects on risk of proposed options?
• What level of uncertainty and variability
  analysis is appropriate?
• Stage 2 Risk Assessment
• Stage 3 Confirmation of Utility
• Does the assessment have the attributes called
  for in planning?
• Does the assessment provide sufficient
  information to discriminate among risk-management
  options?
• Has the assessment been satisfactorily peer
  reviewed?

41
Phase IIIRisk management

• What are the relevant health or environmental
  benefits of the proposed risk-management options?
  
• How are other decision-making factors
  (technologies, costs) affected by the proposed
  options?
• What is the decision, and its justification, in
  light of benefits, costs, and uncertainties in
  each?
• How should the decision be communicated?
• Is it necessary to evaluate the effectiveness of
  the decision? If so, how should this be done?

42
The framework

• Systematically identifies problems and options
  that risk assessors should evaluate at the
  earliest stages of decision-making.
• Expands the array of impacts assessed beyond
  individual effects (e.g., respiratory effects) to
  include broader questions (e.g., health status
  and ecosystem protection).
• Provides a formal process for stakeholder
  involvement.
• Increases understanding of the strengths and
  limitations of risk assessment by decision-makers
  at all levels, e.g., by making uncertainties and
  choices more transparent.
• Maintains the conceptual distinction between risk
  assessment and risk management articulated in
  the Red Book.

43
Framework recommendation

• EPA should adopt a framework for risk-based
  decision-making that embeds the Red Book risk
  assessment paradigm into a process with
• initial problem formulation and scoping,
• upfront identification of risk-management
  options, and
• use of risk assessment to discriminate among
  these options.

44
Stakeholder involvement

• Many stakeholders believe that the current
  process for developing and applying risk
  assessments lacks credibility and transparency.
• Greater stakeholder involvement is necessary to
  ensure that the process is transparent and to
  ensure risk-based decision-making proceeds
  effectively, efficiently, and credibly.
• Stakeholder involvement needs to be an integral
  part of the risk-based decision-making framework.
• It is important that EPA adhere to its own
  guidance on stakeholder involvement particularly
  in the context of cumulative risk assessment, in
  which communities often have not been adequately
  involved.

45
Stakeholder recommendations

• EPA should establish a formal process for
  stakeholder involvement in the framework for
  risk-based decision-making with
• time limits to ensure that decision-making
  schedules are met
• incentives to allow for balanced participation of
  stakeholders including impacted communities and
  less advantaged stakeholders.

46
Concluding thoughts

• The committee felt that, in spite of limitations,
  risk assessment remains essential to EPAs
  mission to ensure protection of public health and
  the environment.
• The committee hopes that the recommendations and
  the proposed framework for risk-based
  decision-making will provide a template for the
  future of risk assessment in EPA and strengthen
  the scientific basis, credibility, and
  effectiveness of future risk-management
  decisions.