Completing the Forecast

1
Completing the Forecast
Characterizing and Communicating Uncertainty for
Better Decisions Using Weather and Climate
Forecasts
Board on Atmospheric Sciences and Climate
2
Background

• Sponsor NOAA-NWS
• Focus estimation and communication of
  uncertainty in weather, hydrological, and
  short-term climate forecasts
• Process 5 meetings, in Boulder and D.C.
  (including at AMS broadcast and WAF conference),
  with almost 50 presenters and other contributors
• The study recognizes the diverse roles of
  participants in the weather and climate
  enterprise, as described in the Fair Weather
  report, and diverse needs of forecast users.
• Recommendations are primarily to NWS, but
  potentially also to other components of NOAA,
  other government agencies, and nongovernmental
  sectors (e.g., commercial meteorologists, media,
  weather risk management institutions, private
  companies).

3
Task

• Provide guidance on how to identify and
  characterize needs for uncertainty information
  among various users of forecasts. Support this
  with examples of needs.
• Identify limitations in current methods for
  estimating and validating forecast uncertainty,
  relating these limitations to users needs and
  recommend improvements or new methods and
  approaches.
• Identify sources of misunderstanding in
  communicating forecast uncertainty, including
  vulnerabilities dependent on the means of
  communication used, and recommend improvements in
  the ways used to communicate forecast
  uncertainty.
• Recognizing the breadth and depth of this task,
  NWS advised the committee at its opening meeting
  to "teach us how to fish as opposed to giving us
  a fish."

4
Committee

• RAYMOND J. BAN (Chair), The Weather Channel,
  Inc., Atlanta, GA
• JOHN T. ANDREW, California Department of Water
  Resources, Sacramento
• BARBARA G. BROWN, National Center for Atmospheric
  Research, Boulder, CO
• DAVID CHANGNON, Northern Illinois University,
  DeKalb, IL
• KONSTANTINE GEORGAKAKOS, Hydrologic Research
  Center, San Diego, CA
• JAMES HANSEN, Massachusetts Institute of
  Technology, Cambridge
• RONALD. N. KEENER, JR., Duke Energy, Charlotte,
  NC
• UPMANU LALL, Columbia University, New York, NY
• CLIFFORD F. MASS, University of Washington,
  Seattle
• REBECCA E. MORSS, National Center for Atmospheric
  Research, Boulder, CO
• ROBERT T. RYAN, NBC4, Washington, DC
• ELKE U. WEBER, Columbia University, New York, NY
• NRC Staff
• PAUL CUTLER, Study Director
• LEAH PROBST, Research Associate
• ROB GREENWAY, Senior Program Assistant

5
Review Process

• Review Oversight
• George Frederick, Vaisala, Inc.
• Kuo-Nan Liou, University of California, Los
  Angeles
• Reviewers
• James Block, DTN-Meteorlogix, Minneapolis,
  Minnesota
• David Budescu, University of Illinois, Champaign
• Simon Chang, Naval Research Laboratory, Monterey,
  California
• Holly Hartmann, University of Arizona, Tucson
• Kathryn Laskey, George Mason University, Fairfax,
  Virginia
• Daniel P. Loucks, Cornell University, Ithaca, NY
• Tom Skilling, WGN-TV News, Chicago, Illinois
• Mort Webster, University of North Carolina at
  Chapel Hill
• Lawrence Wilson, Meteorological Service of
  Canada, Quebec

6
Report Contents

• Summary
• 1 Introduction
• 2 Uncertainty in Decision Making
• 3 Estimating and Validating Uncertainty
• 4 Communicating Forecast Uncertainty
• 5 Overarching Recommendations

7
Background

• Uncertainty is a fundamental characteristic of
  weather, seasonal climate, and hydrological
  prediction, and no forecast is complete without a
  description of its uncertainty.
• Nonetheless, for decades, users of weather,
  seasonal climate, and hydrological (collectively
  called hydrometeorological) forecasts have been
  conditioned to receive incomplete information
  about the certainty or likelihood of a particular
  event.

8
Background

• Uncertainty was included in 19th Century
  forecasts
• Determinism in the 20th Century due to
• progression of the science toward exactness
• numerical weather prediction
• satellite observations and models
• media
• This legacy remains strong

9
An Opportunity

• NWS and others in the Enterprise have recognized
  the need to view uncertainty as a fundamental
  part of forecasts.
• By partnering with other segments of the
  Enterprise to understand user needs, generate
  relevant and rich informational products, and
  utilize effective communication vehicles, NWS can
  take a leading role in the transition to
  widespread, effective incorporation of
  uncertainty information into hydrometeorological
  predictions.

10
Overarching Recommendations

• There are nine overarching recommendations, all
  with equal priority.
• Detailed recommendations appear in Chapters 2, 3,
  and 4 that add further specificity and breadth.
• All recommendations should be considered in the
  context of NOAAs Policy on Partnerships in the
  Provision of Environmental Information.

11
Recommendation 1

• The entire Enterprise should take responsibility
  for providing products that effectively
  communicate forecast uncertainty information.
  NWS should take a leadership role in this effort.

12
Recommendation 2

• NWS should improve its product development
  process by collaborating with users and partners
  in the Enterprise from the outset and engaging
  and using social and behavioral science
  expertise.

13
Recommendation 3

• All sectors and professional organizations of the
  Enterprise should cooperate in educational
  initiatives that will improve communication and
  use of uncertainty information. In particular,
• 1) hydrometeorological curricula should include
  understanding and communication of risk and
  uncertainty,
• 2) ongoing training of forecasters should expose
  them to the latest tools in these areas, and
• 3) forecast providers should help users,
  especially members of the public, understand the
  value of uncertainty information and work with
  users to help them effectively incorporate this
  information into their decisions.

14
Recommendation 4

• NWS should develop and maintain the ability to
  produce objective uncertainty information from
  the global to the regional scale.

15
Recommendation 5

• To ensure widespread use of uncertainty
  information, NWS should make all raw and
  post-processed probabilistic products easily
  accessible to the Enterprise at full spatial and
  temporal resolution. Sufficient computer and
  communications resources should be acquired to
  ensure effective access by external users and NWS
  personnel.

16
Recommendation 6

• NWS should expand verification of its uncertainty
  products and make this information easily
  available to all users in near real time.
• A variety of verification measures and approaches
  (measuring multiple aspects of forecast quality
  that are relevant for users) should be used to
  appropriately represent the complexity and
  dimensionality of the verification problem.
• Verification statistics should be computed for
  meaningful subsets of the forecasts (e.g., by
  season, region) and should be presented in
  formats that are understandable by forecast
  users.
• Archival verification information on
  probabilistic forecasts, including
  model-generated and objectively generated
  forecasts and verifying observations, should be
  accessible so users can produce their own
  evaluation of the forecasts.

17
Recommendation 7

• To enhance development of new methods in
  estimation, communication, and use of forecast
  uncertainty information throughout the
  Enterprise, and to foster and maintain
  collaboration, confidence, and goodwill with
  Enterprise partners, NWS should more effectively
  use testbeds by involving all sectors of the
  Enterprise.

18
Recommendation 8

• The committee endorses the recommendation by the
  NRC Fair Weather report to establish an
  independent advisory committee and encourages
  NOAA to bring its evaluation of the
  recommendation to a speedy and positive
  conclusion.

19
Recommendation 9

• NWS should dedicate executive attention to
  coordinating the estimation and communication of
  uncertainty information within NWS and with
  Enterprise partners.

20
Key points not to forget

• Broad applicability within NOAA and other
  agencies
• This will take time (turn the aircraft carrier)
• Lets get started now
• NWS evolution - time is right to incorporate
  uncertainty information into process
• Uncertainty is another component of improving the
  forecast its not detached from the primary
  mission of forecast improvement
• Uncertainty is not an add-on it is in the
  nature of the product