MURI Quarterly Meeting 1/31/02

1
MURI Quarterly Meeting 1/31/02
APL Presentations
2
Overview

• Three presentations
• David-Thoughts on the METOC Task
• Scott- Ensemble Verification
• Keith- Visualization Framework
• APLs POC
• David-MURI proj. mgnt, CTA, Navy METOC ops
• Scott-Mesoscale modeling, ensem., verification
• Keith-METOC visualization workflow
• Jim-Statistical issues w/ uncertainty

3
Updates

• Recent Visits
• Susan David-NPMOF Whidbey Nov
• Scott David-FNMOC Pt Magu Dec
• David NPMOC San Diego, USS Constellation,
  NAVSPECWAR Mission Support Center (SEALS) Jan
• General impression- all forecasters deal w/
  uncertainty but that uncertainty is not conveyed
  to user

4
Updates (cont.)

• What are we doing?
• Conducting CTA other investigations to
  understand how uncertainty affects the domain
• Evaluating verification strategies for ensemble
  systems
• Exploring alternative design strategies a
  visualization framework

5
METOC Task Analysis Literature Review

• Task analysis, as opposed to task description,
  should be a way of producing answers to questions
  (i.e., identifying potential performance failures
  or training needs and indicating how these
  problems might be solved.)
• Annett (2000)

6
METOC Task Analysis Literature Review

• Hoffman (1991) provides good review of task
  analysis for forecaster domain and human factor
  design considerations for Advance Meteorological
  Workstation, but
• Task analysis of researchers, not forecasters
• There has been a change in the chartroom paradigm

7
METOC Task Analysis A General Model

• July 2000- Human Systems Checklist for METOC
  Forecasting (Appendix A)
• Work in Progress (Appendix B)
• Information Networks- here be uncertainty

8
A Specific METOC Task

• The Terminal Aerodrome Forecast (TAF)
• KNUW 200909 15025G35KT 9999 FEW018 SCT045
  QNH2967INS
• TEMPO 1018 -RA SCT015 BKN040 BKN100
• BECMG 1820 14015G25KT 9999 SCT020 BKN060 BKN100
  BKN200 QNH2973INS
• TEMPO 2209 SHRA BKN020 BKN060 OVC100

9
A Specific METOC Task

• Flight Weather Briefing Form (DD 175-1)

Where the rubber meets the road!
10
Visualizing UncertaintyinMesoscale Meteorology
Thoughts On Verifying Ensemble Forecasts 31 Jan
2002 Scott Sandgathe
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
36km Ensemble Mean and Selected Members SLP,
1000-500mb Thickness 2002 Jan 2200Z
15
12km Ensemble Mean and Selected Members SLP,
Temperature, Wind 2002 Jan 2200Z
16
Verification of Mesoscale Features in NWP
Models Baldwin, Lakshmivarahan, and Klein 9th
Conf. On Mesoscale Processes, 2001
17
Tracking of global ridge-trough patterns from
Tribbia, Gilmour and Baumhaufner
18
Current global forecast and climate models
produce ridge-trough transitions however, the
frequency of predicted occurrence is much less
than the frequency of actual occurrence
19
Creating Concensus From Selected Ensemble
Members - Carr and Elsberry
20
Necessary Actions for Improved Dynamical Track
Prediction
(48 h)
Large Spread (806 n mi) Large Error
Small Spread (229 n mi) Large Error
No forecaster reasoning possible. Help needed
from modelers and data sources to improve
prediction accuracy
Recognize erroneous guidance group or
outlier, and formulate SCON that improves on NCON
Large Spread (406 n mi) Small Error
Small Spread (59 n mi) Small Error
Recognize situation as having inherently low
predictability must detect error mechanisms
in both outliers to avoid making SCONgtgtNCON
No forecaster reasoning required -- use
the non-selective consensus (NCON)
21
References
Cannon, A. J., P.H. Whitfield, and E.R. Lord,
2002 Automated, supervised synoptic map-pattern
classification using recursive partitioning
trees. AMS Symposium on Observations, Data
Assimilation, and Probabilistic Prediction,
pJ103-J109. Carr. L.E. III, R.L. Elsberry, and
M.A. Boothe, 1997 Condensed and updated version
of the systematic approach meteorological
knowledge base Western North Pacific.
NPS-MR-98-002, pp169. Ebert, E.E., 2001 Ability
of a poor mans ensemble to predict the
probability and distribution of precipitation.
Mon. Wea. Rev., 129, 2461-2480. Gilmour, I., L.A.
Smith, R. Buizza, 2001 Is 24 hours a long time
in synoptic weather forecasting. J. Atmos. Sci.,
58, -. Grumm, R. and R. Hart, 2002 Effective use
of regional ensemble data. AMS Symposium on
Observations, Data Assimilation, and
Probabilistic Prediction, pJ155-J159. Marzban,
C., 1998 Scalar measures of performance in
rare-event situations. Wea. and Forecasting, 13,
753-763.
22
Visualization in the METOC Environment

• R. Keith Kerr

23
Visualization

• A broad definition in the context of our work
• The mental representation of concepts (spatially,
  temporally, and operationally) that serve to
  refine and enhance the efficiency and accuracy of
  a defined suite of tasks executed within a
  particular workflow context.

24
The Visualization Framework

• Must blend
• Capturing of the workflow process for a suite
  of individual tasks that constitute a product
• Integrate a varied range of workflows within
  common user-interface paradigms

25
Framework (cont.)

• Might involve
• Automated reasoning and process control
• Analytic plots
• Geospatial (map-based) plots
• Data resource management
• Presentation tools
• Specialized viewing environments

26
Visualization Flow
Mental task model
Ontological representation
Reasoning (inference) engine
Process controller
METOC interface components
Products
27
Software Engineering Goals

• Design component architecture for visualization
  of METOC information
• Help implement useful research results within
  software prototype
• Integrate prototype within METOC Information
  Management Framework
• Install, maintain and support prototype in chosen
  test environment(s)
• Research implementations of cognitive paradigms
  within workflow software

28
Current Development Efforts

• Developing design requirements based on task
  analysis
• Refining design of previously developed
  components (data retrieval, inference, etc.)
• Working with METOC community to define a modern
  information framework
• Investigating relationship between ontology and
  the reasoning engine

29
Current Development Efforts

• Developing design requirements based on task
  analysis
• Refining design of previously developed
  components (data retrieval, inference, etc.)
• Working with METOC community to define a modern
  information framework
• Investigating relationship between ontology and
  the reasoning engine

30
Platform-independent,Three-tiered Services
Arbitrary Data Sources Web, METOC data bases,
models, Local archives, etc.
METOC Center(s) Server(s) Java Enterprise
Environment (Servlets, Server Pages, WebStart,
Applets), Process Management, Reasoning engine
Client Displays Both static and dynamic
interaction, Local process management and
reasoning, METOC product creation tools,
workflow monitoring
31
XIS one size fits all?

• Extremely sophisticated programming model
• Excellent information handling and abstraction
  facilities
• Already adopted by some Naval units and DII/COE
  certified
• But..as of today, lacks METOC annotational tools
  and fine-grained user interactivity with very
  large data sets (models)

32
XIS Viewpoint