Uncertainty Computation,Visualization, and Validation

1
Uncertainty Computation,Visualization, and
Validation

• Suresh K. Lodha
• Computer Science
• University of California, Santa Cruz
• lodha_at_cse.ucsc.edu
• (831)-459-3773

2
Personnel

• Suresh K. Lodha
• PhD, CS, Rice University, 1992
• Scientific information visualization,
  uncertainty quantification visualization, multi
  -modal visualization, computer-aided geometric
  design
• Uncertainty research supported by National
  Science Foundation Department of Energy ASCI
  Program
• (LLNL, LANL, SNL)

3
Overview

• Uncertainty representation computation
• Data/information fusion
• Quality-of-service issues
• Uncertainty visualization
• Uncertainty validation

4
Uncertainty Visualization Pipeline
5
Sources of Uncertainty

• Sensor and human limitations
• Noise, clutter, jamming etc.
• Modeling assumptions
• Algorithm limitations
• Data compression
• Communication errors
• Visualization-induced errors

6
Uncertainty Representation

• Uncertainty formalisms used by the fusion
  community
• Probability
• Dempster-Shafer evidence theory
• Fuzzy sets and possibility theory
• Uncertainty representation in visualization
  research
• Confidence intervals
• Estimation error
• Uncertainty range

7
Uncertainty Computation(Previous Work)

• Data/Information Fusion
• Knowledge-based systems
• Random sets (Goodman, Nguyen, Mahler)
• Visualization
• NIST/ NCGIA 91 (Beard et al.)
• BattleSpace 98 (Durbin et al.)
• Visualization Software 96 (Globus, Uselton)
• Scientific Visualization 96 -- (Lodha, Pang,
  Wittenbrink)

8

• Any battlefield necessarily deals with
  uncertainty, and it is necessary to determine
  ways to represent and encode the confidence level
  that exists for each piece of battlefield data.
  Durbin et al 98 (NRL)

9
Designing Error Metrics

• True vs. measured/observed/anticipated
• Observed vs. simulated
• High resolution vs. low resolution
• Continuous vs. discrete
• Individual source vs. multiple sources
• Static vs. dynamic
• Time-independent vs. time-critical
• Error-free vs. error-prone communication

10
Examples of Error Metrics

• Local metrics
• -- distance metric
• -- curvature metric
• -- sampling-number or depth metric
• (distribution of error)
• Global metrics
• -- Topology metric

11
Uncertainty Metrics Isosurfaces
12
Uncertainty Metrics Fluid Flow Topology
Original 332 cp
65
55
13
Research Issues

• Representations and data structures for
  uncertainty measures
• Design and integration of error metrics
• Uncertainty-aware and uncertainty-reducing data
  processing (algorithms and models)
• Common consistent uncertainty representation over
  a distributed mobile network ?

14
Uncertainty Visualization

• How to convey uncertainty to human users?
• Uncluttered display
• Intuitive metaphors for mapping
• Data characteristics
• Multi-modality
• Do NOT hide processes that produce problems for
  the human users?
• Visualize the abstraction (e.g uncertainty
  pipeline, graphical models)

15
Uncertainty Visualization

• Display devices /environment
• screen space (monitor, PDA, workbench,..)
• mobility
• Modality
• vision
• audio
• haptics

16
Uncertainty Visualization

• Data types/ characteristics
• scalar/vector/tensor
• discrete/continuous
• static/dynamic
• Levels of fusion
• data-level (raw/abstract)
• image-level (physical phenomena)
• feature-level (compressed view)
• decision-level (super-compressed)

17
Uncertainty Visualization (continued)

• Techniques
• glyphs
• deformation
• transparency
• texture
• linking
• superimposing/backgrounding
• augmented reality
• modality

18
Unc Viz Example 3Fluid Flow Visualization
19
Unc Viz Example 5 Geometric Uncertainty
20
Uncertainty Visualization Example 4
21
Research Issues

• Uncertainty mapping and metaphors for different
  modalities, data types and fusion tasks
• Display support for a variety of uncertainty
  metrics/formalisms
• Interactive display for uncertainty-source -gt
  task analysis
• Integration and analysis of uncertainty for
  decision-making?

22
Uncertainty Validation

• Does addition of uncertainty information help
  human users in making decisions?
• Can humans integrate qualitative and quantitative
  (or heterogeneous information) when there is
  uncertainty?
• Task definitions
• Careful design of experiments
• Usability studies
• Statistical analysis

23
Uncertainty Validation

• Task definitions
• primary level tasks (raw estimation)
• secondary level tasks (correlation or simple
  spatio-temporal relationships)
• higher level tasks
• Examples
• feature existence (binary decision)
• feature recognition (finite multiple choices)
• target aiming (zone-centered decision within a
  specified space-time region)

24
Validation Strategies

• Formative vs. summative studies
• With or without uncertainty mapping
• Representative sampling of tasks, data and
  uncertainty mappings
• Constrained, interactive or free-form environment
• Within-subjects/between-subjects and tabular
  designs

25
Uncertainty Validation (Previous Work)

• Validation of user interfaces (CHI 90s)
• Validation of multi-modal mappings (Melara,
  Marks, Massaro (UCSC))
• Validation of uncertainty mappings

26
Uncertainty Validation Example 1 (with M. Hansen)

• Protein structural alignment
• (intuitive metaphors)

27
Uncertainty Validation Example 1 (continued)

• Protein structural alignment -- accuracy of
  discrimination

28
Uncertainty Validation Example 2 (GIS)
Rainbow
Saturated
29
Uncertainty ValidationTasks(averaging,
comparisons)
30
Uncertainty Validation (with Wittenbrink
Pang)

• Vector uncertainty glyph evaluation

31
Research Issues

• Construction of user evaluation environments
• Conduct user evaluation studies for efficiency
  and accuracy
• Data analysis and statistical testing
• Feedback loop to improve performance
• Integrated decision tool combining uncertainty
  approaches in visualization and command and
  control?

32
Concluding Remarks I

• Provide human users with uncertainty information
• Representation and computation of uncertainty
• Uncertainty-aware and uncertainty-reducing
  algorithms and models
• Uncertainty visualization
• Visualization of uncertainty pipeline or hidden
  processes or abstract models
• (continued)

33
Concluding Remarks II

• Effective and clutter-free visualization of
  uncertainty along with the data/information
• Sensitive to data characteristics/ fusion level/
  tasks/ display environments (intuitive and
  cognitively accurate metaphors)
• Multi-modality
• Usability studies

34
Collaboration

• Uncertainty representation/ fusion
• (UCSC, Syracuse ?? GTech, UCB,USC)
• Uncertainty visualization
• (UCSC, Gtech ?? UCB, USC)
• Multi-modal interaction
• (UCSC ?? USC, GTech)
• Other MURIS/ DoD?