Effective visualization of timevarying data using cognitionbased principles

1
Effective visualization of time-varying data
using cognition-based principles

• Alark Joshi
• University of Maryland Baltimore County

2

• "The purpose of computing is insight, not
  numbers", Richard Hamming (1962)

3
Motivation

• Datasets are getting larger
• Scanners are getting better and generating higher
  resolution data
• Experimental simulations are generating datasets
  that are in the order of terabytes and in some
  cases even petabytes
• Data is increasingly time-varying (ultrasound,
  weather forecasting, fluid flow)

4
Motivation

• Higher resolution provides improved accuracy for
  decision making and increased insight into the
  data
• Visualizing such datasets is challenging due to
  their size as well as the time-varying nature
• Current techniques rely on the users ability to
  visualize change over time

5
Application Domains

• Challenges faced by experts in the following
  applications domains
• Computational fluid dynamics
• Medical visualization
• Weather forecasting

6
Computational Fluid Dynamics

• Computational fluid dynamics is defined as the
  study of the dynamics of flow
• Wind flow over an airplane wing
• Outflow from a nozzle into tank, turbines etc
• Flow is measured or simulated to generate
  time-varying data
• A single volume could easily have a resolution of
  2563 and there could possibly be hundreds or
  sometimes thousands of time-steps of data
• The size of each volume coupled with the number
  of time steps increases the size of the entire
  dataset considerably.

Image credits http//ilona.uni-mb.si/hribersek/hm
-cfd2.html
7
Flow visualization

• Current approaches to visualizing time-varying
  data for every timestep
• Standard volume rendering techniques
• Isosurface rendering

Image from Visualizing and Tracking Features in
3D Time varying datasets, Xin Wang, Ph.D. Thesis,
1999
8
Flow visualization

• Rely heavily on the user's ability to identify
  and track regions of interest over time.
• Number of snapshots generated can be quite high
  (100-3000), requiring considerable effort to
  track features.

Image from Volume Tracking by D. Silver and Xin
Wang, 1996
9
Visualization of turbulent vortex data
Video courtesy Deborah Silver,
http//www.caip.rutgers.edu/xswang/feature/index.
html
10
Flow visualization

• In an experiment by Pylyshyn, it was found that
  observers can track a maximum of five
  independently moving objects at the same time.
• Speed increases Performance decreases
• Number of objects increases Performance
  decreases
• Time-varying datasets commonly have 20-30 features

Zenon W. Pylyshyn. Seeing and Visualizing, 2003.
11
Flow visualization challenges

• Effectively tracking features over time
• Visualizing motion of features using a single or
  a smaller subset of images
• Better ways to understand inter-feature
  relationship

12
Medical visualization

• Datasets are getting larger in size due to high
  resolution scanners
• A single CT/MRI scan consists of a slice of
  resolution 512x512 and 1024/2048 such slices
• Finer resolution is better for radiologists and
  doctors to make informed decisions

Image credits - http//support.vitalimages.com/img
1_big.jpg
13
Medical visualization

• Standard volume rendering techniques are used to
  visualize these datasets
• These techniques are currently being used for
• Virtual colonoscopy
• Surgery and treatment planning
• Diagnosis purposes
• Interactivity in such techniques is greatly
  hampered by the size of the dataset

Image credits - http//www.lifespanmedical.com.au/
patients/v_colonoscopy.htm and http//www.medscape
.com/viewarticle/405410
14
Medical Visualization challenges

• Change in Morphology
• Monitoring the size of a tumor over an extended
  period to determine efficacy of radiation therapy
  
• Radiation therapy is a non-surgical alternative
• Involves 7 weeks of therapy usually given 5 days
  a week
• Change in content of the tumor
• Change in tumor vessel content over time
• Change in the stromal content over time

Source http//www.ncbi.nlm.nih.gov/books/bv.fcgi?
ridcmed.section.6535 and http//familydoctor.org/
264.xml
15
Weather visualization

• Weather prediction is crucial to minimize human
  and financial loss
• Data collected by domain experts is in the order
  of tens of gigabytes per day

Video courtesy http//www.research.ibm.com/weathe
r/DT.html
16
Weather visualization

• Expected local weather conditions during the next
  day or two are critical factors in planning
  operations and making effective decisions
• Most application domains work in a reactive
  manner due to the lack of accurate predictions
  (transportation, local government, insurance and
  so on)
• Accurate weather forecasts can be used to improve
  operational efficiency and safety

17
Weather visualization challenges

• Study of hurricanes has become very crucial
• Conditions that cause the transformation of a
  category 3 hurricane to a category 5 hurricane
• Path prediction for evacuation purposes
• Entrainment in a hurricane that reduces the
  intensity of a hurricane
• Requires tools to explore and study the evolution
  and transformations undergone by a hurricane

18
Weather visualization challenges

• Study the interactions and relationships between
  attributes like pressure, temperature, winds and
  so on
• Visualizing important events is beneficial in the
  study of hurricanes
• Opening of the eyewall
• Change in attributes such as pressure, cloud
  water, winds at crucial time intervals, when a
  hurricane changes its category

19
Related Work

• Flow (CFD) visualization
• Samtaney et al 94
• Silver and Wang 96, 97
• Ma and Shen 00
• Medical Visualization
• Levoy 90, Cabral et al 94
• Kaufman 96
• Guthe et al. 02

Image credits Silver and Wang 96 Kniss et al
01
20
Related Work

• Weather Visualization
• Wilhelmson et al 90
• Davis et al 01, 02
• Illustrative visualization
• Lu et al. 03
• Rheingans and Ebert 01
• Burns et al 05

Image credits Wilhelmson 90 and Burns 05
21
Related Work

• Simplification
• Clark 76
• Luebke et al 02
• Luebke and Hallen 01
• Cohen et al 98

Image credits Cohen et al 97
22
Illustrations
Context
Context
Increased abstraction

• Through illustration we have the potential to
• Interpret physical reality
• Distil the essential components of a scene
• Accentuate the important information
• Minimize the secondary details
• Hierarchically guide the attentional focus

Context
Image credits http//www.bartleby.com/107/illus38
5.html
Source V. Interrante, The Visualization
Handbook, 2004.
23
Illustrations

• We propose the use of illustration-based
  principles to
• Visualize Motion over time
• Visualizing positions of older timesteps
• Using cognition-based simplification to aid
  visualizing motion over time
• Visualize Morphological Change
• Visualize Change in attribute values

24
Illustrations to depict motion

• Illustrators have been using techniques to depict
  change over time

Images from Edward MacCurdy. The Notebooks of
Leonardo Da Vinci, 1954 Kawagishi et al.,
Cartoon blur Non-photorealistic motion blur
25
Visualize Motion over time Illustration-based
techniques

• Enable users to pick and track features of
  interest successfully
• Visualizing the motion experienced by a feature
  over time using a single image (or a small set of
  images)
• Augmenting animations of time-varying data
  visualizations with illustration-based cues

26
Line Ribbon based techniques

• Illustrators have used line and ribbon-based
  techniques to convey motion

Flow Ribbons
Speedlines
Images from Kawagishi et al., Cartoon blur
Non-photorealistic motion blur and Understanding
Comics by Scott McCloud, 1994
27
Depicting past positions

• A trailing silhouette shows past positions of the
  object.
• Illustrators have often used techniques where
  they use a blurred, desaturated image to depict
  an older time step whereas a more brighter, more
  detailed image represents a newer time step.

Opacity-based technique
Strobe silhouettes
Image from Kawagishi et al, Cartoon blur
Non-photorealistic motion blur Understanding
Comics by Scott McCloud, 1994
28
Visualize Motion over time Cognitive
simplification

• Cognitive simplification can aid
  illustration-based visualization
• A multiresolution representation can trade
  cognitive overload and effective visualization
• Preserving regions of importance in the process
  of simplification

29
Simplification

• Automatically building hierarchical model
  representations to balance visual fidelity and
  interactivity

Inspired by chapter on Model Simplification by J.
Cohen and D. Manocha in the Visualization
Handbook, 2004. Image credits D. Luebke, A
Developers Survey of Polygonal Simplification
Algorithms. IEEE Computer Graphics Applications
(May 2001).
30
Cognitive Simplification

• Preprocessing datasets to identify cognitively
  significant features
• Generating simplified representations of the
  dataset that maintain those features
• Reduce visual clutter and draw the users
  attention to regions of interest

31
Path abstraction

• Speedlines technique requires an
  illustrative-simplification of the path followed
  by the feature

Unsimplified path Naïve
path Cognitive path
simplification simplification
Faithful to the path, connecting alternate
positions, connecting one of four positions
32
Shadow Simplification

• Shadows provide increased depth cues and
  communicate spatial relationships Hubona 99
• Experiments have shown that soft shadows cast by
  a mesh simplified to 1 its original size was
  acceptable to 90 of its users (Sattler 05)
• Illustrative visualization will be augmented with
  simplified shadows to convey depth and
  orientation cues

Image from Exploitation of human shadow
perception for fast shadow rendering by Sattler
et al, 2005
33
Silhouette Simplification

• We propose to simplify the silhouette based on
  its age
• Older timesteps have a low detail silhouette and
  vice versa

Image from Kawagishi et al, Cartoon blur
Non-photorealistic motion blur
34
Partially-obscured feature simplification

• In the flow ribbons technique, underlying
  features were partially-obscured by the ribbons
  to convey motion
• Illustrators provide abstract representations of
  obscured features

Image from Understanding Comics by Scott McCloud,
1994
35
Change Visualization

• Visualizing change over time is crucial to
  understanding and studying a time-varying dataset
• We propose to identify new techniques to
  visualizing
• Change in morphology
• Growth over time in a single image
• Change in attribute values (like pressure,
  temperature etc.)

36
Morphological change

• Effectively convey change in structure or form of
  a feature using a single image (small subset of
  images)
• Tumor growth and treatment planning
• Transformation undergone by a fluid flow feature
  (bifurcation, amalgamation, dissipation,
  emergence)

37
Morphological change

• Visualizing the change (growth) undergone by a
  feature of interest

Significant snapshots
Image credits http//catalog.nucleusinc.com/genera
teexhibit.php?ID10165ExhibitKeywordsRawTL3273
7A2
38
Change in Attribute values

• Representing the change in the values of an
  attribute in a single image (small subset of
  images)
• Change in value of pressure, wind speed in
  regions of the hurricane
• Increase in the content of dry air in the eye of
  the hurricane (can drastically reduce the
  intensity of a hurricane)
• Change in the content of a tumor over time
  increase in the stromal content of the tumor cells

39
Change in Attribute values

• Using color, texture based methods to convey
  change over time
• Illustration-based approaches

Image credits Temporal visualization of planning
polygons for efficient partitioning of
geo-spatial data, Shanbhag, P. Rheingans, P.
desJardins, M, 2005 Flow map layout, Doantam
Phan Ling Xiao Yeh, R. Hanrahan, P. Winograd,
T, 2005.
40
Plan of Work
41
Expected Research contribution

• Illustration-based techniques to depict motion
• Novel techniques for feature tracking
• Cognitively simplified multiresolution
  representations of data
• Illustration-based techniques to visualize change
  in morphology and attribute values

42
Expected Research Contribution

• Flow visualization
• Improved feature tracking using
  illustration-based techniques
• Medical Visualization
• Novel techniques to visualize change in structure
  and growth over time
• Improved illustrative rendering with shadows
• Weather Visualization
• Cognitively simplified multiresolution data for
  effective visualization
• Change in attribute values for accurate
  prediction and decision making

43

• Next-generation tools will need to employ more
  ingenious approaches including more
• sophisticated data models,
• Multiresolution techniques,
• Level-of-detail views,
• Hierarchical data representation,
• Region-of-interest rendering, ...
• Don Middleton, Tim Scheitlin, National Center
  for Atmospheric Research and Bob Wilhelmson,
  NCSA.

44
Acknowledgements

• Dr. Deborah Silvers group at Rutgers University
• Dr. Lynn Sparling, Dr. Miodrag Rancic, Hai Zhang
  at the Physics Department at UMBC
• Scott McCloud, Kunio Kondo and Harper Collins
  Publishers for the Illustrations
• This work has been funded by NSF grant numbers
  0121288 and 0081581.

45
Questions?