David C' Banks

1
Realistic Illumination for Displaying 3D Medical
Data

• David C. Banks
• UT/ORNL Joint Institute for Computational
  Sciences
• HMS/BWH Department of Radiology

2
Visualizing Fluid Flow
M Sussman, K Beason, D Banks
3
Visualizing Fluid Flow
K Beason, D Banks
4
Visualizing Nanochemistry
Smith, Sablin, Hussaini, Futch, Beason, Banks
5
Visualizing Neurons
Beason, Banks, Dienst, Ouimet
6
Visualizing Neurons
Beason, Banks, Dienst, Ouimet
7
Visualizing MRI Data
Holmes, Beason, Futch, Banks
8
Visualizing DT MRI Data
Kindlmann, Banks
9
Realistic Illumination

• Local versus Physically-Based Illum
• Illumination applied to Brain Data
• Quantifying Perception
• Making PBI Available

10
Graphics and Visualization

• 3D Graphics
• Shine lights on geometry, make image
• Visualization
• Convert data to geometry, make image

11
Illumination Models

• Local illumination (1970s)?
• Consider geometry _at_ 1 point
• Hardware implementation
• 108 triangles/sec, good for interactive games
• Physically based illumination (1980s)?
• Use light transport equation
• Software solver
• 106 photons/sec, good for non-interactive films

12
Local and PB Illumination
Beason, Banks
13
Local and PB Illumination
Beason, Banks
14
Illumination for Film

• 2004 academy award (technical achievement) for
  subsurface scattering

Surface only
15
Illumination for Film

• 2004 academy award (technical achievement) for
  subsurface scattering
• Makes skin realistic for CG films

Subsurface
Surface only
16
Visualizing Head Trauma

• 3D game

17
Visualizing Head Trauma

• 3D Game
• MRI visualization tool

18
Current SW for Data Vis

• Commercial sw for sci/med vis doesnt offer
  physically-based illumination
• Users dont demand it

19
Reasoned Use of Illumination

• witheach

20
Reasoned Use of Illumination

• witheach unless (veryGoodReason) apply
  PBI

21
Reasoned Use of Illumination

• witheach unless (veryGoodReason) apply
  PBI otherwise apply NPI

22
Actual Use of Illumination

• witheach

23
Actual Use of Illumination

• witheach if film/game then apply PBI

24
Actual Use of Illumination

• witheach if film/game then apply
  PBI if sci/med vis then apply LI
• Why is it this way?

25
Why Vis uses Local Illum

• Scientists sw developers worry that images are
  merely pretty or glitzy

Viz-o-matic, Lytle
26
Critique of Visualization

• Intended message
• Limit geometry, animation, narration, colors
• Received message
• Dont make image look any nicer

27
Survey of Current Fiber Vis

• Curves

dTV, Univ Minnesota
28
Survey of Current Fiber Vis

• Curves

AFNI
29
Survey of Current Fiber Vis

• Curves

National Taiwan University Hospital
30
Survey of Current Fiber Vis

• Curves Planes

DTIStudio
31
Survey of Current Fiber Vis

• Curves Surface Plane

National Taiwan University Hospital
32
Survey of Current Fiber Vis

• Tubes

Syngo, Siemens Medical Solutions
33
Survey of Current Fiber Vis

• Tubes plane

Cedars Sinai Medical Center
34
Survey of Current Fiber Vis

• Tubes Sheets

Teem, BWH
35
Survey of Current Fiber Vis

• Tubes ROI

MedINRIA
36
Survey of Current Fiber Vis

• Tubes Volume Plane

DTI Task Card
37
Survey of Current Fiber Vis

• Tubes surface plane

Bogazici University, Turkey
38
Realistic Illumination

• Local versus Physically-Based Illum
• Illumination applied to Brain Data
• Quantifying Perception
• Making PBI Available

39
Basic Idea Apply PBI

• PBI makes 3D scenes look real
• Use it to make data look like a real brain
• What does a real brain look like?

40
What White Matter Looks Like

• Neurosurgeon white matter is like flan

41
Problem with Realism

• May be accurate, but hides structure

Pudding, from a mold
42
Problem with Realism

• Add structure envisioned by surgeon

Rice noodles, gelatin
43
Imagined Realism

• Klingler preparation reveals fibers
• Harden with formalin freeze thaw

Gluhbegovic
44
PBI for Fibers

• Display structures that have meaning
• Render as though physically real

Local illumination
45
Ex Random Clusters
46
Ex Random Clusters
47
Ex Random Clusters
48
Ex Random Clusters
49
Ex Random Clusters
50
Ex Fascicles

• Local illumination
• Physically-based illumination

51
Ex Fascicles
ODonnell, Westin, Banks, Kindlmann
52
Ex Ganglioglioma
53
Realistic Illumination

• Local versus Physically-Based Illum
• Illumination applied to Brain Data
• Quantifying Perception
• Making PBI Available

54
PBI Applied to Fiber Tracts

• Hypothesis Surgeon/radiologist better
  comprehends shape from PBI than LI
• Test1 geometric queries
• Test2 gaze tracking

55
Gaze Tracking

• Yarbus, 1967

56
Gaze Tracking

• Yarbus, 1967
• (1) free exploration

57
Gaze Tracking

• Yarbus, 1967
• (1) free exploration
• (2) guess material status of the family

58
Gaze Tracking

• Yarbus, 1967
• (1) free exploration
• (2) guess material status of the family
• (3) guess the age of individuals

59
Gaze Tracking

• Occipito-frontal fascicle

60
Gaze Tracking

• Fixation points when image is viewed

61
Realistic Illumination

• Local versus Physically-Based Illum
• Application to Brain Data
• Quantifying Perception
• Making PBI Available

62
Ordinary Light, 2D Function
63
Flattened Light, 2D Function
64
PBI Texture, 2D Function
65
Comparison for 2D Function
66
PBI Texture, 3D Function
Blanco, Beason, Banks, Holmes
67
Use with Commercial Tool
68
Summary

• Physically-based illumination is widely used for
  displaying cars, dinosaurs, superheroes
• Local illumination is widely used for displaying
  scientific and medical data
• Strange, because PBI seems better
• Our strategies
• Demonstrate PBI on actual datasets
• Test for perceptual differences between PBI, LI
• Create PBI vis tools that are fast, compatible

69
Thanks

• Beason, Futch, Grant, Blanco, Song, Ouimet,
  Hussaini, Piekarewicz,  Smith, Westin, Kikinis,
  ODonnell, Kindlmann, Golby, Talos
• NSF, NIH

70
(No Transcript)
71
Realism in Data Visualization

• Scientists sw developers worry that images are
  merely pretty or glitzy

Severe storm, NCSA
72
Local and Global Illumination
Yagi, Banks, Kindlmann
73
Illumination on Graph
Samples of graph of h projected to D
74
From 3D to 4D
75
Interpolation to 3D Texture
76
Texture-mapping

• Apply 3D texture to level set

77
Example Bump Box
78
Pre-computing PBI