Feature Aligned Volume Manipulation for Illustration and Visualization

1
Feature Aligned Volume Manipulation for
Illustration and Visualization

• Carlos D. Correa, Deborah Silver
• Rutgers, The State University of New Jersey
• Min Chen
• University of Wales, Swansea, UK

2
Motivation

• Hand-drawn illustrations often include
  manipulating parts of an object
• They often contain cuts
• They allow feature sensitive operations
• They often represent virtual operations (do not
  necessarily conform to reality)

Nucleus Inc
Antonio Scrantoni and Paolo Mascagni, 1833. U.S.
National Library of Medicine
3
Motivation (cont.)

• We refer to such manipulation as lllustrative
  Deformation
• Priority to interactivity, operatability and
  quality
• As opposed to physically-based deformation, this
  can be thought of as a top-down approach
• This type of deformations provides an intuitive
  depiction of internal structure.
• It serves as an abstraction of different stages
  of a procedure, e.g. a surgical operation.
• It is useful in surgery illustration/planning,
  education, and as a visualization tool in
  general.

4
Feature Alignment

• Traditional volume deformations are continuous
  and treat volumes as an homogeneous collection of
  points Westermann et al. 2001, Rezk-Salama et
  al. 2001
• McGuffin 2003 introduced 3D widgets with
  pre-computed segmented data to allow feature
  sensitive manipulation of volumes. Can this
  approach be extended to direct volume rendering?
• Recent approaches allow the definition of cuts
  Correa,2006. However, cuts appear flat as no
  semantics are introduced ? axis alignment
• Cuts in general are difficult to model in
  computer graphics. Require costly
  re-tessellations. This is further complicated
  when cuts have to be aligned with certain
  features.

5
Axis Alignment

• Treating volumes as homogeneous collections of
  voxels leads to axis alignment of cuts.
• Difficult to see features of interest

6
Goal

• To render deformations while preserving features
  of interest, by aligning cuts to a given
• Distance from surface ? surface alignment
• Feature based on segmentation ? segment alignment

Illustration
Illustrative Deformation
CT Dataset
7
Rendering Pipeline (axis aligned cuts)
Select operator
OPERATORS
TRANSFORMATION
Sample and Deform
via inverse space warping
Adapted from Correa et al. 2006
8
Feature-Aligned Rendering Pipeline
MASKS
Definition of features using a volumetric mask
Apply mask
Select operator
OPERATORS
TRANSFORMATION
Sample and Deform
Adjust opacity/lighting according to alignment
9
Operators

• Inspired by surgical tools and procedures
• Generic they can be applied to any dataset
• Defined as a 3D texture. Iconic representations
  are obtained when applied to a volumetric cube
  (or cylinder)

OPERATORS
10
Volume Transformation
TRANSFORMATION
Sample and Deform
Sample
Apply transformation
Estimate normal
Compute lighting
Subject to Alignment Mask
11
Modeling Deformation and Cuts

• Forward transformation is simple but limited for
  volumes ? undersampling unless space between
  points is interpolated (for cuts, it requires
  re-tessellation)
• Inverse transformation. Solves sampling problem,
  but discontinuous deformations are not a 11
  mapping.

Backward Transformation. Note introduction of
special value to model discontinuity
Forward Transformation
12
Modeling Feature Alignment

• Define a smooth mask M
• Binary masks may cause aliasing
• M(p) gt 0.5 ? p is non operatable
• M(p) lt 0.5 ? p is operatable
• Three cases for inverse transformation

• Not affected by mask apply inverse mapping

• Point inside mask it is not transformed

• Point outside mask but maps back into mask empty
  space left by feature

13
Definition of Features
MASKS

• Distance-based vs. Segmentation based
• (1) Surface Alignment Features are defined with
  the shape. Distance field of surface of object
  defines a series of shells, which define
  features. Useful when no segmentation is
  available.

Outer shell (t-d)
M 0.0
Features outer surface (DT t)
M 0.5
Features interior (td)
M 1.0
14
Mask Definition

• (2) Segment Alignment Mask is already defined as
  a segment
• Usually segmentations are discrete (binary for 1
  segment), For proper rendering without aliasing,
  a smooth definition is required
• Using a smoothing operator
• Using Distance Field of segmented part

M 0.0
M 0.5
M 1
15
Rendering and Lighting

• Cuts are now along a certain feature.
• For surface alignment, a new surface appears.
  Pre-computed normals are not necessarily
  perpendicular to that surface
• For segment alignment, gradient already defines
  almost correctly one surface. However, it
  cannot define properly the underside of the cut

16
Normal Adjustment Surface Alignment
Normals are oriented depending on density, not
necessarily aligned with surface shell. This
can be fixed by blending of normals
?T
?T
?DT
17
Normal Adjustment Segment Alignment
Normals on the underside of a cut point in the
opposite direction
?T
-?T
?T
18
Results (1) Peeling of Skin
SEGMENT
AXIS
SURFACE
Original Dataset
19
Results (2) Frog Dissection
Original Dataset
SEGMENT
AXIS
SURFACE
20
Results (3) Hand Surgery
Original Dataset
AXIS
SURFACE
SEGMENT
21
Results (4) Forefoot Retractor
Original Dataset
AXIS
SEGMENT
SURFACE
22
Implementation Details

• Based on discontinuous displacement mapping
  Correa et al. 2006, using texture based volume
  rendering
• Operators are stored as 3D textures (size is much
  smaller than size of dataset). Feature Mask is
  also stored as a 3D texture
• Interactive results (Pentium XEON 2.8GHz Quadro
  FX 4400 (512 MB) d 1

23
Video
24
Applications

• Medical and Biological Illustration.
• Operators are metaphors of the tools used in
  dissection
• Surgical Planning
• Manipulation of operators allows the generation
  of deformations and cuts in various stages of a
  procedure
• Improved Visualization
• Cutaway views with arbitrary cut geometry
• FocusContext, Distortion Lens

25
Conclusions

• Volume deformation techniques often treat volumes
  as homogeneous collection of voxels. When
  modeling cuts and breaks, they appear to be axis
  aligned, which results in decreased realism and
  limited use.
• It is possible to extend volume deformation to
  align cuts with certain features of interest.
  These can be defined as shells of the surface
  of the object using the distance transform, or as
  the product of segmentation
• Feature alignment can be implemented efficiently
  on commodity hardware. Proper handling of cut
  information to reduce aliasing, and adjustment of
  normals near the surfaces of cuts are necessary
  to produce high quality rendering of cuts.

26
Future Work

• Merge illustrative deformation with illustrative
  rendering
• NPR techniques can be used to emphasize new
  surfaces due to cuts or to exaggerate deformation
  (e.g. rendering of stress lines)
• Inclusion of rigid constraints for enhanced
  deformations, collision avoidance.
• More intuitive user interface and manipulation
  widgets to create and place operators.

27
Thanks

• Acknowledgements
• Volumetric datasets are courtesy of Lawrence
  Berkeley Laboratory, UNC Chapel Hill, University
  of Iowa, U.S. National Library of Medicine,
  Viatronix Inc. and Vienna University of
  Technology. The illustrations are courtesy of
  U.S. National Library of Medicine and Nucleus
  Medical Art, Inc.
• We want to thank Dr. Stanley Trooskin, Dr. Sid
  Roychowdhury and Dr. Marsha Jessup for valuable
  input on surgical and medical illustration.
• Further Information
• http//www.caip.rutgers.edu/cdcorrea

28
Thanks