Slicer Overview

1
Slicer Overview

• Steve Pieper, PhD

2
Outline

• Slicer
• Overview and History
• Architecture / Implementation
• Image I/O and Management
• Segmentation Tools (EM, Editor)
• DTMRI Tools
• fMRI Tools
• Applications (IGT, Clinical Research)

3
What is 3D Slicer?

• 3D Slicer is
• An end-user application for 3D medical image
  computing research and Image Guided Therapy
• A platform for research where new techniques can
  be plugged into a useful framework
• A freely-downloadable program with source and
  binaries for Windows, Linux, Solaris and
  (increasingly) Mac OSX
• NOT an FDA approved medical device and CANNOT be
  used clinically without proper research controls
  (IRB etc.)
• NOT finished some parts will work better than
  others and it is constantly evolving

4
Surgical Planning Example

• Dr. Jose Miguel Selman, Clinica Las Condes,
  Santiago, CHILE
• Temporal Lobe Cavernoma
• MR Cortex and White Matter Extracted by
  FreeSurfer (MGH software, interface created for
  BIRN)
• Registration to CT and Visualization in Slicer

5
Virtual Endoscopy Example

• Delphine Nain, MIT AI Lab, now at Georgia Tech
• Automatic or Manual Path Planning
• Animated Camera and Controls

http//www.ai.mit.edu/projects/medical-vision/virt
ual-endoscopy/
6
Slicer Background

• SPL Image Guided Surgery and Visualization
  (Kikinis, Westin, Hata, Halle, others)
• Slicer Application Pulled Together by Dave Gering
  1997-1999 with VTK and Tcl
• Further Development and Architecture by Lauren
  ODonnell 1999-2001
• Ongoing Development of Slicers Base Primarily by
  Steve Pieper and Nicole Aucoin
• Many Modules and Contributions by Various Authors
• BWH, MIT, MGH, Georgia Tech, UCSD, JHU
• Now being used as a reference application
  platform for NA-MIC

7
Why Develop with Slicer?

• Start with a Powerful Platform
• Remove Obstacles to Problem Solving
• Access to Every Layer of Source Code Numeric,
  Graphic, Network, etc.
• Commit to an Environment that will Always be
  Available
• Not Tied Up in IP of Old Institution
• Not Tied to Proprietary Platform with License
  Fees
• Keep Your Own Work Available to You
• Solve a Problem Once, Always Find it with Google

8
Slicer Today

• 500K Lines of Code
• Cross-Platform Tcl/Tk GUI
• VTK/ITK Based C Computing
• www.slicer.org
• 166 on slicer-users
• 117 on slicer-devel
• gt5000 Registered Downloads

9
Architecture

• Plug-in Modules consist of Tcl and C code with
  cross-platform file layout for building and
  loading
• Standard naming convention and hooks to add GUI
  and processing components
• Personally, I find the speed of C and the
  interactive development of Tcl to be a
  near-perfect environment

10
Features

• Load Medical Image Data MR, CT in DICOM, GE,
  Analyze
• XML-based File Format MRML (Medical Reality
  Markup Language)
• Interactive Editor Draw, Threshold, Math
  Morphology
• Automated Segmenters EM Segmentation, Fast
  Marching, Level Sets
• Visualization Model Building, Stereo Rendering,
  Animation
• Registration Manual, ITK, CNI
• Measurement Fiducial-Based, Volumetric,
  Polyhedral Intersection, Vessel Cross-Section,
  Osteotomy Planning
• IGT Tracked Probes, Real-Time Images, Robot
  Control
• Additional Application-Specific Features in
  Modules

11
Administration

• Project Housed at Surgical Planning Lab, Brigham
  and Womens Hospital / Harvard Medical School,
  Many MIT CSAIL (formerly AI Lab) Students and
  Faculty Involved
• CVS, Mailing Lists, etc at the SPL
• Funded Projects Supporting Slicer Base
  Development
• Neuroimage Analysis Center (NAC) NIH Center at
  SPL
• Biomedical Informatics Research Network (BIRN)
  NIH Supported Neuroimaging Collaboration
• Computer Integrated Surgical Systems and
  Technology (CISST) NSF Supported Robotics
  Collaboration Headed by JHU
• Virtual Soldier DARPA Organ Simulation
  Collaboration
• National Alliance for Medical Image Computing
  (NA-MIC)
• Module Development Supported by
  Application-Oriented Grants

12
Image/Scene Management

• XML-Based MRML File Stores Scene Description
• Volumes (Images, Label Maps)
• Models
• Hierarchical Affine Transforms
• Scene Data (Cameras, Colors, Fiducials, etc).
• Manipulated in World Coordinates based on Patient
  RAS
• Automatically Extracted from DICOM or GE Files

13
Image Formats

• DICOM, GE, Headerless, Analyze
• Best Support for MR
• CT and RGB Support limited
• Real-Time from OpenMR and US
• Time Series
• Analyze Sequence
• BXH Files
• BIAC XML Header (Brian Image Analysis Center,
  Duke)
• Like MRML for fMRI Integration Work Ongoing

14
User Segmentation Tools

• Label Map Editor
• Draw on Orthogonal Planes
• Connected Component Island Tools
• Math Morphology
• Image Masking and Logical Operations
• Level Set, Fast Marching

15
EM Segmenter

• Segmentation tool designed for fully automatic,
  high-quality parcellation of the brain
• Segmentation of cortical and ventricle
  substructures
• Multi channel input

• Hierarchical segmentation based on anatomy
• Multi threaded
• User friendly interface

For further information see Pohl et al.
Incorporating Non-Rigid Registration into
Expectation Maximization Algorithm to Segment MR
Images, MICCAI 2002, pp. 564-572
16
Models

• Triangle Meshes from Label Maps
• Marching Cubes, Decimation, Smoothing
• Model Hierarchies
• Clipping By Slice Planes

Image Developmental Neuroinformatics, Simon
Warfield
17
Registration

• Interactive Manual Transform Editing
• Landmark Based Alignment
• Rigid Intensity Registration
• Mutual Information Metric
• ITK Implementation
• Non-Rigid Registration
• VTK-Based Demons Method
• Available by Request to CNI

18
Diffusion Tensor Tractography

• Multiple MR Gradient Acquisitions
• Sensitive to Brownian Diffusion of Water
• Cell Membranes Restrict Diffusion
• Post Processing to Extract Probable White Matter
  Tracts
• Actual Tracts are Far Below the Resolution of the
  Scan

Images Provided by Westin, Park, ODonnell et al
19
DTMRI Tools

• Convert Gradient Images to Tensors
• Generate Scalars
• ADC, FA, etc
• Visualize Glyphs
• Tractography
• User Guided
• From ROI

20
Segmentation and Tractography

• Parcellation
• Freesurfer (MGH)
• Tractography
• DoDTI (H.J. Park)
• Visualization
• Slicer
• Full Integration with Slicer Planned

21
DTMRI Examples

• White Matter Atlas
• Under Development in Slicer by James Fallon, UCI

22
Slicer fMRI Goals

• Advanced GUI and Interactive Visualization
  Environment
• iBrowser
• Platform for Activation Detector Research
• fMRIEngine
• Framework for fMRI Integration with Other
  Modalities

23
fMRI Integration in Slicer
Slicer environment
ibrowser

• implemented functionality
• planned functionality

reorient
Non-rigid registration
motion correct
smooth

• Segmentation tools
• Image analysis tools
• Multi-modality visualization

fMRIEngine
mask
signal modeling
contrast design
GLM (and other) activation detection
threshold
Interrogate data
24
interval browser overview

• volumetric (Analyze, DICOM, BXH, etc.)

data types/formats spatial processing visu
alization information
reorienting, smoothing, normalization, motion
correction, non-rigid registration, etc.

• index, animate, record
• GUI presents a graphical schematic of loaded data

• help
• tool tips

W. Plesniak, S. Pieper, W. Wells
25
ibrowser time-series GUI
animation viewing
name order visibility copy delete hold viewer
FG viewer BG
manual indexing
GUI controls for indexing, animating, operating
on sequences of volumes GUI panel provides a
graphical schematic of loaded volume sequences.
W. Plesniak, S. Pieper, W. Wells
26
ibrowser GUI updates Viewer
W. Plesniak, S. Pieper, W. Wells
27
ibrowser design
vtkImageInterval
Proposed Ibrowser architecture
main Slicer GUI
vtkIntervalCollection
Create MRML nodes
load/ select
. . .
derive new
index
vtkImageIntervalDrop
vtkImageMultipleInputFilter
spatial processing
composite
vtkImageToImageFilter
main viewer
interpolate
populate schematic
vtkImageData
vtkImageTwoInputFilter
Set name, order, opacity, visibility,
interpolation func.
data schematic
Select, index, step, or animate
Record animation, save screenshot
persistent browser
28
ibrowser other applications
visualizing animated cardiac perfusion study
29
ibrowser current / future work
interval browser early version in Slicer 2.4
Data loading Analyze, DICOM, BXH Data
organization persistent GUI organizes and
indexes multi-volume data Data processing
multi-volume window, level and threshold
reorient shuffle MRML extension in progress
vtkMRMLVolumeCollectionNode vtkMRMLVolumeGroupNod
e vtkMRMLVolumeRefNode Ongoing future work
additional processing capabilities as non-rigid
registration representing collections of other
data types (models, events, etc.) as illustrated.
design mock-up
30
fMRIEngine overview

• GLM-based
• MI-based

activation detection model specification v
isualization information

• design matrix
• signal modeling
• spatial priors

• voxel time-course and experimental protocol
• activation overlays

• help
• tool tips

H. Liu, W. Plesniak, S. Pieper, W. Wells, C.
Wible
31
fMRIEngine current status

• Data loading Loads Analyze (3D and 4D), DICOM,
  BXH (BIAC XML Header) format data or imports
  from Interval Browser
• Protocol specification input block design via
  GUI or load/save in text file
• Activation computing GLM detection, currently
  supports block design protocol and single
  regressor

H. Liu, W. Plesniak, S. Pieper, W. Wells, C.
Wible
32
fMRIEngine visualization

• generates color-coded parametric map of
  activation
• 3D visualization of activation in the context of
  subjects own anatomy or in a standardized
  morphological space
• provides interactive activation filtering and
  background masking.

H. Liu, W. Plesniak, S. Pieper, W. Wells, C.
Wible
33
fMRIEngine data inspection

• interactive inspection of voxel timecourse.

Measured voxel timecourse over entire protocol
Average voxel timecourse for all volumes in each
of two experimental conditions
H. Liu, W. Plesniak, S. Pieper, W. Wells, C.
Wible
34
fMRIEngine design
Proposed fMRIEngine architecture
vtkImageInterval
vtkImageData
vtkActivationVolumeGenerator
create MRML data vol
. . .
load/ select
main viewer
modeling
compute
mask
voxel timecourse
activation plot
reference waveform
GLM/MI
protocol
vtkPlotActor2D
vtkStimulusGenerator
HRF
vtkActivationDetector
35
fMRIEngine future work
fMRIEngine early version in Slicer 2.4ongoing
future work
Visualizing FSL data currently reads
FSL-generated output in Slicer, including
activation detection results, pop-up voxel
timecourse plots and FSLs HTML analysis report
exploring additional interoperation with
FSL. Model specification developing GUI tools
for creating design matrix and specifying
contrasts, additional signal modeling, and
reviewing the model Extending native I/O plans
to develop native I/O routines to support other
image formats, including NIfTI and
MINC. Activation detection beginning work to
implement multiple regression, MI-based
activation detection, the incorporation of
spatial priors, and ROI analyses
36
Mutual Information Detector

• Under Development at BWH/MIT (Wells, et al)
• To Detect Partially Responding Voxels
• Treat Paradigm and Voxel as Random Variables
• Initial Results Indicate MI can Detect Complex
  Relationships Missed by GLM
• Slicer Implementation (fMRIEngine) Planned
• Future Work
• Activation Detectors Incorporating Structural and
  DTI Information

• I(u,v) h(v) h(vu) h(u) h(uv)
• h(v) -Ev log2(P(v))
• h(vu) -Eu Ev log2(P(vu))

37
Application Pre-Operative Map

• Structural
• MRI Tumor Segmentation
• DTI
• Diffusion Tensor Imaging
• fMRI
• Functional MRI
• MEG
• Magneto Encephlogram
• Anatomy Atlas
• Textbook Information

38
Pre-Op Map Example
39
Resources

• www.slicer.org
• www.na-mic.org/Wiki
• www.na-mic.org/Bug
• www.na-mic.org/Testing
• slicer-devel_at_bwh.harvard.edu
• slicer-users_at_bwh.harvard.edu