3D Slicer

1
3D Slicer

• Ron Kikinis, M.D.
• Steve Pieper, Ph.D.

2
Acknowledgments

• F. Jolesz, C. Tempany, A. Golby, P. Black, S.
  Wells, CF. Westin, M. Halle, N. Hata, T. Kapur,
  A.Tannenbaum, M. Shenton, E. Grimson, P.Golland,
  W.Schroeder, J. Miller, N. Aucoin, A.
  Yarmarkovich, W. Lorensen, X. Tao, K. Hayes, S.
  Barre, W. Plesniak, R. Gollub, S. Pujol and many
  more.

3
Topic 3D Slicer Capabilities

• Display
• Informatics
• Computing
• Applications
• Background Future

4
What is 3D Slicer?

• A platform for image analysis and visualization
• Current Releases 2.7 and 3.2
• 2.7 most features and documentation
• 3.0 focus of current activity
• A freely-downloadable program
• Source code and executables available for
  Windows, Linux, and Mac OS X
• All Code Reusable
• Slicer is a research platform
• NOT an FDA approved medical device
• NOT finished some parts will work better than
  others

5
Slicer Features

• I/O
• Image DICOM, NIfTI, Analyze, Meta, NRRD, MGZ...
• Surface vtk, vtp, stl, freesurfer, fiber bundle
• Coordinate Systems All Data is
  Patient-Referenced
• Visualization Volume Rendering, Surfaces, Slices
  Planes, Clipping, Volume Overlays...
• Filtering Denoising, Nonlinear/Aniostropic
  Smoothing, Format Conversion
• Registration Multimodal (e.g. CT/MR), Rigid,
  Affine, BSpline. Apply Transforms to other
  volumes.

6
More Slicer Features

• Segmentation Manual Editor, Semi-Automated
  (Region Growing), Statistical Classifiers, Atlas
  Based
• Diffusion Imaging DICOM Import, Tensor Tools,
  Tractography
• Quantification Volume Measurements, Points,
  Lines
• Real-time Networked Trackers and Volume I/O
• And more...
• Meshing, Fiducials, Batch Processing, Remote Data
  I/O, Extensible in C/Tcl/Python, Slicer
  Daemon...

7
Integrated Scene

• XML-Based MRML File Stores Scene Description
• Volumes (Images, Label Maps)
• Models
• Hierarchical Affine Transforms
• Scene Data (Cameras, Colors, Fiducials, etc).
• Patient-Centric
• Manipulated in World Coordinates based on Patient
  RAS (Right-Anterior-Superior)

Provided by N. Archip et al
8
Display
9
Display

• Live Demo of CT Viewing
• Features
• Lightbox, CompareView, Data Fusion, Layouts

10
Informatics
11
Informatics BIRN

• Morphometry Population Statistics
• XNAT Archive
• FreeSurfer Batch Analysis
• QDEC GLM on Server
• Statistical Results via XAR File
• Interactive Exploration
• Drill Down
• Download to Slicer from XNAT Server

Data from Morphometry BIRN
12
FetchMI Medical Informatics

• Two-Way Communication
• Query
• Annoation
• Full Scene Upload Download
• All Slicer MRML Features Supported via URL

W. Plesniak and N. Aucoin
13
QueryAtlas

• Ontology-Based Interaction with Morphometric and
  Functional Data
• Integrated Interpretation of Anatomy and
  Individual or Group Statistics
• Cross-Ontology Translation and Ontology Browsers
  (FreeSurfer, UMLS, BIRNLex, NeuroNames, IBVD)
• Context Based Web and Journal Searches
• Anatomy Terms, Diagnoses, Clinical Indicators,
  Free Terms
• Browser Integration

W. Plesniak and M. Halle
14
Computing Analysis
15
EM Segmenter

• Multi-Channel Atlas Based Segmentation
• Probabilistic Atlas
• Hierarchical Definition of Anatomy and Image
  Properties
• Interactive Slicer Module to Determine Parameters
• Batch Mode for Population Analysis

K. Pohl
16
Interactive Editor

• Tools for Precise Manual Segmentation
• Interactive Steering of Powerful Segmentation
  Routines

S. Pieper and C. Mendoza
17
Each module has
an entry in the module menu
a panel of user interface controls
J. Miller and B. Lorensen
18
Integration Options
Slicer Libs ModuleDescriptionParser vtkITK MRML C/C Non-slicer specific support libraries
Slicer Base Application logic Widgets C Common infrastructure for Slicer applications
Built in modules Slice viewers Models Fiducials Transforms C Full access to Slicer internals
Loadable modules Query Atlas QDEC Volume rendering ChangeTracker EMSegment C/Python/Tcl Full access to Slicer internals
Scripted modules Editor Teem Two Tensor Tractography VMTK Python/ Tcl Limited access to Slicer internals
Command line modules Registration C/C/Java/MATLAB/Python/Tcl/Fortran Restricted access to Slicer internals
Daemon OpenIGTLink Stochastic Tractography C/C/Java/MATLAB/Python/Tcl/Fortran Access to MRML
19
Non Interactive Computation

• BatchMake
• Language for Describing Experiments
• Efficiently Structure Computations to Generate
  Population Statistics
• Aylward, Jomier, Kitware, Inc.
• GridWizardEnterprise
• Interface to Distributed Computations
• caGrid Gateway in Design
• Ruiz, Grethe, UCSD

20
Applications
21
Trans-Rectal Robotic Prostate Biopsy

• Core needle biopsy definitive diagnostic for
  prostate cancer
• TRUS has been Gold standard for guiding biopsy
• MRI/MRS offers high sensitivity for localizing
  tumor
• Robotic access required inside scanner1,2

Objective Develop and integrate end-to-end
application module inside 3D Slicer to perform
robotic prostate biopsy intervention
1 Krieger A, Susil RC, Menard C, Coleman JA,
Fichtinger G, Atalar E, Whitcomb LL, Design of A
Novel MRI Compatible Manipulator for Image Guided
Prostate Intervention, IEEE Trans. Biomed. Eng.
2005 52(2)306-313 2 Susil RC, Ménard C,
Krieger A, Coleman JA, Camphausen K, Choyke P,
Ullman K, Smith S, Fichtinger G, Whitcomb LL,
Coleman NC, Atalar E, Transrectal Prostate Biopsy
and Fiducial Marker Placement in a Standard 1.5T
MRI Scanner, J Urol. 2006 Jan175(1)113-20
Fichtinger
22
Calibrate
Fichtinger
23
Segment
Fichtinger
24
Target
Fichtinger
25
Navigate
Fichtinger
26
Change Tracker

• Watchful Waiting of Brain Tumors
• Longitudinal Volumes
• Change Analysis Algorithms
• Leverages Slicer Infrastructure
• Registration
• CompareView
• Volume Rendering
• MRML Scene Files
• Workflow Wizard
• Ron Will Give Demo

With the Brain Science Foundation
Pohl and Federov
27
Longitudinal Registration

• Radiotherapy of B-Cell Lymphoma
• 13 MRI, Intensity Normalized and Registered to
  Cured Final Scan
• ITK Mattes MI Registration in Slicer2
• Video courtesy by Ervin Berenyi and Andras Jakab,
  Department of Medical Laboratory and Diagnostic
  Imaging, University of Debrecen Medical School
  and Health Science Center.

Berenyi and Jakab
28
MRSI

• MRSI Volume Analysis and Display
• Multi-Spectral, Multi-Modal
• MRSI Choline Map
• Turnover of Cell Membranes
• MRSI NAA Map
• Integrity of CNS Tissue
• Structural MRI
• Integrated Slicer Module for MRSI Calculations

B. Menze
29
IGT Neurosurgery Planning

• Fiducial Tractography
• Interactive Probing of Fiber Bundles
• Peritumoral Fibers
• fMRI Display in Context with Structural and
  Diffusion Volumes

JJ. Lemaire
30
Mapping
Provided by Archip, Warfield
Alignment of all pre-operative datasets to the
intra-operative images achieved during the
neurosurgery.
Archip N, Clatz O, Whalen S, Kacher D, Fedorov A,
Kot A, Chrisochoides N, Jolesz F, Golby A, Black
PM, Warfield SK. Non-rigid alignment of
pre-operative MRI, fMRI, and DT-MRI with
intra-operative MRI for enhanced visualization
and navigation in image-guided neurosurgery.
Neuroimage. 2007 Apr 135(2)609-24
31
OpenIGTLink

• December 2007 Concept
• January 2008 Prototype and Name
• July 2008 BrainLab VVLink with Yale
  (Papademetrios)
• December 2008 Real Time MR Control
• February 11, 2009 Initial Clinical Application
  with Dr. Alex Golby
• Multi-Site Collaboration coordinated by NCIGT
  (Jolesz, Hata et al)

Golby, Hata, Liu, Tokada, Ibanez, Papademetrios
32
Image Guided Ablation

• Pre-Op MR
• Tumor and Kidney Segmentation
• Registered with Intra-Op CT
• Needle Placement
• Treatment (Cryo) Monitoring

Sota, Elhawary
33
Case 1, 60 y/o, Male, CT guided Cryo ablation
for rt. Renal cell carcinoma
Image Guided Ablation
Intra-Op. CT
Pre-Op. MRI
Sota, Elhawary
34
Intra-Op. CT (compare with next)
3D image
Axial image
Coronal image
Zoomed Axial
Sota, Elhawary
Tumor
Tumor
35
Registered MRI
Sota, Elhawary
36
Consolidated view w. needle
Ice Ball
Probes
Tumor
Tumor
Kidney
Sota, Elhawary
37
Liver Lesion Treatment

• Needle Placement for CT-Guided Radio Frequency
  Ablation (RFA)
• Unresectable Hepatocellular Carcinoma (HCC) and
  Liver Metastases
• Open Source Environment
• Slicer3 and IGSTK
• Automated Liver Segmentation
• Path Planning Tools for Multiple Overlapping
  Lesions
• Validation on Swine Model

Sota, Elhawary, Cleary, Hata
38
Perfusion / DCE

• 4D Image Analysis and Visualization
• CompareView
• GPGPU (Cuda) Volume Rendering
• Lung Perfusion Application
• Dr. Hatabu
• Prostate Dynamic Contrast Enhancement
• Dr. Tempany

Tokuda, Hata, Hatabu, Tempeny
39
Radiotherapy Visualization

• Gamma Knife Planning and Visualization
• Registered CT and DTI
• Segmentation, Tractography, and Cropped Volume
  Rendering
• 2nd Place in Kitware Visualization Contest
• Images courtesy by Ervin Berenyi and Andras
  Jakab, Department of Medical Laboratory and
  Diagnostic Imaging, University of Debrecen
  Medical School and Health Science Center.

Berenyi and Jakab
40
Small Animal Imaging Initiatives

• File readers and tools for ultrasound and
  microPET added to Slicer3
• Registration algorithms for consistent results is
  challenging because of large-scale tissue
  deformation in small animals
• Segmentation efforts in process

Volume rendering of subcutaneous tumor segmented
from ultrasound using Slicer
GPU Accelerated ray-casting in Slicer for mouse
lung dataset
Collins, Liu, Lisle
41
Optical Microscopy Initiatives

• Microscopy Segmentation
• A microscopy-specific file format has been added
  to Slicer to support efficient interchange of
  datasets
• NCI-developed segmentation algorithm to be
  published and released as open-source through
  NA-MIC

Microscopy dataset in Slicer
Volume rendering in Slicer
Collins, Liu, Lisle
42
Shared Editing and Archiving Tools

• Shared tool infrastructure across multiple
  laboratories and applications
• Focuses expertise on smaller set of tools and
  increase data exchange

Microscopy
3D Slicer
Dataset Archive at NCI-Frederick
Small Animals
Angiogenesis
3D Slicer
...
Collins, Liu, Lisle
43
Background Future
44
Slicer Background

• 3D Slicer Goals
• Translation Platform to get Medical Computing
  Technology to DBP Researchers
• Provide Reference Implementation using NA-MIC Kit
• Outreach to New Applications
• 80 Rewrite from slicer2 to slicer3
• First Slicer3 svn commit January 26, 2006

45
FOSS A Public Highway

• Open-source is like a Public Road System
• Provides Infrastructure for a Variety of Uses
• Driveways can Lead to Anything
• a Public Park
• a Private Facility
• FOSS Free Open Source Software

Provided by Pieper, Kikinis
46
Dissemination and Training

• National and International Events
• MIT, MGH, UNC, EPFL, NIH, UNM, UCSD, TUM
• All Materials on Wiki
• Clinical and Technical Tracks
• Workshops
• MICCAI 2005, 2006, 2007, 2008..
• OHBM, RSNA, Munich, NCI

47
Project Weeks

• Eight So Far (June 2005 January 2009)
• Summers MIT Stata Center
• Winters Salt Lake City
• gt 100 Participants
• Universities Companies
• Wiki and Telephone Preparation
• Project Teams Sit and Work Together
• Face to Face Communication

48
Progress in 2008

• Numbers 2007
• Subversion Commits 3,407
• Lines of Code 371,428
• Bugs Features
• 154 Submitted
• 63 Closed
• Active Developers 33

• Numbers 2008
• Subversion Commits 2,971 (8,317)
• Lines of Code 735,536
• Bugs Features
• 239 Submitted
• 129 Closed
• Active Developers 53
• 3D Slicer Version 3.2
• Released August 8, 2008

find . -iname \.h -o -iname \.cxx -o -iname
\.tcl -o -iname \.java -o -name \.py grep -v
svn xargs wc (does not include libraries or
modules in external repositories) svn log
grep "r" cut -d " " -f 3 sort uniq wc
49
Next Steps

• Slicer becomes subject-centric platform for
  visualization and analysis. Add the following
  capabilities
• DICOM, HL7, IHE Acquisition and Patient
  Information
• Clinical Context Currently Missing from Slicer
• AIM Descriptions of Anatomy, Imaging Parameters,
  Observations
• Access to Human Expert Contribution
• Atlases Normative Data about Target Anatomy and
  how it Appears in Images
• Access to Normative Data Relevant to Subject
• Slicer Contributes
• Integrated Representation of Images and Derived
  Data
• Interactive Visualization
• Sophisticated Editing and Analysis Tools
• Community of Algorithm and Application Developers