3D Slicer Architecture and Implementation

1
3D SlicerArchitecture andImplementation

• Steve Pieper, PhD

2
Goals

• NA-MIC Kit Software and Methodologies for
  Medical Image Computing
• Facilitate Research
• Promote Interoperability
• Stable, Cross-Platform Run Time Environment
• Full Set of Core Features
• Avoid Duplicated Effort
• Flexible Module Architecture
• Plug-ins should be As Simple As Possible

3
Overview

• MRML, Logic, GUI
• Core Libraries and Dependencies
• Some Details of MRML
• Modules

4
Slicer3 Observer MVC Pattern

• MRML (Model)
• For Scene Description and Application State
• MRML Nodes are Persistent and Undoable
• Scene and Nodes are Observable
• Logic Encapsulate VTK and ITK Pipelines
  (Controller)
• Observe MRML to Configure Pipelines
• Help Create/Manage Nodes
• No UI Components (no Widgets, Actors, Mappers,
  Renderers or RenderWindows)
• GUI (View)
• Observe and Edit MRML
• Interact with User and Display Hardware
• Modules Should Follow Same Conventions

GUI
Widgets
Renderers
Edit
Observe
Logic
Observe
Edit
Observe
Edit
MRML Nodes
Observe means generic event mechanisms are
used to pass information. Edit means code can
directly call methods. Example GUI can call
methods in Logic classes,but Logic cannot call
GUI methods. MRML cannot call Logic or GUI
methods. There can be many observers for any
event.
5
Example EM in Slicer3
MRML
Logic
GUI

• Manage MRML nodes
• API for access to parameters
• Manage hierarchy
• Segmentation algorithm

• Window to parameter set
• Wizard
• Interaction with images

• Global Parameters
• Hierarchical parameters
• Image data
• Segmentation output

6
Example EM Batch Tool
MRML
Logic
CLI

• Manage MRML nodes
• API for access to parameters
• Manage hierarchy
• Segmentation algorithm

• Command Line Parsing
• Load Parameter Description MRML Scenes

• Global Parameters
• Hierarchical parameters
• Image data
• Segmentation output

7
Library Dependencies
8
vtkITK, vtkTeem
Slicer3

• Teem Multidimensional Raster Image Library
  (Gordon Kindlmann)
• Includes NRRD File Format
• ExtensiveTensor Manipulation Utilities
• vtkTeem, vtkITK Create VTK Compatible Filters
  using ITK and Teem Code for Implementation
• Includes vtkITKArchetype Readers and Writers
  Implemented with ITK I/O Factories

SlicerBaseGUI
SlicerBaseLogic
MRML
KWWidgets
vtkTeem
vtkITK
ITK
VTK
Tcl/Tk
Teem
9
MRML
Slicer3

• MRML Medical Reality Markup Language
• Library Provides Central Data Representation for
  Slicer3
• Application State is Explicitly Described in MRML
• GUI and Modules
• Observe MRML to Learn of Changes to State
• Manipulate State to Reflect User Interaction and
  Calculation Results

SlicerBaseGUI
SlicerBaseLogic
MRML
KWWidgets
vtkTeem
vtkITK
ITK
VTK
Tcl/Tk
Teem
10
SlicerBaseLogic
Slicer3

• Utility Code for Implementing Base Application
  Functionality
• Input/Output
• Volume Reslicing
• Image Filters
• No Rendering, and No GUI Code

SlicerBaseGUI
SlicerBaseLogic
MRML
KWWidgets
vtkTeem
vtkITK
ITK
VTK
Tcl/Tk
Teem
11
SlicerBaseGUI
Slicer3

• KWWidgets Subclasses
• vtkSlicerApplication
• Custom KWWidgets
• Rendering Utility Code
• VTK 3D Widget Subclasses

SlicerBaseGUI
SlicerBaseLogic
MRML
KWWidgets
vtkTeem
vtkITK
ITK
VTK
Tcl/Tk
Teem
12
Scene Description
Provided by D. Gering
13
MRML Concepts

• Node
• Unit of Organization for MRML
• Group of Variables Representing an Object or
  Concept State
• Scene
• Slicer Application has Single Scene Instance that
  has Collection of Nodes and API for
  Access/Create/Delete
• Undo/Redo Scene Snapshots
• Scene Swaps Nodes in to / out of Current State
• Logic/GUI Auto-Update through Observers
• Serialization / Deserialization
• Nodes Responsible for Read/Write of XML Version
  of State
• Scene Read/Write Analogous to Snapshots

14
MRML Concepts (cont)

• Observers / Events
• Scene and Nodes Invoke Custom Observable Events
  for Specific Actions (e.g. NodeAddedEvent) or
  ModifiedEvents for General Updates
• Node API Allows Disabling ModifiedEvent
  Invocation to Allow Groups of Operations with
  Single ModifiedEvent
• Node API Includes MRMLObserverManager Class to
  Simplify Addition/Removal of Multiple Event
  Observers
• Scene Provides EventBroker to Support Observer
  Introspection, Asynchronous Invocation, Logging,
  and Event Compression

15
MRML Concepts (cont)

• IDs References
• Each Node is Given a Unique ID for Retrieval in
  Scene
• Nodes Can Refer to Other Nodes by ID
• E.g. TransformableNodes can Refer to a
  TransformNode by ID
• E.g. ModelNode can Refer to ModelDisplayNode by
  ID
• Scene Maintains ReferencedIDs and
  ReferencingNodes Lists to Maintain One-to-One
  Relationship During Scene Import

16
Node Types

• Displayable/Storable/Transformable
• Display/Storage/Transform
• Hierarchy
• Parameter
• Selection
• Slice
• SliceComposite
• Color
• Fiducial
• Snapshot
• Command Line Module
• Other Module-Specific Nodes

17
Observer MVC Example

• Event User Picks Add Volume in Volumes GUI
  (KWWidget)
• Volumes GUI calls AddArchetypeVolume in
  VolumesLogic
• Volumes Logic Creates VolumeNode and
  VolumeStorageNode, Reads Data, and Adds to Scene
• Scene Invokes NodeAddedEvent
• NodeSelector Widgets Update Menus
• Volumes GUI Sets Active Volume on SelectionNode
• Slice Viewers Updated to New Volume

GUI
Widgets
Renderers
Edit
Observe
Logic
Observe
Edit
Observe
Edit
MRML Nodes
Notes Event invocation happens synchronously in
Slicer 3.2, meaning volume is read and updated
before AddArchetypeVolume method returns.
Asynchronous event processing may become the
default in the future. GUI Currently propagates
selection to each slice composite node in the
scene. In the future the slice GUIs should be
observing the scene and optionally display newly
loaded volumes.
18
Coordinate Systems

• IJK (Index Coordinates of Volume)
• vtkMRMLVolumeNode uses vtkImageData to store
  voxels, with Origin 0,0,0 and Spacing 1,1,1
• RAS (Right-Anterior-Superior)
• vtkMRMLVolumeNodeGetIJKToRASMatrix provides
  vtkMatrix4x4 that includes origin, spacing, and
  direction vectors to map to patient space
• Origin is at the center of the voxel
• Vectors Point in Specified Direction (i.e. R goes
  from Left to Right)
• LPS (Left-Posterior-Superior)
• Used by ITK code and DICOM
• Slicer Automatically Converts RAS-gtLPS

19
Transforms

• TransformToParent
• vtkMRMLTransformableNode Instances Contain
  TransformNodeID Reference to a vtkMRMLTransformNod
  e
• Allows Many Nodes Inside Each Transform and
  Transform Hierarchies
• TranfrormNode Defines how to Go from Child to
  Parent Space (e.g. a ModelNode points to a
  TransfromNode that Scales it to Global RAS
  Coordinates)
• Transforms Represented by vtkGeneralTransforms
  (linear/nonlinear)
• (TransformFromParent not yet implemented)
• Nonlinear Registration Results Often Better
  Represented as Resampling Transforms, Opposite of
  Current Modeling Transforms

20
Modularity Goals

• Keep the base package lean and mean
• Modules have individual identity
• Per-module web site, svn, downloads, mailing
  lists, wiki
• Allow users to assemble their own set of tools
• Customized Bundles by task or application
• Easy to download compatible extensions
• Analogous to Firefox extensions
• Integrate extension builds into
  developer/nightly/release processs
• NITRC Supplement to NA-MIC helping to pay for
  needed infrastructure (Neuroimaging Informatics
  Tools and Resources Clearinghouse)
• NITRC can host neuroimaging projects (gforge
  implementation)

21
Base Features vs. Modules
Features Base Modules
Visualization MRML Models, Volumes, Volume Rendering, Fiducials, Continuum Meshes, Labeled Data Can create custom MRML Nodes and behavior
Filtering None Implemented as Modules using ITK or other Libraries
Registration Transform Display and Edit, Save/Restore Calculate Transforms, Resample Data
Segmentation Label Maps, Parcellated Surfaces Segmentation Algorithms in ITK or other Libraries
Quantification Label, Image, Volume Statistics Numpy access to MRML Applications in Python or MATLAB
Real-time Integration VTK Rendering, KWWidgets framework, Tracker Support (as Transforms) Direct Manipulation of the MRML Scene 2D/3D Widgets Device Interfaces, OpenIGTLink Module
Diffusion Imaging DWI, DTI, Fiber Bundles Tractography, Clustering, Atlases
Applications Bundles of Modules in Distribution Registration, Editor, some Filters, Customized Extensions, Domain-specific code, Optimized Interfaces
22
Module Terminology

• Built-In Modules
• Few Core Features Linked to Slicer3.cxx
• Non-Core Modules becoming Loadable
• Loadable Modules
• Dynamically Discovered and Loaded
• Can Access Applications MRML Scene Instance and
  Call SlicerBaseGUI and Related APIs
  (vtkRendering, KWWidgets)
• Scripted Modules
• Like Loadable, but Written in Tcl or Python
• Command Line Modules
• Can Run as Stand Alone Executables
• Provide Standard Command Line Parsing, which
  Allows Slicer to Generate GUI Automatically
• Can be Compiled as Shared Library for Dynamic
  Link to Slicer

23
Loadable Modules

• http//www.slicer.org/slicerWiki/index.php/Slicer3
  How_to_implement_an_Interactive_Module_GUI
• http//www.slicer.org/slicerWiki/index.php/Slicer3
  Loadable_ModulesHOWTO
• Create vtkSlicerModuleGUI Subclass
• Override Virtual Methods for CreateGUI,
  ProcessMRMLEvents, etc.
• Provide Custom vtkMRMLNode Subclass(es) to
  Represent, Save and Restore state
• Create Logic Classes to Implement Module
  Functionality
• Create CMake Configuration Files to Build and
  Install Shared Libraries for Runtime Discovery
  and Packaging

24
Scripted Modules

• A Generic ScriptedModule Module Provides Mapping
  of C Method Entry Points to a Corresponding
  Naming Convention for Scripts
• Scripts are Dynamically Discovered at Runtime
• A Generic ScriptedModuleNode Allows Save/Restore
  of Keyword / Value Pairs to Represent Module
  State
• Full API of Slicer, MRML, KWWidgets, VTK,
  available (except where not wrappable these APIs
  have been designed for scriptability)
• Editor is extensive Tcl-based Module
• Several Python examples are included in current
  svn trunk
• Python modules can also access Numpy routines for
  Numerical processing (Volumes and other
  DataArrays are directly mapped to Numpy arrays
  for easy manipulation)

25
Command Line Modules

• Use the GenerateCLP command (provided with
  Slicer) to convert XML description of program
  arguments into a C .h file
• Main program of module uses PARSE_ARGS macro
  provided by .h file to instance variables
  corresponding to parameters defined in XML file
• PARSE_ARGS also defines a --xml argument that
  returns original XML string on stdout
• Slicer parses the XML to create GUI for each
  module
• Are Covered Extensively in the HelloWorld
  Tutorial
• Note Command Line Modules can be written in any
  language. C utilities are provided, but can be
  emulated / recreated as needed.