Programming for Image Processing/Analysis and Visualization using The Visualization Toolkit

1
Programming for Image Processing/Analysis and
Visualization using The Visualization Toolkit
http//noodle.med.yale.edu/seminar/seminar.html

• Xenios Papademetris
• papad_at_noodle.med.yale.edu
• BML 325, 5-7294

2
Course Structure

• Using VTK using scripting languages
• Understand Toolkit Structure
• Use existing algorithms (incl. local extensions)
• Learn Tcl/Tk
• Extending VTK
• C/Object Oriented Programming/Design
• Cross-Platform Issues
• Coding Guidelines

3
Schedule Part 2

1. Review of Part 1 and Course Overview
2. C Pointers/Classes, Object Oriented Programming
3. Adding new VTK Commands/Cmake
4. Image-to-image filters/ surface to surface
   filters
5. Case Study I -- Iterative Closest Point surface
   matching
6. Case Study II A Simple Segmentation Algortihm

4
VTK Pipeline (I)
File Output
Sources
Filters
Mappers
Props
vtkDataSet e.g. Images vtkImageData Surfaces
vtkPointData
vtkDataSet
5
VTK Pipeline (II)
Props
Render Window
Renderer
Props
Props (e.g. Actor/Volume)
vtkCamera, vtkLight
vtkRenderWindowInteractor
vtkProperty
6
Datasets

• Organizing structure plus attributes
• Structured points
• Rectilinear Grid
• Structured Grid

7
Data Representation(vtkDataSet)
Point Attributes (vtkPointData) Point Properties
(e.g. intensity)
Points (vtkPoints) Define Location
Arrays of Numbers (one per point or
cell) vtkDataArray
Data Set
Cells (vtkCellArray) Define Topology
Cell Attributes (vtkPointData) Cell Properties
(e.g. normal)
8
Cells

• Cell is defined by an ordered list of points
• Triangle, quadrilateral points specified counter
  clockwise
• Others as shown

6
3
7
5
4
1
3
2
2
Hexahedron
1
Tetrahedron
0
0
9
Representing arrays with vtkDataArray

• vtkDataArray is an abstract superclass for
  classes representing arrays of vectors called
  tuples (or numbers treated as vectors of length
  1). Each tuple consists of a set of numbers or
  components.
• Derived Classes include vtkUnsignedCharArray,
  vtkShortArray, vtkFloatArray, vtkDoubleArray etc.
• Can function either as a dynamic array (lower
  performance) or a fixed length array
• All data in VTK is stored ultimately in one of
  the many derived classes of vtkDataArray,
• e.g. in the case of vtkImageData the intensities
  are stored in a vtkDataArray having dimensions
  equal to the number of voxels and vector length
  typically equal to 1 (3 for color images, Number
  of Frames for multiframe data such as fMRI
  cardiac etc.)

10
vtkDataArray Hierarchy
11
A concrete example vtkFloatArray
Mode 1 Fixed Length Array To create in TCL
type vtkFloatArray arr arr SetNumberOfComponents
1 arr SetNumberOfTuples 20 arr SetComponent
10 0 10.0 arr SetTuple1 11 9.0 set b arr
GetComponent 10 0 This creates an array of 20
(Number of Tuples) vectors each having size 1
(Number of Components) We access elements in
this array be using the SetComponent and
GetComponent methods. All indices start at 0.
12
A concrete example vtkFloatArray
Mode 2 Dynamic Array To create in TCL
type vtkFloatArray arr arr SetNumberOfComponents
1 arr InsertNextTuple1 5 arr InsertNextTuple1
10 set b arr GetComponent 1 0 This creates
a dynamic array of vectors each having size 1
(Number of Components). The InsertNextTuple
command allocates memory dynamically and places
the value there Note also (InsertNextTuple2,
InsertNextTuple3,4,9 for multi-component arrays)
13
Images are Simple Data-sets
Point Attributes (vtkPointData) Point Properties
(e.g. intensity)
Points (vtkPoints) Define Location
Arrays of Numbers (one per point or
cell) vtkDataArray
vtkImageData
Cells (vtkCellArray) Define Topology
Cell Attributes (vtkCellData) Cell Properties
(e.g. normal)
14
vtkImageData

• vtkImageData is the basic VTK class for storing
  images.
• It is defined by 4 key elements
• Dimensions -- these define the size of the image
• Origin -- the position in 3D space of point 0 0 0
• Spacing -- the voxel dimensions
• Scalar Type -- the type of the image (e.g.
  float, short etc)
• An 4x4x4 image has 4x4x464 points and 3x3x327
  cubic cells (both are implicitly defined)

15
Manually Creating an Image in TCL

• vtkImageData img
• img SetDimensions 10 10 2
• img SetOrigin 0 0 0
• img SetSpacing 0.78 0.78 1.5
• img SetScalarType VTK_SHORT
• img SetNumberOfScalarComponents 1
• img AllocateScalars
• Intensity values can be accessed using the scalar
  array i.e.
• Point 0 is (0,0,0), point 1 is (1,0,0), point 10
  is (0,1,0), point 100 is (0,0,1)
• set data img GetPointData GetScalars
• data SetComponent 10 0 5.0
• set v data GetComponent 10 0
• Set v2 img GetScalarComponentAsFloat 0 1 0 0
  
• ( this unfortunately is the nearest vtk comes to
  a getvoxel, no set voxel command)

16
Multi-Platform Program Structure
Hardware Layer (Windows/Linux/Unix/Mac OS X)
17
A First Example(http//www.vtk.org/example-code.p
hp)
Creating a sphere in Tcl, C or Java, using the
underlying C VTK libraries.
18
TCL part 1
First we include the VTK Tcl packages which
will make available all of the vtk commands
from Tcl. The vtkinteraction package defines a
simple Tcl/Tk interactor widget. package require
vtk package require vtkinteraction create
sphere geometry vtkSphereSource sphere sphere
SetRadius 1.0 sphere SetThetaResolution 18 sphere
SetPhiResolution 18 map to graphics
library vtkPolyDataMapper map map SetInput
sphere GetOutput actor coordinates geometry,
properties, transformation vtkActor
aSphere aSphere SetMapper map aSphere
GetProperty SetColor 0 0 1 blue
19
TCL part 2
actor coordinates geometry, properties,
transformation vtkActor aSphere aSphere SetMapper
map aSphere GetProperty SetColor 0 0 1
blue create a renderer add the
sphere vtkRenderer ren1 ren1 AddActor
aSphere ren1 SetBackground 1 1 1 Background
color white create a window to render
into vtkRenderWindow renWin renWin AddRenderer
ren1 create an interactor vtkRenderWindowIntera
ctor iren iren SetRenderWindow renWin Render
an image since no lights/cameras specified,
created automatically renWin Render
20
C part 1
include "vtkSphereSource.h" include
"vtkPolyDataMapper.h" include "vtkActor.h" inclu
de "vtkRenderWindow.h" include
"vtkRenderer.h" include "vtkRenderWindowInteracto
r.h" void main () // create sphere geometry
vtkSphereSource sphere vtkSphereSourceNew()
sphere-gtSetRadius(1.0) sphere-gtSetThetaResol
ution(18) sphere-gtSetPhiResolution(18) vtkPol
yDataMapper map vtkPolyDataMapperNew()
map-gtSetInput(sphere-gtGetOutput()) // actor
coordinates geometry, properties, transformation
vtkActor aSphere vtkActorNew()
aSphere-gtSetMapper(map) aSphere-gtGetProperty()-
gtSetColor(0,0,1) // sphere color blue
21
Java part 1
import vtk. public class test // in the
static constructor we load in the native code
// The libraries must be in your path to work
static System.loadLibrary("vtkCommonJava")
System.loadLibrary("vtkFilteringJava")
System.loadLibrary("vtkIOJava")
System.loadLibrary("vtkImagingJava")
System.loadLibrary("vtkGraphicsJava")
System.loadLibrary("vtkRenderingJava")
public static void main (String args) //
create sphere geometry vtkSphereSource sphere
new vtkSphereSource() sphere.SetRadius(1.0) sp
here.SetThetaResolution(18) sphere.SetPhiResolut
ion(18) // map to graphics objects vtkPolyData
Mapper map new vtkPolyDataMapper() map.SetInpu
t(sphere.GetOutput()) // actor coordinates
geometry, properties, transformation vtkActor
aSphere new vtkActor() aSphere.SetMapper(map)
aSphere.GetProperty().SetColor(0,0,1) // color
blue
22
Comment on Language Selection

• VTK can be used from any one of C/Java/Tcl and
  Python
• Some features not accessible from the scripting
  languages
• In Java/Tcl/Python our own code can
• Be used to join standard VTK elements in a
  pipeline
• Call existing VTK routines
• In C we can do all of the above plus
• Write new filters/sources etc as required (which
  can in turn be used from the other languages)

23
Programming Languages

• Interpreted vs Compiled Languages
• Note that classic distinction is blurring thanks
  to on-the-fly compilers (e.g. Tcl 8.x)
• Interpreted Languages
• e.g. BASIC, Matlab, Tcl, Python, Perl
• Fast development, computationally inefficient
• Compiled Languages
• e.g. C/C, Fortran, Java(?)
• Computationally more efficient but extra step in
  development cycle

24
Interpreted Languages

• Program execution is performed by the interpreter
  or shell e.g.
• TCL (tcsh/wish)
• Allows for interactive command execution
• Faster development cycle because there is no
  lengthy compilation step
• Slower performance as each command needs to be
  parsed before it is executed
• Simpler to use!

25
Compiled Languages

• Compiler converts source code into executable.
• Typically allows for standalone programs
  (especially if statically linked more later)
• Slower development cycle because there is a
  compilation step.
• Less prone to syntax errors as compiler flags
  these out.
• More complex to use some understanding of lower
  level computer fundamentals is necessary.

26
Compiling a simple C program(single source
file cosine.c)

• include ltmath.hgt
• include ltstdio.hgt
• int main(int argv,char argv)
• printf(Sin(40)5.3f\n,
• sin(40.0M_PI/180.0))

(In case you are unsure this program simply
prints the value of the cosine of 40 degrees,
the ugly 40.0M_PI/180.0 construct is needed as
the cos function requires its arguments to be in
radians)
27
Step 1 creating the object file(compiling)

• (This is assuming a UNIX setup)
• In the same directory as the file cosine.c type
• g c cosine.c
• This if everything works out will produce the
  object file cosine.o

28
Step 2 creating the executable(linking)

• (Again assuming a UNIX setup)
• In the same directory as the file cosine.o type
• g o cosine cosine.o -lm

Statement to link in the math library libm.so. In
unix shared libraries are assumed to begin with
lib which is not included in the link
directive.
Output File
29
Step 3 -- Execution

• Typing cosine on the command prompt, will
  (roughly)
• 1. Load the program cosine into memory
• 2. Load the system shared libraries into memory
  (e.g. libC and libm)
• 3. Start the program by calling the main function.

30
More on Libraries

• Typically many different programs are built using
  standard building blocks e.g.
• Math Library
• I/O Library
• Graphics Library
• Custom libraries e.g. to read/write certain
  medical image formats
• Libraries can be created as static or dynamic.
• Static each program has its own copy of the
  library embedded in the executable (typically .a
  or .lib)
• Dynamic one copy of the library for the whole
  system (typically .dll or .so)

31
Compiling a more complicated C program and
library

• Project consists of 5 files
• utility.h -- header file defining the
  specifications of the utility code
• Utility.cpp actual code for the utility code
• Print.h header file for the printing code
• print.cpp implementation of the printing code
• Main.cpp -- main program that uses code in
  utility.cpp and print.cpp
• Plan
• Create a library containing the code in
  utility.cpp and readwrite.cpp
• Link the library with the object file main.o to
  create the executable

32
Utility.h and Utility.cpp

• Utility.h
• include ltmath.hgt
• float deg2rad(float t)
• Utility.cpp
• include utility.h
• float deg2rad(float t)
• return tM_PI/180.0

33
print.h and print.cpp

• print.h
• include ltmath.hgt
• void printnumber(float t)
• print.cpp
• inlcude print.h
• void printnumber(float t)
• printf(The number is 5.2f\n,t)

34
Main.cpp

• main.cpp
• include print.h
• include utility.h
• include math.h
• int main (int argc,char argv)
• float t40.0
• float traddeg2rad(t)
• float costradcos(trad)
• printnumber(costrad)

35
Compling and LinkingMethod 1 no libaries

• g -c utility.c produces utility.o
• g -c print.c produces print.o
• g -c main.c produces main.o
• g -o main main.o utility.o print.o lm
• This yields the final executable main
• Note that the following one-step procedure is
  also possible but only useful for really small
  projects
• g -o main main.cpp utility.cpp print.cpp lm

36
Compling and LinkingMethod 2 static library
method

• g -c utility.c produces utility.o
• g -c print.c produces print.o
• ar -cr libutil.a print.o utility.o produces
  libutil.a
• (a ranlib step may also be needed)
• g -c main.c produces main.o
• g -o main -lutil lm
• This yields the final executable main by linking
  it with the library libutil.a and the math
  library libm.so. The size of main is
  approximately equal to
• Size(main,o) size(libutil.a) overhead
• The executable main will run just fine even if
  libutil.a is deleted.

37
Compling and LinkingMethod 3 shared library
method

• g -c utility.c produces utility.o
• g -c print.c produces print.o
• g -shared -o libutil.so print.o utility.o
  produces libutil.so
• g -c main.c produces main.o
• g -o main -lutil lm
• This yields the final executable main by linking
  it with the library libutil.so and the math
  library libm.so. The size of main is
  approximately equal to
• Size(main,o) overhead
• The executable main will NOT run if libutil.a is
  deleted.
• The LD_LIBRARY_PATH variable also needs to be set
  to point to the location of libutil.so

38
Automating compiling/linkingThe make command

• The compiling process of even the simplest
  project involves the typing of a large number of
  commands.
• The process is automated on UNIX using the make
  command and its corresponding macro file
  (makefile)
• On Windows (using Visual C) the same is done
  using either nmake (and makefiles) or project
  files in Visual Studio
• VTK uses a higher level program called cmake
  (www.cmake.org) which produces output for either
  unix makefiles or windows visual C project
  files/makefiles.
• We will only describe cmake next week.

39
Extending VTK the gameplan

• Gather our own code into one or more shared
  libraries
• Automatically wrap the library to generate
  interface code to make our library accessible
  from TCL (Java/Python also possible)
• Load our library into the tcl interpreter in
  effect making our code appear as additional TCL
  commands
• Use TCL as a glue language to test/call/debug our
  C code.

40
Next Week

• Discuss pointers and memory allocation issues
• Discuss Object-Oriented Design Philosophy in more
  detail
• Constructors/Destructors
• Inheritance
• Abstract Classes
• Virtual Functions
• Overriding Functions
• Discuss reference counted allocation/de-allocation

41
Homework

• If you are not familiar with basic C/C get a
  book and read about it. I will discuss the object
  stuff in detail but you should be familiar with
• Basic syntax
• Loop/conditional operators
• Procedures and functions
• Input/output statements
• Variable types
• If you have not already go through the slides
  from the last seminar series and freshen up on
  some of the material there. I will be assuming
  some familiarity with it.