Spatial Processing

1
Spatial Processing

• Chris Rorden
• Spatial Registration
• Motion correction
• Coregistration
• Normalization
• Interpolation
• Spatial Smoothing
• Advanced notes
• Spatial distortions of EPI scans
• Image intensity distortions
• Matrix mathematics

2
Why spatially register data?

• Statistics computed individually for voxels.
• Only meaningful if voxel examines same region
  across images.
• Therefore, images must be in spatially registered
  with each other.

3
Spatial Registration

• We use spatial registration to align images
• Motion correction (realignment) adjusts for an
  individuals head movements.
• Coregistration aligns two images of different
  modalities from the same individual.
• Normalization aligns images from different
  people.

4
Within-subject registration

• With-in subject registrations
• Assumption same individual, so there should be a
  good linear solution.

Motion correction
Coregistration
Registration of the fMRI scans (across time)
Registration of fMRI scans with high resolution
image.
5
Rigid Body Transforms

• Translation

• Rotation

• By measuring and correcting for translations and
  rotations, we can adjust for an objects movement
  in an image.

6
How many parameters?

• Each transform can be applied in 3 dimensions.
• Therefore, if we correct for both rotation and
  translation, we will compute 6 parameters.

• Translation

• Rotation

• Yaw

• Pitch

• Z

• X

• Roll

• Y

7
Motion Correction

• Motion correction aligns all in time series.
• Translations and rotations only
• rigid body registration
• Assumes brain size and shape identical across
  images.

8
Motion Correction

• Motion Correction (Realignment) is crucial
• We want to compare same part of the brain across
  time.
• If we do not MC, there will be a lot of
  variability in our data.
• Mathematically, MC is easy
• We assume that all images show the same brain, so
  rigid body transform is sufficient.
• All images have the same contrast.

9
Motion Correction Cost Function
2
When aligned, Difference squared 0

Reslice
Target
2
When unaligned, Difference squared gt 0

Reslice
Target
cfmi.georgetown.edu/classes/BootCamp/
10
Local Minima

• Search algorithm is iterative
• move the image a little bit.
• Test cost function
• Repeat until cost function is do not get better.
• Search algorithm can get stuck at local minima
  cost function suggests that no matter how the
  transformation parameters are changed a minimum
  has been reached

Value of Cost Function
Local Minimum
Global Minimum
Translation in X
cfmi.georgetown.edu/classes/BootCamp/
11
Motion correction cost function

• Motion correction uses variance to check if
  images are a good match.
• Smaller variance better match (least squares)
• Iterative moves image a bit at a time until
  match is worse.

Image 1
Difference
Image 2
Variance (Diff²)
12
Intensity unwarping

• Motion correction creates a spatially stabilized
  image.
• However, head motion also changes image intensity
  some regions of the brain will appear
  brighter/darker.
• SPM EPI unwarping corrects for brightness
  changes (right)
• FSL You can add motion parameters to statistical
  model (FEAT stats page).
• Problem We will lose statistical power if head
  motion is task related, e.g. pitch head every
  time we press a button

• Above motion related image intensity changes.

13
Coregistration

• Coregistration is more complicated than motion
  correction
• Rigid body not enough
• Size differs between images (must add zooms).
• fMRI scans often have spatial distortion not seen
  in other scans (must add shears).
• Variance cost function will fail relative
  contrast of gray matter, white matter, CSF and
  air differences between images.

14
Coregistration

• Coregistration is used to align images of
  different modalities from the same individual

• Uses mutual information cost function Note
  aligned images have neater histograms.
• Uses entropy reduction instead of variance
  reduction as cost function.

15
Coregistration

• Used within individual, so linear transforms
  should be sufficient
• Typically 12 parameters (translation, rotation,
  zooms, shear each in 3 dimensions)
• Though note that different MRI sequences create
  different non-linear distortions

T1 image
Coregistered FLAIR
16
Between-subject Normalization

• Allows inference about general population

Subject 1
Template
Subject 2
Average activation
Normalization
17
Why normalize?

• Stereotaxic coordinates analogous to longitude
• Universal description for anatomical location
• Allows other to replicate findings
• Allows between-subject analysis crucial for
  inference that effects generalize across
  humanity.

18
Normalization

• Normalization attempts to register scans from
  different people.
• We align each persons brain to a template.
• Template often created from multiple people (so
  it is fairly average in shape, size, etc).
• We typically use template that is in the same
  modality as the image we want to normalize
• Therefore, variance cost function.
• If different groups use similar templates, they
  can talk in common coordinates.

Popular MNI Templatebased on T1-weighted scans
from 152 individuals.
19
Coordinates - normalization

• Different peoples brains look different
• Normalizing adjusts overall size and orientation

Normalized Images
Raw Images
20
Coordinates - Earth

• For earth (2D surface) we use latitude and
  longitude
• Origin for latitude is equator
• Explicit defined by axis of rotation
• Origin for longitude is Greenwich.
• Arbitrary could be Paris
• What is crucial is that we we agree on the same
  origin.

21
Coordinates - stereotaxic

• For the brain, left-right side is obvious.
• Interhemispheric Fissure analogous to equator
• How about Anterior-Posterior and
  Superior-Inferior?
• We need an origin for these coordinates.

22
Coordinates - Talairach

• Anterior Commissure (AC) is the origin for
  neuroscience.
• We measure distance from AC
• 57x-67x0 means right posterior middle.
• Three values left-right, posterior-anterior,
  ventral-dorsal

23
Coordinates - Talairach

• The AC is not enough
• We need second origin to define horizontal plane.

?
?
?
24
Coordinates - Talairach

• Axis for axial plane is defined by anterior
  commissure (AC) and posterior commissure (PC).
• Both are small regions that are clear to see on
  most scans.

? PC
? AC
25
Coordinates - Talairach
position relative to anterior commissure -Xleft,
Xright -Yposterior, Yanterior Zsuperior,
-Zinferior 57x-67x0 right posterior region
26
Templates

• Original Talairach-Tournoux atlas based on
  histological slices from one 69-year old woman.
• Single brain may not be representative
• No MRI scans from this woman
• Modern templates were at some stage aligned to
  images from the Montreal Neurological Institute.
• MNI space slightly different from TT atlas
  (larger in every dimension).

27

• SPM uses modality specific template
• MNI T1 template, plus custom templates
• FSL uses MNI T1 template for all modalities
• Requires intra-modal cost functions

T1 T2
PET
28
Affine Transforms (aka linear, geometric)

• Zoom

• Shear

• Translation

• Rotation

29
Affine Transforms

• Co-linear points remain co-linear after any
  affine transform.
• Transform influences entire image.

30
Spatial Processing

• Non-linear transforms can match features that
  could not aligned with affine transforms.
• SPM uses basis functions.

31
Nonlinear functions and normalization
Scans from 6 people

• Linear Only

Linear Nonlinear
http//imaging.mrc-cbu.cam.ac.uk/imaging/SpmMiniCo
urse
32
Nonlinear basis functions

• Here are the functions SPM uses.
• They can be combined to create subtle deformations

33
Spatial Processing

• Affine Transforms are robust they influence the
  entire brain
• Note that non-linear functions can have local
  effects.
• This can improve normalization
• This can also lead to image distortion.
• E.G. In stroke patients, the injured region may
  not match the intensity of the template

34
Non-linear transforms
Stroke image
Stroke variance image
Template image
35
Non-linear transforms

• If you work with pathological brains, only use
  non-linear transforms appropriately

Linearonly
LinearNonlinear
MaskLinearNonlinear
Template
36
Sulcal matching

• Normalization conducted on smoothed images.
• We are not trying to precisely match sulci (would
  cause local distortion).
• Sulcal matching only approximate
• www.loni.ucla.edu/thompson/

Post-normalization alignment of calcarine sulcus,
precentral gyrus, superior temporal gyrus.
37
Alternatives

• SPM/FSL normalization will roughly match
  orientation and shape of head.
• Good if function is localized to proportional
  part of brain
• Poor if function is localized to specific sulci
  (e.g. early visual area V1 tied to calcarine
  fissure).
• Alternatively, use sulci as cost function (Goebel
  et al., 2006).
• Image below mean sulcal position for 12 people
  after standard normalization (left) followed by
  sucal registration (middle).
• Note This technique improves sulcal alignment,
  but distorts cortical size.

38
Alternatives

• SPM and FSL normalize overall brain shape.
• Individual sulci largely ignored.
• What are different normalization strategies?
• Sulci are crucial for some tasks (Herschls gyrus
  and hearing)
• Perhaps less so for others (e.g. Amunts et. al
  2004 with Brocas variability)

39
Interpolation

• Top and bottom images each rotated 12º.
• Top image looks jagged, bottom looks smooth.
• Difference is in the interpolation used in
  reslicing.

40
Interpolation
cfmi.georgetown.edu/classes/BootCamp/

• Reslicing data after spatial registration will
  require interpolation.
• Rotations, zooms, etc mean that there is not a
  perfect source voxel for each output voxel.

41
Interpolation
?

• How do we estimate values that occur between
  discrete samples?
• Four popular methods
• Nearest neighbor
• Linear
• Spline
• Sinc

?
?
42
Linear Interpolation

• For neuroimaging we usually use linear
  interpolation.
• Much more accurate than nearest neighbor.
• There is some loss of high frequencies spline
  or sinc interpolation are better but much slower
  to compute.
• Since we spatially smooth data after spatial
  registration, we will lose high frequencies
  eventually.

1D Linear Interpolation Weighted mean of 2 samples
2D Bilinear Interpolation Weighted mean of 4
samples
3D Trilinear Interpolation Weighted mean of 8
samples
43
Linear Interpolation High Frequency Loss
Original

• Linear interpolation tends does not preserve high
  frequencies
• Multiple successive resampling will lead to
  blurry image
• Solution Minimize number of times the data is
  resliced.

cfmi.georgetown.edu/classes/BootCamp/
44
Advanced Interpolation

• Spline and Sinc interpolation can retain high
  frequency information.
• Especially useful if multiple transformations
  will be applied.
• Computationally much slower to apply.
• Not necessary if you will heavily blur your data
  with a broad smoothing kernel.

Sinc Function
cfmi.georgetown.edu/classes/BootCamp/
45
Smoothing

• The need for spatial registration (motion
  correction, registration, normalization) is
  obvious.
• However, intentionally blurring images seems
  unintuitive we are throwing away information.

46
Smoothing

• Smoothing has several benefits
• Each voxel is a noisy measure. A blurred image
  minimizes noise and amplifies coherent signal.
• Voxels are arbitrary neighbors should show
  similar signal.
• The statistics lectures will describe additional
  benefits
• Reduces the number of independent statistics
• Makes our data more normal fits the
  assumptions of our statistics.

47
A smoothing kernel

• A kernel describes how neighboring samples
  influence a sample.
• We will start describing a one-dimensional filter.

0.25,0.5,0.25

• Output values are equal to 50 of the source
  value plus 25 of each neighbor.
• This acts to smooth the data.
• Kernels tend to sum to 1
• Values gt1 will amplify the signal
• Values lt 1 will attenuate the signal

48
A smoothing kernel

• A wider kernel is influenced by more neighbors
  more blur, less noise.

0.1,0.2,0.4,0.2,0.1

• Output values are equal to 40 of the source
  value plus 20 of immediate neighbors and 10
  their neighbors.
• This acts to heavily smooth the data.

49
An edge-detection kernel

• We can do interesting image processing with
  kernels.

-1,0,1

• Output values are equal the difference between
  the two neighbors.
• The symbols mean the output is the absolute
  value (always sign)
• This detect edges.

50
2D Kernels

• We can apply kernels to 2D and 3D images.

.1
.1
.1
.1
.2
.1

.1
.1
.1

• Output values are equal to 20 of the source
  value plus 10 of each neighbor.
• This acts to smooth the data.

51
2D Edge Detection Kernels

• We can also do edge detection in 2D or 3D using
  kernels.

52
Spatial Smoothing

• Each voxel is noisy. However, neighbors tend to
  show similar effect. Smoothing results in a more
  stable signal.
• Smooth also helps statistics smoothed data tends
  to be more normal fits our assumptions. Also,
  allows RFT thresholding (see Statistics lecture).

Gaussian Smoothing

53
FWHM

• Smoothing referred to a convolution the output
  intensity based on neighbors.
• The relative weighting of the neighbors is
  referred to as the kernel.
• The most popular kernel is the gaussian function
  (a normal distribution).
• The full width half maximum adjusts the amount
  of gaussian smoothing.
• FWHM is a measure of dispersion (like standard
  deviation or variance)
• Large FWHMs lead to more blurry images.
• For fMRI, we typically use a FWHM that is x2..x3
  our original resolution (e.g. 8mm for 3x3x3mm
  data).
• However, the FWHM tunes the size of region we
  will be best able to detect.
• E.G. If you want to look for a brain region that
  is around 10mm diameter, use a 10mm FWHM.

Dispersion Differs
54
Smoothing

• Spatial smoothing useful for between-subject
  analyses.
• Spatial normalization is only approximate
  smoothing minimizes individual sulcal
  variability.
• Smoothing controls for variation in functional
  localization between people.

None 4mm 8mm 12mm
55
Smoothing Limits Inference

• Extra activation observed comparing strong with
  light taps.
• After smoothing we can not distinguish between
• Increased activation of the same population of
  neurons
• Recruitment of more neighboring neurons.
• Example note that after smoothing broad low
  contrast looks line looks like focused high
  contrast line.

56
Smoothing Alternatives

• Gaussian smoothing is great for normal data
  assumes few outliers.
• Outliers will contaminate neighbors.
• If your data has dropouts or high-frequency
  artifacts, consider alternative filters.
• Median filters
• FSLs SUSAN

Gaussian Smoothing
Median Filter
57
Spatial unwarping

• The head distorts the magentic field.
• Shimming attempts to make field level
  homogeneous.
• Even after shimming, there will be varying field
  strengths.
• Specifically, regions with large density changes
  (sinus/bone of frontal lobe).
• This inhomogeneity leads to intensity and spatial
  distortion.

www.bruker-biospin.de/MRI/applications/medspec_hca
lc.html
58
Spatial unwarping

• We can measure field in homogeneity.
• This can be used to unwarp images (FSLs B0
  unwarping, SPMs FieldMap).

• Structural

• Unwarped EPI

• Raw EPI

59
Bias correction

• Inhomogeneity also leads to variability in image
  intensity.
• Bias correct anatomical scans (e.g. SPMs
  segmentation, N3).
• Field homogeneity issues more severe with higher
  field strength.
• Parallel Imaging (collecting MRI with multiple
  coils) can dramatically reduce effects.

60
Euler angles

• Most people like to describe spatial transforms
  as Euler angles (yaw, pitch, roll).
• In neuroimaging, we use mathematical matrices to
  describe linear spatial transforms.
• Very compact just 12 numbers can describe the
  result of an infinite number of linear transforms
  without losing precision.
• Avoids gimbal lock problem of Euler angles.

61
Euler transforms

• We like to talk about yaw, pitch and roll.
• However, order of these transforms is crucial.
• Consider a plane that pitches 90º and then rolls
  90º
• Different direction to a 90º roll followed by 90º
  pitch

62
Euler angles

• Euler angles order is important
• Mathematically, we can not explain all rotations
  with only three numbers for yaw, pitch and roll
  (we also need to specify the order).
• While directions make sense from pilots point of
  view, they can be confusing from the ground (e.g.
  the first rotation of 90º shifts the frame of
  reference for following rotations).

E.G. Computer warnings of Apollo 11 lunar lander
used moons surface as frame of reference. Only 3
gimbals were used threatening gimbal lock
problems.
63
Mathematical Matrices

• SPM, FSL, etc use matrix mathematics to compute
  linear spatial transforms.
• We will start with a matrix for 2D space. This
  has six numbers that concern us (3 columns, 2
  rows).
• To transformed horizontal position for x
• fx (xi)(yj)k
• To transformed vertical position y
• fy (xl)(ym)n

i
j
k
l
m
n
64
The identity matrix

• The identity matrix has 1 on the diagonal, and
  zeros in all other positions.
• With this matrix, the input and output are
  identical

1
0
0

0
1
0
65
Translations

• Translations are performed by adding a value to
  the last column.
• Top-left value moves left-right
• Bottom-left value moves up-down

Horizontal Position
1
0
0

0
1
20
Vertical Position
66
Shear

• You can skew an image by adding/subtracting a
  value at the position multiplied by the
  orthogonal direction.

Horizontal Shear
1
0
0

-0.2
1
0
Vertical Shear
67
Scaling

• Zooms are performed by multiplying all the values
  in a row by your scale factor.
• Top row shrinks/stretches horizontally
• Bottom row shrinks/stretches vertically
• Use a negative value to mirror-flip in the axis.

1.3
0
0

0
1
0
68
Rotations

• To rotate a 2D matrix by an angle ?
• cos(?) sin(?)
• -sin(?) cos(?)
• Note translation values (right column) set the
  center for pivoting.

.93
-.37
0

.37
.93
0
69
Combining Matrix Tranforms

• Our six numbers can store all of the possible
  rotations, shears, translations and scaling.
• Simply multiply previous matrix with our
  transform (order crucial).
• E.G. Zoom an image we have rotated and
  translated

.93
-.37
5
1.2
-.48
7

-5
.37
.93
-5
.37
.93
70
2D Matrices are 3x3

• In actual fact, our 2D matrices have 9 values
• To multiply matrices together, the first matrix
  must have exactly as many rows as the second
  matrix has columns.
• So it is useful to use matrices with equal
  numbers of rows and columns.
• However, last row is always 0 0 1
• Therefore, the identity matrix really looks like
  this

1
0
0
0
1
0
0
0
1
71
3D Matrices are 4x4

• We can generate 4x4 matrices that will allow us
  to work with 3D images.
• A 4x4 matrix, but the last row always 0 0 0 1
• fx (xi)(yj)(zk)l
• fy (xm)(yn)(zo)p
• fz (xq)(yr)(zs)t

72
Matrices and 3D space

• 3D matrices work just like 2D matrices
• The identity matrix still has 1s along the
  diagonal
• Translations are the values in the final column
  (constants)
• Zooms are done by scaling all values of a row.
• Shears are values added to the relevant
  orthogonal value
• Rotations use sine/cosine in dimensions of plane.
• Our twelve numbers can store all of the possible
  rotations, shears, translations and scaling.
• Simply multiply previous matrix with our
  transform (order crucial).