ECSE6963 Biological Image Analysis

1
ECSE-6963Biological Image Analysis

• Lecture 8
• Nuclear Medicine, Image Pre-Processing
• Badri Roysam
• Rensselaer Polytechnic Institute, Troy, New York
  12180.

Center for Sub-Surface Imaging Sensing
2
Recap CT vs. MRI

• CT
• Cheap Fast
• Good resolution with bone
• Hard to distinguish soft tissues without contrast
  agent
• Cant distinguish atoms beyond their X-Ray
  cross-section
• X-Rays harmful to body

• MRI
• Expensive Slow
• Can distinguish bone and various soft tissues
• Can distinguish specific atoms
• No known health hazards to MR imaging
• Avoidable injury hazard if magnetic objects
  in/around body

3
Main advantages of MRI

• Structural Functional Imaging Possible
• Differentiation between various kinds of soft
  tissue.
• X-rays pass through soft tissue without much
  absorption
• High sensitivity to early pathological changes
  makes early detection possible.
• Studies of blood vessels and flow without use of
  contrast
• just oxygen level of blood gives contrast
• 3-D, allowing Multi-planar display
• i.e. axial, sagittal, coronal, and oblique.
• Multi-channel output
• Enables better segmentation
• No known biological hazards
• Magnetic fields dont ionize, unlike X-rays
• Exceptions people with pacemakers and/or
  implanted metallic objects cant be imaged safely

4
Nuclear Medicine

• Basic Idea
• Inject patient with radio-isotope labeled
  substance (tracer)
• Chemically the same, but physically different
• Detect the radioactive emissions (gamma rays)
• Super-short wavelength
• But, cant achieve the implied high resolution
• Detection technology limitations
• Not enough photons!
• Use filtered back-projection to reconstruct the
  3-D image
• Like fluorescence microscopy, except we dont
  need excitation

5
SPECT PET
PET image Showing a tumor

• Major Functional imaging tools
• SPECT Single-photon Emission Computed Tomography
• cheap and low-resolution
• Tells us where blood is flowing
• PET Positron Emission Tomography
• expensive and higher-resolution

6
SPECT Instrument

• The gamma camera is a 2-D array of detectors
• One or more gamma cameras are used to capture 2-D
  projections at multiple angles
• Use filtered back-projection to reconstruct 3-D
  image!
• Actual sinograms appear noisy due to the fact
  that we dont have enough photons
• Quantum-limited imaging

3-camera SPECT instrument
7
PET Idea
Gamma Photon 1
Nucleus (protonsneutrons)

• Basic Idea
• Nucleus emits a positron
• A short-lived particle
• Same mass as electron, but opposite charge
• Positron collides with a nearby electron and
  annihilates
• Two 511 keV gamma rays are produced
• They fly in opposite directions (to conserve
  momentum)

BANG
electrons
Gamma Photon 2
8
Emission Detection
Ring of detectors

• If detectors A B receive gamma rays at the
  approx. same time, we have a detection
• Hard sensor and electronics design problem,
  expensive

9
Image Reconstruction

• We can organize our set of detections as a set of
  angular views
• Use filtered back-projection algorithm!

10
PET Images

• Single-channel images
• Noisy, and blurry
• Not ideal for segmentation
• Segment MRI/CT for defining anatomy
• Register the images
• Measure activity

11
Better Algorithms

• Filtered back-projection algorithm
• produces a background artifact, discussed earlier
• Noisy reconstruction
• The Maximum Likelihood algorithm produces a
  better reconstruction for the same data

Filtered Back-Projection
Maximum Likelihood
12
Hybrid Imaging Instruments

• Structure imaging
• CT Magnetic Resonance Imaging
• Ultrasound Imaging
• Functional Imaging
• Nuclear Imaging
• Positron Emission Tomography
• Single-Photon Emission Computed Tomography
• Combined Modalities
• Functional structural imaging
• 1999 image of the year, U. of Pittsburgh

13
Image Analysis Steps
Image Acquisition
Image Reconstruction Pre-processing
Image Segmentation
Morphometry Higher-Level Analysis

• Segmentation is the hardest most critical step
  to image analysis
• Much easier to eliminate defects in images prior
  to segmentation

14
Imaging Defects

• Types of defects
• Grayscale distortions
• Noise
• Saturation and Cutoff
• Loss of contrast
• Blur
• Loss/movement of edges
• Non-uniformity of imaging
• Spatial non-uniformity
• Temporal non-uniformity
• Geometric distortions
• Inadequate Sampling

15
Avoiding Image Defects

• Make sure the instrument (e.g. microscope) is
  clean, aligned, and properly adjusted
• Calibrate the instrument
• Make every effort to eliminate or minimize
  disturbances using physical instrumentation
• Mount the microscope on an anti-vibration table
• Put an optical filter in the illumination path to
  suppress light in parts of the spectrum that
  dont contribute to a better image
• Cool parts that are affected by heat

16
Image Defect Removal

• Best to avoid them in the first place!
• Adjust the instrument for maximum performance
• Once the pixels are recorded, we lose a lot of
  flexibility
• Next best thing Use image processing methods to
  deal with them after the fact
• It is really impossible to remove defects.
• We can at best suppress them to a limited
  extent
• There is, invariably, a price to pay, e.g.,
  artifacts
• We need to know the cause of the defect to do the
  best possible job
• Being able to construct a mathematical model most
  helpful

17
Why Correct Image Defects?

• Better Visualization
• So as to not fool later processing programs
• Sometimes lead to better measurements

Image Acquisition
Image Reconstruction Pre-processing
Image Segmentation
Morphometry Higher-Level Analysis
18
Noise

• What is noise anyway?
• The pixel values in an image can be thought of as
  representing information about an object
• An informative pattern
• Noise is any variations from the informative
  pattern
• It is uninteresting
• It is a kind of nuisance or disturbance
• Noise is distinct from texture
• It is informative, though random
• Cant really remove it
• The pixel values include the disturbance, so
  cant isolate it

19
Noise

• Many sources, e.g.
• Quantum noise
• Not enough photons ( lt 50 per pixel)
• Thermal noise at CCD camera front end
• Digitizer noise
• Errors in converting signal to digital form with
  fixed bit length
• Noise due to computing errors
• Roundoff errors, especially when working with
  small numbers of bits per pixel
• Flickering illumination
• Vibrations in system (often, due to cooling fans)
• Errors in image compression/transmission
• Images often compressed for transmission
• Lossy compression algorithms introduce
  sub-visual noise-like artifacts

20
Common Image Observation Model

• S(x,y) f(I(x,y)) N(x,y)
• S(x,y) observed by sensor
• I(x,y) the true underlying image
• f(.) is a distortion function
• N(x,y) additive noise

21
Frame Averaging

• Often, N(x,y) is Gaussian distributed, and
  independent from one pixel to the next
• (mean µ, std dev ?)
• we can subtract the bias (µ)
• we can frame average N times to reduce std. dev
  by .
• When N(x, y) is non-Gaussian
• Frame averaging can still be done, but a median
  average may work better
• In general, we must use mathematical model for
  the noise and figure out if averaging can be
  performed, and then the kind of averaging

22
When we cannot average frames

• Moving Object(s)
• Motion-compensated frame averaging possible if
  the motion vector is known
• Restrict averaging to the non-moving regions
• We look for opportunities to perform averaging in
  space instead
• Assumptions
• Pixels are much smaller than the objects of
  interest
• Images are generally smooth

23
Other Complications

• N(x,y) may not be uniform across the image
• µ(x,y) and ?(x,y)
• Example The foreground and background regions
  may be affected differently by the noise process.
• N(x,y) may also be time varying
• µ(x,y,t) and ?(x,y,t)
• N(x,y) may not be Gaussian

24
Quantum Noise

• Common in confocal and fluorescence microscopy
  Nuclear Medicine
• Weak fluorophores, tight pinholes, small
  radiation dosage, detector limitations
• Observed image is a realization from a Poisson
  point process in space with intensity ?(x,y).
• Mean Variance ?(x,y)
• Rule of thumb
• If ?(x,y) ? 50, then a Gaussian noise model
  suffices
• Example, photographic film
• Frame averaging the method of choice

25
Multiplicative Noise

• S(x,y) g(I(x,y)) N(x,y)
• Also called modulation noise
• Generally, harder to deal with...
• One method Take logarithms and get an additive
  description
• Another Consider geometric means instead
• Generally, requires sophisticated statistical
  analysis
• Often, this N(x,y) not random
• Example A flickering illumination lamp

26
Quantum Noise in Fluorescence Microscopy
27
Quantization Noise

• Results when an analog quantity is converted to a
  discrete n-bit number
• If were off by 1 bit, error is

• If we assume uniformly distributed additive
  error,
• Mean 0
• Variance ?2/12

28
Neighborhood Averaging

• new pixel value average of its previous
  neighbors values
• How do we compute the average?
• mean/median/mode
• weights
• How do we pick the neighbors?
• How we make the above choices can make a big
  difference in terms of
• quality of results
• artifacts introduced

29
Choosing the Neighbors
4 nearest neighbors
8 nearest neighbors

• Neighborhood Size
• Larger neighborhood leads to more extensive
  averaging
• Neighborhood shape
• Should match object shapes

30
Mean Filtering
3 3 kernel
9 9 kernel
Original Image
Moral Too big a neighborhood leads to blurring
31
Treating Neighbors Differently

• Simplest Idea
• Give lesser importance to neighbors that are
  farther away
• E.g., Gaussian weights, discretized and scaled

0
0
1
1
1
1
1
0
0
0
1
2
3
3
3
2
1
0
1
4
1
1
2
3
6
7
6
3
2
1
1
3
6
9
11
9
6
3
1
4
12
4
1
3
7
11
12
11
7
3
1
1
3
6
9
11
9
6
3
1
1
4
1
1
2
3
6
7
6
3
2
1
3 3, ? ? 0.4
0
1
2
3
3
3
2
1
0
9 9, ? ? 1
0
0
1
1
1
1
1
0
0
32
Gaussian Smoothing
3 3 Gaussian, Variance ? 0.4
Original Image
Magnify for a closer look
33
Gaussian Smoothing
9 9, Uniform Weighting
9 9 Gaussian, Variance ? 1.0
Magnify for a closer look
34
Bad Neighbors

• Some forms of noise produce drastic and highly
  localized changes in pixel values
• Outliers
• Can have huge effect on the averages
• Recognizing and eliminating these outliers is one
  other way to treat neighbors differently

35
The Median Filter

• The median The middle value of a bunch of
  numbers
• Just sort the numbers and extract the middle
  number
• Robust to outliers
• Introduces no new values
• e.g., does not make image dimmer
• Will smooth (in the extreme, posterize) upon
  repeated application
• Does not shift boundaries
• There are many variations on the median filter
• E.g., center weighted median

36
Median Filtering
3 3 kernel
9 9 kernel
Magnify for a closer look
37
Center-weighted Median
Basic Idea Just repeat the values that we want
to emphasize
5 5 center weighted median
5 5 median
Magnify for a closer look
38
Max/Min Filters

• Basic Idea
• Noise has the effect of creating abrupt changes
  in the grayscale values of adjacent pixels
• The extreme values in a neighborhood can form the
  basis for useful operations
• Min Can help us ignore bright outliers - salt
• Max Can help us ignore dim outliers pepper
• Suppose our objects are bright against a dark
  background
• Min will erode our objects
• Max will fatten (dilate) our objects

39
Example
Min Filtering (radius 1 pixel)
Max Filtering (radius 1 pixel)
Original Image
40
Max/Min Filters

• Also called grayscale erosion and dilation
• We can use them together to smooth images
• First pass pixel value replaced by
  maxneighbors
• Second pass pixel value replaced by
  minneighbors
• What happens if we reverse the order (i.e., min
  followed by max)?
• The size and shape of the neighborhood (kernel)
  makes a big difference.

41
Grayscale Closing
3 3 rectangular kernel
9 9 rectangular kernel
Magnify for a closer look
42
Closing with circular kernel
9 9 circular
9 9 rectangular kernel
Magnify for a closer look
43
Opening
3 3 rectangular
9 9 rectangular
Magnify for a closer look
44
Opening with Circular Kernel
9 9 circular
9 9 rectangular
Magnify for a closer look
45
Summary
Image Acquisition
Image Reconstruction Pre-processing
Image Segmentation
Morphometry Higher-Level Analysis

• Survey of common imaging defects
• Find ways to avoid them in the first place by
  optimizing specimen preparation and image capture
• Noise
• An important type of defect
• Methods to deal with unavoidable image noise
• Introduction to adaptive smoothing
• Next Class
• Adaptive, image smoothing algorithms

46
References on MRI

• Main MRI Reference
• http//www.cis.rit.edu/htbooks/mri/inside.htm
• Other MRI References
• http//www.spincore.com/nmrinfo/mri_s.html
• http//dmoz.org/Science/Chemistry/Nuclear_Magnetic
  _Resonance/Theory_of_NMR_and_MRI/Basic_NMR_and_MRI
  _Theory/

47
References on SPECT PET

• PET
• http//www.crump.ucla.edu/lpp/lpphome.html
• SPECT Imaging
• http//www.physics.ubc.ca/mirg/intro.html
• SPECT Image Atlas
• http//brighamrad.harvard.edu/education/online/Bra
  inSPECT/BrSPECT.html

48
Reference for Pre-processing

• Chapter 4 of the textbook
• Most materials are available online from the
  authors web page
• http//css.engineering.uiowa.edu/dip/LECTURE/lect
  ure.html
• Frame averaging example
• http//micro.magnet.fsu.edu/primer/java/digitalima
  ging/processing/imageaveraging/

49
Summary

• Discussion of major medical instruments
• Structure imaging
• Function imaging
• Next Class
• Image Pre-processing methods

50
Instructor Contact Information

• Badri Roysam
• Professor of Electrical, Computer, Systems
  Engineering
• Office JEC 6046
• Rensselaer Polytechnic Institute
• 110, 8th Street, Troy, New York 12180
• Phone [phone]
• Fax [phone]/2433
• Email roysam_at_ecse.rpi.edu
• Website http//www.rpi.edu/roysab
• Course Material Website http//www.ecse.rpi.edu/c
  enssis/BioCourse
• Weekly Teleconference Wednesdays at 1200pm
  beginning on September 11. Dial 1-888-872-2038
  and use the guest passcode 0611.
• NetMeeting ID (for off-campus students)
  128.113.61.80
• Assistant Betty Lawson, JEC 6049, (518) 276
  8525, lawsob_at_rpi.edu