CSci 6971: Image Registration Lecture 1: Introduction January 13, 2004

1
CSci 6971 Image Registration Lecture 1
IntroductionJanuary 13, 2004
Prof. Chuck Stewart, RPI Dr. Luis Ibanez, Kitware
2
Outline

• Syllabus
• Registration problem
• Applications of registration
• Components of a solution
• Thematic questions underlying registration
• Software toolkits

3
Syllabus - Topic

• Image registration
• Determining the mapping between two images of the
  same object, similar objects, the same region or
  similar regions
• All aspects of the problem will be covered
• Underlying mathematics
• Images
• Algorithms
• Implementations
• Applications
• Special emphasizis on software toolkits

4
Syllabus - Instructors
Prof. Chuck Stewart 107 Amos Eaton Rensselaer
Polytechnic Institute 518-276-6731 stewart_at_cs.rpi.
edu
Dr. Luis Ibanez Kitware Corporation
(518)-371-3971 x 112 luis.ibanez_at_kitware.com
5
Syllabus - Distributed Course

• NSF Center for Subsurface Sensing and Imaging
  Systems (CenSSIS) Course
• Four universities
• Boston University,
• Northeastern University,
• Rensselaer Polytechnic Institute,
• University of Puerto Rico Mayaguez
• Lectures live at Rensselear
• Lectures recorded in voice-annotated powerpoint
  for remote students
• Lectures missed due to weather or travel will
  also be available via voice-annotated powerpoint

6
Syllabus - Office Hours

• At Rensselaer
• Tuesday and Friday immediately following lecture
• Distributed office hours
• Web-meeting
• Anyone can participate

7
Syllabus - Prerequisites

• Data structures
• Calculus
• Linear algebra
• Vectors and matrices
• Experience working with images
• C programming experience
• Templates!

8
Syllabus - Requirements

• Weekly homework assignments and programming
  projects
• Extended programming project (due Tuesday, April
  6)
• 10-page research paper (due Tuesday, May 4)
• Each is worth 30 of the semester grade
• Late assignments will not be accepted without
  prior arrangement or a verified personal
  emergency
• Last 10 is for acting as a scribe during
  lecture. This is a pass/fail requirement that
  off-campus students automatically pass.

9
Syllabus - Course Materials

• Voice-annotated powerpoint lectures will be
  placed on CenSSIS website
• Software toolkits will include tutorials
• Reading materials will also be placed on the
  CenSSIS website

10
Syllabus - Topics

• Introduction
• Mathematical background
• First examples
• Intensity-based registration and ITK
• Feature-based registration and the CenSSIS/RPI
  toolkit
• Initialization techniques
• Multiresolution techniques
• Mutual information
• Deformable registration
• Video registration and image mosaics
• Trends and research questions

11
Syllabus - Academic Integrity

• Students may discuss homework and programming
  assignments
• Solutions must be written in students own words
• Extended programming project and research paper
  must be individual work with appropriate
  citations
• A serious incident will result in failing the
  course

12
Registration Problem Definition
13
Example Mapping Function
q (912,632)
p (825,856)
Pixel scaling and translation
14
Registration Problem Definition
q (912,632)
q T(pq)

• Problems
• Form of mapping function T
• Unknown mapping parameters q
• Unknown correspondences, p,q

Chicken-and-egg problem
15
Applications Multimodal Integration

• Two or more different sensors view same region or
  volume
• Different viewpoints
• (Some specialized sensors have two or more
  coincident modalities, so registration is not
  needed.)
• Different information is prominent in each image
• The images may even have different dimensions!
• Range images vs. intensity images
• CT volumes vs. fluoro images

16
Example MR-CT Brain Registration
MR
CT

• MR (magnetic resonance) measures water content
• CT measures x-ray absorption
• Bone is brightest in CT and darkest in MR
• Both images are 3d volumes

Source http//www-ipg.umds.ac.uk/d.hill/hhh/10/10
.pdf
17
MR-CT Registration Results
Superimposed images, with bone structures from CT
in green
Aligned images
18
Retinal Angiogram and Color Image
19
Applications Image Mosaics

• Many, partially overlapping images
• No one gives a complete view
• Goal stitch images together
• Requires
• Limited camera viewpoint such as rotation about
  optical center
• Simple surface geometry such as plane or
  quadratic

20
Retinal Image Mosaics
21
Sea-Floor Mosaics
Courtesy Woods Hole Oceanographic Institution
22
Spherical Mosaics
Images from Sarnoff Corporation
23
Applications Building 3d Models

• Range scanners store an (x,y,z) measurement at
  each pixel location
• Each range image gives a partial view
• Must register range images and texture map them
• Applications
• Reverse engineering
• Digital architecture and archaeology

24
Examples
http//www1.cs.columbia.edu/allen/NEW/workshop.ht
ml
25
Applications Change Detection

• Images taken at different times
• Following registration, the differences between
  the images may be indicative of change
• Deciding if the change is really there may be
  quite difficult

26
Retinal Change Example
27
Regions Showing Change
28
Applications Video Super-Imposed on 3d Model
Taken from Sarnoff Corporation research
29
Other Applications

• Multi-subject registration to develop organ
  variation atlases.
• Used as the basis for detecting abnormal
  variations
• Object recognition - alignment of object model
  instance and image of unknown object
• Industrial inspection
• Compare CAD model to instance of part to
  determine errors in manufacturing process

30
Steps Toward a Solution

• Analyze the images
• Determine the appropriate image primitives
• Determine the transformation model
• Geometric and intensity
• Design an initialization technique
• Develop constraints and an error metric on the
  transformation estimate
• Design a minimization algorithm
• Develop a convergence criteria

31
Software Toolkits

• ITK
• Medical image processing, segmentation, and
  registration toolkit
• C, heavily templated, data flow architecture
• Registration stresses intensity-based approaches
• VXL
• Computer vision applications
• C, moderate templating
• Registration stresses feature-based approaches
• CenSSIS tool suite is a hybrid of these
• They can be used together
• Programming styles are different

32
Summary Pervasive Questions

• Three questions to consider in approaching any
  registration problem
• What intensity information or image structures
  is/are consistent between the images to be
  registered?
• What is the geometric relationship between the
  image coordinate systems?
• What prior information can be used to constrain
  the domain of possible transformations?

33
Looking Ahead Lecture 2 - Friday, January 16

• Mathematical background, part 1
• Vectors and matrices
• No meeting on Rensselaer campus
• Voice-annotated lecture will be posted Thursday
  night, January 15

34
Homework Problem

• Due Tuesday, January 20 at 12 noon (via email to
  Professor Stewart)
• Problem
• Find an application of registration, preferably
  in a research area of interest to you. In a
  short write-up (less than a full page), describe
  the problem and attempt to sketch answers to the
  three Pervasive Questions posed.