Processing and Reconstruction of Cryogenic Electron Microscope Tomography Images

1
Processing and Reconstruction of Cryogenic
Electron Microscope Tomography Images

• Automatic tracking of fiducial markers across
  very low SNR images

Fernando Amat Farshid Moussavi Mark Horowitz LBL
meeting-September 2006
2
Cryogenic Electron Microscope Tomography

• Take 100 electron microscope images at different
  tilt angles and with finite dose budget (low SNR)
• Align, reproject 2D images, do 3D reconstruction.
• Quality of 3D reconstruction directly related to
  quality of 2D preprocessing.

3
Brief problem statement
Caulobacter Images
?
Very low SNR, faint features. Use fiducial
markers. Automatically find accurate
correspondences in images for alignment.
4
Outline

• A Probabilistic Framework solution
• Results
• Future work
• Discussion/Conclusions

5
Steps of Preparation for Reconstruction

• Incomplete and unreliable data at first
• Incorrect decisions cause more incorrect
  decisions downstream (errors propagate)

Robust probabilistic framework
6
Probabilistic Framework

• Maximum Likelihood Estimation. We want to find
  assignment to variables X ,T,y that maximizes
• P(X ,T,y O)
• X , the set of 3D marker locations R3xM,
• y, the set of trajectories across images R2xMxN
• T, set of microscope parameters
• O, the set of observed peaks in the 2D images
  R2xMxN
• (Mnumber of contours, Nnumber of images)

7
Probabilistic Framework (contd)
P(X ,T,yO)P(X ,Ty,O) P(yO)
Projection model Correspondence

• But we have the observed peaks, not the
  trajectories.
• Correspondence is a discrete problem.
• Projection model estimation is a continuous
  optimization problem.
• We need to split the problem

8
Probabilistic framework block diagram
Finds O (peaks) Finds argmax P(yO)
Finds argmax P(X ,Ty,O) y X
,T
9
Feature Extraction Location

• Template matching using normalized cross
  correlation or mutual information
• We focus on gold beads, but still get false
  matches gt need to tolerate errors
• Fiducial template is the ONLY input to the
  algorithm

Red dots are marker candidates
10
Correspondence

• What is probability p(M-gtK) that i-th peak in
  image 1 corresponds to j-th peak in image 2?
• Bipartite graph matching problem- O(N!)
• Scores for individual matches may not be
  informative enough (look at groups of matches)

How to make decisions with all this uncertainty?
Markov Random Fields
11
Correspondence using Loopy Belief

• Loopy belief approximates a joint distribution
  over n variables

ABCD

• As a collection of pairwise and singleton
  distributions

• O(kM)-gtO(M2)gtCan now consider assignments to M
  markers jointly

12
Correspondence using Markov Random Fields

• Discrete problem in nature
• We estimate joint pairwise correspondence for all
  peaks in image 1 and 2 at the same time
• Use simple geometric constraints
• No use of projective model-gtrobust to distortions
• Invariant to translations-gtno need of prealign
  images
• Complexity is exponential in number of peaks
• Use approximate techniques which treat a joint
  distribution over M variables as a collection of
  pairwise distributions (complexity becomes O(M2))

13
New definition of pairwise potentials based on
relative distance

• Any vector 1-gt2 inside the blue area gets a
  potential higher than 0
• This definition respects angle orientation and
  direction. It is not symmetric
• It assign high potentials only to very similar
  configurations
• Small drawback it still needs a loose distance
  constraint to avoid finding
• pairs of markers with similar orientation in the
  other side of the picture
• Parameter b should be 2 by default (but we can
  selected in the config file)

14
Results in real dataset Caulo19
Only clear winners (high probability
correspondences) are selected
15
Results in real dataset Caulo19
Only clear winners (high probability
correspondences) are selected
16
Projection model

• Find the solution to
• D(x,yT-RtX Y Z 1T) (1)
• x,y known points from correspondence
• Rt projective model (partially unknown)
• X Y Z 3D markers position (unknown)
• D() cost function
• (1) is the ML solution to P(X ,Ty,O) assuming
  certain error model distribution for reprojection
  errors (related to D())

17
Projection model (contd)

• We use D()Huber penalty
• Huber is a mixture between L2 norm and L1 norm
• Fits correct correspondence as least squares (L2)
  but allows outliers (L1)
• Makes easier to detect bad correspondences from
  LBP

18
Results of robust model estimation
19
Results of robust model estimation
20
Results of robust model estimation
21
Results tracking contours
22
Results tracking contours
23
Results tracking contours
24
Results tracking contours
25
Results tracking contours
26
Results tracking contours statistics
27
Results Caulo 19 tomogram
Manual reconstruction by Luis R. Comolli
28
Results Caulo 19 tomogram
Fully automatic reconstruction
29
Results CyKR-He1 tomogram
Manual reconstruction by Luis R. Comolli
30
Results CyKR-He1 tomogram
Fully automatic reconstruction
31
Results

• Other tomogram reconstructions for different
  specimens are available.
• They are not shown here to keep the talk short.

32
Future work

• Occlusion solve problems in high tilt angles for
  group of markers
• Speed up the process
• Extend Markov Random Fields correspondence to
  multiple images
• Iterate correspondence and 3D model estimation
  using Expectation-Maximization if results are not
  satisfactory in one single pass

33
Discussions/Conclusions

• Fully automated process to align images with
  fiducial markers only a template of a marker is
  needed as an input
• Accuracy results comparable to manual alignment
  in very low SNR images
• Robust to distortions and error propagation