Arbitrary and Dynamic Patterning in a Programmable Array Microscope

1
Arbitrary and Dynamic Patterning in a
Programmable Array Microscope

• (Controlled Light Exposure Programmable Array
  Microscopy)
• Wouter Caarls, Anthony H.B. de Vries, Donna J.
  Arndt-Jovin, Thomas M. Jovin
• Laboratory of Cellular Dynamics
• Max Planck Institute for Biophysical Chemistry
• Goettingen, Germany

2
What is the PAM?

• Optically sectioning microscope using conjugate
  structured illumination and detection through a
  spatial light modulator
• Reduction of offset by capturing the light
  rejected by the pinholes (non-conjugate image)
• Arbitrary pinhole patterns
• Optimized for different samples and objectives
• Could be chosen automatically
• Patterns can be augmented with masks
• Selective photoactivation and photobleaching
• Adaptive imaging

3
Dual-path PAM
4
Confocal image generation
C - 0.5NC
(after registration)
Conjugate (duty 1/3)
Non-conjugate (duty 2/3)
Confocal
5
Photobleaching

• Bleaching occurs above and below the point being
  illuminated
• Also when that point is not part of the sample
• And even when the point lies to the side of the
  sample, due to high NA objectives

NA 1.45
In regular images, much of the experienced
photobleaching is unnecessary!
6
Controlled Light Exposure Microscopy (CLEM)

• Main idea switch off unneeded illumination to
  avoid bleaching
• Dont illuminate background
• Dont illuminate bright foreground
• Available on (Nikon) laser scanning systems

Marsupial/Rho123 Scalebar 10um
Hoebe et al., Nature Biotechnology, 2007
7
CLE-PAM

• Discretized integration time decisions
• Per-image decision instead of per-pixel
• Allows filtering to avoid noise artifacts

Illumination time Black/orange/red/white Stop
after 1/2/3/4 frames
8
Border effects

• Decision to illuminate a pixel affects
  neighboring pixels, due to illumination PSF
• Leads to border effects near decision boundaries

Desired illumination
Actual illumination
Naïve reconstruction
9
Solving border effects

• Dilate illumination, i.e. illuminate more than
  necessary
• Only use that part of the image which is far
  enough away from the dilated border of
  illumination

Desired illumination
Dilated illumination
Useful illumination
10
Background effects

• CLE-PAM lowers the signal-to-noise ratio in the
  background
• At very low decision times, this leads to very
  noisy backgrounds
• Especially problematic in maximum intensity
  projections

Single-slice CLEM image
Maximum intensity projection
11
Solving background effects

• Basic idea behind CLE-PAM is that background is
  not important
• Trade SNR for spatial resolution by Gaussian
  filtering
• Choose sigma such that resulting SNR is equal to
  expected SNR with full illumination.

Original MIP
Background-smoothed MIP
The foreground is not affected!
12
Time-domain extension

• Extend CLEM decision into time domain
• If a pixel is background now, it will probably be
  background in the next frame
• Additional threshold, below the low threshold
• Dilated decision boundary to account for movement
• Looks like sample is pushing the illumination

0 1 2
3 4
Time
13
Reduced photobleaching
13m
0m
4m
9m
HeLa cells, mitotracker 200nM 20min, normal
illumination (67ms)
HeLa cells, mitotracker 200nM 20min, CLE-PAM
illumination (max 67 ms)
14
Reduced photobleaching, cont.
Using CLE-PAM, the time until half the original
fluorescence is bleached is extended by a factor
of 2.2
I mean(img) mean(bkg) For
TD-CLEM, unilluminated pixels are set to
mean background
Normal t1/2
CLE-PAM t1/2
15
Conclusions

• The Programmable Array Microscope allows dynamic
  adjustment of illumination
• CLEM avoids unnecessary photobleaching by
  reducing the illumination outside the sample
• With the PAM, CLEM can easily be integrated into
  a full-field system
• Border effects are solved by dilating the
  illumination
• Background is smoothed for presentation
• Extension to the time domain
• Photobleaching is reduced by more than a factor
  of 2