1
Live Tomographic Reconstructions

• Alun Ashton
• Mark Basham

2
Overview

• Incentive
• Experimental Changes
• Beamline Overview
• Reconstruction Hardware
• Conclusions

3
Incentive

• Higher resolution cameras are forcing larger
  collection times.
• I12 at DLS will use a 4000x2500 pixel detector
  (PCO 4000) capable of approximately 2 frames per
  second (Technically 4-5 frames). Assuming the
  best case scenario this will require around 30-40
  minutes to collect the data(4000 images i.e. the
  width of the detector).
• Even with good cluster performance, it could
  still take around 30 minutes to make the 80GB
  reconstructed volume (4000x4000x2500)
• Having to wait an hour from start to finish
  before being able to see what has been imaged is
  frustrating, and could waste beam time due to
  misalignments etc.

4
Trying something different

• Recent work has focused on fast and accurate
  reconstructions.
• Make good use of complete data sets to correct
  for various anomalies.
• This has to happen after the collection has been
  completed (on an 80Gb file)
• This work focuses on providing a reconstruction
  during the data collection.
• Less quality, and smaller reconstructions (e.g.
  1000x600) to allow visualisation.
• Gives the user the ability to see the volume
  after only a few minutes.

5
Traditional Tomography
Direction of the beam
Ingoing Path
Outgoing Path
6
Traditional Tomography
Each new acquisition collects the next segments
data
7
Traditional Tomography
8
Traditional Tomography
9
Traditional Tomography
10
Traditional Tomography
All the data has been collected and the final
full size and quality reconstruction can be
produced
11
Experimental changes

• To make the maximum use of this methodology there
  needs to be some experimental changes.
• The main change is in the way that the data is
  collected
• This requires certain hardware requirements
  mainly a sample stage capable of continuous
  rotation
• The acquisition is then preformed as follows

12
Continuous rotation
Ingoing Path
Outgoing Path
13
Continuous rotation
Each new acquisition skips 4 segments and then
collects the 5th
14
Continuous rotation
15
Continuous rotation
Initial lowest quality reconstruction can now be
calculated
16
Continuous rotation
Because its transmition you can flip the image
and then its going the right way. So becomes red
17
Continuous rotation
18
Continuous rotation
19
Continuous rotation
Refined reconstruction can now be calculated and
replaces the previous one.
20
Continuous rotation
This is where skipping 4 becomes important.
21
Continuous rotation
22
Continuous rotation
Refined reconstruction can now be calculated and
replaces the previous one.
23
Continuous rotation
24
Continuous rotation
25
Continuous rotation
26
Continuous rotation
Refined reconstruction can now be calculated and
replaces the previous one.
27
Continuous rotation
28
Continuous rotation
29
Continuous rotation
All the data has been collected and the final
full size and quality reconstruction can be
produced
30
Progressive Collection

• Advantages
• A full reconstruction can be preformed after only
  a fifth of the acquisition time, albeit at
  reduced resolution.
• As there is a gap between acquisitions, the full
  sector can be integrated, then the gap can be
  used for camera readout.
• Disadvantages
• Requires custom software to reconstruct, or
  convert to classical data.
• If the gap is too large, acquisition time can be
  increased.

31
Integration Time
Time
Traditional
Sector 1
Sector 2
Acquisition
Detector to memory
New
S1
S2
Acquisition
Detector to memory
32
Architecture
Digital camera
Beamline Control PC
Camera Control and Live Reconstruction PC
Central Storage
Cluster Computing Resources
33
Architecture
Start-up
34
Architecture
Start-up
35
Architecture
Collect Images
36
Architecture
End of collection
37
Architecture
Produce full reconstruction
38
Differences to normal setups
39
Computing Hardware for the live reconstruction

• Standard PC server
• Passes the data on to the central storage
• Scales and applies the flat field correction to
  the images as they come in.
• Runs the Host program for the Tesla
• Tesla Graphics Processor Unit
• Takes the scaled and corrected images
• Filters the images.
• Provides the Back Projection.

40
Why the TESLA

• Tomography is inherently very parallelisable
• Tesla requires around 100,000 concurrent threads
  to make it effective
• This then allow for in general a single GPU to
  run 40 times faster on these problems than a
  single CPU or 10 times faster then a QuadCore.
• Space and power are saved in this case as a 1U
  Tesla Unit can effectively replace 20 dual
  processor quad core machines, for tomographic
  reconstruction.
• This also allows our beamline machine to pack the
  punch required to make the live reconstructions
  possible.

41
Conclusions

• This methodology for collecting tomographic data
  should give the users much more insight into
  there samples and more time to make decisions
  about collections.
• The Tesla GPU is a good way of increasing the
  speed of Tomographic reconstruction, and has been
  proven in various different labs around the
  world.
• We can modify the way in which the experiment is
  preformed to make the most use or influence the
  choice of hardware, such as the modifications
  made to allow for camera readout and continuous
  rotation stage.

42
Acknowledgements

• Manchester University
• Valeriy Titarenko, Albrecht Kyrieleis, Phil
  Withers, Mark Ibson.
• Diamond Light Source
• Michael Drakopoulos, Thomas Connolley
• Architecture
• Piercarlo Grandi, Nick Rees, Bill Pullford
• Mark Basham, mark.basham_at_diamond.ac.uk