Rapid Diagnosis of Acute Heart Disease by Cloud-based High Performance Computing for Computer Vision

1
Rapid Diagnosis of Acute Heart Disease by
Cloud-based High Performance Computing for
Computer Vision

• Oleksii Morozov
• Physics in Medicine Research Group
• University Hospital of Basel
• Switzerland

April 8, 2010
2
The Heart

• Life-sustaining pump 2500000 L/year of vital
  blood
• Coronary artery disease (CAD) is most frequent
  cause of heart malfunction and death
• World largest killer (WHO)
• 29 of global death
• 17100000 lives/year

3
Cardiology yesterday

• Tools with relatively low information content

4
Cardiology today

• More tools, more information
• Subjective decision mostly relying on experience
  of a doctor

5
Cardiology tomorrow

• More advanced technologies
• Multidimensional information
• High quality, high resolution data
• Multimodal information
• Quantitative, objective, integrative computer
  based analysis
• Worldwide-networked standards and databases

• Problems
• Need for high performance computing in a
  distributed environment but only for a fraction
  of the time
• Global storage network for storing large datasets

6
Cardiac Ultrasound

• One of the modern tools for evaluation of the
  heart function
• HF sound waves No Radiation/Ionization
• Safe, Non-invasive, Fast, Portable, Cheap
• - Rather low signal to noise ratio

7
Cardiac UltrasoundDiagnostic value

• Heart wall assessment

8
Cardiac UltrasoundDiagnostic value

• Pumping function

9
Cardiac UltrasoundDiagnostic value

• Valve function

10
3D Cardiac Ultrasound

• Explore heart in 3D
• Freehand ultrasound (Manual sweeping)

11
3D Cardiac Ultrasound

• Explore heart in 3D
• Freehand ultrasound (Manual sweeping)
• Mechanical sweeping ultrasound (Motor driven)

12
3D Cardiac Ultrasound

• Explore heart in 3D
• Freehand ultrasound (Manual sweeping)
• Mechanical sweeping ultrasound (Motor driven)
• Live 3D ultrasound (2D arrays with electrical
  sweeping)

13
Cardiac Ultrasound

• Ultrasound machine a transducer a
  supercomputer

50000 500000 USD
Idle 90 of the time
14
Computational problems in Cardiac Ultrasound

• Signal reconstruction

Non-uniformly sampled measurements
Complete gridded or continuous data
representation
15
3Dtime signal reconstruction

• Inherent non-uniformity of scanning
• Spatial non-uniformity
• Serialism in scanning

16
3Dtime signal reconstruction

• Inherent non-uniformity of scanning
• Spatial non-uniformity
• Serialism in scanning
• Non-uniformity in synchronization (ECG)

17
3Dtime signal reconstruction

• Inherent non-uniformity of scanning
• Spatial non-uniformity
• Serialism in scanning
• Non-uniformity in synchronization (ECG)
• Body motion artifacts (breathing)

I(x,y,z,t) ?
4D non-uniform data
18
3Dtime signal reconstructionA spline solution

• B-spline non-uniform interpolation by
  Arigovindan, Unser (EPFL, Switzerland 2005)
• Robust global interpolation handles oversampling
  and undersampling (gaps) in the data
• Sparse and well-conditioned alternative to the
  optimal RBF solution
• Enjoys multiresolution properties (way to fast
  solving)
• Parallelizability of solving process
• Successfully applied to 2D problems

19
3Dtime signal reconstructionA spline solution

• Obstacles in 3D/4D
• Complexity is exponentially dependent on the data
  size 128 x 128 x 128 x 18 gt 78752009856
  non-zeros (312 Gbyte in single precision)
• Tensor based approach by Morozov, Hunziker, Unser
  2009
• Tensor decomposition of the problem
• Relaxed storage requirements
• Feasibility on standard workstations
• 9 millions of measurements with size 128 x
  128 x 128 x 18 -gt 30 minutes on my dual core
  laptop

20
3Dtime signal reconstructionA spline solution

• Tensor based approach applied to ultrasound data
  from continuously rotating transducer

21
Computational problems in Cardiac Ultrasound

• Tissue/blood motion estimation
• Doppler Ultrasound imaging (State of the art)
• Semi-quantitative measurements
• Full motion reconstruction
• Generalization of B-spline reconstruction to
  vector valued data (Arigovindan, Unser 2005)
• Employing additional constraints from physics of
  fluids (incompressibility, Navier-Stokes
  equations)

22
Computational problems in Cardiac Ultrasound

• B-spline based tissue motion reconstruction

• Continuous
• Fully quantifiable
• Can be combined with Doppler for better
  robustness

23
Computational problems in Cardiac Ultrasound

• Blood flow reconstruction
• Resolves ambiguity of Doppler measurements
• Continuous
• Fully quantifiable

24
Pathway to distributed supercomputing

• Multicore (IBM Power7) claimed 260 GFLOP/chip
• Cluster (UniBasel) 34500 GFLOP/400 cores
• GPGPU (ATI 4870X2) 2000 GFLOP/card
• GPGPU array
• FPGA accelerator cards dozens of GFLOP/chip, up
  to 512 chips per system, low power
• In exploration within ICES Microsoft project
• Cloud - Microsoft Azure

25
Cloud Ultrasound Processing Service

• Reasons
• Processing of large multidimensional multimodal
  medical data requires vast computational power
• Building/maintaining own HPC infrastructure is
  overly expensive
• Relatively rare use of HPC power (few times per
  day)
• Availability at multiple points of care (medical
  practices and hospital emergency rooms)
• Unified storage/access of the multimodal medical
  data

26
Cloud Ultrasound Processing Service
Record data
User
Cloud
4D acquisition with real-time on board
visualization
Visualization/Analysis parameters
Interactive web-based visualization of the result
Rendered images and quantitative information
27
Cloud Ultrasound Processing Service

• Record data
• Raw data
• 180 beams x 500 samples x 100 frames x 10 sec -gt
  85 Mb
• Additional information (geometry) -gt few Kb
• Lossless compressed DICOM
• Low latency response to the user by sending first
  a subpart of the data for coarser resolution
  reconstruction

28
Cloud Ultrasound Processing Service Signal
reconstruction

• Problem is very large for solving using direct
  solvers -gt use iterative solver

Ci1 Ci OP(Ci) OP linear operator
2 iterations
50 iterations
80 iterations
29
Cloud Ultrasound Processing Service Signal
reconstruction

• Iteration can be distributed relative to the grid

dx
dx, dy grid spacing
C1,1i1 C1,1i OP1,1(Ci) C1,2i1
C1,2i OP1,2(Ci) C2,1i1 C2,1i
OP2,1(Ci) C2,2i1 C2,2i OP2,2(Ci)
dy
Completely independent output
Ck,m solution subpart dedicated to a compute
unit
OPk,m() operator applied by a k,ms compute
unit
30
Cloud Ultrasound Processing Service Signal
reconstruction

• Data dependency

OPk,m() uses data outside the bounds of Ck,m

• Extents of dependent input data 3 samples for
  cubic spline
• At each iteration this data is transferred among
  adjacent units
• Performance limiting factor

Ck,m
31
Cloud Ultrasound Processing Service Signal
reconstruction

• Data dependency
• Data size 512 x 512 x 512 x 64
• Single precision 32 GB
• Infiniband QDR 12X ( 12GB/s )

Number of units Size of dependent data per unit, MB Total data transfers for single iteration, MB Maximal number of iterations/s (excluding CPU time)
64 72 4608 166
128 48 6144 250
256 30 7680 400
512 18 9216 666
1024 12 12288 1000
32
Cloud Ultrasound Processing Service Signal
reconstruction

• Computational load
• Data size 512 x 512 x 512 x 64
• Intel Quad Core 2.67 GHz (30 GFLOP/s in single
  precision)
• PC3-10600 DDR3-SDRAM (30 GB/s)
• 115000000 data samples
• Total requirements 5000 GFLOP, 3000 GB
  of memory transfers

Number of units Maximal number of iterations/s (including inter-unit communication)
64 0.24
128 0.48
256 0.96
512 1.92
1024 3.84
33
Cloud Ultrasound Processing Service Signal
reconstruction

• Multiresolution
• Coarse to scale propagation getting general
  from coarser scales and improving details on
  finer scales
• Inherent spline inter-scale relation

34
Cloud Ultrasound Processing Service Signal
reconstruction

• Multiresolution in solving algorithm
• Coarse to scale propagation getting general
  from coarser scales and improving details on
  finer scales
• Inherent spline inter-scale relation
• Multigrid solving algorithm

35
Cloud Ultrasound Processing Service Signal
reconstruction

• Multiresolution in solving algorithms
• Coarse to scale propagation getting general
  from coarser scales and improving details on
  finer scales
• Inherent spline inter-scale relation
• Multigrid solving algorithm
• Few iterations needed at each scale to get
  reasonably good solution
• With each coarser scale the cost of iteration
  decreases exponentially
• In total requires much less computational load
  than pure iteration

36
Cloud Ultrasound Processing Service Signal
reconstruction

• Total requirements per compute unit
• Data size 512 x 512 x 512 x 64
• 6 scales including finest scale
• 16 iterations at each scale
• Motion reconstruction algorithms require
  9 times more computational load

Number of units Memory, GB CPU time, s
64 2.7 62
128 1.36 31.2
256 0.68 15.6
512 0.34 7.8
1024 0.17 3.9
37
Cloud Ultrasound Processing Service

• Costs estimation

Data size 512x512x512x64
Storage, GB 32
Number of instances 1024
Compute hours 0.0011
hourinstance 1.13

• Switzerland (1/1000 of world population)
• 700 cardiologists/7000000 population
• 1000 echocardiograms per year per cardiologist
• Multiple views(3) per patient
• Multiple analyses (3) per view

7000000 use cases/year 3043 use
cases/hour Full loaded 3 x 1024
instances (Uniform load in Switzerland)
38
Collaborative Research enabled by the Cloud

• Globally available service for processing,
  storing and accessing medical data
• Standardized DICOM interface for unifying data
  access
• Involve all interested parties around the world
• World-wide large scale trials are possible
• Getting more statistics for rare cases
• Building reference datasets for known cases

39
Conclusion

• An approach to rapid diagnosis of heart disease
  using cloud based distributed computing
• Replace ultrasound machines supercomputer by a
  cloud service for remote processing and storage
• Miniaturization of the medical equipment and
  decrease of its costs
• Availability of advanced analysis technologies
  for objective analysis
• Availability at multiple points of care
• Unified storage and access of medical data
• Enables collaborative research