Update on the CAR Evaluation of Irreversible Compression for Medical Images

1
Update on the CAR Evaluation of Irreversible
Compression for Medical Images

• David A. Koff MD
• Peter Bak PhD
• Paul Brownrigg MBA
• Luigi Lepanto MD
• Tracy Michalak MSc
• Harry Shulman MD
• Andrew Volkening MSc

2
Background

• Storage volume
• Even if the cost of storage is dropping, the
  volume of data to store keeps increasing, mainly
  CT Scan and MRI.
• 35 million diagnostic imaging exams are performed
  annually in Canada.
• With a legal retention period of 7 years
• Use of irreversible compression could save M100
  million per year.

3
Background

• and transmission times
• and if access to high bandwidth gets more
  available in local hospital networks, it is still
  premature to expect any health professional to
  use 100 mbps connections on their computers.
• EHR networks cannot support large medical images
  and timely access to diagnostic images requires
  adequate level of compression.

4
The CAR PACS committee position

• The Canadian Association of Radiologists
  PACS/Teleradiology committee has accepted the
  principle of irreversible (lossy) compression
  for use in primary diagnosis and clinical review,
  using DICOM JPEG or JPEG-2000 compression
  algorithms, at specific compression ratios set by
  image type.

5
The CAR PACS committee position

• The most commonly used techniques for lossy
  compression and decompression are JPEG and more
  recently DICOM JPEG 2000.
• The JPEG 2000 algorithm allows higher levels of
  compression than JPEG at the same image quality.
• However, at lower levels of compression (e.g.
  301 in chest radiography), both methods produce
  equal results. (Slone et Al.)
• This fact should be kept in mind as it is likely
  that most users will play it safe and opt for a
  compression ratio below the maximum possible for
  any given modality.

6
The CAR PACS committee position

• This assessment is based on
• 2 literature reviews
• 2 legal opinions
• Before making a final decision, more evaluation
  is required.

7
Literature review

• Lossy compression is a clinically acceptable
  option for the compression of medical images.
• In comparison to wavelet, JPEG exhibits poorer
  ratios, PSNR values, artifacts and
  flexibilitybut new JND research contradicts
  research to date.
• Large matrix images can be compressed up to 501
  (using wavelet) 251 seems to be an optimum
  ratio.
• Small matrix images can be compressed up to 201
  (using wavelet) 101 seems to be an optimum
  ratio.
• The extent of allowable lossy compression ratio
  is dependent on the modality of the image and the
  nature of the imaged pathology and anatomy.
• JPEG2000 performed as well as most other wavelet
  schemes but with the added benefit of being an
  ISO (and DICOM) standard.

8
Literature review
9
Legal opinion

• 2 legal reviews
• Penny Washington of Bull, Housser and Tupper
• Mitchel McInnes of University of Western Ontario
• The conclusion is that use of lossy compression
  does not present any greater risk to the
  physicians then the typical risks that we live
  with in normal circumstances.
• To date, the use of Lossy Compression has not yet
  been considered in a court of law in the United
  States or the Commonwealth.

10
Legal opinion

• Provided that
• Appropriate ratios are used
• There is no clinically significant loss of data
• Lossy compression is used in primary reading
  (avoid altering records after primary reading)
• The technology is not adopted recklessly and due
  diligence is applied, such as
• Literature reviews
• Education
• Supervision, and
• Technology is used appropriately

11
The CAR PACS committee position

• The Canadian Association of Radiologists, with
  assistance from the Fraser Health Authority and
  Infoway, has decided to conduct a clinical
  evaluation to assess the most appropriate
  compression ratios in JPEG and JPEG 2000.
• Guidelines will be presented to the CAR steering
  committee and recommended for adoption based on
  the results of our large scale evaluation study.

12
The evaluation project

• Mandated by the Canadian Association of
  Radiologists in order to evaluate the
  acceptability of lossy compression and establish
  guidelines for its members.
• Supported by Canada Health Infoway, an
  independent not-for-profit organization
  responsible for fostering the development and
  adoption of the EHR in Canada.

13
Methodology
14
Methodology

• The goal of our study is to evaluate objectively
  if lossy compression can be used for all or part
  of the modalities and body parts assessed.
• We dont try to break records, but to find the
  lowest acceptable common denominator.
• We dont test readers skills.

15
Methodology

• Based on previous studies, our evaluation
  results from the association of 2 accepted
  methods
• Diagnostic accuracy evaluation with ROC analysis
• Image comparison with original-revealed forced
  choice (JND)

16
Methodology
5 modalities (CR/DR, CT, US, MR, NM) and 7
radiological areas (Angio, Body, Breast, Chest,
MSK, Neuro, Pediatrics)
17
Methodology

• Images are compressed in JPEG and JPEG 2000 at 3
  different ratios, close to acceptability, based
  on previous studies, below and beyond.

18
Methodology

• Diagnostic Accuracy objective measurement of
  diagnostic accuracy using blinded evaluation
  methods.
• ROC Analysis Receiver Operating Characteristic
  Analysis measures difference in perceived quality
  between original and reconstructed image through
  a confidence rating assigned by trained observers.

19
Diagnostic accuracy

• Images are a mixed of normal cases and identified
  pathologies
• (4 abnormal1 normal)
• Images presented full screen compressed in JPEG
  and JPEG 2000 at 1 of the 3 different ratios or
  in original size.

20
Diagnostic accuracy

• Using normal images is important for two reasons
  
• It allows us to examine the possibility of
  artifacts that simulate abnormalities in a normal
  image
• It addresses the possibility of bias that could
  occur if readers know that each image contains
  some abnormality.

21
Diagnostic accuracy

• The entire set of images is randomized to readers
  in such a way that each reader does not see the
  same image twice under different levels of
  compression this eliminates bias due to
  recognizing the image.
• A restricted number of pathologies are listed in
  a drop-menu from which the reader has to chose.

22
Diagnostic accuracy

• Reviewer specifies in which sector of the image
  s/he sees the pathology and gives a confidence
  rating on a scale from 1 to 5 (1definite absence
  of lesion and 5 definite presence).

1
2
3
4
23
Image comparison with original-revealed forced
choice (JND)

• Each compressed image is paired with the original
  
• Observer is asked to rate the degradation in
  image quality, which could impair interpretation,
  on a scale of 1 to 6.

301
Original
24
Image comparison with original-revealed forced
choice (JND)
25
Methodology

• 3 readers per session, 27 sessions, 81 readers in
  total from all across Canada.
• Must have appropriate workstations (minimum 1.5
  MPixel upon modality) and videocards (Matrox
  type).
• The workstation must be connected to the Internet
  ideally.
• Appropriate reading environment with correct
  ambient light.

26
Methodology

• The reader receives a CD/DVD with 70 images or
  image stack of no more than 20 images for CT
  scan.
• The reading session should not take more than 2
  hours.
• The session can be interrupted at any time and
  resumed automatically where it stopped.
• The answers are filled on-line and directly
  transferred to our server.
• If user cannot connect his workstation to the
  Internet, an alternate paper-based solution is
  provided.

27
The team
28
Sunnybrook

• David Koff Principal Investigator
• Tracy Michalak Project Manager
• Andrew Volkening PACS administrator
• Harry Shulman Scientific advisor
• Engineering team for software development 3
  programmers and a tester with DICOM and medical
  imaging experience.

29
Steering committee

• Peter Bak Infoway
• Paul Brownrigg Fraser Health Authority
• Luigi Lepento CAR
• Jane Van Essen Infoway

30
Readers

• Approximately 100 radiologists.
• Readers representing all provinces.

31
Technical developments
32
Technical developments

• An industry recognized compression engine from
  Pegasus Inc. (Tampa, FL) PICTools with JPEG 2000
  and Lossy JPEG.
• DICOM communication MergeCOM-3 from Merge-Cedara

33
Technical developments

• A standalone administrator application allowing
• Database of anonymized cases
• Database of participating radiologists
• Compression engine
• Creation and compilation of worklists
• CD/DVD engraving

34
Technical developments

• A specially designed viewing software allows to
  display images to review at the different levels
  of compression then side by side paired with the
  original when revealed.
• Limited processing functions are provided
• Zoom
• Pan
• Window-leveling
• Reset

35
(No Transcript)
36
(No Transcript)
37
Timeframe
38
Timeframe

• Initially
• Software April 28
• Radiologist recruitment April 28
• Test software May 12
• Completion of data acquisition August 25
• Findings presented at RSNA November 26

39
Timeframe

• Revised
• Radiologist recruitment April 28
• Test software July 31
• Completion of data acquisition November 20
• Findings January 30

40
Timeframe

• Late, but stronger
• Evaluation of methodology
• Evaluation of compression
• Evaluation of source code

41
Challenges
42
Challenges

• Technical development longer than expected
• Different types of images
• Complexity of database
• Shuffle cases
• Shuffle compression levels
• Track answers
• Multiplicity of reading environments
• Different monitors and video cards

43
Challenges

• Data acquisition
• Creating a database of 2500 cases has required
  more time than expected, as images have been
  provided by multiple sites and there has been
  some ownership issues on some datasets.

44
Challenges

• Specific issues
• Lossy JPEG accepts only 8 and 12 bit images,
  therefore necessity to convert 16 bit images into
  12 bit images.
• Compatibility issue with CR images from AGFA CR

45
Challenges

• Specific issues
• Conversion of Quality factor in size ratio and
  quality control to ensure accuracy of results.
• Image quality issues, mainly in pediatrics due
  to
• Low radiation dose
• Low acquisition times (MR)

46
Challenges

• Connectivity
• For several sites, we have been unable to connect
  the workstations to the Internet, mainly due to
  IT resilience to open firewalls.
• This has now been solved for our largest site in
  BC, using a different port on our server (port
  400).

47
Status
48
Status

• As of February 8, 41 readers (on 81 required)
  have completed their evaluation.
• This does not include the CR study, for which we
  have been delayed due to compatibility problems,
  and does not include readers from BC with whom we
  had connectivity issues.
• These issues have now been solved.

49
Status
50
Status

• Statistical analysis has started.
• No result yet for the Diagnostic Accuracy with
  ROC analysis.
• Preliminary results for the original revealed
  forced choice comparison showed that at the
  compression ratios chosen, there is no
  significant difference between original and
  compressed image.

51
Conclusion

• We are on track.
• Results of the study are to be released first
  half of 2007.
• Guidelines are expected by the end of 2007.
• Thank you to all for your support and
  participation.