George Cernile, Manager A'I' Technology Group - PowerPoint PPT Presentation

Title: George Cernile, Manager A'I' Technology Group


1
Advancements in Automated Synoptic Reporting

• George Cernile, Manager A.I. Technology Group
• Artificial Intelligence In Medicine Inc.

2
Agenda
3
Pathology Report

Synoptic Report
CLINICAL HISTORY/MACROSCOPY Right mastectomy and
axillary tissue. A right mastectomy specimen
with overlying skin measuring 220mm x 85mm and
underlying breast tissue measuring 220mm x 100mm
x 70mm. The axillary tail measures 125 x 60mm.
The nipple is slightly retracted and located
centrally. The superior margin is painted red,
the inferior margin painted green and the deep
cut margin is painted blue. Cut sections of the
underlying breast tissue shows an ill-defined
grey white yellow lesion with patchy areas of
haemorrhage measuring 35 x 35 x 35mm located
immediately below the nipple, 20mm from the
inferior margin, 45mm from the deep cut margin,
50mm from the superior margin, 85mm from the
medial margin and 100mm from the lateral cut
margin. A1 - nipple, B1 - upper outer quadrant,
C1 - upper inner quadrant, D1 - lower outer
quadrant, E1 - lower inner quadrant, F1, G1 -
tumour composite blocks, H1, I1 - tumour
composite blocks, J1 - deep cut margin, K1 -
superior margin, L1 inferior margin, M4 - lymph
nodes, N4 - lymph nodes, O - 3 serial slices,
lymph node, P - 3 lymph nodes.   MICROSCOPY This
right mastectomy specimen demonstrates an
invasive ductal carcinoma with the following
pathological features   TUMOUR HISTOLOGY
GRADE The tumour is of an infiltrating poorly
differentiated ductal carcinoma of non-otherwise
specified type. The tumour is poorly defined and
extremely infiltrative, comprising poorly-formed
tubules, nests or strands of cuboidal tumour
cells displaying high grade nuclei. The tumour
cells are set within fibrotic desmoplastic
stroma. Many lactiferous ducts are entrapped
within the tumour. Frequent tumour mitoses are
seen. Microcalcification is seen in some
neoplastic tubules.   Tumour grade (Modified
Bloom-Richardson Scoring System) Tubular
formation 3 Nuclear atypia
3 Tumour mitoses 2 Total
score 8 (Grade III)   TUMOUR
LOCATION, SIZE AND EXTENT The tumour is located
5mm below the nipple and has a macroscopic size
of 35mm across. The border of the tumour is
poorly circumscribed and infiltrative.   INTRA-LYM
PHOVASCULAR OR PERINEURAL TUMOUR PERMEATION Focal
intralymphatic tumour permeation is noted. No
perineural tumour invasion is seen in sections
submitted.
4
Pathology Report

Synoptic Report
CLINICAL HISTORY/MACROSCOPY Right mastectomy and
axillary tissue. A right mastectomy specimen
with overlying skin measuring 220mm x 85mm and
underlying breast tissue measuring 220mm x 100mm
x 70mm. The axillary tail measures 125 x 60mm.
The nipple is slightly retracted and located
centrally. The superior margin is painted red,
the inferior margin painted green and the deep
cut margin is painted blue. Cut sections of the
underlying breast tissue shows an ill-defined
grey white yellow lesion with patchy areas of
haemorrhage measuring 35 x 35 x 35mm located
immediately below the nipple, 20mm from the
inferior margin, 45mm from the deep cut margin,
50mm from the superior margin, 85mm from the
medial margin and 100mm from the lateral cut
margin. A1 - nipple, B1 - upper outer quadrant,
C1 - upper inner quadrant, D1 - lower outer
quadrant, E1 - lower inner quadrant, F1, G1 -
tumour composite blocks, H1, I1 - tumour
composite blocks, J1 - deep cut margin, K1 -
superior margin, L1 inferior margin, M4 - lymph
nodes, N4 - lymph nodes, O - 3 serial slices,
lymph node, P - 3 lymph nodes.   MICROSCOPY This
right mastectomy specimen demonstrates an
invasive ductal carcinoma with the following
pathological features   TUMOUR HISTOLOGY
GRADE The tumour is of an infiltrating poorly
differentiated ductal carcinoma of non-otherwise
specified type. The tumour is poorly defined and
extremely infiltrative, comprising poorly-formed
tubules, nests or strands of cuboidal tumour
cells displaying high grade nuclei. The tumour
cells are set within fibrotic desmoplastic
stroma. Many lactiferous ducts are entrapped
within the tumour. Frequent tumour mitoses are
seen. Microcalcification is seen in some
neoplastic tubules.   Tumour grade (Modified
Bloom-Richardson Scoring System) Tubular
formation 3 Nuclear atypia
3 Tumour mitoses 2 Total
score 8 (Grade III)   TUMOUR
LOCATION, SIZE AND EXTENT The tumour is located
5mm below the nipple and has a macroscopic size
of 35mm across. The border of the tumour is
poorly circumscribed and infiltrative.   INTRA-LYM
PHOVASCULAR OR PERINEURAL TUMOUR PERMEATION Focal
intralymphatic tumour permeation is noted. No
perineural tumour invasion is seen in sections
submitted.
5
Comparison to manual review

• Manual extraction of Synoptic data from text
  15-20 min. per report
• Manual extraction with Synoptex Assist
• 2 min. per report
• Fully automated extraction
• 2 3 seconds per report

6
How smart is it?

• Specimen type Hemicolectomy, right,
  partial excision of urinary
• bladder, and
  salpingo-oophorectomy, right
• Orientation
• Clinical information Colon, right
• Anatomical landmarks Ileum proximal
• Surgical markings Absent
• Pathlogy markings Ink margins of
  urinary bladder wall
• Ileum, terminal
• Length 8 cm
• Serosa Unremarkable
• Wall Thickness 0.4 cm
• Mucosa Slightly edematous
• Vermiform appendix Absent
• Colon, right
• Length 22 cm
• Serosa Cecum Puckering and
  attached portion of urinary

• Data can be found in many formats
• Data aggregate detection
• Pattern matching
• Heuristics for logical consistency
• Language processing

7
(No Transcript)
8
(No Transcript)
9
Agenda
10
Measuring Synoptex Performance

• How often does Synoptex return the expected
  value?
• How often does Synoptex return a non-expected
  value?
• Is there a tradeoff between completeness and
  accuracy?
• How does Synoptex improve with new iterations?

11
How do we measure performance?

• Sensitivity and Specificity measure the
  performance of a binary classification system
• Present/Not present
• True/False
• In the Synoptex case, this could only be applied
  to a single value, for example histologic type
• Need a new method to evaluate Synoptex that is
  analogous to Sensitivity and Specificity

• Sensitivity and Specificity?
• Synoptex has many variables and many checklists

12
Accuracy Model step 1
C Correct Sum of correctly returned values
by Synoptex. ? (all correctly found values)
I Incorrect Sum of incorrectly returned
values by Synoptex ? (all incorrect values)
M Missed Sum of values the were missed
by Synoptex ? (all missed values)

• Build a manual reference data set
• Define three measures from the manually reviewed
  reference data

13
Accuracy Modelstep 2
Required Correct Missed C M Returned
Correct Incorrect C I

• Define two new concepts with the variables

C Correct (all correct values) I Incorrect
(all incorrect values) M Missed (all missed
values)
14
Accuracy Model
Completeness Correct / Required C / (C
M) The ability to find the correct data when it
is in the path report. Accuracy Correct /
Returned C / (C I) The ability to find only
the correct data when it is in the path report.
Accuracy can be seen as a measure of exactness or
fidelity.
Sensitivity
C Correct I Incorrect M Missed
Specificity
15
Accuracy results - Melanoma
Completeness 81.2 average
Accuracy 82.0 average
16
Accuracy results - Breast
Completeness 93.4 average
Accuracy 93.8 average
17
Agenda
18
The Challenge

• How do we expand capabilities of Synoptex system

19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
Agenda
24
Synoptex Applications

• Standardization of Pathology Reports for
  Registries
• Data Quality Assessments
• Audits
• Mining of Current and Legacy Data
• Export of Indexed Data for Statistical Processing
• Annotated Datasets for Biospecimen Repositories
• Data for Automated Record Matching
• Clinical Trials
• Case Control Studies

25
Pathology Data Completeness Report
Element Analysis
Site Breast   Reports Sample Size
49776 Date Range 2004 - 2007 No. Of
Cancer Reports 8422 No. Of Breast Cancers
1678  
26
Example ER Positivity vs. HER2 Status
- ER
324 (61)
- HER2
148 (68)
(68)
(61)
ER Positive
27
Agenda
28
The challenge

• How frequently are they updated?
• How do we manage updates/versions?
• How can systems be maintained ?
• How can custom additions be kept across versions?

• CAP Checklists version updates

29
CAP electronic Cancer Checklist (CAP eCC)
30
Checklist Editor for CAP eCC
31
How is this related to Synoptex?

• Interoperability
• Synoptex output same format as Manual checklists
• SNOMED codes
• CKEYS
• Synoptex data can be viewed with STS checklist
  system

32
Synoptex uses same database as (CAP eCC)
Export
Synoptic Database
Export
33
Automatic Updates
Knowledge Base Manager
CAP eCC updates
Import
Export
Export
Synoptic Database
34
Synoptex Benefits

• Renders Pathology Reports Machine Readable
• Facilitates Research
• Identifies Some Collaborative Stage Elements
• Saves Labor in Deriving Synoptic Reports both
  manually and fully automated
• Verifies Report Content (Audits)
• Compare reporting accuracy hospital-to-hospital
• Apply Synoptex to other data sources
• Surgical reports
• Medical records
• Radiology reports
• Hematology reports

35
The End

• Thank You