Automatic Detection of Anatomical Features in Bronchoscopy

1
Automatic Detection of Anatomical Features in
Bronchoscopy

• Indriyati Atmosukarto
• CSE577 Project

2
Motivation
3
Motivation

• Current bronchoscope can only reach within first
  five airways
• Based on CT scans, doctors mentally build model
  of patients lung in their head
• During bronchoscopy doctors keep track of
  bronchoscopes location in their head

4
Motivation

• HIT Lab developing an ultrathin endoscope which
  can reach first 8 airways
• Doctors would not be able to keep track of
  bronchoscope location in the lungs
• Virtual bronchoscopy with electromagnetic does
  not work all the time (calibration, small airways)

5
Objective

• Given a video of bronchoscopy
• Automatically detect location of bronchoscope in
  lung based on anatomical feature (bifurcation)
  and camera motion
• Automatically count level of airway

6
Approach

• Identifying bifurcations
• EM Segmentation
• Identifying camera motion
• EM Segmentation
• Optical Flow

7
Approach EM Segmentation

• Cluster each pixel based on intensity
• convert RGB to HSV and use V of each pixel
• Extract bifurcation cluster
• Remove small clusters using threshold
• Fill holes in bifurcation cluster
• Look at adjacent frames,
• Number of bifurcation clusters and size determine
  camera motion

8
Approach EM Segmentation

• If number of clusters increase OR areas of
  clusters increase
• gt FORWARD
• If number of clusters decrease
• gt BACKWARD (bifurcation merge)
• If areas of clusters decrease
• gt BACKWARD

9
Approach Optical Flow

• Look at flow vectors
• Camera moving forward (zoom-in)
• flow vectors moving towards centre
• Camera moving backward (zoom-out)
• flow vectors moving outwards from centre
• Camera moving left/right (pan)
• flow vectors moving left / right

10
Experiments - Setup

• Video of 4 independent sequences

11
Experiments EM Segmentation

• Modified matlab source code from Enrique
• Quantitative test determines 4 clusters
• Small bifurcations mistakenly removed
• threshold issue
• Background sometimes clustered as bifurcation
• crop image

12
Experiments EM Segmentation
13
Experiment EM Results

• Can identify bifurcations in all 130 frames
• Can identify camera motion for 60 of frames
• More difficult to identify camera moving backward

14
Experiments Optical Flow

• Code www.cs.ucf.edu/vision/source.html
• Hierarchical Lucas Kanade (pyramids)
• pyramid 3, Window size 3, iterations 3
• Optical flow on whole image
• Optical flow on bifurcations only
• Reduce homogeneous areas
• Optical flow on edges of bifurcations
• Try out different frame distances

15
Experiments Optical Flow
16
Experiments Optical Flow
17
Experiments Optical Flow
Can still identify zoom-in/camera moving forward
as flow vectors are quite regular
bifurcation_pixel_edge_test0.pgm_bifurcation_pixel
_edge_test1.pgm_1.HierarchicalLucasKanade.jpg
18
Experiments Optical Flow
Flows do not really show zoom out behavior?
bifurcation_pixel_edge_test53.pgm_bifurcation_pixe
l_edge_test54.pgm_1.HierarchicalLucasKanade.jpg
19
Experiments Optical Flow results

• Flow vectors are not as orderly as expected
  eventhough few points were used
• Difficult to really conclude motion of camera
  using optical flow

20
Other things Ive tried

• Curvelinear features

21
Other things Ive tried

• Extracting SIFT features

22
Extensions

• Combine information from both EM and optical flow
• For EM Segmentation, look at more neighboring
  frames (include tracking)
• Use learning
• Try out approach on real data ( color lighting)
  and different patient demography

23
Extensions

• Extend to real time application
• Provide a user interface for application
• Incorporate auto-correction or re-calibration to
  reset accumulation of errors
• URL http//www.cs.washington.edu/research/VACE/in
  driwww/577/html