KLTKanadeLucasTomasi feature tracking algorithm in embedded hardware

1
KLT(Kanade-Lucas-Tomasi) feature tracking
algorithm in embedded hardware

• Yamini Balasukumaran
• Rekha Dilli Babu

2
Project Goal

• Track mice using embedded hardware
• Implement KLT algorithm for tracking
• Efficient embedded system implementation

3
Project Plan

• Study the theory for KLT (Good features to
  track).
• Study the given KLT implementation details(Vision
  group at Stanford University)
• Familiarize with the TMS320C5510 DSP kit.
• Implement the KLT in TMS320 to track mice.

4
KLT Algorithm TheoryGood features to track

• Feature selection deals with extracting
  attributes that result in some quantitative
  information of interest.
• Tracker does not track single pixels but windows
  of pixels.
• Good window is one that can be tracked well.
• The goodness of each pixel is measured as the
  minimum eigenvalue of the 2 by 2 gradient matrix
  computed from the tc-gtwindow_width by
  tc-gtwindow_height window around the pixel
• Eigen values indicate whether there is a constant
  intensity profile, unidirectional pattern or salt
  and pepper textures.

5
KLT Algorithm Implementation

• Functions Has functions to select good features,
  track the selected features from one frame to
  another(KLTSelectGoodFeatures,KLTTrackFeatures).
• Can customize the tracker by manually setting
  various parameters like mindist, Window_width,
  window_height, min_eigenvalue, subsampling.
• Structures KLT_TrackingContext (to customize
  tracker), KLT_Feature, KLT_FeatureList.

6
KLT Implementation Details

• KLT_Feature
• typedef struct
• KLT_locType x
• KLT_locType y
• int val
• KLT_Feature

• KLT_TrackingContext
• typedef struct
• int mindist
• int window_width, window_height
• KLT_BOOL sequentialMode
• KLT_BOOL smoothBeforeSelecting
• KLT_BOOL writeInternalImages
• int min_eigenvalue
• float min_determinant
• float min_displacement
• int max_iterations
• float max_residue
• void pyramid_last_grady
• KLT_TrackingContext

7
KLT Implementation DetailsKLTSelectGoodFeatures

• Gradients are computed in x and y direction and
  are used to select the features
• tc-skippedPixels can be used to speed up the
  process
• Goodness of pixel measured as minimum Eigen value
  of gradient matrix
• Pixels are sorted in ascending order of goodness
• nFeatures are selected
• Meet mindist and min_eigen value.

• KLTSelectGoodFeatures()
• void KLTSelectGoodFeatures(
• KLT_TrackingContext tc,
• KLT_PixelType img,
• int ncols,
• int nrows,
• KLT_FeatureList fl)

8
Hardware - TMS320C5510 DSK kit

• Chosen because of availability with the idea
  that the design could be ported to another
  appropriate hardware.
• TMS320C5510 DSK kit has fixed point DSP
  processor(200 Mhz ), has support for DMA.
• Supports Multiply accumulate hardware(MAC)
  instruction - intrinsic functions
  sum
  (ppp) (kernel.datak)
  sum _smac(sum,ppp,kernel.datak)
  
• On chip hardware timer .

9
Implementation of Feature selection(Embedded
system)

• Measured the execution time of the
  KLTSelectFeatures function using the high
  resolution timer functions.
• Start CLK_gethtime()
• functioncall()
• stop CLK_gethtime()
• time_used stop - start
• num_clocks_count
  CLOCK_FREQ_MS/CLK_countspms()
• num_clocks_used
  num_clocks_count time_used
• tottime (num_clocks_used/200000)
  
• In the case of 45 by 22 frame image this
  value was 51ms for the KLTSelectFeatures
  function.

10
Implementation of Feature selection(Embedded
system)

• File-IO execution time (3 usec) was found to be
  negligible compared to the Select-features
  function(51 ms).
• The periodic function scheduled for 70 ms. (14
  frames per sec)
• Separate the signal processing functions from the
  file formatting functions. If implemented on a
  system like C5471 which has an ARM and DSP then
  it would be easier to implement.

Periodic-function Generator (every 70 ms)
Image-file Pre-processor
KLt-Feature-Selection
11
Results

• Location X1 Y1 X2 Y2
• KLT reference implementation 22 13 31 13
• Embedded implementation 22 12 29 12

12
Tracking mice
13
Scope for improvement

• Choose a Floating point processor.
• Set-up which has streaming data interface
• Parallel processing to improve speed up (e.g..
  for computing horizontal and vertical gradients)
• Change the algorithm to save on memory
  allocations.

14
Conclusion

• Implemented the feature selection portion of the
  algorithm with limited scope.

15
References

• http//vision.stanford.edu/birch/klt/
• Jianbo Shi and Carlo Tomasi. Good Features to
  Track. IEEE Conference on Computer Vision and
  Pattern Recognition, pages 593-600, 1994.
• TMS320C5510 Reference Manuals