Sensor network based vehicle classification and license plate identification system

1
Sensor network based vehicle classification and
license plate identification system

• Jan Frigo, Vinod Kulathumani
• Ed Rosten, Eric Raby
• Sean Brennan

2
Scenario

• Facility monitoring
• detect suspicious vehicles entering secure area
• deployed at key access points / check posts or
  along length of a road
• Vehicle classes
• Personal such as car, SUV
• Heavy loads such as pickup trucks,
• Military vehicles such as ATV, hummers and huge
  log trucks
• Platforms
• Mica2 motes
• ARM processor stargates

3
Objectives

• Classify vehicles with
• High reliability
• low latency
• low energy
• Extract license plate image

4
Challenges

• Small scale deployment at each access point (lt 10
  units)
• Vehicles last in influence region for a short
  time (1-2 seconds)
• Spectral signature of vehicles changes with time
• Resource constrained devices
• Vehicles moving at variable speeds
• Environment
• Temperature
• Physical barriers (trees, winding road,
  environmental)
• System Power

5
Seismic Acoutic Node Architecture
Network
2 GHz
900 MHz
Mica2
Stargate
Geophone
Microphone
6
Seismic-Acoustic Field Experiment
7
Frequency characteristics of seismic detection

• Geophone placed 50 ft away from road to avoid
  acoustic interference

8
Seismic detection

• Sample at 100 Hz
• 16 bits samples using MDS320 board
• Estimate energy of 12-25Hz band
• Haar wavelets up to level 2
• Energy average of coefficients of band 2
• Haar wavelet computed on 128 samples every 10 ms
• 10 new samples each round
• 118 samples from previous round
• Compute variance of the energy on moving window
  of size 20
• Use variance threshold to detect vehicle

9
Seismic detection performance

• Seismic detection triggers acoustic sampling and
  / or processing
• Energy efficient
• Person walking 2 ft does not trigger detection
• Person thumping feet (running) lt 10 ft away
  triggers detection
• Can be isolated using temporal characteristics

10
Acoustic classification

• FFT used to obtain spectral characteristics
• Fixed point FFT implemented on stargate
• Classifier trained using FFTs computed on
  stargate
• Identify best feature vector characteristics to
  distinguish between vehicle classes
• Use Fisher linear discriminant analysis (FLDV)
  for classification
• Pairwise classifier
• Select order of classification that maximizes
  accuracy
• Input obtained vectors into stargate for
  classification

11
Acoustic classification

• FFT computed every 1/8 of a second
• 512 samples FFT
• 12 samples from previous round
• 8Hz resolution
• Consider frequencies gt 64 Hz
• Mic response varies at lt 60 Hz
• Temporal variation in response lt 60 Hz (probably
  due to wind)
• Closest 1.5 seconds of data used as training
  samples

12
Mean FFT coefficients for Vehicle Classes
13
Acoustic classification

• Classification order that maximizes accuracy
• Presence of vehicle
• Hummer vs car and truck
• Car vs truck
• Presence
• Use average energy of 200-360 Hz band
• Moving window variance (size 20) based detection
• 200-360 Hz band less sensitive to high frequency
  chirp and wind noise

14
Classification using FLDV

• Hummer vs car / truck
• Feature vector 1 ratio of energies of 80 112
  Hz and 350 500 Hz bands
• Feature vector 2 ratio of energies of 250 300
  Hz and 350-500 Hz bands
• Ratios less sensitive to mic response and
  distance from road
• FLDV uses training samples to compute best
  projection vector

15
Acoustic classification

• Integrating output
• Approaching vehicle characteristics differ from
  closest point
• Classifier operates for 10 seconds as the
  vehicle approaches and passes node
• Classifier designed such that
• Low probability of car being classified as truck
  or hummer at any instant
• Low probability of truck being classified as
  hummer at any instant
• gt 5 consecutive truck classifications within a
  test run-gt vehicle is truck
• gt 5 consecutive hummer classifications within a
  test run -gt vehicle is hummer

16
License Plate Recognition
Classify Pixels
Find bounding boxes
Extract and resample plates
Find and filter regions
Send image

• Sending only license plates over the network
  requires very little bandwidth.
• Resampling license plate to fixed size reduces
  network usage when vehicles are close.
• Computationally expensive OCR is run on remote
  host
• Algorithms use integer arithmetic only

17
Performance

• Vehicle classification
• Within 2 seconds of object passing zone
• Reliability gt 90
• 2.26 watts power consumption when computing
• Will last about 12 hours if continuously
  computing on 4.8V 4200mAH cell
• License plate recognition
• Detection accuracy gt 95
• Suppresses 90 of image content
• Latency 5.1 seconds mainly for image capture
• Higher power consumption can perform 5582 trials
  on 4.8V 4200maH cell

18
License plate detection algorithm

• Combination of
• Viola jones detector object detection
• Decision tree classifier license plate
  segmentation
• Analyse using Haar Wavlet like features
• Efficient to compute using an integral image
• Integral image is also required for resampling
  license plate
• Computational time is independent of feature size
• Train decision tree classifier to recognize
  license plate pixels
• Decision trees are very efficient
• Only integer arithmetic required for evaluation
• Tune the tree to rapidly reject most pixels very
  quickly

19
Ongoing and Future Work

• Increase power efficiency
• Embedded FPGA implementations for in situ
  computing
• Estimated 5 100x power savings and 30 100x
  speed up in run-time performance over COTS
• Node Architecture combination of
• new ARM processor technology on next generation
  mezzanine board
• Igloo FPGA on sensor board
• Low power analog acoustic circuitry
• Design of cooperative analog-digital signal
  processing systems
• Upto 300X power savings
• Identify optimal balance between nodal
  computation, in-network processing and central
  computation