Sensor and Vision Research for Potential Transportation Applications

1
Sensor and Vision Research for Potential
Transportation Applications
Research Initiatives for Nano and Hi-tech
Research January 18, 2006

• Zhigang Zhu
• Visual Computing Laboratory
• Department of Computer Science
• City College of the City University New York

2
Multimodal Human/Vehicle Signatures

• Collaboration with
• AFRL
• Prof. Tom Huang at UIUC
• Prof. George Wolberg at CCNY
• Goal
• Remote sensing/hearing
• Multimodal signatures
• Challenges
• Noisy LDV signals
• Low-res, non-front faces
• Target detection??
• Transportation???

3
Major Sensor Acquisition

• Polytec Laser Doppler Vibrometer (LDV)
• OFV-505 Sensor Head, OFV-5000 Controller
• Tip-Tilt Stage VIB-A-P05 with Telescope
• FLIR ThermoVision Camera
• ThermoVision A40M Infrared Camera
• Canon Color/IR PTZ Camera
• Canon VC-C50i Low Light IR PTZ Camera
• Other video cameras
• Omnidirectional cameras, stereo head
• Camcorders, webcam, CMOS sensors

4
Polytec Laser Doppler Vibrometer

• Sensor Head OFV-505
• HeNe (Helium-Neon) laser, l632.8 nm, Wlt1 mW
• OFV-SLR lens (f30mm) 1.8 m 200 m
• Any surfaces, Automatic focus
• Controller OFV-5000
• Low pass (5, 20,100 kHz), high pass (100Hz)
• RS-232 interface for computer control
• Velocity Decoder VD-06
• Ranges 1, 2, 10 and 50 mm/s/V
• Resolution 0.02 mm/s under 1mm/s/V range
  (2mv/20V)
• 350 kHz bandwidth analog output
• 24 bit, 96 kHz max. digital output on S/P-DIF
  interface

5
Dynamic Pushbroom Stereo Mosaics
Sponsored by AFRL PIs Zhigang Zhu, George
Wolberg - CUNY City College Robert Haralick
CUNY Graduate Center
Results Content-Based 3D Mosaics
Concept Dynamic Pushbroom Stereo
Innovative Claim
Description of Research
Video Registration, representation and 3D
static/dynamic content extraction from video
sequences of dynamic urban scenes taken from
aerial or ground vehicles.
(1) Pushbroom stereo mosaics for 3D dynamic
scenes (2) Natural stereo matching primitives for
3D urban scenes and moving targets (2)
Content-based 3D mosaic rep for accurate 3D and
motion
Objectives
Expected Contribution to Research Area
(1) Rapid panoramic stereo mosaic construction
(2) Accurate 3D reconstruction and parametric
rep. (3) Robust moving target extraction and
estimation
An efficient and more accurate representation for
large scale scenes A new natural matching
approach for both urban structures and moving
targets
6
Key Results Accurate 3D reconstruction and
motion detection on mosaics from real world video
7
Under-Vehicle Inspection
Joint Work with UMass Computer Vision Lab

• Car drives over a 1D array of cameras

Video
Stereo Mosaics revealing occlusions
8
Gamma-Ray Cargo Inspection
Images from SAlC Mobile Vehicle and Cargo
Inspection System (VACIS)
3D View
3D measurements from 2 pushbroom gamma-ray images
9
Summary

• Novel Sensors
• Color, thermal, vibration
• Novel Algorithms
• Stereo mosaicing 3D/ motion detection
• Emerging 3D Applications
• Security, surveillance safety
• in transportation