IST199929017 OMNIVIEWS Omnidirectional Visual System

1
IST-1999-29017 OMNIVIEWSOmnidirectional Visual
System

• Final Review
• September 27-28, 2001
• Lisbon

2
Agenda of the meetingFirst Day
  1430 Reviewers private meeting 1500
Welcome Jose' Santos-Victor 1510 Goal of
Review Pekka Karp 1520 Omniviews Main
Achievements Giulio Sandini 1600 Coffee
Break 1630 Mirror design principles Branislav
Micusik 1650 Mirror design tools Jose'
Santos-Victor, Claudia Decco 1710 Demos
Introduction 1720 Surveillance Demo Tomas
Pajdla, Alex Bernardino 1800 Transmission Demo
Pedro Soares, Giulio Sandini 1830 End of
first Day  2030 Dinner  
3
Agenda of the meetingSecond Day
  900 Navigation Demo José Santos-Victor
940 Future Outlook and General Discussion
Giulio Sandini 1010 Reviewers Private Meeting
(with coffee) 1110 Preliminary Evaluation
Report 1140 End of meeting 
4
Main Facts
Open-scheme, Assessment Phase Project
Consortium DIST - University of Genova -
Genova CMP - Czech Technical University in
Prague VISLAB - Instituto Superior Técnico -
Lisbon
Project Start and Duration September 1st 2000
One year
Funding 100 K
5
Projects Main Objective
The main objective of the project is to integrate
optical, hardware, and software technology for
the realization of a smart visual sensor, and to
demonstrate its utility in key application areas.
In particular our intention is to design and
realize a low-cost, digital camera acquiring
panoramic (360) images and performing a useful
low-level processing on the incoming stream of
images.
6
Key Technologies
Retina-like visual sensor
Omnidirectional Mirrors
7
Specific Objectives of Assessment Phase

• Define the optimal profile of a mirror matching a
  retina-like visual sensor. Optimal in the sense
  that direct read-out of panoramic images is
  obtained.
• Demonstrate its utility in key application areas
• If successful present a follow-up proposal

8
Methodologies

• Use the currently available SVAVISCA camera for
  initial experiments
• Design and simulate mirror using SVAVISCA camera
• Realize the OMNIVIEWS mirror for the current
  sensor
• Demonstrate the mirror in key applications

SVAVISCA Pixel layout
Simulated image SVAVISCA camera Hyperbolic mirror
9
Mirrors design principle

• Uniform Cylindrical Projection
• Direct read-out through log-polar mapping

10
OMNIVIEWS Mirror
Mirrors Profile
Experimental Set-up and test images
11
Assessment Criteria

• Direct read-out of panoramic images
• Frame rate
• Resolution and layout of the sensor
• Mirror profile and size
• Lens characteristics
• Camera cost
• Image quality

12
AC1 Direct read-out of panoramic images
Direct read-out from OMNIVIEWS About 30,000
read-out operations
Image Obtained from a conventional camera About
1.8 M operations required 882,000 read-outs (30
times more) 882,000 additions 30,000 divisions
13
AC2 Frame rate
Currently the maximum read-out frequency is fixed
by the cameras interface (PC Parallel port)
limiting the frame rate to about 12
frames/s. More than 25 frames/s is achievable
with a faster interface (e.g. USB or PCMCIA)
14
AC3 Resolution and Layout of the sensor
15
AC4 Mirror Profile and Size
Mirror profile and size meets the original plan
(6 cm.).

• Furthermore
• New technology for mirror realization
• Mirrors design tool of general utility
• Design and realization of mixed-mirror
• Overall size can be reduced

16
AC5 Lens characteristics
Standard C-Mount lenses have been used for all
experiments and demos
No difficulties in principle are envisaged for
the design of smaller size lenses (possibly
including the mirror).
17
AC6 Camera Costs
Cost of obtaining panoramic images is zero in our
case Compared to conventional solutions no
extra-cost for the mirror is required. Lower cost
is possible with the new glass-based
technology. The cost of the sensor is equivalent
to the cost of conventional sensors realized with
the same technology and with the same size.
18
AC7 Image quality
The project will be successful if we demonstrate
that it is possible to create virtual images by
simple reading out the pixels from the proposed
sensor and to use such images in the aimed
applications .

• Topology of images meets the quality criteria
• Evaluation of numerical approximations
• Three demonstrations
• Surveillance (two parts)
• Navigation
• Image Transmission
• Further processing experiments
• Localization using Agam fiducials
• 3D reconstruction

19
Additional remarks

• Software mirror-design tools have been developed
• New kind of mirror have been proposed extending
  the original plan (i.e. the mixed mirror)
• 10 scientific papers have been published
• Plans for the future are clearer.

20
Future Outlook
Draft
21
Objectives

• Miniature Omnidirectional Camera with increased
  performance
• Pre-industrial prototype
• Focused applications

Draft
22
Miniature Camera
Draft

• Increase the resolution (and/or reduce the size)
  of the sensor using currently available (but
  forefront) CMOS technology
• Improve and integrate optical components
• Faster Camera Interface

23
Sensors Layout
Current CMOS 0.35 µm technology min pixel size
6.8 µm 33,000 pixels (equivalent
1060x1060) Forefront CMOS 0.18 µm technology
min pixel size 3.6 µm 100,000 pixels
(equivalent 2000x2000)
Draft
In this case about 100 K read-out operations are
equivalent to 6.3 Million operations required by
a conventional camera with 2000x2000 pixels.
24
(No Transcript)
25
Simulations are worst than actual images
26
Mirrors Technology
Draft
Investigate the integration of current
mirror-lens assembly from an optical design point
of view (application driven) Possibly adopt
cheaper technology such as glass coating
27
Applications

• (Remote) Surveillance (e.g. traffic monitoring
  and emergency call-box for highways)
• Endoscopes for inspection of body cavities
  (pipe-like).
• Sewer inspection systems

Draft
28
Consortium

• Add optic-design expert
• Add silicon designer
• Add Industrial partners for realistic
  requirements (surveillance medical)

Draft