Robot Vision with CNNs: a Practical Example

1
Robot Vision with CNNsa Practical Example
Barcelona, 19/2/03
M. Balsi Dep. of Electronic Engineering La
Sapienza Univ. of Rome, Italy

• P. Vitullo
• P. Campolucci
• G. Apicella
• L. Pompeo
• D. Bellachioma
• S. Graziani

X. VilasísCardona S. Luengo J. Solsona R. Funosas
A. Maraschini A. Aznar V. Giovenale P. Giangrossi
2
Framework of this work

• completely autonomous robot
• simple (cheap) hardware
• vision-based guidance
• short term line following
• longer term navigation in a real environment

3
Architecture

• Cellular Neural Networks to handle all the image
  processing

• Fuzzy-rule-based navigation

4
Cellular Neural Networks

• Fully parallel analog vision chips

• Capable of real-time nonlinear image processing
  and feature detection

• Algorithmically programmable to implement complex
  operations

• On-board image acquisition (focal-plane
  processing)

5
Cellular Neural Networks

• Recurrent Neural (?) Network
• Locally connected ? VLSI-friendly
• Space-invariant synapses (cloning templates)
• small number of parameters explicit design
• Continuous variables analog computing
  (discrete-time model for digital)

6
Topology
Locally connected ? VLSI
Space-invariant synapses
7
Discretetime model

• Binary state variable
• Analog or binary input depending on implementation

8
Application

• Input ports analog arrays u, x(0)
• Output port binary array x(?)
• Analog instruction A,B,I (cloning template)
• Feature detection (nonlinear image filtering)

9
CNN Universal Machine

• Local memory
• Global control (broadcasting cloning templates
  and memory transfer commands)
• Analogic computing stored-program analog/logic
  algorithms

10
Task line following

• The robot is to follow a maze of straight lines
  crossing at approximately right angles

• Functions required by vision module

• Acquiring image, cleaning, thinning lines

• Measuring orientation/displacement of lines

11
Image processing algorithm

• Image acquisition

• Binarization

• Line thinning

12
Image processing algorithm (ctd.)

• Directional line filtering

• Projection

13
Fuzzy control
14
Simulation
15
el cochecito(Barcelona)
16
Visibilia (Rome)
FPGA-based CNN emulator Celoxica RC-100 board
Xilinx Spartan II 200Kgates
PAL B/W CAMERA
PS/2 mouse port
STEPPER MOTOR CONTROLLER
SERVO MOTOR (steering)
microcontroller Jackrabbit
BL1810
Parallel port E
STEPPER MOTOR (advancing)
Rabbit2000 microcontroller
PIC 16F84
LCD
Parallel port A
Serial port D
17
(No Transcript)
18
Celoxica RC-100
19
Jackrabbit BL1810
20
driving
start
Y
hor
store left avail.
vert
hor
N
N
Y
timer0
Y
hor
Y
timergt10s
N
N
Y
follow vert
turn left if avail. else right
diag (L/R)
Y
follow diag
normal driving
N
crossing
21
Continuation of the work

• more realistic tasks
• obstacle avoidance
• navigation in a real-life environment

22
Obstacle avoidance

• using other sensors together with vision, e.g.
  ultrasound
• monocular range evaluation
• local path-finding strategies

23
Hybrid (topological/metric) navigation
24
door recognition
25
Robot Vision with CNNsa Practical Example
Barcelona, 19/2/03
M. Balsi Dep. of Electronic Engineering La
Sapienza Univ. of Rome, Italy balsi_at_uniroma1.it