Automobile Lane Detection System-on-Chip Integrated with Mixed Signal Mode CMOS Image Sensor

1
Automobile Lane Detection System-on-Chip
Integrated with Mixed Signal Mode CMOS Image
Sensor
IEEE 9th International Symposium on Consumer
Electronics (ISCE 2005)
Pei-Yung Hsiao¹, Hsien-Chein Cheng¹, Chun-Wei
Yeh¹, Shih-Shinh Huang², and Li-Chen Fu²
¹ Department of Electronic Engineering Chang Gung
University
² Department of Electrical Engineering National
Taiwan University
2
OUTLINE

• The Problem Motive
• Brief Introduction to Algorithms
• Architecture and Circuits description
• Simulation and Results
• Conclusion

3
Original Idea for the Problem
We are interested in developing a System-on-Chip,
Soc, which can capture image as well as produce
vehicle lane map at the same time.
Capture Image
Land map
4
Motive

• The areas of Intelligent Transportation System,
  ITS, include lane detection, obstacle
  recognition, vehicle detection, car following,
  etc.
• Our goal in this investigation is to develop a
  CMOS imager to achieve real-time image capture
  and lane detection, simultaneously, for
  intelligent automotive driver awareness/assistance
  system.

5
Automotive IC Design
The target chip can be defined as a component
device for intelligent vehicles.
6
Widespread Applications

• The proposed imager without demanding extra ADC
  circuits for signal transformation is a single
  low-cost and compact chip for used in the
  thousands of consumer electronics not limited to
  ITS.

7
Introduction(1/4)

• From the referenced literatures, there are a lot
  of vision-based lane detection algorithms
  proposed in recent 10 years 1-6 (1995-2004).
• In 1995, Kluge and Lakshmanan 3 proposed the
  LOIS (Likelihood of Image Shape) lane detection,
  which is able to detect lanes even in situations
  with shadows or broken lanes by using a
  stochastic optimization procedure.

8
Introduction (2/4)

• In 1995, Broggi 5 proposed an edge-based road
  detection algorithm, while it is effective only
  for the well-painted road.
• In 1999, Takahashi, etc. 4 divided the
  parameter space of the lane model to generate the
  lane marking patterns and then applied the voting
  scheme to find the lane boundary.

9
Introduction (3/4)

• In 2004, Huang, etc 1 proposed an on-board
  vision system for lane recognition and
  front-vehicle detection to enhance driver's
  awareness.
• Regarding to high recognition rate and hard-wired
  regularity, we adopted Peak-Finding based lane
  detection algorithm from Huang, etc 1, which
  has high recognition rate about 96.

10
Introduction (4/4)

• According to Huangs algorithm, we also developed
  an auto-regulated threshold circuit to
  automatically adjust the threshold for the lane
  detector to adapted to different weather
  conditions.

11
Architecture and Circuit description

• Our chip can be divided into three parts, such as
  analogue capturing and processing circuits,
  digital processing circuits and digital control
  unit.
• The analogous circuits include 2-D pixel cell
  array, CDS module, 1-D Gaussian filter and
  Peak-Finding module.
• The digital processing circuits are composed of
  Line Point Allocation module, column selector and
  row selector.

12
SOC for real-time image capture and lane detection
13
Pixel Cell Sensor Array

• The developed sensor array consists of two types
  of pixel cells.
• Our sensor array prototype is made up of 6464
  effective pixels.
• The upper region containing 1664 pixels is
  ignored in back-end processing to promote the
  computing efficiency.
• The other regions are horizontally partitioned
  into three sub-regions. Each sub-region consists
  of 16 rows.
• The 12th row in the sub-region or in the upper
  region is defined as a 1-D sample array.
  Consequently, we have four 1-D sample arrays in
  our sensor array.

14
Normal cell
Sampling cell
15
Dual 1-D Gaussian Filters

• Each 1-D Gaussian filter includes 64 Gaussian
  mask cells and a current divider. Each Gaussian
  mask cell consists of 3 current mirrors in 7
  transistors and two OR gates.
• The Gaussian filter module is used for smoothing
  the selected pixel by referring to a couple of
  right and left neighbors to eliminate noisy
  points in the original image.

16
Dual 1-D Gaussian Filters
17
Peak-Finding Module (1/3)

• The 1st part of the Peak-Finding Module can
  accumulate and average current, Iavg, from the
  aforementioned sample arrays, Ips.
• The averaged current from sample array, Iavg, was
  generated according to the following equation.

18
Peak-Finding Module
Peak-Point Output
Lane-Point Output
Line-Point Allocation
19
Peak-Finding Module (2/3)

• The 2nd part of PFM is called as auto-regulated
  threshold circuit. It compares average current,
  Iavg, with four preconfigured currents, Irefn,
  and then produces the threshold current, Isth.
• The total threshold current, ITH can be noted by
  the following equation.

, where
20
Peak-Finding Module (3/3)

• Inside the auto-regulated threshold circuit, each
  threshold mapping circuit control a threshold
  current. It can be noted by the following
  equation
• According to the 3rd part of the PFM, If the
  current pixel is a peak point, the output value
  will be 1, otherwise, it should be 0.

, where
21
Line-Point Allocation

• The Lane-Point Allocation Module expressed as the
  following equation is composed of two digital
  functions, such as the line segment filter and
  the lane point selector.
• Lane points, Lb(i,j), are obtained, and
  represented by only one pixel width in each row.

22
Timing Diagram
The clock frequency of the Row Selector (clk R)
is 64 times of the Column Selector (clk C). In
this case, the frequency of the Column Selector
is 25MHz and the frequency of the Row Selector is
0.78MHz.
23
HSPICE Simulation (1/2)
(a)
(b)
Peak-Point
(c)
1.5us
(a) is the output current of the current Gaussian
Filter. (b) is the output current of the previous
Gaussian Filter. (c) is the results of the
Peak-Finding Module.
24
HSPICE Simulation (2/2)
(a)
(b)
(c)
(d)
5us
25us
(a) is the simulation results of the current
Gaussian Filter. (b) is the simulation results of
the previous Gaussian Filter. (c) is the
simulation result of the Peak-Finding Module. (d)
is the simulation results of the Lane-Point
Allocation Module.
25
Software Simulation in C
From 1, noises are to be removed by the
followed post processing.
(a) Original image (512x512 gt 320x2401)
(b) Lane map points generated by Peak-Finding
algorithm.
(c) Original image (64x64)
(d) Lane map points generated by Peak-Finding
algorithm.
26
Experimental Results
(a) Original image in 32 32
(c) Result generated by our chip
(b) Result generated by software
27
Chip Layout in 64 64
28
Specification of The Proposed CMOS Imager
Item Values
Pixel Count 64(H) X 64(V)
Pixel Size 18.45 um(H) X 21.8 um (V)
Aperture Size 12.45 um(H) X 9.6 um (V)
Fill Factor 29.7
Image Size 1217.7 um (H) X 1455.05 um (V)
Chip Size 2191.4 um (H) X 2389.8 um (V)
Operation Clock 25MHz
Operation Voltage 3.3 V
Power consumption 159.4mW
29
Conclusion

• Our investigation includes a 2-D image sensor
  array embedded with four modularized circuits
• ----- four 1-D sample array by different pixel
  cell design for accumulating the sampled
  currents
• ----- dual 1-D Gaussian filers coupling as an
  analogue image smoothing module
• ----- an analogue design for Peak-Finding
  function associating with a novel auto-regulated
  threshold operation
• ----- a sophisticated digital implementation for
  Lane-Point allocation.

30
Conclusion, cont.

• A new current-mode mixed signal design of CMOS
  image sensor integrated with Peak-Finding based
  lane detection algorithm is developed.
• The proposed low-cost and one compact chip
  solution can grab the road images from the real
  world and successfully detect the lane markers
  simultaneously, in real time.

31
Reference (1/4)

• 1 Shih-Shinh Huang, Chung-Jen Chen, Pei-Yung
  Hsiao, and Li-Chen Fu, On-Board Vision System
  for Lane Recognition and Front-Vehicle Detection
  to Enhance Driver's Awareness, IEEE
  International Conference on Robotics and
  Automation, vol. 3, 26 April 1 May, 2004,
  pp.24562461.
• 2 Yue Wang, Eam Khwang Teoh and Dinggang She,
  Lane detection using B-snake, , International
  Conference on Information Intelligence and
  Systems, 31 Oct. - 3 Nov., 1999, pp.438 443.
• 3 Kluge, K., and S. Lakshmanan,, A
  Deformable-Template Approach to Lane Detection,
  in I. Masaky, editor, Proceedings IEEE
  Intelligent Vehicle95, Detroit, 25-26 Sept.,
  1995, pp.54-59.
• 4 Takahashi, A., Ninomiya, Y., Ohta, M., and
  Tange, K., A Robust Lane Detection Using
  Real-Time Voting Processor, IEEE/IEEJ/JSAI
  International Conference on Intelligent
  Transportation Systems, 5-8 Oct., 1999,
  pp.577580.

32
Reference (2/4)

• 5 A. Broggi, Robust Real-Time Lane and Road
  Detection in Critical Shadow Conditions,
  Computer Vision, 1995, Proceedings.,
  International Symposium on, 21-23 Nov. 1995
  pp.353-358
• 6 Li, Q., Zheng, N., and Cheng, H.,
  Springrobot A Prototype Autonomous Vehicle and
  Its Algorithms for Lane Detection, IEEE
  Transactions on Intelligent Transportation
  System, vol. 5, no. 4, Dec., 2004, pp.300-308.
• 7 Coulombe, J., Sawan, M. and Wang, C.,
  Variable Resolution CMOS Current Mode Active
  Sensor, IEEE International Symposium on Circuits
  and Systems, vol. 2,  28-31 May, 2000, pp.293
  296.
• 8 Tabet, M., Hornsey, R., CMOS Image Sensor
  Camera with Focal Plane Edge Detection, Canadian
  Conference on Electrical and Computer
  Engineering, vol. 2,  13-16 May, 2001, pp.1129
  1133.

33
Reference (3/4)

• 9 Pei-Yung Hsiao, Yu-Chun Hsu, Wen-Ta Lee,
  Chia-Chun Tsai, and Chia-Hao Lee, An Embedded
  Analog Spatial Filter Design of The Current-Mode
  CMOS Image Sensor, IEEE Transactions on Consumer
  Electronics, vol. 50, no. 3, Aug., 2004,
  pp.945951.
• 10 N. Yang and G. Jianhong, A 256x256 Pixel
  Smart CMOS Image Sensor for Line Based Stereo
  Vision Applications, IEEE Journal of solid state
  circuits, vol. 35, no. 7, July, 2000,
  pp.1055-1061
• 11 Yuan, J. and Svensson, C., High-speed CMOS
  circuit technique, IEEE J. Solid-state Circuits,
  vol. 24, no. 2, 1989, pp. 62-70.
• 12 Byungsoo Chang, Joonbae Park and Wonchan
  Kim,A 1.2 GHz CMOS dual-modulus prescaler using
  new dynamic D-type flip-flops, IEEE Journal of
  Solid-State Circuits, vol. 31, no. 5, May, 1996.
  pp.749-752

34
Reference (4/4)

• 13 Fish, A and Yadid-Pecht, o.,CMOS
  current/voltage mode winner-take-all circuit with
  spatial filtering, The IEEE International
  Symposium on Circuits and Systems, vol. 3, 6-9
  May, 2001, pp.636-639.
• 14 Nakamura, J., Pain, B., Nomoto, T.,
  Nakamura, T. and Fossum, E.R., On-focal-plane
  signal processing for current-mode active pixel
  sensors, IEEE Transactions on Electron Devices,
  vol. 44,  no. 10,  Oct., 1997, pp.1747 1758