Biology Inspired Approximate

1
Biology Inspired Approximate Data Representation
for Signal Processing, Soft Computing and
Control Applications
Emil M. Petriu, Dr. Eng., FIEEE School of
Information Technology and Engineering University
of Ottawa Ottawa, ON., K1N 6N5 Canada http//www.s
ite.uottawa.ca/petriu
WISP2007 - IEEE Int. Symposium on Intelligent
Signal Processing, Alcalá de Henares, Spain, 3-5
Oct.2007
2
Abstract
This paper reviews basics, similarities, and
applications of two biology inspired approximate
data representation modalities stochastic data
representation and fuzzy linguistic variables.
3
Stochastic Data Representation
4
Biological Neurons
5
Looking for a model to prove that algebraic
operations with analog variables can be performed
by logic gates, von Neuman advanced in 1956 the
idea of representing analog variables by the mean
rate of random-pulse streams J. von Neuman,
Probabilistic logics and the synthesis of
reliable organisms from unreliable components,
in Automata Studies, (C.E. Shannon, Ed.),
Princeton, NJ, Princeton University Press, 1956.
6
The random-pulse machine concept, S.T.
Ribeiro, Random-pulse machines, IEEE Trans.
Electron. Comp., vol. EC-16, no. 3, pp.
261-276,1967, a.k.a. "noise computer,
"stochastic computing, dithering deals with
analog variables represented by the mean rate of
random-pulse streams allowing to use digital
circuits to perform arithmetic operations. This
concept presents a good tradeoff between the
electronic circuit complexity and the
computational accuracy. The resulting neural
network architecture has a high packing density
and is well suited for very large scale
integration (VLSI).
7
Analog/Random-Pulse Conversion
8
Random-Pulse/Digital Conversion
The deterministic component of the random-pulse
sequence, conveniently unbiased and rescaled for
this purpose to take values 1 and -1 (instead of
1 and respectively 0) , can be calculated as a
statistical estimation from the quantization
diagram EVRP (1) .pVR0 (-1)
.pVR
(FSV)/(2.FS) - (FS-V)/(2.FS ) V/FS This
finally gives the deterministic analog value V
associated with the binary VRP sequence V
p(VRP) - p(VRP') . FS where the apostrophe
( ' ) denotes a logical inversion.
9
Random-pulse/digital converter using the moving
average algorithm .
10
Analog/random-pulse and random-pulse/digital
conversion of a step input signal
11
Random-Pulse Addition
12
Random-pulse addition
13
Random-pulse multiplication
14
Stochastic Data Representation
Generalized b-bit analog/stochastic-data
conversion and its quantization characteristics
15
The ideal estimation over an infinite number of
samples of the stochastic data sequence VRD is
EVRD (k-1). p(k-1.5)D VR
k . p(k-0.5)D VR
(k0.5)D (k-1) . b k . (1-b)
k -b The estimation accuracy
of the recovered value for V depends on
the quantization resolution, the finite number of
samples that are actually averaged, and on the
statistical properties of the analog dither.
16
(No Transcript)
17
Mean square errors function of the moving average
window size
18
Stochastic-data addition.
19
2-bit stochastic-data multiplier.
20
Example of 2-bit stochastic-data multiplication.
21
Correlator Architectures Using Stochastic Data
Representation
22
(No Transcript)
23
Parallel architecture of a 4-point correlator
using random-pulse data representation.
24
Each calculated correlation point has a 8-bit
resolution. In order to reduce the number of
interface lines, the 8-bit wide outputs of
the four random-pulse/digital converters are
multiplexed. When more modules are connected in
larger structures the 1-bit XRQ input lines of
all modules are connected together while the
1-bit YRQ output of each module is connected to
the 1-bit YRQ input of the next module creating
a longer delay line.
25
Autocorrelation function of a sinusoid calculated
by a 32-point parallel random-pulse correlator
with a 8-bit/point resolution.
26
Correlation function of two sinusoids with the
same frequency but different amplitudes and
phases, calculated by a 32-point parallel
random-pulse correlator with a 8-bit/point
resolution.
27
Correlation function between a sinusoid and
another sinusoid of the same frequency but
corrupted by white noise, calculated by
a32-point parallel random-pulse correlator with
8-bit/point resolution.
28
Neural Network Architectures Using Stochastic
Data Representation
29
Neuron structure
30
Multi-bit stochastic-data implementation of a
neuron body.
Multi-bit stochastic-data implementation of a
synapse
31
Auto-associative memory NN architecture
Recovery of 30 occluded patterns
Training set
32
Fuzzy Logic Control
33
Fuzzy Logic
Pioneered by Zadeh in the mid 60s fuzzy logic
provides the formalism for modeling the
approximate reasoning mechanisms specific to the
human brain. In more specific terms, what is
central about fuzzy logic is that, unlike
classical logical systems, it aims at modeling
the imprecise modes of reasoning that play an
essential role in the remarkable human ability to
make rational decisions in an environment of
uncertainty and imprecision. This ability
depends, in turn, on our ability to infer an
approximate answer to a question based on a store
of knowledge that is inexact, incomplete, or not
totally reliable. Fuzzy Logic, IEEE Computer
Magazine, April 1988, pp. 83-93
34
Fuzzy Logic Control

• The basic idea of fuzzy logic control (FLC) was
  suggested by L.A. Zadeh, A rationale for fuzzy
  control, J. Dynamic Syst. Meas. Control,
  vol.94, series G, pp.3-4,1972.
• FLC provides a non analytic alternative to the
  classical analytic control theory. But what
  is striking is that its most important and
  visible application today is in a realm not
  anticipated when fuzzy logic was conceived,
  namely, the realm of fuzzy-logic-based process
  control, L.A. Zadeh, Fuzzy logic, IEEE
  Computer Mag., pp. 83-93, Apr. 1988.

Early FLCs were reported by Mamdani and Assilian
in 1974, and Sugeno in 1985.
35
Classical control systems are based on a detailed
I/O function OUTPUT F (INPUT) mapping each
high-resolution quantization interval of the
input domain into a high-resolution quantization
interval of the output domain
36
Fuzzy control is based on a much simpler
functional description of the desired I/O
behavior mapping each low-resolution quantization
interval of the input domain into a
low-resolution quantization interval of the
output domain
37
Membership functions and the quantization
characteristics for a 3-set (N, Z, and P) fuzzy
partition of the domain where the analog variable
x is defined. XFQ are the crisp analog values
recovered after a defuzzification of the fuzzy
converted value x. It also shows the truncated
information XQ recovered from an A/D converter
with 3 quantization levels defined over the same
domain for x.
38
The key benefit of FLC is that the desired system
behavior can be described with simple if-then
relations based on very low-resolution models
able to incorporate empirical engineering
knowledge. FLCs have found many practical
applications in the context of complex
ill-defined processes that can be controlled by
skilled human operators water quality control,
automatic train operation control, elevator
control, etc.,
39
Fuzzy Controller for Truck and Trailer Docking
40
INPUT MEMBERSHIP FUNCTIONS
OUTPUT MEMBERSHIP FUNCTIONS
SLOW MED FAST
41
(No Transcript)
42
DEFFUZIFICATION
The crisp value of the steering angle is obtained
by the modified centroidal deffuzification
(Mamdani inference)
I/O characteristic of th Fuzzy Logic Controller
for truck and trailer docking.
43
STOCHASTIC-DATA FUZZY LOGIC CONTROLLERS
There is tenet of common wisdom that FLCs are
meant to successfully deal with uncertain data.
According to this, FLCs are supposed to make do
with uncertain data coming from (cheap)
low-resolution and imprecise sensors. However,
experiments show that the low resolution of the
sensor data results in rough quantization of of
the controller's I/O characteristic
4-bit sensors
7-bit sensors
I/O characteristics of the FLC for truck
trailer docking for 4-bit sensor data (a, b, g)
and 7-bit sensor data.
44
The truck backing-up problem
45
Membership functions for the truck backer-upper
FLC
46
The FLC is based on the Sugeno-style fuzzy
inference. The fuzzy rule base consists of 35
rules.
47
Time diagram of digital FLC's output q during a
docking experiment when the input variables, j
and x are analog and respectively quantizied with
a 4-bit bit resolution
48
FLC architecture using 4-bit stochastic data
representation with low-pass filters placed
immediately after the input A/D converters
49
It offers a better performance than the previous
one because a final low-pass filter can also
smooth the non-linearity caused by the min-max
composition rules of the FLC.
FLC architecture using 4-bit stochastic data
representation with low-pass filters placed at
the FLCs outputs.
50
Time diagram of the stochastic FLC's output
during a docking experiment when 4-bit A/D
converters are used to quantize the dithered
inputs and the low-pass filter is placed at the
FLC's output
51
Truck trails for different FLC architectures (a)
analog (b) digital without dithering (c)
stochastic data representation with uniform
dithering and 20-unit moving average filter
52
(No Transcript)
53
(No Transcript)
54

• Conclusions
• Due to its relatively low hardware complexity
  and high internal noise immunity, the von Neumann
  stochastic data representation represents an
  attractive alternative to the analog and high
  resolution digital data processing techniques for
  many statistical signal processing and soft
  computing applications.
• Because of the smooth linear transitions in
  the membership of overlapping fuzzy sets, the
  fuzzy partition of the analog FLC inputs will not
  introduce any quantization noise. However,
  digital FLCs cannot make do with low-resolution
  data. It is shown that dithering can offer a
  solution to significantly improve the resolution
  of the reduced word-length digital FLCs.

55
Acknowledgement This paper represents a
synthesis of the work carried out by the author
and his collaborators published over the years in
conference proceedings and journals

• E. Pop, E. Petriu, Influence of Reference
  Domain Instability Upon the Precision of Random
  Reference Quantizer with Uniformly Distributed
  Auxiliary Source, Signal Processing, North
  Holland, Vol. 5, 1983, pp. 87-96.
• G. Eatherley, Autonomous Vehicle Docking Using
  Fuzzy Logic, M.A.Sc. Thesis, 1994.
• G. Eatherley, E.M. Petriu, "A Fuzzy Controller
  for Vehicle Rendezvous and Docking," IEEE Trans.
  Instr. Meas., Vol. 44, No. 3, pp. 810-814, 1995.
• E.M. Petriu, G. Eatherley, Fuzzy Systems in
  Instrumentation Fuzzy Control, Proc. IMTC/95,
  IEEE Instr. Meas. Technol. Conf., pp.1-5,
  Waltham, MA, 1995.
• E. Petriu, K. Watanabe, T. Yeap,
  Applications of Random-Pulse Machine Concept to
  Neural Network Design, IEEE Tr.. Instr. Meas.,
  Vol. 45, No.2, 1996, pp. 665-669.
• L. Zhao, Random Pulse Artificial Neural
  Network Architecture, M.A.Sc. Thesis, University
  of Ottawa, Canada, 1998
• J. Mao, Reduction of the Quantization Error
  in Fuzzy Logic Controllers by Dithering, M.A.Sc.
  Thesis, University of Ottawa, Canada, 1998.
• E.M. Petriu, J. Mao, Fuzzy Sensing and
  Control for a Truck, Proc. VIMS-2000, IEEE
  Workshop on Virtual and Intelligent Measurement
  Systems, Annapolis, MD, April 2000, pp. 27-32.
• E. M. Petriu, L. Zhao, S.R. Das, V.Z. Groza,
  A. Cornell, Instrumentation Applications of
  Multibit Random-Data Representation, IEEE Tr.
  Instr. Meas., Vol. 52, No. 1, 2003, pp. 175- 181.
• M. Dostaler, Multi-Level Random Data Based
  Correlator Model, M.A.Sc. Thesis, 2005.

56
Acknowledgement The work reported in this paper
was funded in part by Communications and
Information Technology Ontario (CITO) and the
Natural Sciences and Engineering Research Council
(NSERC) of Canada.
57
Thank you!