Adaptive Filtering and Data Compression using Neural Networks in Biomedical Signal Processing

1
Adaptive Filtering and Data Compression using
Neural Networks in Biomedical Signal Processing

• T-61.181 Biomedical Signal Processing
• 2.12.2004

2
Contents

• Neural Networks
• What are neural networks and where they are
  used?
• Properties of a neural network
• Network topologies
• Learning
• Adaptive filtering
• What is adaptive filtering and when is it
  needed?
• Common algorithms used in adaptive filtering
• Adaptive filtering with neural networks
• Two example cases
• Data Compression
• What is data compression and when is it needed?
• Lossless and lossy compression
• Data compression with neural networks
• An example case

3
Neural Networks

• A neural network is a massively parallel
  distributed
• processor made up of simple processing units
  1
• Why?
• (Artificial) neural networks (ANNs) offer
  several useful properties and capabilities e.g.
  nonlinearity, adaptivity, input-output mapping,
  VLSI implementability etc.
• Where?
• Neural networks can be used in many application
  areas
• for example in
• Pattern recognition
• Vehicle and robot control systems
• Data compression
• Adaptive filtering

4
Neural Networks - Basic Concepts

• The basic building block of a neural network is a
  single
• perceptron, which has many similarities with a
  biological
• neuron

Activation function
weights
summation
Inputs
Output
For more complex problems, which are encountered
in practice, a multilayer network is required.
5
Input Signal and Weights

• Input signals
• An input may be either a
• raw / preprocessed signal or
• image. Alternatively, some
• specific features can also be
• used.
• If specific features are used
• as input, their number and
• selection is crucial and
• application dependent

Weights Weights are connected between an input
and a summing node. These affect to the summing
operation. The quality of network can be seen
from weights Bias is a constant input with
certain weight. Usually the weights are
randomized in the beginning
6
Activation Function and Output
Summation This is just a weighted sum (linear
combination)
Output This depends on the activation
function. There may be several outputs
Activation Function There are different
activation functions used in different
applications. The most common ones are
Hard-limiter
Piecewise linear
Sigmoid
Hyperbolic tangent
7
Different Neural Networks
Neural networks can be characterized by the
topology of the network. Single layer
feedforward networks Multilayer feedforward
networks (hidden layers) Recurrent networks
(feedbacks)
Networks may be either partially or fully
connected
A fully connected, multilayer feedforward neural
network
8
Learning
The learning can be supervised (e.g.
backpropagation), unsupervised (e.g. competitive
learning) or reinforcement learning
The backpropagation-algorithm is probably the
most common algorithm used for training
neuron j is an output node
neuron j is a hidden node
There is some optimal point where the training
should be stopped in order not to overtrain the
network.
9
Adaptive Filtering

• Why?
• If the properties of noise change in time
• If the frequencies of the signal and the noise
  overlap
• How?
• By changing the system
• parameters according to
• some algorithm
• Where?
• Few biomedical application examples 36
• Fetal ECG extraction (ECG signal)
• 50 Hz mains (ECG-signal)
• Eye movement artefacts (EEG-signal)

10
Adaptive Filtering - Basic Concepts
In adaptive filters, there are many different
algorithms used for computing the weights. These
are the two common ones 6
Least Mean Square (LMS)
Recursive Least Squares (RLS)
Most effective in terms of computation
storage requirements
Superior convergence properties
learning parameter
forgetting factor
recursive way to compute the inverse matrix
11
Adaptive Filtering with Neural Networks

• Why neural networks?
• High-speed computational capacity. The
• weight coefficients can be computed in
• real-time 5
• Any nonlinear filter can be realized with by
• an MLP-network. Used if noise is
• non-Gaussian or not additive 5

LMS Can be implemented with one linear neuron
RLS Computational complexity is reduced
O(N n ) O(N) and the result of the
algorithm can be calculated in
real-time 5
2
2
12
Adaptive Filtering with Neural Networks (2)
The weights of a neural network are usually
updated with backpropagation algorithm or
temporal backpropagation algorithm 1
1) Propagate input signal through the network and
calculate error e(n)
2) Neuron j in the output layer
A a group of neurons connected to the neuron j
3) Neuron j in a hidden layer
l hidden layer index (from output layer)
The filter coefficients are actually the weights
of the neurons The response of the adaptive
filter is affected by weights of the
neurons thresholds used activation functions
13
Example Case (ECG) Fetal ECG Extraction
Problem The spectra of mother and fetal signals
overlap each other. In addition the amplitude of
MECG is much higher 9
14
Example Case (EEG) Visual Evoked Potential
Estimation
Problem The number of VEPs required in ensemble
averaging is large due poor SNR. The averaging
smoothes differences (amplitude latency)
between VEPs 10
The VEP signals have a non-linear behaviour,
which can be matched better with NNs than
conventional adaptive filters. As a result, fewer
VEPs are required 10
15
Data Compression

• The goal of data compression is to represent an
  
• information source as accurately as possible
  using
• the fewest number of bits. 1
• Why?
• Measured raw data requires a lot of
• memory / channel capacity.
• How?
• Redundancies in the data are exploited
• Where?
• There are many applications for example in
• image video signal,
• speech
• biomedical signal processing

16
Data Compression Basic Concepts

• Compression rate, measures how much the signal
  can be compressed
• from the original one.

Compression methods which are used can be
lossless (e.g. RLC), lossy (e.g. JPEG) or both.
Lossy compression
High compression rates can be achieved (e.g.
28,67)
Too high compression ratio results in distortions
The exact replica of an original signal cannot be
recovered anymore
Size 10,5 MB
Size 375 kB
The compression of ECG signal must always be
lossy in order to be efficient enough
17
Data Compression Methods
There are many ways for compressing data, for
example
Linear Predictive Coding (LPC) A sample s(n) is
approximated with a linear combination of
previous samples
Principal Component Analysis (PCA) Gives the
optimal solution in the field of data
compression. It projects the d-dimensional data
into a lower dimension in a optimal way
(sum-squared error sense). The d eigenvectors
have the largest variances
18
Data Compression with Neural Networks
Why neural networks?
High-speed computational capacity. The
compression can be done in real-time 5 The
nonlinear mapping properties of an MLP can be
used to implement nonlinear predictors in
prediction coding. This is much more powerful
than using linear predictors 511 Experimen
tal knowledge of the networks can be exploited
5
LPC Solution can be provided at much higher
speed.
PCA Adaptive PCA extraction, more simple to
calculate 5
19
Data Compression with Neural Networks
The neural networks used in data compression have
massively parallel structures and high-degree of
interconnections 5
The compression ratio depends on the ratio of
neurons on input layer and on hidden layer.
Output layer is as big as input layer 13
The actual compressed data is obtained from the
weights and activation levels of the network 12
20
Example Case (ECG)
Problem The long time (Holter) monitoring of ECG
requires a lot of memory.
fs 250 Hz ADC 11 bits time 24 h 226,6
Mbits
BPNN (Back Propagation Neural Network)
PCANN (Principal Component Analysis Neural
Network)
21
Example Case (ECG) (2) - Results
The compression rate of 33 is achievable with
this method
PRD (Percent Root Mean Square Difference)
22
Summary (1) Neural Networks

• They offer many advantages e.g. non-linearity,
  VLSI
• implementability and adaptivity
• The basic building block of a network is a
  perceptron,
• which resembles a physiological neuron
• To generate an output, a weighted sum from the
  inputs of
• the network is calculated and then an activation
  function is
• used.
• There are many different network
• topologies for different applications
• Neural networks can learn (adaptability)
• either in a supervised or unsupervised
• manner. Many different learning
• algorithms have been developed.

23
Summary (2) Neural Networks in Adaptive Filtering

• Adaptive filtering is needed when spectra of
  signal and
• noise overlap
• Neural networks can be used
• effectively for adaptive filtering,
• since they adapt to nonlinear and
• time-varying features of a signal.
• The filter weights adapt faster with NNs than
  with common
• adaptive filter algorithms. Also signal artefacts
  are handled
• better 7.
• The exact algorithm knowledge is not required,
  thus any
• non-linear adaptive filter is implementable with
  NNs
• Today the research is more focused on wavelets

24
Summary (3) Neural Networks in Data Compression
Data compression is often needed because of
limited storage capacity (memory) or transmission
bandwidth
Neural networks can be used to data compression
and their advantages over traditional methods
are very fast, nonlinear operation and learning
capability.
As disadvantages, they must be teached for each
patient separately, which takes time and the
resulting compression is lossy.
The neural network structure is highly parallel
and interconnected in data compression
applications.
25
References

• 1 S. Haykin, Neural Networks 2nd Ed.1990
• 2 Data Compression.com
• lthttp//www.data-compression.com/index.shtmlgt
  11.11.2004
• 3 D. Perez, Adaptive Filtering, University of
  Strathclyde
• lthttp//www.spd.eee.strath.ac.uk/interact/AF/aftu
  torial/apps/appsindex.htmlgt 11.11.2004
• 4 V. Sebesta, The Utilization of Neural
  Networks in
• Medical Informatics
• lthttp//new.euromise.org/english/material/95102023
  /sebesta.htmlgt 15.11.2004
• 5 F. Luo, Applied Neural Networks for Signal
  Processing, 1997
• 6 E.Ifeachor, Digital Signal Processing
  Practical Approach, 1997

26
References (2)

• 7 A.Yilmaz M.J.English, Adaptive Non-Linear
  Filtering of ECG Signals Dynamic Neural Network
  Approach, Artificial Intelligence Methods for
  Biomedical Data Processing, IEEE
• Colloquium on , 26 Apr 1996
• 8 M.B.I. Reaz L.S. Wei, An Approach of Neural
• Network Based Fetal ECG Extraction. Enterprise
• Networking and Computing in Healthcare Industry,
  2004. HEALTHCOM 2004. Proceedings. 6th
  International Workshop on , 28-29 June 2004
• 9 G. Camps, M. Martinez E. Soria, Fetal ECG
  Extraction Using an FIR Neural Network. Computers
  in Cardiology 2001, 23-26 Sept. 2001

27
References (3)

• 10 K.S.M. Fung, F.K. Lam, F.H.Y. Chan, P.W.F.
  Poon, J.G.Liu,
• Adaptive Neural Network Filter for Visual
  Evoked Potential
• Estimation. Neural Networks, 1995.
  Proceedings., IEEE
• International Conference on  ,Volume 5 , 27
  Nov.-1 Dec.
• 1995
• 11 R. Kannan, C. Eswaran, N. Sriraam, Neural
  Network Based
• Methods for ECG Data Compression. Neural
  Information
• Processing, 2002. ICONIP '02. Proceedings of
  the 9th
• International Conference on , Volume 5
  , 18-22 Nov. 2002
• 12 A. Iwata, Y. Nagasaka, N. Suzumura, Data
  Compression of
• the ECG Using Neural Network for Digital
  Holter Monitor.
• Engineering in Medicine and Biology Magazine,
  
• IEEE, Volume 9 , Issue 3 , Sept. 1990

28
References (4)

• 13 Y. Nagasaka, A. Iwata, Performance
  Evaluation of BP and
• PCA Neural Networks for ECG Data Compression.
  Neural
• Networks, 1993. IJCNN '93-Nagoya. Proceedings
  of 1993
• International Joint Conference on , Volume 1
  , 25-29 Oct.
• 1993