An Object Oriented Approach To Bio-Feedback Applications For Disabled People

1
An Object Oriented Approach To Bio-Feedback
Applications For Disabled People
ICBEM 2000
Luigi Bianchi1,5, Fabio Babiloni2, Febo
Cincotti3, Serenella Salinari4, Maria Grazia
Marciani 1,3 1 Dip. Neuroscienze, Università
Tor Vergata, Rome, ITALY 2 Dip. Fisiologia
Umana e Farmacologia, Università La Sapienza,
Rome, ITALY 3 IRCCS, Fondazione S. Lucia, Rome,
ITALY 4 Dip. Informatica e Sistemistica,
Università La Sapienza, Rome, ITALY 5
Brainware, Rome, ITALY
E-mail icbem_at_luigibianchi.com
2
Introduction (1)
ICBEM 2000
One of the main problems encountered in the
development of computer-based systems for
handicapped people is that it is very difficult
to optimize them in a wide range of situations.
This happens mainly because every potential user
has residual capabilities that are specific to
his condition and that make him in some way
unique. It should be extremely useful to use all
of the voluntarily controlled activities (VCA)
such as eye movements, muscle contractions, EEG
activity, etc. as inputs for a processing unit
that could take care of recognizing them and then
translating them into desired tasks like opening
doors, playing-back some pre-recorded phrases,
using the telephone and so on. Unfortunately,
building a system for even a well-defined
pathology might be difficult optimal use of
residual capabilities in different subjects could
require modifying it at a level that is
incompatible with practical needs.
3
Introduction (2)
ICBEM 2000
However, while the number and type of the
voluntarily controlled biological signals is
specific to every patient, the number of tasks
that one wants to execute is quite homogeneous.
Furthermore, even if the nature of the utilized
signals may vary among different pathological
situations, the way in which a biofeedback system
works is quite stereotyped after the data have
been acquired, there is a DSP pre-processing
stage that performs some basic operations on the
input signals, then a classification stage in
which some features are extracted (FE),
manifested in some way to the user and eventually
attributed to one of the subject VCA
(discriminator), and finally, a stage in which a
task can be executed.
4
Introduction (3)
ICBEM 2000
Finally it is very frequent that these systems
provide different operating modalities such as
training, testing, setup, and running. This is a
situation in which an object oriented programming
(OOP) approach reaches his best results
inheritance, virtual functions and templates are
all instruments created to reuse code and to
allow generic programming. In our case it is
possible to describe the operative flow that is
common to all biofeedback applications leaving to
be defined only those aspects that are specific
to the single implementation, such as the
algorithms and the classification rules. Then, in
a separate step, several systems can be realized
just defining the points that are intentionally
left unspecified. Here we describe an object
oriented software framework that can be used as a
skeleton for the practical implementation of a
wide range of Bio-feedback systems. It is
important to notice that the proposed solution
does not make any assumption on both the
operating system and the hardware used.
5
Bio-Feedback System
ICBEM 2000

• A Bio-Feedback system is usually composed of
  various hardware and software modules that
  interact among themself and that implement
  different tasks. The main sub-systems are
• A Data Acquisition Board that collects biological
  signals
• A hardware that receives collected data (e.g. a
  Personal Computer) and that interacts with the
  environment, driving special devices
• Some algorithms that extract the desired features
  from the biological signals
• Different operating modalities, such as training,
  exercise, running.
• Several supporting utilities for configuring the
  system in all the aspects.

6
Modalities (1)
ICBEM 2000

• Another aspect that is always encountered in
  different systems is the fact that they must
  operate in more modalities such as training,
  testing, running and setup. The training modality
  is relative to a period in which some parameters
  relative to the user are extracted in order to
  tune the system to him. During this phase, one
  asks to the subject to perform an action several
  times in order to collect enough reference data
  that will be used later on by the classifier.
  This procedure is repeated for all the VCAs that
  the patient intends to use.

 
Ref
1
a) training
The running modality is devoted to the
identification of a VCA and to the triggering of
an event that might start an action in the case
of a successful classification.
Ref
2
...
VCA 1
Ref
m
VCA 2
...
DSP
ACQ
F.E.
VCA m
b) running
Ref
1
Ref
2
Task
1
...
VCA 1
state
Task
2
Ref
m
VCA 2
...
Task
...
DSP
ACQ
F.E.
Discr.
driver
Task
n
VCA m
7
Modalities (2)
ICBEM 2000
The testing modality is somehow in the middle
between the previous two the system asks to the
subject to perform one of the VCAs, and then try
to classify it. After that, an internal score is
updated and an event is triggered in order to
signal a success or a failure. In this way it is
possible to evaluate the performance of the whole
system for every single performed task. The
setup modality is relative to the hardware
configuration and to the loading and saving of
the references and other parameters computed
during previous training sessions.
8
Algorithms
ICBEM 2000
Even if Bio-Feedback systems are mainly
characterized by the classification rules and the
feature extraction algorithms, most of them share
the same DSP engines at least at a pre-processing
stage. This includes FFT computation and digital
filtering. Furthermore, advanced matrix
computation routines are commonly required for
the complete implementation of a particular tool
such as the ones based on Signal Space Projection
or on the Fisher Discriminant Linear
Classifiers. Other algorithms which are based on
Neural Networks, still need a pre-processing
stage for DSP. Anyway, the workflow of all these
systems are still the one illustrated in the
introduction section, or at least it is easily
re-conducible to it.
9
Hardware-Data Acquisition
ICBEM 2000

• Actually there are several implementations for
  the realization of the various systems. They are
  mainly characterized by
• Platform used (PC, Macintosh, Palm, proprietary,
  etc)
• Operating System (Win32, WinCE, MacOS, Linux,
  Unix, BeOS, proprietary, etc)
• A/D and D/A converter (proprietary, National
  Instruments, Analog Devices, etc)
• Biological signal treated (EMG, EEG, Voice,
  Breath, etc)
• Amplifiers
• I/O protocols and peripheral (RS232, microphones
  and speakers, TTL triggers, digital cameras, USB,
  IEEE488, FireWire, etc)

10
A modern approach
ICBEM 2000

• A different way to realize a versatile and easily
  configurable Bio-Feedback system is to make few
  assumptions about it. In particular it must
• be an open architecture
• operate with a high hardware abstraction level
• be easily configurable
• be easily customizable
• maximize the software source code reuse
• maximize the software binary code reuse
• interact with the tools and devices already
  available
• facilitate the diffusion of new algorithms.

11
A modern Answer (1)
ICBEM 2000
Object Oriented Programming can give a lot of
answers to the previously listed key-points. Out
implementation uses C for the following
reasons C compatibility, in which almost all the
operating system are written Speed, because c
generated binary code is generally faster than
the one generated by other languages ones such as
Java Diffusion, because it is the most widely
used Hardware control, because the vast majority
of peripherals and IO board are easily driven in
C Language features, such as virtual functions,
inheritance, template, that allows generic
programming Cross-platform portability, because
it is possible to re-compile an ANSI C program
under almost every platform.
12
A modern Answer (2)
ICBEM 2000
Another key issue is that it is fundamental to
keep separate the hardware IO aspects from the
implementation of the different algorithms. This
goal can be achieved by making a simple
assumption the acquired data must be stored on a
buffer whose size is large enough to hold the
data required for the real-time processing. It is
not important how the data are collected it is
sufficient to notify in some way when and where
new biological acquired data are available. Other
relevant information are related to every single
acquired channel, such as gain, sampling rate,
and coordinate position. This last can be used by
special algorithms such as those based on
Laplacian estimation for example in some Brain
Computer Interface (BCI) systems.
13
A modern Answer (3)
ICBEM 2000
To solve all these problems, a hierarchy of
classes have been implemented, in order to keep
into account different requirements. The first
class is the BFEnv, which implements the
operative flow of an Bio-Feedback application as
described in the introduction. All the timing
aspects are treated here, and special virtual
functions such as DecisionRule() and
ComputeAlgorithm() must be defined to
characterize a specific system. Other virtual
functions such as OnSuccess() and OnFailure() are
automatically triggered every time a recognition
based on the specific DecisionRule and
ComputeAlgorhitm implementations occurs. Another
object derived from BFEnv, called BFSpeEnv, adds
some FFT capabilities, while BFAdvEnv gives also
matrix computation routines. Note that all the
source described here is cross-platform and to be
considered freeware for any non commercial use
and can be obtained at www.luigibianchi.com.
14
SSP implementation (1)
ICBEM 2000
MS Windows
Linux
Bio-Feedback Software Development Kit (with
matrix computation and basic DSP routines)
SSP
Mac OS
BeOS
At this point we have decided to implement a
Signal Space Projection (SSP) algorithm. In order
to do that, we have decided to adopt MS Windows
as operating system, and the EEG signal as
biological input.
15
SSP implementation (2)
ICBEM 2000
Anyway, the implementation was performed into two
separate levels the first one for all the
necessary processing, including the DecisionRule
and ComputeAlgorithm implementation (and this
still save the cross-platform compatibility) the
second one to add all aspects that are dependent
on the operating system used such as the Graphic
User Interface (GUI) and other external tool
support such as Text to Speech, Voice Command
capabilities (SAPI 4.0) and Dual Monitor support.
This last feature was necessary to give to
different views to the user (cursor movement) and
to the technician (full EEG signal display with
real-time 3D spectral maps) In an immediate
future step, accessibility options integrated in
all the Win32 platform will be also used. All
the implementations were done using Borland
CBuilder 4 compiler.
16
SSP implementation (3)
ICBEM 2000
It is important to note that all the
implementation without the GUI and some other OS
specific aspects, required less that 200 lines of
C code, so divided 20 lines for pseudo-inverse
matrix computation 30 lines to implement the
DecisionRule and the ComputeAlgorithm
functions 60 lines for constructors,
destructors, allocation and de-allocation 80
lines for other things, including dual monitor
support, in order to give to different views to
the user (cursor movement) and to the technician
(full EEG signal display with real-time 3D
spectral maps)
17
SPP Screenshot
ICBEM 2000
18
Conclusions
ICBEM 2000
A model for a generic biofeedback system was
described. Its main advantages are the net
distinction of the functional blocks that are
relative to the hardware, the input signals, the
algorithms and the type and degree of
disabilities. In this way it is possible to reuse
a huge amount of code, thus minimizing the time
required in the development and adaptation of a
system to a wide number of patients.