A wearable BrainComputer Interface

1
A wearable Brain-Computer Interface

• Master Thesis Presentation
• By
• Payam Aghaeipour
• November 08

2
Story begins

• 10 months ago
• University of Sydney
• Web Engineering Group (EE Department)
• http//www.weg.ee.usyd.edu.au/index.html
• Supervisor Dr. Rafael A. Calvo
• Examiner Dr. Peter Sjödin

3
Outline

• BCI Background
• Goals of Wearable BCI
• Existing Solutions (their limitations)
• Architecture
• Mobile BCI (Video Demonstration)
• Implementation challenges
• Analysis of Results
• Conclusion and Future Work

4
What is BCI?

• Brain-Computer Interfaces (BCI)
• Interaction between the human neural system and
  machines
• Goal
• Enabling people (especially disabled) to
  communicate and control devices by mere thinking.
• BCI is a control system

5
BCI Components, Signal Acquisition

• Brain signals can be collected in different ways,
  one of these methods is EEG (Electroencephalograph
  y)
• Non-invasive
• Mu Rhythms
• In awake people, even when they are not producing
  motor output, motor cortical areas often display
  812 Hz EEG activity (Mu Rhythm)
• Movement or preparation for movement typically
  causes a decrease in mu rhythms (motor imagery)
• Right versus left hand
• Right hand versus tongue
• Left hand versus foot

6
BCI Components, Signal Processing

• Feature Extraction
• The Translation Algorithm
• These algorithms adapt to each user on 3 levels
• First time, the algorithm adapts to the users
  signal features
• Periodic online adjustments (tired, happy, etc)
• The adaptive capabilities of the brain. The brain
  has the ability to modify the signals features to
  improve BCI operation!

7
BCI Components, Output Device

• Any controllable machines
• For answering yes/no questions
• For word processing at slow
• Wheelchair
• Virtual Reality
• Usually, Computer screen and the output is the
  selection of targets or cursor movement

8
BCI Categories

• Dependent Vs Independent BCI
• Dependent BCI generation of the EEG signal
  depends on the other physical movement (e.g. gaze
  direction)
• Independent BCI only depends on the users intent
• Synchronous Vs. Asynchronous BCI
• Synchronous EEG signals are processed in fixed,
  predefined time windows
• Asynchronous Continuous sample-by-sample
  analysis and feature extraction (more complex)
• No output signal during the resting or during
  unintentional brain states
• Our System EEG, Independent and Synchronous

9
Wearable BCI

• Mobility
• Communication technologies
• Bluetooth
• 802.11
• GSM/GPRS
• PDA instead of stationary computer
• Dry Electrode instead of wet (reducing montage
  time)
• Making the BCI transparent
• No need to change electrodes for a reasonable
  long time

10
Existing Solution

• Successful Story, Wearable BCI
• A successful transition of the whole BCI system
  to the portable device
• No machine learning
• Limited computational power (limited signal
  processing)
• BCI2000
• A general-purpose system for (BCI) research
• Source Module (new device new driver)
• Signal Processing Module (reusable, No Machine
  Learning)
• User Application Module (UDP/IP support to be
  running in any machine)
• Operator Module (controls the whole process)
• Platform
• Microsoft Windows 2000/XP
• C language

11
Our Architecture
12
Mobile BCI Playing Breakout on the mobile phone
13
BluesenseAD

• 8 analog to digital channels
• Sampling frequency up to 4000Hz
• Low power consumption
• Compliance of safety issues for humans brain
• Virtual Serial port
• Bluesense packets are like AT commands (not
  compatible)
• New Driver
• No existing library

14
BluesenseAD Driver Design
15
BluesenseAD Driver Implementation

• New Diver in BCI2000
• C, event driven, serial communication
• Sample Scenario, Connection Establishment

16
Distributed Output Device

• Client/Server Application Module
• BCI2000 provides a way to directly communicate
  with an external device through UDP
• PDA may not support UDP
• Machine learning server (e.g. WEKA) (not inside
  BCI2000)
• Keep computation as little as possible in
  portable device
• Implemented in Java SE (Network Programming)

17
Breakout-Video Game

• Should be Simple
• No distraction ? no strategy
• Green bar
• 2 control signals (left, right)
• Synchronous BCI, Timing
• J2ME
• Development environment ! running environment
• Simulator (NetBeans IDE)
• Both UDP and TCP
• low-level networking support ? MIDP 2.0
  specification
• Multithread
• Sender (always sleep), receiver, game environment

18
BluesenseAD Evaluation and Validation

• Delay in BluesenseAD Driver
• Acceptable delay lt 0.5 Sec
• Signal collection
• A/D conversion
• Transmission
• Receiving and decision by PC and end user program
  (BCI2000)
• Sampling frequency, 128 or 256Hz

19
Delay measurement

• The same triangular signal from the signal
  generator to both
• BluesenseAD
• National Instruments data collector (NI USB-6251)
  
• More than a million samples/sec
• Timestamp which indicates the absolute time
• the sampled has been picked
• The Bluesense driver
• Timestamp that indicates the absolute time
• the data has been received in BCI2000

Delay T1-T2
Bluesense Time, T1
NI Time, T2
20
Finding Extrems

• Averaging near the peaks

21
Delay Parameters

• Sampling Frequency
• Block Size (BCI2000)
• Number of active channels
• Example one active channel
• Exception
• Bluesense Behavior

22
Delay in 4 and 8 active channels

• No exceptions!

23
Summary of Results

• Changes of driver delay based on parameters

24
BluesenseAD Scalability

• More active channels Less sampling frequency
• If sampling rate goes beyond supported value
  corrupted signal
• Low computational power (microcontroller)
• Packet lost (low communication speed)

25
BluesenseAD Scalability (Cont.)

• Maximum sampling frequency for various number of
  sampling channels

26
Video Game, Breakout, Evaluation

• Parameters Definition
• Trial Number of experiment running
• Hit Number of hitting the bar to the green
  indicator
• Failed Number of moving the bar to the opposite
  direction of green indicator
• Aborted Number of experiments that does not lead
  to Hit or Failed
• Rate (Hit Abort)/Trial

27
Method

• EEG
• Four male subjects (20-40 years old)
• No experience using the game
• Little experience using BCI systems
• Sampling rate of 250 Hz
• Using a band-pass filter of 0.1100 Hz (10 dB)
• An amplification ranging from 10 000 to 20 000
• Right and left movements based on mu rhythm
• Focus on moving the left/right arm
• Game experiment/Cursor Task alternatively
• Breakout 20 trials, Cursor Task 2 minutes20
  trials
• 4 sessions each
• The outcome of both experiments was compared to
  show that the Game performance Cursor Task
  outcome

28
First Subject Results

• Game performance Cursor Task outcome
• Can be used in BCI Experiments

29
Conclusions

• Distributed framework
• Controlling the Breakout video game through brain
  signals
• BluesenseAD ? No ribbon cable
• Acceptable delay, sampling frequency
• Reliable
• Robust
• Game Server Application
• TCP support
• less dependent on BCI2000
• Simplicity
• Video Game
• Suitable for BCI experiment
• Can be run on Java-enabled handsets

30
Future Work

• Asynchronous BCI
• Virtual Reality
• Bluesense, Sniff Mode
• Bluesense, Security
• Conference Paper (Accepted)
• Payam Aghaei Pour, Tauseef Gulrez, Omar Al-Zoubi
  and Rafael A. Calvo. Brain-Computer Interface
  Next Generation Thought Controlled Distributed
  Video Game Development Platform. IEEE
  Computational Intelligence and Games Symposium.
  Perth, Australia.

31
Questions

• More Information
• KTH
• http//www.tslab.ssvl.kth.se/thesis/node/901
• University of Sydney
• http//www.weg.ee.usyd.edu.au/projects/penso
• Email
• payama_at_kth.se