Some Mathematical Ideas for Attacking the Brain Computer Interface Problem

1
Some Mathematical Ideas for Attacking the Brain
Computer Interface Problem

• Michael Kirby
• Department of Mathematics
• Department of Computer Science
• Colorado State University

2
Overview

• The Brain Computer Interface (BCI) Challenge
• Signal fraction analysis
• Takens theorem and classification on manifolds
• Nonlinear signal fraction analysis
• Conclusions and future work

3
NSF BCI Group

• Chuck Anderson (PI), Computer Science, Colorado
  State
• Michael Kirby (Co-PI), Mathematics, Colorado
  State
• James Knight, Ph.D. Student, Colorado State
• Tim OConnor, Ph.D. Student, Colorado State
• Ellen Curran, Medical Ethics and Jurisprudence,
  Dept. of Law, Keele University, Staffordshire, UK
  
• Doug Hundley, Consultant, Department of
  Mathematics, Whitman
• Pattie Davies, Occupational Therapy Department,
  Colorado State
• Bill Gavin, Dept. of Speech, Language and Hearing
  Sciences, University of Colorado

Geometric Pattern Analysis and Mental Task
Design for a Brain-Computer Interface
4
SourceForge https//sourceforge.net/projects/csue
eg/

• Development Status 1 - Planning
• Environment Other Environment
• Intended Audience Science/Research
• License GNU General Public License (GPL)
• Natural Language English
• Operating System Linux, SunOS/Solaris
• Topic Artificial Intelligence, Human Machine
  Interfaces, Information Analysis, Mathematics,
  Medical Science Apps.

5
Chuck Anderson
6
Pattie Davies
7
BCI Headlines in the News

• Computers obey brain waves of paralyzed,
  Associated Press, appearing in MSNBC News, April
  6, 2005
• Brainwaves Control Video Games, BBC March 2004
• Brainwave cap controls computer, BBC December
  2004
• Brain Could Guide Artificial Limbs
• Patients put on thinking caps, Wired News,
  January 2005
• Monkey thoughts control computer, March 2002

8
Lou Gehrigs Disease (ALS)

• Amyotrophic Lateral Sclerosis (ALS) , or
  Locked-In Syndrome, is an extreme neurological
  disorder and many patients opt against life
  support.
• Most commonly, the disease strikes people between
  the ages of 40 and 70, and as many as 30,000
  Americans have the disease at any given time.
  (ALS Association website).
• Genetic factors appear to only account for 10
  percent of all ALS cases. ALS can strike anyone,
  anytime.
• There are no effective treatments and no cure.
• Brain activity appears to remain vigorous while
  muscle control atrophies degeneritively and
  completely.

9
Gulf War Veterans and ALS

• The following information is from a news release
  sent out by the Department of Veteran Affairs on
  December 10, 2001.  (ALS Association Web
  posting.)
• According to a news release on December 10, 2001
  from the Department of Veteran Affairs,
  researchers conducting a large epidemiological
  study supported by both the Department of
  Veterans Affairs and the Department of Defense
  have found preliminary evidence that veterans who
  served in Desert Shield-Desert Storm are nearly
  twice as likely as their non-deploying
  counterparts to develop amyotrophic lateral
  sclerosis. 

10
The Brain Computer Interface (BCI)

• A means for communication between person and
  machine via measurements associated with cerebral
  activity, e.g., EEG, fMRI, MEG.
• We assume that no muscle motion is employed such
  as eye twitching or finger movement.

11
Low-Cost EEG
12
History of EEG

• Duboi-Reymond (1848) reported the presence of
  electrical signals
• Caton (1875) measured feeble currents on the
  scalp
• Berger (1929) measured electrical signals with
  EEG
• 1930-50s EEG used in psychiatric and neurological
  sciences relying on visual inspection of EEG
  patterns
• 1960s-70s witness emergence of Quantitative EEG
  and confirmation of hemispheric specialization,
  e.g., left brain verbal and right brain spatial.
• 1980s observation of biofeedback

13
Characteristics of Brainwaves

• Delta waves 0,4 Hz associated with sleep. Also
  empathy.
• Theta waves 4, 7.5 associated with reverie,
  daydreaming, meditation, creative ideas
• Alpha waves 7.5,12 prevalent when alert and
  eyes closed. Associated with relaxed positive
  feelings.
• Beta waves 12Hz associated with active state,
  eyes open.

14
Reasons Why EEG Should Not Work for BCI

• Electrical activity generated by complex system
  of billions of neurons
• Brain is a gelatinous mass suspended in a
  conducting fluid
• Difficult to register electrode location
• Artifacts from motion, eyeblinks, swallows,
  heartbeat, sweating
• Food, age, time of day, fatigue, motivation of
  subject

15
Why EEG Can Work for BCI

• Many EEG studies have reported reproducible
  changes in brain dynamics that are task
  dependent!
• People are able to control their brainwaves via
  biofeedback!

16
Biofeedback

• Patients may correct their waveforms to achieve
  a normal state.
• Kamiya demonstrated the controllability of alpha
  waves in 1962.
• Communication in morse code by turning alpha
  waves on and off.
• Stress management and sleep therapy.
• Move a pac-man by stimulating alpha and beta
  waves.
• Note that artifacts are a serious problem for
  real-time biofeedback applications.

17
Motivation for Our Work

• Current biofeedback training requires 10 weeks to
  move a cursor.
• Typing requires 5 minutes/letter with 90
  accuracy.
• Although there has been some mathematical work
  the field has been dominated by experiment and
  heuristics.
• Suggestions by clinical EEG experts that
  understanding EEG problem will have a strong
  mathematical component.
• Tremendous potential for results.

18
EEG Data Set Mental Tasks

• Resting task
• Imagined letter writing
• Mental multiplication
• Visualized counting
• Geometric object rotation
• Keirn and Aunon, A new mode of communication
  between man and his surroundings, IEEE
  Transactions on Biomedical Engineering,
  37(12)1209-1214, December 1990

19
Lobes of the Brain

• Frontal Lobes
• Personality, emotions, problem solving.
• Parietal lobes
• Cognition, spatial relationships and
  mathematical abilities, nonverbal memory.
• Occipital lobes
• Vision, color, shape and movement.
• Temporal lobes
• Speech and auditory processing, language
  comprehension, long-term memory.

20
Electrode Placementand Sample Data
21
Geometric Filtering of Noisy Time-Series

• Given a data set
• The Q fraction of a basis vector is defined
  as where

22
Signal Fraction Optimization

• Determine ? such that D(?) is a maximum.
• Solution via the GSVD equation

23
(No Transcript)
24
SVD filter
Original Signal
Signal fraction filter
25
SVD basis
GSVD basis
26
SVD reconstruction
GSVD reconstruction
27
Blind Signal Separation

• Unknown (tall) m n signal matrix S
• Unknown mixing n n matrix A
• Observed m n data matrix X
• Task recover A and S from X alone.
• In general it is not possible to solve this
  problem.

28
Signal Fraction Analysis Separation

• Theorem The solution to the signal fraction
  analysis optimization problem solves the signal
  separation problem X SA given
• 1) is observed
• 2)
• 3)
• In particular,
• Where is the ? solution to the GSVD problem for
  signal fraction analysis.

29
Original signals (unknown)
Mixed signals (observed)
30
FastICA separation
Signal fraction separation
31
(No Transcript)
32
Artifact Removal

• Given the separated signals ? X ? we may filter
  the ith column of ? by setting
• Where Id is the identity matrix with the ith row
  set to zero. The filtered version of the data is
  now
• Where recall the original data is

33
Signal Fraction Filters
34
Constructing Signal Fraction Filter
35
(No Transcript)
36
Benefits of Signal Fraction Analysis

• Can identity sources of noise such as
  respirators, eyeblinks, cranial heartbeat, line
  noise etc
• Filtering works over short periods of the signal,
  i.e., can remove artifacts from a time series of
  length 500.
• Can use generalizations of the signal to noise
  ratio to separate quantities of interest.
• Simple and fast to compute.

37
Classification on Manifolds

• Insert slide from Istec meeting

manifold H(x) 0
dist(A,B) large but H(A)H(B)0
38
Dynamical Systems Perspective

• Assume a system is described by the dynamical
  equations
• and that the solutions reside on an attracting
  set, e.g., a manifold. What can be said about
  the full system if it is only possible to observe
  part of the system? In the extreme, imagine we
  can only observe a scalar value

39
Time Delay Embedding

• We may embed the scalar observable into a higher
  dimensional state space via the construction
• So now it is clear that

40
Takens Theorem (simplified)

• Given a continuous time dynamical system with
  solution on a compact invariant smooth manifold M
  of dimension d, a continuous measurement function
  h(x(t)) can be time-delay embedded in to
  dimension 2d1 such that there is a
  diffeomorphism between the embedded attractor and
  the actual (unobserved) solution set.

41
The Lorenz Attractor
Given a data point (x,y,z) we know which lobe by
the sgn of x. But what if we only observe the z
value? The lobe can be classified using Takens
theorem and Time delay embedding.
42
Do EEG data lie on an attractor?
43
Elephants in the Clouds?
Random data
Classification rate
44
Super Resolution Skull Caps

• How many electrodes are needed? 6, 16, 32, 128,
  256, 512? We should be able to answer this
  question by means of evaluating an objective
  function.
• Through attractor reconstruction, time delay
  embedding techniques may practically enhance the
  resolution of skull caps leading to significant
  savings in time and equipment.
• Colleagues working on EEG studies in children are
  very enthusiastic about this!

45
Manifolds and Nonlinear Methods(work with
Fatemeh Emdad)

• Veronese embeddings of the data
• Degree 1 (x,y)
• Degree 2 (x2, xy, y2)
• Degree 3 (x3, x2y, xy2, y3)
• Degree 1 (x,y,z)
• Degree 2 (x2, xy, xz, y2, yz, z2)
• Degree 3 (x3, x2y, x2z, xy2, xz2, xyz, y3,
  y2z, yz2, z3)
• Such embeddings are behind one variant of kernel
  SVD.

46
Kernel SVD versus Kernel SFA

• Numerical Experiments
• KSVD (KPCA) degree 1, 2, 3, 4
• KSFA degree 1, 2, 3, 4
• Objective compare mode classification rates
  using knn for k 1,, 10.

47
KSFA, KPCA degree 1
48
KSFA, KPCA degree 2
49
KSFA, KPCA degree 3
50
KSFA, KPCA degree 4
51
Relative Performance
52
Conclusions and Future Work

• Present a geometric subspace approach for signal
  separation, artifact removal and classification.
• Provided evidence that brain dynamics might
  reside on an attractor and that time-delay
  embedding enhances classification rates.
• Illustrated a nonlinear extension to signal
  fraction analysis and compared with similar
  extension to svd.
• These ideas are presented in the context of EEG
  signals but are quite general and can be applied
  to images.