Independent Component Analysis

1
Independent Component Analysis

• From PCA to ICA
• Bell Sejnowski algorithm
• Kurtosis method
• Demonstrations

2
(No Transcript)
3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
Bell and Sejnowski 1995
Consider yg(x)noise with f depending on
w I(yx)H(y)- H(yx) H(yx)E_x E_yx -log
P(yx)
12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
ICA based on KurtosisOja and Hyvarinen
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
Independent Component Analysis
An overview of applications of ICA to biological
data and general data mining, Computational
Neurobiology Laboratory Salk Institute, La Jolla
CA (April, 1999). Enter Enter to advance,
up-arrow to rewind.

• Perform blind separation of signals recorded
  at multiple sensors

• Use minimal assumptions about the characteristics
  of the signal sources.

32
Principle Maximize Information

• Q How to extract maximum information from
  multiple visual channels?

• ICA produces brain-like visual filters for
  natural images.

• A ICA does this -- it maximizes joint entropy
  minimizes mutual information between output
  channels (Bell Sejnowski, 1995).

Set of 144 ICA filters
33
ICA versus PCA

• Independent Component Analysis (ICA) finds
  directions of maximal independence in
  non-Gaussian data (higher-order statistics).

• Principal Component Analysis (PCA) finds
  directions of maximal variance in Gaussian data
  (second-order statistics).

34
Example Audio decomposition
Perform ICA
Mic 1
Mic 2
Mic 3
Mic 4
Terry
Scott
Te-Won
Tzyy-Ping
Play Mixtures
Play Components
35
Electroencephalography (EEG)
Artifacts

• ICA separates brain signals from artifacts.

Brain signals

• Permits study of brain activity in noisy
  conditions.

• Allows monitoring of multiple brain processes.

36
Functional Brain Imaging

• Functional magnetic resonance imaging (fMRI) data
  are noisy and complex.

• ICA identifies concurrent hemodynamic processes.

• Does not require a priori knowledge of time
  courses or spatial distributions.

37
Data Mining

• ICA was applied to Armed Forces Vocation Aptitude
  Battery (ASVAB) test scores and Navy Fire Control
  School grades.

• Two ICA components contributed to final school
  grade.

• ICA may suggest more efficient and balanced
  selection criteria.

38
This presentation by

• Scott Makeig, Naval Health Research Center, San
  Diego
• Tzyy-Ping Jung, Institute for Neural Computation,
  UCSD, La Jolla CA
• Te-Won Lee, Salk Institute, La Jolla CA
• Sigurd Enghoff, Salk Institute
• Terrence J. Sejnowski, Salk Institute UCSD

39
From Barak Pearlmutter

• Contextual ICA
• The first demo applies the Contextual ICA blind
  source separation algorithm. Lucas Parra and I
  digitally extracted ten five-second clips from
  ten audio CDs. These were digitally mixed,
  without time delays or echos, and with random
  gains, to form the output of a synthetic
  microphone. Ten such microphone outputs were
  synthesized. These synthetic microphone outputs
  formed the input to the Bell-Sejnowski
  Independent Components Analysis algorithm. The
  sources are somewhat separated in the output of
  the Bell-Sejnowski ICA algorithm, but not fully.
• The same synthetic microphone outputs were then
  used as input to our new cICA algorithm (see my
  publications page for technical details). The
  sources are almost fully separated in the output
  of cICA.