Basic Research Challenge:

1
(No Transcript)
2
(No Transcript)
3
(No Transcript)
4

• Basic Research Challenge
• gt To figure out how the brain works lt
• What qualifies as an understanding, to first
  order?
• Deep knowledge of tasks being performed,
• Knowledge of algorithms implemented to carry out
  those tasks,
• Knowledge of the functional architecture,
• Knowledge of the detailed circuit diagram, in
  terms of the relevant functional components.

5

• Basic Research Challenge
• gt To figure out how a sensory system works lt
• What qualifies as an understanding, to first
  order?
• Deep knowledge of tasks being performed,
• Knowledge of algorithms implemented to carry out
  those tasks,
• Knowledge of the functional architecture,
• Knowledge of the detailed circuit diagram, in
  terms of the relevant functional components.
• What are the immediate problems were faced with?
• To determine the relevant stimulus space, and
• To determine the encoding scheme through which
  info about that stimulus space is represented
  internally.
• So what is needed?
• Assumption-free algorithms, feasible
  computational approaches toward solving those
  algorithms, and massive amounts of data.

6
What information about the stimulus waveform is
encoded in the spike train, and how is that
information represented???
Stimulus waveform
Intracellular neuron recording
7
Encoding of complex dynamic stimuli

• Rate codes The number of impulses (spikes) in a
  small temporal windows is a function of the
  stimulus.
• Population vector codes A linear function of the
  number of spikes from several neurons is
  proportional to the stimulus.
• Single spike codes The position, or latency, of
  each spike represents the same stimulus feature.
  The stimulus is recovered as temporal
  superposition of features.
• Pattern codes Particular spike patterns
  represent specific stimulus features.

8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
Analysis of simple cases (small spaces,
unlimited sampling)

• The relationship between stimulus (X, air current
  direction) and response (Y, spike rate) as a
  joint histogram. The deterministic relationship
  is shown in red.
• b-e) Successive refinements of the quantizations.
  
• The optimal mutual information as a function of
  reproduction size.

14
Looking for the probabilistic dictionary to the
neural code
Q(YX)
stimulus X(?)
response Y(t)
Q(XY)
15
What is neural coding?

• A coding scheme is a map between an (input)
  probability measure space X and a (neural
  response) probability measure space Y.
• The map is probabilistic on elements of the
  original spaces.
• The map is almost deterministic on codeword
  classes in (X,Y).
• A stimulus in X is represented by any of the
  responses in the same equivalence class. A
  response in Y is decoded as any of the stimuli
  from the same equivalence class.
• Our goal is to recover such a coding scheme.

16

• What are the possible codewordfeature classes in
  this stimulusresponse set?

17

• What is the mapping between the functional
  architecture and the biological architecture?
• What is the correlate of an information channel
  within a nervous system?
• A single synapse?
• Single dendrite?
• Single neuron?
• An ensemble of neurons?
• A single independent component of an ensemble
  activity mode?

18

• What is the mapping between the functional
  architecture and the biological architecture?
• What is the correlate of an information channel
  within a nervous system?
• A single synapse?
• Single dendrite?
• Single neuron?
• An ensemble of neurons?
• A single independent component of an ensemble
  activity mode?
• Is information multiplexed over the channels?
• If so is the information independent, or is it
  additive?

19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
How do we think the brain analyzes sensory
information? 1) Information is captured
peripherally, and 2) is represented internally as
a map of relevant parameters (either through
projection or construction), and 3) is projected
inward and upward to subsequent processing
stages as pictures of the external world. 4)
Hierarchical, multi-stream processing of mapped
information is fed forward and transformed
through successively higher processing stages
performing successively more complex
computations. (Lateral and feedback circuitry
mediates contrast gain control, adaptive
filtering feature detection, etc.) 5) At the
top (somewhere?) all of the transformed inputs
are re-integrated to construct a model of the
universe, and 6) We make up our minds about
which aspects of the model to pay attention to.
24
What if the nervous system works the other way
around, through predictive coding (e.g., map
seeking circuits or adaptive resonance)?
E.g. 1) A generative, adaptive model of the
universe is constantly being constructed
internally, throughout the brain at all
processing stages, and 2) projected outward
toward the sensory periphery from each processing
stage, 3) compared reiteratively at each stage
with what is coming inward from the sensory
periphery, in a manner that a) nulls out anything
that matches the internally-generated expectation
and b) adapts to slow changes in the statistics
of sensory input signals, but c) presents
anything that is unexpected. 4) gtgtgt What you
are capable of attending to is precisely this
difference between the internal model and the
actual data steam about the universe, modulated
by your concept of causality in the universe. 5)
That difference image is distributed throughout
the entire brain anatomically, and 6) maybe is
updated synchronously throughout the entire
brain, rather than sequentially.
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
Message from the front A neural code must be
capable of supporting calculations sufficient for
useful cognitive calculations, and those
calculations must be capable of neural
implementation. One example If you have
calculating machinery and a coding scheme capable
of supporting the following 8 operations
(presented with more rigor on pp. 16-17 in Map
Seeking Circuits), then it can support
significant cognitive/perceptual
computations combine, match,map, inverse map,
compete, attenuate, scale, and a
nonlinearity What sort of neuronal code satisfies
these requirements? One possibility is phase
encoding of vector elements, and David will show
you what it looks like in detail. He will also
show an isomorphic circuit using phase encoding,
and the neuronal machinery corresponding to each
of the above operations. gtgtgt One of his major
messages to us A neural code should be
considered along with the operations it is
capable of supporting.
29
(No Transcript)
30

• So where are we?
• 1) In order to figure out how a nervous system
  works, we need to understand neural coding in
  that system.
• 2) A first essential step toward understanding
  neural coding is to derive measures for cell
  stimulus/response properties that do not have
  imbedded assumptions about the nature of neural
  encoding. Alternatively, the first step should be
  capable of trying out all possible coding
  schemes, without excluding the actual scheme that
  might be implemented. Duh!
• One productive approach involves the measurement
  of mutual information between stimulus and
  response sets under different assumptions about
  the code and stimulus universe.
• 3) When we analyze neurons with
  information-theoretic metrics, we find that
  individual nerve cells encode less information
  than classical characterizations might lead us
  to believe.
• 4) In applying this approach, we cant assume
  that a single neuron corresponds to a single
  channel.
• 5) A neural code should be considered along with
  the operations it has evolved to support.

31
Prevailing Hypothesis about the functional
significance of cortical oscillations I.
Cortical oscillations are an epiphenomenon they
are only the shadows of the ensemble activity
of cells engaged in meaningful computations, but
do not convey useful information independent of
the ensemble activity patterns themselves. I.e.,
1) Cortical function is mediated by dynamic
modulation of coherent firing in groups of
neurons. 2) Neurons associate rapidly into
functional groups in order to perform
computational tasks, at the same time becoming
dissociated from concurrently activated but
groups engaged in other tasks. (e.g., each
oscillation could be the correlate of a single
reiterative, generative model update cycle in the
map seeking or adaptive resonance or
binding process, or of a winnerless-competition
-attractor sparsification process.) 3)
Macroscopic field potentials are a consequence of
this emergent coherent grouping, but nothing
about them conveys information decoded explicitly
by any subsequent processing operation.
32
Totally different hypothesis Hopfield, (1995)
Pattern recognition computation using action
potential timing for stimulus representation.
Nature 376 (6535) I. Cortical oscillations are
still an epiphenomenon but they emerge from the
ensemble activity of cells that are NOT engaged
in meaningful computations. I.e., 1) Neurons
involved in a common computational task associate
rapidly into functional groups that fire in a
complex sequence that is explicitly uncorrelated
with the rest of the 40Hz drones. 2)
Information about sensory parameters is
represented by the explicit times at which action
potentials occur within that relevant ensemble,
rather than by the ensemble firing rate of the
cells. 3) These time delays are organized so that
the features recognized by feature-detecting
neurons, although occurring at different times,
produce signals which arrive simultaneously at a
recognition neuron which then responds
maximally. 4) The comparison of patterns over
sets of analogue variables is done by a network
using different delays for different information
paths. I.e., the scheme computes in this analogue
representation by combining information through
pathways with different delays.
33
Notes on Hopfield pattern decoder model An
essential feature of the model is an oscillatory
subthreshold variation of the membrane potential
for the encoding neurons. In the absence of an
input current Ij(t), the cell exhibits only
subthreshold membrane potential oscillations.
When the input current exceeds Io, action
potentials will be generated. The dead time and
level of input can together prevent the cell from
generating two action potentials within one
period of the oscillation. Different cells will
respond with different time delays to any given
analogue input pattern. The time ?j each neuron j
fires in advance of the maximum of the
subthreshold oscillation is determined by its
input current, thus encoding the analogue
information Ij as a time advance ?j. The firing
frequency for all active neurons is simply f. The
time pattern to be decoded in this case
originates with the encoding scheme itself, and
is created even for a constant input stimulus,
rather than originating with the nature of the
time-dependent stimulus itself. In general, when
a pattern consists of a set of features occurring
in a given time relationship it can be recognized
by a processing system which uses time delays and
coincidence detection. These time delays are
organized so that the features recognized by
feature-detecting neurons, although occurring at
different times, produce signals which arrive
simultaneously at a recognition neuron which then
responds maximally.   (E.g., barn owl auditory
system azimuthal localization of sound source
via interaural delay.)  NOTE 1 Synchronized
action potentials play a very different role in
the processing described here from that which
they have been hypothesized to play in the visual
system Eckhorn, R. et al. Biol. Cybern. 60,
121-130 (1988), Gray, C. M. Singer, W. PNAS.
U.S.A. 86, 1698-1702 (1989). If information is
encoded as hypothesized, then neurons that
receive little input fire with very little time
advance (or none at all), and will appear
synchronized. These are generally the neurons
with the least useful information about an odor.
The neurons representing the strong components of
the odor fire earlier in time and will not appear
synchronous.  NOTE 2 Single-electrode recording
is completely incapable of elucidating the nature
of this representation /computation model.
Because weakly driven cells do not respond at
all, and strongly driven cells all fire at the
same frequency, conventional electrophysiology
would simply conclude that the cells are broadly
tuned.
34
Worries 1. What if what we think is waaay
wrong?
35
Worries 1. What if what we think is waaay
wrong? 2. What if the way we think about how
nervous systems work is so far wrong that we
cant get there from here.
36
Worries 1. What if what we think is waaay
wrong? 2. What if the way we think about how
nervous systems work is so far wrong that we
cant get there from here. 3. What if the
analytical tools we are using are not compatible
with the correct way of thinking about how
nervous systems work?
37
Worries 1. What if what we think is waaay
wrong? 2. What if the way we think about how
nervous systems work is so far wrong that we
cant get there from here. 3. What if the
analytical tools we are using are not compatible
with the correct way of thinking about how
nervous systems work? 4. What if you kids all
decide to diddle with phenomenological models
instead of helping figure out how the brain
works?
38

• Biggest Technological barriers in studies of
  neural coding
• a) We need to record from a large fraction of
  all possible nerve cells in the ensemble.
• b) We need to be able to decipher the symbols
  from raw signals.
• Two processing stages
• i) Spike discrimination
• ii) Symbol extraction
• 1) De-multiplexing
• 2) Symbol discovery
• c) We need huge amounts of data recorded for long
  times under conditions of stationarity.
• d) The stimulus set must be complete and natural.
  
• e) Need to be able to interact with the system
  in real time (i.e., perform an experiment!!!)

39
(No Transcript)
40

• Critical enabling capabilities 1
• massive data stream acquisition
• Long-term, simultaneous recording of the activity
  patterns from a large proportion of the neurons
• From 100 cells (invertebrates) to 10,000 cells
  (vertebrates)
• gtgtgt Need sensor arrays that are denser, smaller
  and better integrated with the nervous system!!
• Recordings must have high temporal resolution (to
  allow unambiguous identification of superimposed
  spikes from multiunit recordings)
• 50KHz/channel at 16 bit accuracy 800
  Kbits/sec.channel
• gtgtgt 100 electrodes 80 Mbits/sec
• gtgtgt 10,000 electrodes 8 Gbits/sec

41

• Critical enabling capabilities 2
• for long durations
• Recordings must be maintained for long duration
• one 50KHz/channel at 16 bits 400 GigaBytes/hour
• gtgtgt 100 electrodes 40 TeraBytes/hour
• gtgtgt 10,000 electrodes 4 PetaBytes/hour
• Yikes!
• Alternate approach
• Store ( process) only the times of spike
  occurrence on-line
• one nerve cell average of about 100
  spikes/second
• one neuron-channel at 0.1 ms resolution 10k
  bits/sec
• gtgtgt 100 electrodes 1 Mbits/sec (vs. 80
  Mbits/sec)
• gtgtgt 10,000 electrodes 100 Mbits/sec (vs. 8
  Gbits/sec)

42

• Critical enabling capabilities 3
• on-line interactive data processing
• Must be able to manipulate experimental prep in a
  closed-loop mode
• SO gtgt Hardware, software and advanced algorithms
  for
• massive data-stream acquisition
• on-line analysis
• interactive and programmable automated control
• NOTE massive datastream acquisition and on-line
  analysis should be integrated with one another
• 1. Raw data stream must be converted to
  independent signal channels
• 2. Signals must be converted to symbols in the
  Shannon sense