Center for Computational Biology

1
Computational Issues when Modeling Neural Coding
Schemes
Albert E. Parker
Center for Computational Biology Department of
Mathematical Sciences Montana State University
Collaborators Alexander Dimitrov John P.
Miller Zane Aldworth Thomas Gedeon Brendan
Mumey
2
Neural Coding and Decoding.

• Goal Determine a coding scheme How does neural
  ensemble activity represent information about
  sensory stimuli?
• Demands
• An animal needs to recognize the same object on
  repeated exposures. Coding has to be
  deterministic at this level.
• The code must deal with uncertainties introduced
  by the environment and neural architecture.
  Coding is by necessity stochastic at this finer
  scale.
• Major Problem The search for a coding scheme
  requires large amounts of data

3
How to determine a coding scheme?

• Idea Model a part of a neural system as a
  communication channel using Information Theory.
  This model enables us to
• Meet the demands of a coding scheme
• Define a coding scheme as a relation between
  stimulus and neural response classes.
• Construct a coding scheme that is stochastic on
  the finer scale yet almost deterministic on the
  classes.
• Deal with the major problem
• Use whatever quantity of data is available to
  construct coarse but optimally informative
  approximations of the coding scheme.
• Refine the coding scheme as more data becomes
  available.
• Investigate the cricket cercal sensory system.

4
Information Theoretic Quantities A quantizer or
encoder, Q, relates the environmental stimulus,
X, to the neural response, Y, through a process
called quantization. In general, Q is a
stochastic map The Reproduction space Y is a
quantization of X. This can be repeated Let Yf
be a reproduction of Y. So there is a
quantizer Use Mutual information to measure the
degree of dependence between X and Yf. Use
Conditional Entropy to measure the
self-information of Yf given Y
5
Stimulus and Response Classes
6
The Model

• Problem To determine a coding scheme between X
  and Y requires large amounts of data
• Idea Determine the coding scheme between X and
  Yf, a squashing (reproduction) of Y, such that
  Yf preserves as much information (mutual
  information) with X as possible and the
  self-information (entropy) of Yf Y is maximized.
  That is, we are searching for an optimal
  mapping (quantizer)
• that satisfies these conditions.
• Justification Jayne's maximum entropy
  principle, which states that
• of all the quantizers that satisfy a
  given set of constraints, choose
• the one that maximizes the entropy.

7
Equivalent Optimization Problems

• Maximum entropy maximize F(q(yfy))
  H(YfY) constrained by I(XYf ) ? Io Io
  determines the informativeness of the
  reproduction.
• Deterministic annealing (Rose, 98) maximize
  F(q(yfy)) H(YfY) - ? DI(Y,Yf ).Small ?
  favor maximum entropy, large ? - minimum DI.
• Simplex Algorithm
• maximize I(X,Yf ) over vertices of
  constraint space
• Implicit solution

8
(No Transcript)
9
Modeling the cricket cercal sensory system as a
communication channel
Nervous system
Signal
Communicationchannel
10
Wind Stimulus and Neural Response in the
cricket cercal systemNeural Responses (over a 30
minute recording) caused by white noise wind
stimulus.
X
Y
Time in ms. A t T0, the first spike occurs
Some of the air current stimuli preceding one of
the neural responses
Neural Responses (these are all doublets) for a
12 ms window
T, ms
11
QuantizationA quantizer is any map f Y -gt Yf
from Y to a reproduction space Yf with finitely
many elements. Quantizers can be
deterministic or
Yf
Y
12
Applying the algorithm to cricket sensory data.
1 2
Yf
1 2 3
Yf
Y
13
High Performance Computing

• Tools
• Bigdog an SGI Origin 2000
• MATLAB 5.3
• Parallel Toolbox
• Algorithms
• Model Building
• Optimization
• Bootstrapping

14
Conclusions

• We
• model a part of the neural system as a
  communication channel.
• define a coding scheme through relations between
  classes of stimulus/response pairs.
• Coding is probabilistic on the individual
  elements of X and Y.
• Coding is almost deterministic on the
  stimulus/response classes.
• To recover such a coding scheme, we
• propose a new method to quantify neural spike
  trains.
• Quantize the response patterns to a small finite
  space (Yf).
• Use information theoretic measures to determine
  optimal quantizer for a fixed reproduction size.
• Refine the coding scheme by increasing the
  reproduction size.
• present preliminary results with cricket sensory
  data.

15
(No Transcript)