What is Coding

1
What is Coding?

• Coding is the conversion of information to
  another form for some purpose
• Source Coding The purpose is lowering the
  redundancy in the information. (e.g. ZIP, JPEG,
  MPEG2)
• Channel Coding The purpose is to defeat channel
  noise

2
Channel Coding

• Channel encoding The application of redundant
  symbols to correct data errors
• Modulation Conversion of symbols to a waveform
  for transmission
• Demodulating Conversion of the waveform back to
  symbols, usually one at a time
• Decoding Using the redundant symbols to correct
  errors

3
Block Diagram of a general Communication System
4
Memoryless Channel Model

• Encoding cmG, G -generator matrix
• Decoding cHT0, H-parity check matrix

5
Soft Output Channels

• Soft information(bit estimate, reliability)
• Measure of reliability
• likelihood ratio

6
Iterative Decoding

• Linear codes with sparse parity check matrices
• Iterative decoding
• Message passing algorithm

7
Graph Model for a Linear Code

• n-k equations in n variables
• Example

v1 v2 v3 v4 v5 v6
v7
Variables
e1 e2 en

Checks
c1 c2 c3
8
Simple Soft Message Passing Algorithm

• For each connected variable-check pair (v,c)
• Find product of reliability signs of all
  variables connected to check node c (excluding
  node v).
• Find the magnitude of the smallest reliability of
  variables connected to check node c (excluding
  node v).
• Combine sign and minimum reliability to update
  reliability of the variable node.
• Repeat until all parity checks are satisfied

9
Performance
BER, B
SNR, s
From R. Urbanke, Iterative coding systems
10
Error floor (finite size BP-approximate)
Error floor prediction for some regular (3,6)
LDPC Codes using a 5-bit decoder. From T.
Richardson Error floor for LDPC codes, 2003
Allerton conference Proccedings.
No-go zone for brute-force Monte-Carlo
numerics. Estimating very low BER is the major
bottleneck in coding theory/practice
11
How to evaluate Code Performance?

• Need to consider Code Rate (R), SNR (Eb/No), and
  Bit Error Rate (BER)
• For given (a) channel (b) coder (c) decoder to
  estimate BER by means of analytical and/or
  semi-analytical methods
• BER is small and it is mainly formed at some very
  special bad configurations of the noise

12
Instantons for (155,64,20) code Gaussian channel
Phys. Rev. Lett -- Nov 25, 2005
menu
13
Sum-Product Algorithm(Kschischang et. al.)

• Converges if factor graph is a tree

14
Sum-Product Algorithim

• Given the log-likelihoods of (xj)1?j?m find the
  log-likelihood of y, L(y).

l1 l2 lj lm-1 lm 1

• There is need for look-up tables

15
Min-Sum Algorithm

• Simpler decoding algorithm
• Asymptotically performs the same as sum-product
• Note that
• So we can replace the r message update formula
  with
• This greatly reduces complexity, since now we
  dont have to worry about computing the nonlinear
  ? function.

16
Structured Codes

• Needed for good placement and routing
• The parity check matrix of these codes is
  completely determined by a small set of
  parameters, and can lend itself to a very low
  complexity implementation
• Related work
• Projective geometries (Kou, Lin, Fossorier)
• Subgroups of the multiplicative group of a prime
  field (Tanner, Srkdhara, Fuja )
• Ramanujan graphs (Rosenthal, Vontobel)
• Steiner systems (MacKay and Davey)
• Designs (Johnson, Weller)

17
A Parity-Check Matrix

• An array of permutation matrices

18
References
Iterative Decoding Papers 1   R. G. Gallager,
Low-Density Parity-Check Codes. Cambridge, MA
MIT Press, 1963. 2   G. Battail, M.C.
Decouvelaere, and P. Godlewski, Replication
decoding, IEEE Trans. Inform. Theory, vol IT-25,
pp. 332-345, May 1979. 3   J. Hagenauer, E.
Offer, and L. Papke, Iterative decoding of
binary block and convolutional codes, IEEE
Trans. Inform. Theory, vol 42., no. 2, pp.
439-446, March 1996. 4   F. R. Kschischang, B.
J. Frey, and H.-A. Loeliger, Factor Graphs and
the Sum-Product Algorithm, IEEE Trans. on
Inform. Theory, Vol. 47, No. 2, pp. 498 519,
Feb. 2001. 5  T. Richardson, A. Shokrollahi,
and R. Urbanke, Design of provably good
low-density parity check codes, International
Symposium on Information Theory, 2000.
Proceedings, p. 199, June, 2000. 6   T.
Richardson and R. Urbanke, The Capacity of
Low-Density Parity-Check Codes Under
Message-Passing Algorithm, submitted to Trans.
Inform Theory. 7    R. J. McEliece, D. J. C.
MacKay, and J.-F. Cheng, Turbo Decoding as an
Instance of Pearls Belief Propagation
Algorithm, IEEE J. Select. Areas Commun., Vol.
16, pp. 140152, Feb. 1998. 8    R. M. Tanner,
A Recursive Approach to Low Complexity Codes,
IEEE Trans. Inform. Theory, Vol. IT-27, pp.
533547, Sept. 1981.
Structured LDPC Code Papers 1    Y. Kou, S.
Lin, and M.P.C. Fossorier, Low-Density
Parity-Check Codes Based on Finite Geometries A
Rediscovery and New Results, IEEE Trans. Inform.
Theory, Vol. 47, No. 7, pp. 2711-2736, Nov. 2001.
2     S. J. Johnson, and S. R. Weller,
Regular Low-Density Parity-Check Codes from
Combinatorial Designs, in Proc. 2001 IEEE
Information Theory Workshop, pp. 90-92,
2001. 3  M.A. Margulis, Explicit
group-theoretic constructions for combinatorial
designs with applications to expanders and
concentrators," Problemy Peredachi Informatsii,
vol. 24, no. 1, pp. 51-60, 1988. 4     M.
Blaum, P. Farrell, and H. van Tilborg, Array
Codes, in Handbook of Coding Theory, V.Pless,
W.Huffman Eds., Elsevier, 1998. 5    E.
Eleftheriou, S. Olcer Low-Density Parity-Check
Codes for Digital Subscriber Lines, IEEE
International Conference on Communications, ICC
2002, Vol. 3, pp. 1752 1757, 2002. 6   J. L.
Fan, Array Codes as Low-Density Parity-Check
Codes, in Proc. 2nd International Symp. on Turbo
Codes and Related Topics, Brest, France, pp.
543-546, Sept. 2000.
19
Encoding Algorithm

• Hc0, mm(1) m(2)m(t-1)T

?
?