Error correction on a tree: Instanton approach

1
Boulder 04/15/04
Error correction on a tree Instanton approach
Misha Chertkov (LANL)
In collaboration with V. Chernyak (Corning) M.
Stepanov (UA, Tucson) B. Vasic (UA, Tucson)
Thanks I. Gabitov (Tucson/LANL)
Submitted to Phys.Rev.Lett.
2
Introduction

• Forward-Error-Correction (FEC). Channel
  Noise.
• Coding.
• Low Density Parity Check codes (LDPC) -
  Tanner graph
• Decoding.
• Marginal-A-Posteriori (MAP) Stat Mech
  interpretation
• Belief Propagation (BP) Message Passing
  (MP)
• Post-Error-Correction Bit-Error-Rate (BER).
• Optimization. Shannon transition/limit .
  Error floor - Evaluation.

Our objectives

• Tree as an approximation BP is exact . From
  LDPCC to a tree
• BER in the center of the tree
• High Signal-to-Noise Ratio (SNR) phase. Hamming
  distance.
• Symmetry. Broken Symmetry.
• Instantons/phases on the tree.

What is next?
3
Forward-Error-Correction
N gt L RL/N - code rate
Coding
Decoding
noise
white
channel
example
Gaussian symmetric
menu
4
(linear coding)
Low Density Parity Check Codes
N10 variable nodes
Parity check matrix
Tanner graph
MN-L5 checking nodes
spin variables -
- set of constraints
menu
5
Decoding (optimal)
menu
6
Sub-optimal but efficient decoding
Belief Propagation (BP) Gallager63Pearl
88MacKay 99
solving Eqs. on the graph
Iterative solution of BP Message Passing (MP)
QmN steps instead of Q - number
of MP iterations m - number of checking nodes
contributing a variable node
What about efficiency? Why BP is a good
replacement for MAP?
(no loops!)
menu
7
Post-Error-Correction Bit Error Rate (BER)
Probability of making an error in the bit i
probability density for given magnetic
field/noise realization
measure for unsuccessful decoding
Foreword-error-correction scheme/optimization

• describe the channel/noise --- External
• suggest coding scheme
• suggest decoding scheme
• measure BER/FER
• If BER/FER is not satisfactory (small enough)
  goto 2

menu
8
Shannon transition/limit
BER, B
SNR, s
From R. Urbanke, Iterative coding systems
menu
9
Error floor
Error floor prediction for some regular (3,6)
LDPC Codes using a 5-bit decoder. From T.
Richardson Error floor for LDPC codes, 2003
Allerton conference Proccedings.
No-go zone for brute-force Monte-Carlo
numerics. Estimating very low BER is the major
bottleneck of the coding theory
menu
10
Our objective For given (a) channel (b) coder
(c) decoder to estimate BER/FER by means
of analytical and/or semi-analytical methods.
Hint BER is small and it is mainly formed at
some very special bad configurations of the
noise/magnetic field Instanton/saddle-point
approach is the right way to identify the bad
configurations and thus to estimate BER!
menu
11
Tree -- no loops -- approximation
Analogy Bethe lattice (1937)
MAP
BP
Belief Propagation is optimal (i.e. equivalent to
Maximum-A-Posteriori decoding) on a tree (no
loops)
Gallager 63 Pearl 88 MacKay 99 Vicente,
Saad, Kabashima 00 Yedidia, Freeman, Weiss 01
menu
12
From a finite-size LDPCC to a tree
1) Fix the variable node where BER needs to be
calculated 2) Choose shortest loop on the graph
coming through the 0th node. Length of the
loop is (n1). 3) Count n-generations from the
tree center and cut the rest.
Regular graph/tree is characterized by m -
number of checking nodes connected to a
variable node k - number of variable nodes
connected to a checking node n -
number of generations on the tree
m2,k3,n4
menu
13
BER in the center of the tree
Effective action
Remarks 1) Optimal configuration/instanton
depends on SNR, s 2) There are
may be many competing instantons
3) Looking for instantons pay attention to the
symmetry
Instanton equations!
menu
14
High Signal-to-Noise-Ratio (SNR) phase
Original code word 1 on the entire tree
The next closest code word
-1 on the colored branches,
1 on the remaining variable nodes
Hamming distance between the two code words
number of the colored variable nodes

Analogy with a low-temperature phase in
stat-mech High SNR value of effective action
self energy
menu
15
Low SNR -- symmetric -- phase
Symmetric phase at any node on the tree
depends primarily on the generation (counted from
the center)
instanton equations
for j0,,n-2
zero momentum configuration/approximation
guarantees estimation from above for eff. action
menu
16
Broken symmetry
Low SNR (high temp)
High SNR (low temp)
Remark Broken symmetry instantons may be related
to the near codewords suggested by
Richardson 03 in the context of the
error-floor phenomenon explanation
In general There are many (!!!) broken
symmetry instanton solutions
menu
17
Instanton phases on the tree
m4, l5, n3. Curves of different colors
correspond to the instantons/phases of different
symmetries.
truth
menu
18
Full numerical optimization (no symmetry breaking
was assumed !!!)
Area of a circle surrounding any variable node is
proportional to the value of the noise on the
node.
m2 l3 n3
menu
19
What is next?

• We plan to develop and extend this instanton
  approach to
• Regular codes with loops. This task will require
  developing a perturbation theory with respect to
  the inverse length of the closed loop and/or with
  respect to the small density of closed loops.
• Other types of codes, e.g. convolutional, turbo,
  etc.
• Calculation of the Frame Error Rate (FER),
  thereby measuring the probability of making an
  error in a code word.
• Finite-number of iteration in message-passing
  version of the BP algorithm. The particular
  interest here lies in testing how BER in general
  and the error floor phenomena in particular
  depend on the number of iterations.
• Other types of fast but, probably, less efficient
  decoding schemes.
• Other types of uncorrelated channels (noise),
  e.g. binary eraser channel.
• Correlated channels, with both positive and
  negative types of correlations between
  neighboring slots. This is particularly relevant
  for linear and nonlinear (soliton) transmission
  in fiber optics communications.
• Accounting for Gaussian fluctuations (i.e. second
  order effects) around the instantons.

menu
20
Truth
main slide
menu