Modern Coding Theory: LDPC Codes

1
Modern Coding Theory LDPC Codes

• Hossein Pishro-Nik
• University of Massachusetts Amherst
• November 7, 2006

2
Outline

• Introduction and motivation
• Error control coding
• Block codes
• Minimum distance
• Modern coding LDPC codes
• Practical challenges
• Application of LDPC codes to holographic data
  storage

3
Errors in Information Transmission
Digital Communications Transporting information
from one party to another, using a sequence of
symbols, e.g. bits.
Received bits Corrupted version of the
transmitted bits
Noise interference received sequence may
be different from the transmitted one.
Transmitted bits
4
Magnetic recording
Errors in Information Transmission Cont.
Some of the bits may change during the
transmission from the disk to the disk drive
5
Errors in Information Transmission Cont.
These communication systems can be modeled as
Binary Symmetric Channels (BSC)
Receiver
Sender
Each bit is flipped with probability p
6
Pioneers of Coding Theory
Bell Telephone Laboratories
Richard Hamming
Claude Shannon
7
Error Control Coding Repetition Codes
Error Control Coding Use redundancy to reduce
the bit error rate
BSC
1001010101
1011000101
Bit error probability p0.01
Example

• Three-fold repetition code send each bit three
  times

8
Repetition Codes Cont.
Encoder Repeat each bit three times
codeword
BSC
Decoder majority voting
Corrupted codeword
9
Repetition Codes Cont.
(0,0,0)
Encoder
(0)
codeword
BSC
(1,0,0)
Decoder
(0)
Corrupted codeword
Successful decoding!
Decoding majority voting
10
Decoding Error
Decoding Error Probability Prob2 or 3
bits in the codeword received in error

• Advantage reduced bite error rate
• Disadvantage we lose bandwidth because each bit
• should be sent three times

11
Error Control Coding Block Codes
Information block
ngtk

• Encoding mapping the information block to the
• corresponding codeword
• Decoding an algorithm for recovering the
• information block from the corrupted codeword
• Always ngtk redundancy in the codeword

12
Error Control Coding Block Codes
K code dimension, n code length
This is called an (n,k) block code
13
Code Rate

• In general an (n,k) block code is a 1-1 mapping
  
• from k bits to n bits

• R shows the amount of redundancy in the codeword
• Higher R Lower redundancy

14
Repetition Codes Revisited
For repetition code k1, n3, R1/3
Encoder
K1
n3
BSC
Decoder
The repetition code is a (3,1) block code
15
Block Codes Cont.

• There are two valid codewords in the repetition
  code

Valid codewords

• A (5,3) block code (n5, k3, R3/5)

16
Block Codes Cont.
All n-tuples
All k-tuples
(0,0)
(0,1)
(1,0)
(1,1)
17
Good Block Codes

• There exist more efficient and more powerful
• codes.
• Good Codes
• High rates lower redundancy (depends on the
• channel error rate p)
• Low error rate at the decoder
• Simple and practical encoding and decoding

18
Linear Block Codes
A linear mapping

• Linear block codes
• Simple structure easier to analyze
• Simple encoding algorithms.

Generator matrix G
19
Linear Block Codes

• There are many practical linear block codes
• Hamming codes
• Cyclic codes
• Reed-Solomon codes
• BCH codes

20
Channel Capacity
Channel capacity (Shannon) The maximum
achievable data rate
Shannon capacity is achievable using random codes
21
Shannon Codes
Random mapping
All n-tuples
All k-tuples
(0,0)
(0,1)
(1,0)
(1,1)
22
Shannon Random Codes

• As n (block length) goes to infinity, random
  codes achieve the channel capacity, i.e,
• Code rate R approaches C, while the decoding
  error probability goes to zero

23
Error Control Coding Low-Density Parity-Check
(LDPC) Codes

• Ideal codes
• Have efficient encoding
• Have efficient decoding
• Can approach channel capacity
• Low-density parity-check (LDPC) codes
• Random codes based on random graphs
• Simple iterative decoding

24
t-Error-Correcting Codes

• The repetition codes can correct one error in the
  codeword however, it fails to correct higher
  number of errors.
• A code that is capable of correcting t errors in
  the codewords is called a t-error-correcting
  code.
• The repetition code is a 1-error-correcting code.

25
Minimum Distance

• The minimum distance of a code is the minimum
  Hamming distance between its codewords
• Mindist(u,v) u and v are codewords
• For the repetition code, since there are only two
  valid codewords,

26
Minimum Distance Cont.
All vectors of length n (n-tuples)
27
Minimum Distance Cont.
Higher minimum distance Stronger code
Example For repetition code t1, and
.
28
Modern Coding Theory

• Random linear codes can achieve channel capacity
• Linear codes can be encoded efficiently
• Decoding of linear codes NP hard
• Gallagers idea
• Find a subclass of random linear codes that can
  be decoded efficiently

29
Modern Coding Theory

• Iterative coding schemes LDPC codes, Turbo codes

Encoder
BSC
Iterative Decoder

• Iterative decoding instead of distance-based
  decoding

30
Introduction to Channel Coding

• Noisy channels

Noisy channel
Information bits
Corrupted bits
1001010101
10e10e01e1
Example binary erasure channel (BEC)
Other channels Gaussian channel, binary
symmetric channel,
31
Low-Density Parity-Check Codes

• Defined by random sparse graphs (Tanner graphs)

Check (message) nodes
Variable (bit) nodes
Simple iterative decoding message-passing
algorithm
32
Important Recent Developments

• Luby et al. and Richardson et al.
• Density evolution
• Optimization using density evolution
• Shokrollahi et al.
• Capacity-achieving LDPC codes for the binary
  erasure channel (BEC)
• Richardson et al. and Jin et al.
• Efficient encoding
• Irregular repeat-accumulate codes

33
Standard Iterative Decoding over the BEC
Standard Iterative Algorithm Repeat for any
check node If only one of the neighbors is
missing, recover it
34
Standard Iterative Decoding Cont.
f
f
f
f

0
e
e
1
e
1
1
0
1
35
Algorithm A Cont.

• The algorithm may fail

f

0
1
e
e
1
e
36
Practical Challenges Finite-Length Codes

• In practice, we need to use short or
  moderate-length codes
• Short or moderate length codes do not perform as
  well as long codes

37
Error Floor of LDPC Codes
Capacity-approaching LDPC Codes suffer from the
error floor problem
38
Volume Holographic Memory (VHM) Systems
39
Noise and Error Sources

• Thermal noise, shot noise,
• Limited diffraction
• Aberration
• Misalignment error
• Inter-page interference (IPI)
• Photovoltaic damage
• Non-uniform erasure

40
The Scaling Law

• The magnitudes of systematic errors and thermal
  noise are assumed to remain unchanged with
  respect to M
• (number of pages) and SNR is proportional to

• SNR decreases as the number of pages increases.
• There exists an optimum number of pages that
  maximizes
• the storage capacity.

41
Raw Error Distribution over a Page

• Bits in different regions
• of a page are affected
• by different noise
• powers.
• The noise power is
• higher at edges.

42
Properties and Requirements

• Can use large block lengths
• Non-uniform error correction
• Error floor Target BERlt
• Simple implementation Simple decoding

43
Ensembles for Non-uniform Error Correction
Bits from the first region
44
Ensemble Properties

• Threshold effect
• Concentration theorem
• Density evolution

45
Ensemble Properties

• Stability condition (BEC)

46
Design Methodology

• The performance of the decoder is not directly
  related to the minimum distance.
• However, the minimum distance still plays an
  important role
• Example error floor effect
• To eliminate error floor, we avoid degree-two
  variable nodes to have large minimum distance.
• For efficient decoding, and also for simplicity
  of design, we use low degrees for variable nodes.

47
Performance on VHM
Rate.85 Avg degree6 Gap from capacity at
BER 1e-9 0.6dB
48
Storage Capacity
Information theoretic capacity for soft-decision
decoding .95Gb
1
LDPC soft .84 Gb
LDPC hard .76 Gb
0.8
Storage capacity (Gbits)
RS hard .52Gb
0.6
0.4
0.2
0
2000
4000
6000
Number of pages
49
Conclusion

• Carefully designed LDPC codes can result in
• significant increase in the storage capacity.
  
• By incorporating channel information in design
  of LDPC codes
• - Small gap from capacity
• - Error floor reduction
• - More efficient decoding

50
Modern Coding Theory

• The performance of the decoder is not directly
• related to the minimum distance.
• However, the minimum distance still plays an
• important role
• Example error floor effect