Asymptotically good binary code with efficient encoding

1
Asymptotically good binary code with efficient
encoding Justesen code

• Tomer Levinboim
• Error Correcting Codes Seminar (2008)

2
Outline

• Intro
• codes
• Singleton Bound
• Linear Codes
• Bounds
• Gilbert-Varshamov
• Hamming
• RS codes
• Code Concatention
• Examples
• Wozencraft Ensemble
• Justesen Codes

3
Hamming Distance

• Hamming Distance between
• The Hamming Distance is a metric
• Non negative
• Symmetric
• Triangle inequality

4
Weight

• The weight (wt) of
• Example (on board)

5
Code

• An (n,k,d)q code C is a function such that
• For every

6
Code (parameters)

• (n,k,d)q
• Parameters
• n block length
• k information length
• d minimum distance (actually, a lower bound)
• q size of alphabet
• C qk or klogqC

7
Code (parameters div n)

• Asymptotic view of parameters as n?8
• The rate
• Relative minimum distance
• Thus an (n,k,d)q can be written as (1,R,d)q
• Notation (n,k,d)q vs. n,k,dq latter reserved
  for linear code (soon)

8
Trivial Code Example

• FEC3 write each bit three time
• R ?
• d ?
• how many errors can we
• Detect ? (d-1)
• Correct ? t, where d2t1

9
Goal

• Would like to
• Maximize d correct more
• Maximize R send more information
• conflicting goals - would like to be able to
  construct an n,k,dq code s.t. dgt0, Rgt0 and both
  are constant.
• Minimize q for practical reasons
• Maximize number of codewords while minimizing n
  and keeping d large.

10
Singleton Bound

• Let C be an n,k,dq code then
• k n d 1
• equivalently
• R 1 d o(1)
• Proof project C to first k-1 coordinates
• On Board

11
Visual intuition

• On board...
• Ballq(x,r)
• rd
• rt (where d2t1)
• Volq(n,r) Ballq(x,r)

12
Linear Codes
13
Linear Codes

• An n,k,dq code CFqK?Fqn is linear when
• Fq is a field
• C is linear function (e.g., matrix)
• Linearity implies
• C(axby) aC(x) bC(y)
• 0n member of C

14
Linear Codes (example)

• FEC3
• 3,1,32
• Hadamard longest linear code
• n,logn, n/22
• e.g., - 8,3,42
• (H - Matrix representation on board)
• Dimensions
• Asymptotic behavior

15
Linear Codes minimum distance

• Lemma if CFqK?Fqn is linear then
• Note for clarity Cx means C(x)
• Proof
• - trivial
• - follows from linearity (on board)

16
Reed-Solomon code

• Idea oversample a polynomial
• Let q be prime power and Fq a finite field of
  size q.
• Let kltn and fix n elements of Fq,
• x1,x2,..xn
• Given a message m(c0..ck-1) interpret it has the
  coefficients of the polynomial p

17
RS Codes

• Thus (c0..ck-1) is mapped to (p(x1),..p(xn))
• Linear mapping (Vandermonde)
• Using linearity, can show for x?0
• ? RS meet the Singleton bound
• Proof on board
• ( of roots of a k-1 degree poly)
• Encoding time

18

• Bounds

19
Gilbert-Varshamov Bound Preliminaries

• Binary Entropy
• Stirling
• Implying that

20
Gilbert-Varshamov Bound Preliminaries

• Using the binary entropy we obtain
• On board

21
Gilbert-Varshamov Boundbound statement

• For every n and dltn/2 there is an (n,k,d)q (not
  necessarily linear) code such that
• In terms of rate and relative min-distance

22
Gilbert-Varshamov Bound Proof

• On Board
• Sketch of proof
• if C is maximal then
• And
• Now use union bound and entropy to obtain result
  (we show for q2, using binary entropy)

23
GV-Bound

• Gilbert proved this with a greedy construction
• Varshamov proved for linear codes
• proved using random generator matrices most
  matrices are good error correcting codes

24
Singleton / GV Plot
1
Singleton (upper)
Gilbert-Varshamov (lower)
1
0.5
25
Hamming Bound (Upper)

• With similar reasoning to GV bound but using
• For q2 can show that

26
Bounds plot
Madhu Sudan (Lecture 5, 2001)
27

• Code Concatenation

28
Code Concatenation - Motivation

• RS codes imply we can construct good n,k,dq
  codes for any qpk
• Practically would like to work with small q (2,
  28)
• Consider the obvious idea for binary code
  generated from C simply convert each symbol
  from Sn to log2q,
• Whats the problem with this approach ? (write
  the new code!)

29
Code Concatenation

• Due to Forney (1966)
• Two codes
• Outer Cout N,K,DQ
• Inner Cin n,k,dq
• Inner code should encode each symbol of outer
  code ? k logqQ

30
Code Concatenation

• How does it work ?

Luca Trevisan (Lecture 2)
31
Code Concatenation

• What is the new code ?
• dcon dD Proof
• On board

32
Code Concatenation (Examples)

• Asymptotically
• d ¼ ?
• Rlogn/2n ? 0 ?

33
Good Codes

• Can we explicitly build asymptotically good
  (linear) codes ?
• asymptotically good constant R, d gt 0 as n?8
• Explicit polytime constructable / logspace
  constructible

34
Asymptotically Good Codes
35
Asymptotically Good Codes

• GV tells us that most linear functions of a
  certain size are good error-correcting codes
• Can find a good code in brute-force
• Use brute force on inner-code, where the alphabet
  is exponentially smaller!
• Do we really need to search ?

36
Wozencraft Ensemble

• Consider the following set of codes
• such that
  (?R1/2) (
• Notice that (on board)

37
Wozencraft Ensemble

• Lemma There exists an ensemble of codes c1,..cN
  of rate ½ where N qk-1 such that for at least
  (1-e)N value of i, the code Ci has distance di
  s.t.
• Proof (on board), outline
• Different codes have only 0n in common
• Let yCa(x), then, If wt(y)ltd
• ? y in Ball(0n, d)
• ? there are at most Vol(n,d) bad codes
• For large enough n2k, we have Vol(n,d) eN

38
Wozencraft Ensemble

• Implications
• Can construct entire ensemble in O(2k)O(2n)
• There are many such good codes, but which one do
  we use ?

39
Justesen Code

• Concatenation of
• Cout - RS code over
• a set of inner codes
• Justesen Code C Cout(C1, C2, .. CN)
• Each symbol of Cout is encoded using a different
  inner code Cj
• If RS has rate R ?C has rate R/2

40
Justesen Code - d

• Denote the outer RS code N,K,DQ
• Claim C has relative distance

41
Justesen Code Proof

• Intuition like regular concatenation, but eN bad
  codes.
• for x?y, the outer code induces Sj xj?yj,
• S D
• There are at most eN js such that Cj is bad and
  therefore at least S- eN D- eN (1-R- e)N
  good codes
• since RS implies DN-(K-1)
• Each good code has relative distance d
• d (1-R- e)Nd

42
Justesen Code

• The concatenated code C is an asymptotically
  good code and has a super explicit construction
• Can take q2 to get such a binary code