Error Correcting Codes

1
Error Correcting Codes

• by
• Paul Bates

2
Introduction

• Sending codes
• Pictures from space, satellite transmitting
  pictures to Earth of Mars
• Disturbances in transmission
• Space, earths atmosphere, solar activity this
  goes for all transmissions
• Error correction
• Why it is needed to ensure the accuracy and
  integrity of data
• No electronic data transmission or storage system
  is perfect

3
Vocabulary

• Error correction the process of detecting bit
  errors and correcting them3
• Digital systems of 0s and 1s (bits get
  interpreted as the other)
• A word in binary group of 0s and 1s
• ex) The number 5 in binary 00101 (if using 5
  bits)
• 5 bits of information can represent numbers from
  0-31

4
Overview

• History of error correction
• Pioneers
• Concept of error correction
• How it works
• Examples
• Using the Reed-Muller code
• Applications
• Where can we use error correction?

5
History

• Claude Shannon created the subject of
  information theory in the late 1940s. He
  concluded that the best way to get the most
  storage capacity or the fastest data transmission
  is through powerful error correcting systems.
• At the same time Richard Hamming discovered a
  single bit error correcting code.4

6
History (contd)

• Other researchers include Irving Reed and Gustave
  Solomon (not pictured) who discovered codes for
  arbitrary number of bits but no known decoder for
  the code.
• Elwyn Berlekamp and James Massey discovered
  algorithms to build decoders for multiple error
  correcting codes.4

7
Concept of Error Correction

• Group of bits is referred to as a word
  comprised of 0s or 1s
• ex) 5 00101 (without error correction)
• 01011010010110100101101001011010
• (with added error correction)
• Error correction adds extra or redundant
  letters to words
• The extra letters give structure to words

8
Concept of Error Correction (contd)

• If structure is altered by errors, the word can
  be corrected up to a certain point5
• English language is a good example4
• not all combinations of English letters are real
  words
• errors while transmitting English words can be
  found if they are not in the dictionary
• errors can be corrected by determining which
  legitimate English word is closest to the
  received word
• error correcting systems work similar to this

9
How frequent are bit errors?

• Some electronic devices produce more bit error
  rates than others4
• for example, optical disks and magnetic tape have
  higher error rates than magnetic disks
• fiber optic cable has a low memory rate
• Bit error rate
• Magnetic disks 1 bit error per 1,000,000,000
• Optical disks 1 bit error per
  100,000

Number of bit errors
Total bits transferred
10
How frequent are bit errors? (contd)

• Measuring bit error rate how many bit errors
  per unit of time
• Magnetic disks if they transfer 1 million bits
  per second, on average there is 1 bit error every
  thousand seconds or roughly every 16 ½ minutes
• If you transfer 1 billion bits per second, on
  average that is
• 1 bit error every second
• Currently, some storage drives transfer 40
  million bits per second so a bit error occurs
  once every 25 seconds
• Without error correction, most storage devices
  would be unreliable so in the future, error
  correction for faster drives is a necessity

11
The Hadamard Matrices

• The Reed-Muller code relies on the Hadamard
  Matrix defined as

• 1
• 1 -1

H1
Hn1 Hn H1
- Denotes the Cartesian Product of matrices
12
The Hadamard Matrices (contd)

• 1
• 1 -1

H1

• Substitute H1 into H2

H1 H1 H1 - H1

• Take the (-) sign on the bottom right
• into account

H2

• 1
• 1 -1

• 1
• 1 -1

• 1
• 1 -1

• 1
• 1 -1

• 1
• 1 -1

• 1
• 1 -1

• 1
• 1 -1

-1 -1 -1 1
-

• Repeat this process until desired word length
  has been achieved!

13
The Hadamard Matrices (contd)

• Next replace
• 1 by 0 and -1 by 1 to obtain a visual
  representation of the matrices1
• In the upcoming square grids,
• 0 white
• 1 black

14
The Hadamard Matrices (contd)

• To encode a word
• ex) 1 bit word (numbers 0-1) where n 1

Row 0 number 0
0 0 0 1
H1
Row 1 number 1
n number of bits
0
2n 21 2 rows ( 0-1 )
M1
1
15
The Hadamard Matrices (contd)

• To encode a word
• ex) 2 bit word (numbers 0-3)

Row 0 number 0
0 0 0 1
0 0 0 1
Row 1 number 1
H2
0 0 0 1
1 1 1 0
Row 2 number 2
Row 3 number 3
0
n number of bits
2n 22 4 rows ( 0-3 )
1
M2
2
Notice that each row is different than any
other row
3
16
The Hadamard Matrices (contd)

• For example2 continue this process from M1 to M5

M1
(2 bits)
17
The Hadamard Matrices (contd)

• For example2 continue this process from M1 to M5

M2
(4 bits)
18
The Hadamard Matrices (contd)

• For example2 continue this process from M1 to M5

M3
(8 bits)
19
The Hadamard Matrices (contd)

• For example2 continue this process from M1 to M5

M4
(16 bits)
20
The Hadamard Matrices (contd)

• For example2 continue this process from M1 to M5

M5
(32 bits)
21
The Hadamard Matrices (contd)

• A unique property to these matrices is every line
  differs from all others in exactly 16 places! 1

32 bits
2
16 places where each row differs
22
Back to the space problem

• Mariner vehicle takes pictures of Mars and sends
  them back to Earth
• Pictures consist of pixels each on a scale from
  white to black with 32 bits (0-31)

white
0 1 . . . 30 31
shades of gray
black
23
Back to the space problem (contd)

• The pictures utilize the Reed-Muller code1
• 0 - 31 numbers needed (32 bits)

What level matrix do we need?
2n 32
n 5
- We need matrix M5 to encode a 32 bit word
24
Back to the space problem (contd)

• For example a single pixel is sent with a color
  code of 5
• 00101 (without error correction)
• 01011010010110100101101001011010
• (with added error correction)

25
Back to the space problem (contd)

• 01011010010110100101101001011010

The number 5 represents line number 5
M5
26
Back to the space problem (contd)

• What happens if the message gets errors?
• 01011010010110100101101001011010
• 01001010000101100111101101111010

We picked up 7 errors on the way back! Can we
still recover?
27
Back to the space problem (contd)

• Yes! The Hamming Distance states that two binary
  numbers differ in d places.1
• Therefore, users can detect and correct up to

So for 32 bits
d 1
16 1
errors!
up to 7 errors can be corrected!
2
2
28
Back to the space problem (contd)
(received)
(line 5)
25
7
Matches
25
78.1


x 100
Total
32
(received)
(line 29)
19
13
Matches
19
59.4


x 100
Total
32
29
Applications

• Space communication
• Computer disk storage
• Digital media such as music
• Digital wireless
• communications

30
Summary

• Error correction can be a useful tool to limit
  transmission errors
• One method of error correction is the Reed-Muller
  code
• We can correct up to a certain amount of errors
  in the code
• Wide variety of applications for error correction

31
References

• Dewdney, A.K. The New Turing Omnibus. Owl Books,
  New York. 1993 (pages 77-81)
• http//mathworld.wolfram.com/HadamardMatrix.html
• http//dict.die.net/error20detection20and20corr
  ection/
• http//members.aol.com/mnecctek/faqs.html
• http//www.sims.berkeley.edu/rosario/projects/err
  or_correcting_codes.html