Arithmetic Coding

1
Arithmetic Coding

• The bits allocated for each symbol can be
  non-integer
• If pdf(a) 0.6, then the bits to encode a is
  0.737
• For the optimal pdf, the coding efficiency is
  always better than or equal to the Huffman coding
• Huffman coding for a2 a1 a4 a1 a1 a3, total 11
  bits
• Arithmetic coding for a2 a1 a4 a1 a1 a3, total
  11.271 bits

2 1 3
1 1 3
2.32 0.737 3.74
0.7370.737 3
The exact probs are preserved
2
Arithmetic Coding

• Basic idea
• Represent a sequence of symbols by an interval
  with length equal to its probability
• The interval is specified by its lower boundary
  (l), upper boundary (u) and length d (
  probability)
• The codeword for the sequence is the common
  bits in binary representations of l and u
• The interval is calculated sequentially starting
  from the first symbol
• The initial interval is determined by the first
  symbol
• The next interval is a subinterval of the
  previous one, determined by the next symbol

3
An Example
Any binary value between l and u
can unambiguously specify the input message.
½(10)(011)
¼ (010)(0011)
d(?ab?)1/8
4
Implementation of Arithmetic Coding

• Rescaling and Incremental coding
• Integer arithmetic coding
• Binary arithmetic coding

5
Issues

• Finite precision (underflow overflow) As n
  gets larger, these two values, l(n) and u(n) come
  closer and closer together. This means that in
  order to represent all the subintervals uniquely
  we need to increase the precision as the length
  of the sequence increases.
• Incremental transmission transmit portions of
  the code as the sequence is being observed.
• Integer implementation

6
Rescaling Incremental Coding
7
Incremental Encoding
U
U
L
L
L
U
8
Question for Decoding

• How do we start decoding? decode the first symbol
  unambiguously
• How do we continue decoding? mimic the encoder
• How do we stop decoding?

9
Incremental Decoding
0.312(0.6-0.312)0.8
U
1.0
Top 18 of 0,0.8)
0.82
0.8
10
Integer Implementation
11
Integer Implementation

• nj the of times the symbol j occurs in a
  sequence of length Total Count.
• FX(k) can be estimated by
• Define
• we have
• E3 if (E3 holds)
• Shift l to the left by 1 and shift 0 into LSB
• Shift u to the left by 1 and shift 0 into LSB
• Complement (new) MSB of l and u
• Increment Scale3

12
(8.5.7) From 5 Table 4.3, the optimal bl and
gl for a source with arbitrary ?f and ?f2