James L. Massey

1
ITS2002, Natal, Brazil 10 September 2002
Turbo Codes, Space-Time Codes, and What Next?
James L. Massey Prof.-em. ETH Zürich, Adjunct
Prof., Lund Univ. Trondhjemsgade 3, 2TH, DK-2100
Copenhagen, Denmark JamesMassey_at_compuserve.com
2
Questions that we will try to answer

• Turbo-Coding -- What are turbo codes and why
  are they interesting?
• Trellis-Coded Modulation -- What made this
  such an interesting kind of coding?
• Space-Time Coding -- What are space-time codes
  and why are they of interest?
• What kind of hyphenated-coding can we expect
  next?

3
Turbo-Coding began in Geneva at ICC93.
C. Berrou, A. Glavieux and P. Thitimajshima,
Near Shannon limit error-correcting coding and
decoding Turbo codes, Proc. IEEE Intl. Conf. on
Communications, Geneva, 1993, pp. 1064-1070.
Berrou et al. showed, for the first time, that it
was practical to build communication systems that
operate very close to the "Shannon limit," i.e.,
very close to the capacity of the Gaussian
channel. Previously, it was widely believed
that the cut-off rate, R0, was the practical
limit. Many, perhaps most, in the audience
thought that Berrou et al. had made the usual 3
dB mistake of not accounting for the rate loss
due to the redundancy of the code.
4
Berrou et al.s curves that stunned the
communications world in 1993! Performance within
0.7 dB of capacity at a bit error rate of 10-5
with eighteen decoding iterations.
5
Info. bits
Rate 2/3 RSC
Punctured parity bits
Decoupling Trick 256?256 interleaver
Rate 2/3 RSC
Berrou et al. R 1/2 Turbo code
6
Turbocharging
extrinsic information
bit log-likelihood ratio
decoupling trick
Berrou et al.s Turbo Decoder
7
Typical behavior of the bit-error probability for
a Turbo decoder.
Pb
100
10-1
10-2
waterfall region
error floor
Eb /No (dB) normalized
0
1
2
3
theoretical limit
8
Principle 1 Appropriately-coupled
low-complexity codes can be as powerful at low
S/N ratios as a single much more complex code.
It is true that the error floor (or high S/N
ratio behavior) is essentially determined by the
(not very good) minimum distance of the overall
code formed by the coupling of the low-complexity
codes, but, at low S/N ratios, appropriate
coupling of the codes gives a quasi-random
structure to the codewords that allows one to
reach the performance predicted by the random
coding bound.
Appropriate coupling means that the successive
decoding iterations enjoy a rough independence.
9
. . . coding in the beyond-R0 regime is in its
early stages, and theoretical understanding is
still weak. At R0, it seems that a phase
transition occurs from a static regime of
regular code structures to a statistical regime
of quasirandom codes . . .
G. D. Forney, Jr., and G. Ungerboeck,
Modulation and Coding for Linear Gaussian
Channels, IEEE Trans. Inform. Theory, vol.
IT-44, Oct. 1998, pp. 2384-2415.
But there has been some progress!
N. Wiberg, H.-A. Loeliger and R. Kötter, Codes
and Iterative Decoding on General Graphs,
European Trans. Telecomm., vol. 6, Sept./Oct.
1995, pp. 513-525. B. J. Frey, R. Koetter and A.
Vardy, Signal -Space Characterization of
Iterative Decoding, IEEE Trans. Inform. Theory,
vol. IT-47, Feb. 2001, pp. 766-781.
10
Principle 2 Random interleaving of the
information bits between codes is an appropriate
way to couple low-complexity convolutional codes.
S. Benedetto and G. Montorsi, Unveiling Turbo
Codes Some Results on Parallel Concatenated
Coding Schemes, IEEE Trans. Inform. Theory, vol.
IT-42, March 1996, pp. 409-428.
The original Berrou et al. interleaver had a
memory of 65,536 bits and was chosen at random!
But interleavers with only 160 bits of memory
give surprisingly good performance!
F. Daneshgaran and M. Mondin, Optimized Turbo
Codes for Delay Constrained Applications, IEEE
Trans. Inform. Theory, vol. IT-48, Jan. 2002, pp.
293-305, Jan. 2002.
11
Question Couldnt we do better by
maximum-likelihood (ML) decoding of the overall
code rather than by turbo decoding of this code?
Answer Sure, but this misses the point. The
dimension of the state space of the overall code
is so large that it is completely infeasible to
do ML decoding.
Principle 3 Turbo decoding of
appropriately-coupled low-complexity codes is
nearly as good at low S/N ratios as
maximum-likelihood decoding.
If the first iteration improves matters, the
turbo decoder will usually converge to the ML
solution. This explains the waterfall region
performance region.
12
Question What is Turbo-coding? My Answer
Any coding scheme using iterated decoding of
coupled codes.
The work of Berrou et al. touched off a frantic
search for other Turbo coding schemes.
A very good Turbo coding scheme turns out to be
Gallagers old low-density parity-check codes.
R. G. Gallager, Low-Density Parity-Check Codes.
Cambridge, MA, M.I.T. Press, 1963.
Gallagers decoding algorithm for these codes is
in fact a Turbo decoder, but in the 1960s it
was not possible to perform simulations to
determine its performance.
13
What is new about Turbo Coding is the method of
decoding (partially separate, then iterate
alternately), not the kind of codes.
The principles of Turbo decoding are being
applied in other areas! Here are a few recent
instances that caught my eye

• Z. Yang and X. Wang, Blind Turbo Multiuser
  Detection for Long-Code Multipath CDMA, IEEE
  Trans. Commun., vol. COM-50, Jan. 2002, pp.
  56-64.
• D. Reynolds and X. Wang, Turbo Multiuser
  Detection with Unknown Interferers, IEEE Trans.
  Commun., vol. COM-50, April 2002, pp. 616-622.
• M. Tüchler, R. Koetter and A. C. Singer, Turbo
  Equalization Principles and New Results, IEEE
  Trans. Commun., vol. COM-50, May 2002, pp.
  754-767.

14
What an information theorist sees when
he/she looks at a single-sender communication
system.
Information Source
Source Encoder
Channel Encoder
Modulator
Waveform Channel
Information Bits
Information Sink
Source Decoder
Channel Decoder
Demodulator
A prominent trend in communications has been the
combination of two or more functions into a
single unit, which is why we get
hyphenated-coding!
15
Examples of combination
Joint Source/Channel Encoder
Source Encoder
Channel Encoder
Channel Encoder
Codulator
Modulator
Trellis-coded modulation is of this type. So is
space-time coding.
Why not
???
Source Encoder
Channel Encoder
Socodulator
Modulator
16
Why combination is so tempting You cannot lose!
Theorem The best (by any optimality criterion)
achievable with a separate source encoder,
channel encoder and modulator is not better than
what can be done with a single socodulator.
Proof
Source Encoder
Channel Encoder
Modulator
Socodulator
17
Information Source
Source Encoder
Channel Encoder
Modulator
Waveform Channel
Information bits for coding channel
Information Sink
Source Decoder
Channel Decoder
Demodulator
The separation principle asserts that under
mild conditions there is no loss of optimality in
this divide and conquer approach to
communications.
i. e., you usually cannot win by combination! And
you pay a big price in loss of flexibility.
18
There is a conceptual combination that makes
sense!
Modulator
Coding Channel
Waveform Channel
Demodulator
The purpose of the modulation system is to create
a good coding channel!
Those engineers working with the waveform channel
have the most freedom--next comes those working
on the modulation system. The coding engineers
are at the mercy of both groups.
19
Ungerboeck's Most Important Contribution Observin
g that Fourier bandwidth depends only on the
number of modulation components per second (i.e.,
dimensions of signal space per second). ? All
forms of phase-shift keying (binary PSK, QPSK,
8-PSK, etc.) use the same Fourier bandwidth.
Increasing the number of phases or the number of
amplitude levels in a modulation symbol does not
expand bandwidth!
The following coded system uses the same
bandwidth as an uncoded QPSK system
bits
Rate 2/3 Binary Encoder
phase symbols
information bits
8-PSK Modulator
Both give 1 information bit per dimension of
signal space!
20
In the 1980s Ungerboecks Trellis-Coded
Modulation became the new buzzword in
communications engineering and revolutionized
narrowband communication systems. Does
trellis-coded modulation really call for a joint
encoder and modulator? NO! What it calls for is
the design of the coding system to match the
unusual (for people used to think in terms of
Hamming distance) coding channel created by the
multi-phase and/or multi-amplitude modulation
system. Ungerboecks Subset partitioning of
modulation signal sets is one technique for such
matching.
21
010
001
011
000
100
101
111
110
8-phase modulation signal set with natural
mapper. (Hamming distance has little relevance
here!)
22
Trellis-coded modulation is nothing more (or
less) than trellis coding for multi-phase and/or
multi-level modulation.
But why is the system
bits
Rate 2/3 Binary Encoder
phase symbols
information bits
8-PSK Modulator
a good idea, while the system
bits
Rate 1 Binary Encoder
phase symbols
information bits
QPSK Modulator
is a bad idea?
Both use 1 dimension of signal space per
information bit.
23
? signal-set
8-PSK
b)
QPSK
G. Ungerboeck, Channel Coding with
Multi-Level/Phase Signals, IEEE Trans. Inform.
Th., Vol. IT-28, No. 1, Jan. 1982, pp. 55-67.
24
8-PSK modulation creates a significantly better
coding channel than does QPSK in the region of
signal-to-noise ratios where one wishes to
operate efficiently.
Capacity (or R0) curves are the only ones that
should be allowed to be published to describe the
performance of modulation systems (as opposed to
coding systems). In particular, probability of
error versus signal-to-noise ratio curves should
be strictly forbidden!
The purpose of the modulation system is to create
a good coding channel!
25
Space-Time Coding
Some words from scripture
We can view our work as combined coding and
modulation for multi-input (multiple transmit
antennas) multiple-output (multiple receive
antennas) fading channels.
V. Tarokh, N. Seshadri and A. R. Calderbank,
Space-Time Codes for High Data Rate Wireless
Communication Performance Criterion and Code
Construction, IEEE Trans. Inform. Th., Vol.
IT-44, No. 2, Mar. 1998, pp. 744-765.
This seems to be the first use of the terminology
space-time code, which like turbo coding and
trellis-coded modulation, is now a buzzword.
This emphasis on the method of coding motivated
the choice of outage capacity (rather than
Shannons capacity) as the measure of achievable
performance.
26
By combined coding and modulation for
multi-input multiple-output fading channels did
Tarokh, Seshadri and Calderbank mean a joint
coding and modulation system or coordinated
design of the coding and modulation system?
This is not clear.
M. O. Damen, K. Abed-Meraim and J.-C. Belfiore,
Diagonal Algebraic Space-Time Block Codes, IEEE
Trans. Inform. Th., Vol. IT-48 No. 2, Mar. 2002,
pp. 628-636.
An ST block code associates with each
information symbol vector x (x1, . . ., xd) for
d gt 0, an n ? l matrix B(x) with entries bjt, j
1 . . . n, t 1 . . . l, such that bjt is sent
over transmit antenna j at time t.
This sure sounds like joint coding/modulation! Thi
s paper is loaded with error probability versus
signal-to-noise ratio performance curves but
says nothing about capacity of any kind.
27
Wittnebens space-time code
w(t)
Information bits (?1)
unit delay
w(t)
R 1/2 binary encoder
2-bit modulator
A. Wittneben, A new bandwidth efficient transmit
antenna modulation diversity scheme for linear
digital modulation, in Proc. ICC93, pp.
1630-1634.
Space-time coding is just coding for modulation
of multiple-antenna signals.
28
Where are the capacity results for space-time
coding?
I mean Shannons capacity, or at least outage
capacity (which was introduced by Foschini and
Gans for space-time coding and has some
validity), but not other measures of performance
to which people have attached the name capacity.
To evaluate Shannons capacity, you must look
only at the coding channel (created by the
modulator and waveform channel). The code
designer must then figure out how to use the
available capacity efficiently.
The purpose of the modulation system is to create
a good coding channel!
29
E. Teletar, Capacity of Multi-Antenna Gaussian
Channels, European Trans. Telecommun., Vol. 6,
Nov.-Dec. 1999, pp. 585-595.
0 ? P ? 35 dB in 5 dB increments.
r of receive antennas t of transmit
antennas C capacity in nats
There are no codes whatsoever in this paper (and
no plots of error probability versus
signal-to-noise ratio). Teletar was studying the
coding channel! For this he deservedly received
the IT-Society Paper Award!
30
What next?
Quantum-Codes ??? -- These are already
around! Network-Codes ??? -- These will come
soon! There are already the beginnings of
the new field of network information
theory. Extrasensory-Codes ??? -- Well, maybe!
You can be sure that all the new hypenated codes
will not actually be new types of codes, but
rather codes designed for new types of coding
channels!
The purpose of the modulation system is to create
a good coding channel!