Sei sulla pagina 1di 25

Turbo Coding and MAP Decoding - Example 1

Intuitive Guide to Principles of Communications


www.complextoreal.com
Charan Langton

Turbo Coding and MAP Decoding


A step by step example

Extrapolating from Shannon’s theory of communication, an ultimate code would be


one where a message is sent infinite times, each time shuffled randomly. Turbo decoding
is a step in that direction. But this requires infinite bandwidth and who has time for that?
It so turns out that for a decent performance; we do not really need to send the
information infinite number of times, just two or three times provides pretty good results.

The most common form of turbo code uses two (or more) recursive systematic
convolutional (RSC) codes working in parallel on the same information. There is a
parallel structure that uses two or more RSC codes, each with a different interleaver. The
purpose of the interleaver is to offer each encoder an uncorrelated version of the
information, resulting in parity bits from each RSC that are independent. How
independent these parity bits are, is essentially a function of the type and length/depth of
the interleaver, shown below with notation π.

uk
y 1k, s
EC1 y 1k, p

π1 EC2 yk2, p

ykn, p
π n −1 ECn

Figure 1 – A rate 1/(n+1) Parallel Concatenated Convolutional Code (PCC)


Turbo Code

On the receiving side, each decoder is working on the same information, y1,s
k but
n, p
different parity bits, yk produced by the two encoders. This type of structure is called
Parallel Concatenated Convolutional Code or PCCC.

The convolutional codes used in turbo codes usually have small constraint length,
around 3 to 5. Where a longer constraint length is an advantage in stand-alone
convolutional codes, it does not offer better performance for turbo codes (TC) and
increases computation complexity and decoding delay. The codes in PCCC must be

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 2

systematic. The RSC property allows the use of systematic bit as a reference. Then the
independent parity bits from the different encoders are used to assess its reliability.

When two RSC codes are used in a TC, the signal produced is rate 1/3. If there are
three encoders, then the rate is ¼ and so on. Usually two encoders are enough, as
increasing the number of encoders reduces bandwidth efficiency and does not buy
proportionate increase in performance. Figure 1 shows a generic PCCC code.

Turbo codes also come in a serial for, Serial Concatenated Convolutional Code
(SCCC). The SCCC codes have better performance at higher SNRs. Where the PCCC
codes require both constituent codes to be RSC, in SCCC, only the inner code must be
RSC. PCCC codes also seem to have a flattening of performance around 10-6 which is not
present in SCCC. However, this is not a practical problem, since signals usually operate
at low SNRs. The SCCC constituent code rates can also be different as shown below. The
outer code can even be a block code.

PCCC is a special form of SCCC. We can even think of concatenation of


RS/Convolutional codes used in line-of-sight links as a form of SCCC. A Turbo SCCC
may look like as in Fig. 2 with different rate constituent codes.

EC1
π EC2
Rate 1/2 Rate 2/3

Figure 2 – Serially concatenated constituent coding (SCCC)

There are also hybrid versions that use both PCCC and SCCC such as shown in Fig.
3.

EC1
π1

EC2 π2 EC3

Figure 3 – Hybrid Turbo Codes

Another code called Turbo Product Code (TPC) has a very different structure from
PCCC or SCCC. In fact, other than the word turbo in the name, TPC codes have nothing
in common with PCCC or SCCC codes. TPC use block codes instead of convolutional
codes. Two different block codes (usually Hamming codes) are concatenated serially
without an interleaver in between. The function of interleaving is provided by the row-
column mapping from one code to the other. Since the codes are independent and operate
in rows and columns, this alone offers enough randomization that no interleaver is
required.

TPC codes, like PCCC also perform well in low SNR and can be formed by
concatenating any block code. The following shows a TPC code created from
concatenating a (7x5) and a (8x4) Hamming code. The 8x4 code first codes the 4 info bits

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 3

into 8, by adding 4 (p1) party bits. These are arrayed in five rows. Then the 7x5 code
works on these in columns and creates (in this case, both codes are systematic) new parity
bits (p2) for each column. The net code rate is 5/14 ~ 0.33. The decoding is done along
rows and then columns. In real TPC, the block sizes are much bigger than what is shown
here.

Hamming (5x8) (8x5) Hamming (7x8)

(8x4) (7x5)

i i i i p1 p1 p1 p1
i i i i p1 p1 p1 p1
i i i i p1 p1 p1 p1
i i i i p1 p1 p1 p1
i i i i p1 p1 p1 p1
p2 p2 p2 p2 p2 p2 p2 p2
p2 p2 p2 p2 p2 p2 p2 p2

Figure 4 – Turbo Product codes

What makes all these codes “Turbo” is not their structure but the type of decoding
used. It is generally called feedback iterative decoding. This feedback being reminiscent
of turbo charging in automotive terminology. If the structure of a SCCC does not use
iterative coding then it would be just a plain concatenated code, not a turbo code.

Iterative decoding is very powerful. In this tutorial, I give an example of how it helps
to extract the correct results. You can think of it as a 20 questions game, where each
previous guess helps to improve your knowledge of the hidden information.

Maximum a-posteriori Probability (MAP) decoding algorithm


In MPSK or QAM demodulation process, decision about a bit is made by looking at
what decision region the phase or the amplitude falls in. Making decision this way is
called the Maximum Likelihood Detection (MLD).

A similar but better rule is based on knowing the apriori probability of the signal. If a
+1 bit has a probability of 80%, then if the signal falls in the negative decision range, the
MLD will decide it as a -1. However, it is clear that the apriori probability, as here of
80% should be taken into account. A detection/decision method that does this conditional
probability into account is called the Maximum aposteriori Proability (MAP). It is also
called the Minimum Error Rule.

MAP maximizes the output probability based on some knowledge of the input apriori
probabilities and soft output from the demodulator. So it is also called a Soft In – Soft
Out algorithm, or SISO. In communications, this algorithm was first identified by L.
Bahl, J. Cocke, F. Jelinek, and J. Raviv, (Paper titled "Optimal Decoding of Linear
Codes for minimizing symbol error rate", IEEE Transactions on Information Theory, vol.
IT-20(2), pp.284-287, March 1974.). So it is also known as the BCJR algorithm. The
MAP algorithm is related to other algorithms, such as Hidden Markov Model HMM,
which is used in voice recognition, genomics and music processing. Other similar

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 4

algorithms are Baum-Welch algorithm, Expectation maximization, Belief propagation,


Forward-Backward algorithm, and more. BCJR is a complex algorithm, hard to
understand and hard to explain.

The MAP rule uses Baye’s theorem of conditional probability which is expressed this
way.
H1
>
P ( x = +1 | y) P ( x = −1 | y ) (1.1)
<
H2

The right side is probability of receiving a value y, when a +1 was sent. And the left side
is the reverse. Each forms a hypothesis. Since these are conditional probabilities, they can
be written with their apriori probabilities as
H1
>
P ( y | x = +1)P ( x = +1) P( y | x = −1)P ( x = −1) (1.2)
<
H2

The underlined values are the apriori probabilities. This can be expressed as the ratio.
H1
P ( y | x = +1) > P ( x = −1)
(1.3)
P ( y | x = −1) < P ( x = +1)
H2

or
H1
P( y | x = +1)P ( x = +1) >
1 (1.4)
P( y | x = −1)P ( x = −1) <
H2

This is called the operating hypotheses for MAP algorithm. If the ratio is >1, then the bit
is +1 otherwise it is a -1.

If we take log of this equation, we get

P ( y | x = +1) P ( x = +1)
L( y | x) = log + log (1.5)
P ( y | x = −1) P ( x = −1)

L( y | x ) = L( y | x ) + L( x ) (1.6)

The above can be written in more standard form as


L(d ) = Lc ( y) + L(d ) (1.7)

Where d is the bit value and y the received value. The first term on the right side
of the equation is called the Log Likelihood Ratio (LLR) of the channel and the second
term L(x) is the apriori Log Likelihood Ratio of the information bit. Now normally, in
communications, a 0 bit as likely as a 1 bit, so the second term would be zero. (Log
(0.5/0.5) = 0) but when trellis coding is used, the 0 and 1 are not equal likely but depend

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 5

on which state the signal is in. The second number becomes important as we move into
trellis-based decoding.
When using iterative decoding, we can subdivide the first term, called the channel
LLR, into two parts, such that equation 5 becomes
L(d ) = Lc ( y) + L(d ) + Le (d ) (1.8)

Where the last part is called the extrinsic information obtained through the decoding
process.
In addition to MAP algorithm, another algorithm called SOVA, based on Viterbi
decoding is often used for Turbo Codes. SOVA uses Viterbi decoding but instead of
producing hard decision at the end of an iteration, it produces soft values indicating
reliability of the decision. This is then iterated. SOVA is a combination of iterative
decoding with a modified form of Viterbi decoding. SOVA maximizes the probability of
a sequence, whereas MAP maximizes the bit probabilities at each time, even if that
makes the sequence not-possible.

In turbo codes, the MAP algorithm is used iteratively to improve performance.


The number of iteration is often preset. More iteration are done when the SNR is low, and
lesser when SNR is high. Clearly doing 15 iteration maybe a waste of time if signal
quality is good. Instead of making a decision ad-hoc by computing convergence of the
reliability figures, the algorithm is often preprogrammed with a fixed number of
iterations for each SNR. On the average, seven iterations give adequate results and no
more 20 are ever required.

DEC1

π π −1

DEC2

Figure 5 – The decoder start with an initial value and the using its own
parity bits produce an extrinsic value. It passes that value to the next
decoder which does the same thing and passes its back.

The MAP decoding and iterative decoding are two separate concepts. MAP
algorithm is used with trellis based coding systems. The iterative process on the other
hand can be applied to any type of coding including block coding which is not trellis
based.

I am going to concentrate only on PCCC decoding using MAP algorithm. We are


going to do a step-by-step MAP decoding example using a Turbo code in shown
Figure 6 with two RSC encoders. Each RSC has two memory registers. Since it has
only two of these, the coding trellis has four states.

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 6

EC1

uk

y 1k, p
y 1k, s
π EC2

yk2, p
yk2, s

Figure 6 – A rate 1/3 PCCC Turbo code

The rate 1/3 code shown here has two identical Convolutional codes. Note that
since yk2,s is a deterministically reshuffle version of y1,s
k , it is not transmitted. The

second decoder is given yk , and it then deinterleaves it to get its copy of yk2,s which it
1,s

needs for decoding.

The coding trellis for each is given by the figure below. The blue lines show
transitions in response to a 1 and black lines in response to a 0. In notation 1(11), the
first number is the input bit, the next in parenthesis are two code bits. Of these, the
first is what we called the systematic bit, and it is same as the input bit. The second
bit is the parity bit. Each convolutional code uses this trellis for encoding and of
course for decoding.
0(0,0)
s1 = 00 s1 = 00
1(1,1)

0(0,0)
s2 = 01 s2 = 01

1(1,1)

0(0,1)
s3 = 10 s3 = 10
1(1,0)

0(0,1)
s4 = 11 s4 = 11
1(1,0)

Figure 7 – The trellis diagram of the rate 1/2 code RSC code

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 7

Take information bit sequence: 1, 0, 1, 0, 1, 0, 0, which in bi-polar form is shown


in Table 1. Code 1 encodes each of these bits as systematic and parity bits using the
trellis in Fig. 7. The bits are coded in a bipolar format.

Table 1 – Coded bits through first Encoder

Code 1 k=1 k=2 k=3 k=4 k=5 k=6 k=7


Systematic 1 -1 1 -1 1 -1 -1
Parity bit, ENC1 1 1 -1 1 1 -1 -1

The sequence looks like this on the trellis diagram.

s1

s2

s3

s4
k=0 k=1 k=2 k=3 k=4 k=5 k=6 k=7

Figure 8 – Trellis diagram for information (1, -1, 1, -1, 1, -1, -1) through first
encoder

The code 2 receives the same information bits but interleaved by a given pattern.
Interleaving is an important issue in turbo coding. There are several different types of
interleaving patterns other than the row column method you know. Here I have made
up a “pseudo-random” interleaving pattern, shown in Table 2. The deinterleaving
pattern puts the bits back in the original order. Simulations shows that pseudorandom
interleaving works best with turbo codes in AWGN channel.

Table 2 – Interleaving patter from Encoder 1 to Encoder 2 and then


deinterleaving back to Encoder 1 from Encoder 2

Interleaving 1-2 1-3 2-4 3-1 4-5 5-2 6-6 7-7


Deinterleaving 2-1 1-3 2-5 3-1 4-2 5-4 6-6 7-7

For interleaving, the bit in first place goes, place 3, bit in 2nd place goes to 4 and
so on. After reordering the information sequence using this rule (1, -1, 1, -1, 1, -1, -1)
into (1, 1, 1, -1, -1, -1, -1) Encoder 2 codes the new sequence as in Table 3.

Table 3 – Interleaved bits coded by Encoder 2

Code 2 k=1 k=2 k=3 k=4 k=5 k=6 k=7


Systematic 1 1 1 -1 -1 -1 -1
Parity bit, ENC2 1 -1 1 -1 -1 -1 -1

On the trellis diagram this new sequence is coded as shown in Figure 8.

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 8

s1

s2

s3

s4
k=0 k=1 k=2 k=3 k=4 k=5 k=6 k=7

Figure 9 – Trellis diagram for information (1, 1, 1, -1, -1, -1, -1) through
second encoder

The systematic bits from Code 2 are redundant and will not be transmitted. This
gives us a rate 1/3 code if you don’t count the tail bits. The transmitted data is given
in columns or: 1 1 1 -1 1-1 1-1 1 …

Table 4 – Transmitted data, one systematic bit and two parity bits from each
encoder

Transmitted k=1 k=2 k=3 k=4 k=5 k=6 k=7


Systematic 1 -1 1 -1 1 -1 -1
Parity bit, ENC1 1 1 -1 1 1 -1 -1
Parity bit, ENC2 1 -1 1 -1 -1 -1 -1

That’s all for the encoding. On the receiving end, assume that this signal goes
through a AWGN channel with signal Es/N0 = 0.25, which is quite small. A bunch of
errors popup and the received data with some channel gain is received as shown in
Table 5.

Table 5 – Received signal amplitudes (soft data)

Received Data k=1 k=2 k=3 k=4 k=5 k=6 k=7


Systematic bit 2 1 3 -2 2 -4 -5
Parity bit -5 2 -1 -2 1 -2 -1
Parity bit 2 -2 6 -5 -1 5 -6

The parity bits for the second decode have to be de-interlaved which do below
using the rule in Table 3.

Table 6 – Received signal amplitudes (soft data), deinterleaved

Received Data k=1 k=2 k=3 k=4 k=5 k=6 k=7


Systematic bit 2 1 3 -2 2 -4 -5
Parity bit -5 2 -1 -2 1 -2 -1
Parity bit 6 -1 2 -2 -5 5 -6

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 9

The received values are called soft inputs. Although the values in Table 5 are
integers, they don’t have to be. I just picked integers for this example.

We have several errors here, one in systematic bits and four in parity bits.

y1k,s y1k, p
e
L12 (u k )
Le21 (uk ) = 0 DEC1

π −1 π
Lc

DEC2
Le21 (u k )

Figure 10 – Computation of Extrinsic L-values for iteration

Unlike Viterbi and other codes, in MAP algorithm channel SNR effects decoding
and it does that by exaggerating the effect of the systematic bits. So if there is an error
in the systematic bit, it also gets exaggerated. In Table 6, the second systematic bit is
in error, as well as the first and fourth parity bits. The term Lc is the measure of the
signal SNR.

Lc = 4 Es/N0 = 4 x 0.25 = 1.0

Multiply the received signal by Lc which is defined above and is equal to 1.0 for
this example. The magnitude of Lc indicates the condition of the channel. High is
good.

Table 6 – Input signal level multiplied by channel Level


DEC1 Received Data k=1 k=2 k=3 k=4 k=5 k=6 k=7
Systematic bit 2 1 3 -2 2 -4 -5
Parity bit -5 2 -1 -2 1 -2 -1

DEC2 Received Data k=1 k=2 k=3 k=4 k=5 k=6 k=7
Lc ⋅ y 1k, s 2 1 3 -2 2 -4 -5
Lc ⋅ y k2 , p 6 -1 2 -2 -5 5 -6

Decoding

For each time tick k, decoding is done by calculating the L-values of a +1 bit. If it
is positive, the decision is in favor of a +1. Calculation of the L-values or L(uk) is
quite a complex process. The main equation used to calculate the L-value is this. This
equation was developed in Part I.

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 10

∑α 
k −1 (s ') ⋅ β k (s) ⋅ γ k (s ', s)
e

u+
L(uk ) = ⎡ Le (uk ) + Lc ⋅ y1,k s ⎤ + log (1.9)
⎣ ⎦
∑α 
k −1 (s ') ⋅β k (s) ⋅ γ k (s ', s)
e

u−

From this computation, if L(uk) is positive, then uk is decoded as equal to +1.

The first term, Le (uk ) is the a-priori value from Decoder 2. This is the L-value of
the a-priori probability of the bit in question. This sort like fore-knowledge of the bit.
At first, the decoder has no idea what it is. A reasonable thing is to assume it is 0.5.
The second term, Lc y1,sk is computed by multiplying the systematic information with
Lc as in Table 7. This value is the channel L-value and gives an indication of channel
SNR. The third big term with all kinds of items is the a-posteriori probability. This
number is calculated for each trellis segment. The example trellis has eight segments
at each transition, this number is calculated eight times at each time tick. Remember
that we can only have one result for a trellis section, a +1 or -1. The L(uk) calculation
tells us what that number is,

The big equation can be written simply as sum of three pieces of information.

L-apriori – This is our initial guess about a +1 bit in first iteration

L-Channel - This is related to channel SNR and systematic bit and is equal to
Lc ⋅ y1,s
k

Le – this computed term in each iteration is called the a-posteriori L-value.

Eq. 1 can be now written as the sum of these as

= L _ apriori + L _ channel + Le (uk ) (1.10)

The L-channel value does not change from iteration to iteration since is it given
by L _ channel = Lc y1,k s . It may change from sequence to sequence, if the system has a
channel equalizer. Neither the Lc nor the systematic bit changes from iteration to
iteration, So lets called it K. The only two items that change are the a-priori and a-
posteriori L-values.
L(uk ) = L _ apriori + K + L _ posteriori (1.11)

A-priori value goes in, it is used to compute a new a-posteriori probability and
then it can be used to compute L(uk) and is passed to the next decoder. Although in
this example here we compute L(uk) each time, during actual decoding this is not
done. Only a-posteriori metric is computed at each iteration and decoders and keep
doing this either a fixed number of times or until it converges. L-posteriori is also
called Extrinsic information.

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 11

Iteration 1
DEC1 L1 (uk ) = 0.0 + K1 + Le1,1
DEC 2 L2 (uk ) = Le1,1 + K2 + Le1,2
Iteration 2
DEC1 L1 (uk ) = Le1,2 + K1 + Le2,1
DEC 2 L2 (uk ) = Le2,1 + K2 + Le2,2
Iteration 3
DEC1 L1 (uk ) = Le2,2 + K1 + Le3,1
DEC 2 L2 (uk ) = Le3,1 + K2 + Le3,2

In each iteration we compute these two quantities. The Decoder 2 in each


iteration, gets the extrinsic value computed by Decoder 1. It then passes its
extrinsic information to Decoder 1 for the next iteration.
So the whole objective for the calculations is to compute the extrinsic L-value, keep an eye
on it and when it gets big (either negative or positive), compute L(uk) and make a decision.
The computation of these L-values uses an algorithm called Forward-Backward Recursion
and this is of course another name for BCJR and many others.

In equation, we have three terms in the ratio that need to be computed. They are

α k −1 (s ') ⋅ βk (s) ⋅ γ ke (s ', s) (1.12)

The first term is called the forward metric. It is computed by a forward recursion
using the last term called branch metrics. It is given by

αk = ∑αi
s
'
k −1 (s ) γ k (s, s' ) (1.13)

This is a recursive process. The forward branch metric is the product of previous forward
metric times the branch metric. (The reason we call these metrics is that these are not
probabilities in a strict sense, just a measure of probability.) So to compute the forward
metric, we first need to compute branch metrics.

In real situations, if the block is 10,000 bits long, then we would need to compute these
metrics for 10000 times the number of states times 2. The example here has just 7 bits in the
sequence. It has 4 states so we will do this 7 x 4 x 2 = 56 times.

A branch metric is the correlation of the received signal with its trellis values. In a nutshell, if
the received values are the same sign as the bits for a branch, then the correlation will be
high. For each decoder, there are full transition branch metrics which incorporate the
systematic and the parity bits and partial branch metrics which incorporate only the parity
bits.

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 12

π
y1k,s , y1k, p , y k2, p

y1k,s
e
y1k, p
L12 (u k )
DEC1
Initialize
Le21 (uk ) = 0

π −1 π
Lc

Le21 (u k ) DEC2 yk2,s


Decision
L (u k ) y k2, p

Figure 11 – Iteration cycle of MAP Turbo Decoding

1. Computing full branch metrics

The full branch metric is given by the equation

⎡1 ⎤ ⎡ q
⎛ ⎞⎤
∑ ⎜ Lc ⋅ 2 ⋅ y
1 1
γ (s ', s) ∝ exp ⎢ ⋅ Le (c1k ) ⋅ c1k + Lc ⋅ ⋅ y1,k s ⋅ c1k ⎥ exp ⎢ i, p
k ⋅ cki ⎟ ⎥ (1.14)
⎣2 2 ⎦ ⎢ ⎝
⎣ i =2 ⎠ ⎥⎦

The data required is Lc, the systematic value, parity value and trellis coded values.
We compute this for each branch of the trellis for all seven time ticks.

Example: For branch 1-1 (from state1 to state1), k = 1,

y11,s = 2, y11, p =-5, Le = 0

What is Le and why is it zero? Le is our initial L-value. We do not know if the
encountered bit was a 0 or a 1, so we assume that the probability of a 0 is equal to a
probability of 1. The L-value is given by

P (uk = +1)
L(uk ) = ln
P (uk = −1) (1.15)

If 0 and 1 are equally likely, a safe bet, then log of 1.0 is zero. And that is what
Le is. It is the initial value of the Extrinsic L-value. The Extrinsic L-value is the key
information that the decoders share. It works something like this.

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 13

y1k,s y1k, p
e
L12 (u k )
Le21 (uk ) = 0 DEC1

π −1 π
Lc

DEC2
Le21 (u k )

Figure 12 – Feedback nature of the Extrinsic value. The value produced


e
by each decoder1 ( L12 (uk ) ) becomes the input for the next decoder.

This thing – Extrinsic L-value is what goes around from one code to the next.
When normalized. It is also called the a-posteriori probability at the output of a
decoder and becomes the apriori probability for the next decoder in a feedback
manner going round and round and getting larger each iteration. Nice thing about L-
values is that large means better reliability.

Computing Full branch metrics requires five inputs, shown in columns 1-5 below.
Branch value are calculated using equation 10.

Table 7 – Computing Full branch metrics

c1k ck2 Le Lc ⋅ y1,s


k Lc ⋅ yk2,p γ ( s ', s )
-1 -1 0 2 -5 4.4817
-1 -1 0 2 -5 4.4817
-1 1 0 2 -5 0.0302
-1 1 0 2 -5 0.0302
1 1 0 2 -5 0.2231
1 1 0 2 -5 0.2231
1 -1 0 2 -5 33.1155
1 -1 0 2 -5 33.1155

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 14

Table 8 – Full branch metrics for full length of the signal, computed by
DEC1 ⎢2 ⎥
2 ⎣ ⎦ ⎣ i=2 ⎝ 2 ⎠⎦
γ (s ' , s ) k=1 k=2 k=3 k=4 k=5 k=6 k=7
γ (1,1) 4.4817 0.2231 0.3679 7.3891 0.2231 20.0855 20.0855
γ (2 ,3 ) 4.4817 0.2231 0.3679 7.3891 0.2231 20.0855 20.0855
γ (3, 4 ) 0.0302 1.6487 0.1353 1.0000 0.6065 2.7183 7.3891
γ (4 , 2 ) 0.0302 1.6487 0.1353 1.0000 0.6065 2.7183 7.3891
γ (1,3 )
γ ( s ' , s )

0.2231 4.4817 2.7183 0.1353 4.4817 0.0498 0.0498


γ (2 ,1) 0.2231 4.4817 2.7183 0.1353 4.4817 0.0498 0.0498
γ (3, 2 ) 33.1155 0.6065 7.3891 1.0000 1.6487 0.3679 0.1353
γ (4 , 4 ) 33.1155 0.6065 7.3891 1.0000 1.6487 0.3679 0.1353

2. Compute partial branch metrics

These values are based only on parity bits. It is the last part in Eq. 1. We will use
these to compute the total extrinsic L-values.

The equation for calculating the partial metrics is given by

⎡ q
⎛ 1 ⎞⎤
γ e ( s ' , s ) ∆ exp ⎢∑ ⎜ Lc ⋅ ⋅ yki , p ⋅ cki ⎟⎥ (1.16)
⎣ i=2 ⎝ 2 ⎠⎦

Note that there is nothing here that changes from iteration to iteration, so these
only need to be computed once.

Table 9 – Partial branch metrics stay constant for each decoder


γ ( ) γ ( , ) p⎢∑ ⎜
⎣ i= 2 ⎝ 2
γ e
(s ' , s ) k=1 k=2 k=3 k=4 k=5 k=6 k=7
γ e
(1,1) 12.1825 0.3679 1.6487 2.7183 0.6065 2.7183 1.6487
γ e
(2 , 3 ) 12.1825 0.3679 1.6487 2.7183 0.6065 2.7183 1.6487
γ e
(3, 4 ) 0.0821 2.7183 0.6065 0.3679 1.6487 0.3679 0.6065
γ e
(4 , 2 ) 0.0821 2.7183 0.6065 0.3679 1.6487 0.3679 0.6065
γ e
(1,3 ) 0.0821 2.7183 0.6065 0.3679 1.6487 0.3679 0.6065
γ e
(2 ,1) 0.0821 2.7183 0.6065 0.3679 1.6487 0.3679 0.6065
γ e
(3, 2 ) 12.1825 0.3679 1.6487 2.7183 0.6065 2.7183 1.6487
γ e
(4 , 4 ) 12.1825 0.3679 1.6487 2.7183 0.6065 2.7183 1.6487

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 15

Figure 13 - Partial branch metrics on the trellis

Calculation of forward metrics

Encoding always starts in state 1. So we assume that signal can only be in state 1
at time k =0. The initial value of the forward metric is set at 1.0. The other three states
are given value of 0.0. Meaning, these are not possible states. Thereafter the forward
metrics are computed recursively for each state (not branches).

The equation is

∑ α~ k −1 ( s ' )γ k ( s ' , s ) (1.17)


α~ k ( s ) = s'

∑ ∑ α~
s s'
k −1 (s ' )γ k (s ' , s)

Computation of forward metrics will be done in two steps, first we will compute
the numerator at time k, then we will compute the denominator.

s1 1.0 4.4817 4.4817


(.9526)
.2231
s2 .0288
0
(.0063)
4.4817 .6065
.0474
s3 0 (.2231)
.0302
1.6487

s4 0
0
33.1155 (0)
k-1 k k+1

Figure 14 – Computing forward metric

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 16

We will compute the forward metric at state 3, k = 1 and then at state 2, k =2.
Here is how you think about this. There are two branches converging into state 3 at
time k =1. Each of these brings its full branch metric to bear on the converging state.
Computation involves multiplying the branch metrics with the starting state value and
then summing these for all converging branches at the state in question. This is two
branches. We do this for each state based on the specific branches that are coming
into it. We can think of as the amount of traffic these roads are bringing to the
intersection.

We compute the forward state metric at state 3 as

α13 = 1.0 x .2231 + 0.0 x 4.4817 = 0.2231

The only other non-zero forward metric is at state 1, which is

α11 = 1.0 x 4.4817 + 0.0 x 0.2231 = 4.4817

Before we continue on to k = 2, we need to normalize these values, which is


denominator in Eq 13. The purpose of normalization is to control the numerical
overflow which may occur due to the large number of numbers that are being
multiplied together. Before we proceed, we normalize the values at each time. This
gives them more of a flavor of probabilities and makes it easier to understand and
reduces the numerical overload.

To normalize, we use the sum of forward metrics which is 4.7048. Normalizing


the two metrics with this sum, gives us the two values (.9526 and .0474) in col 3,
Table.

Now to move forward to state 2, it has two branches (from s3 and from s4)
converging into it. Its forward metric is calculated by

α11 = 0.0474 x 0.6065 + 0.0 x 0.0302 = 0.0288

We compute these for all four states and we get the following values. The column
is the added to compute the normalizing sum. It is then used in Table to normalize all
forward metrics.

Table 10 – Computation of forward metrics

α~ ( s ) k=0 k=1 k=2 k=3 k=4 k=5 k=6 k=7


α~ (1) 1 4.4817 0.2125 0.0341 0.1622 0.0373 0.3558 0.4780
α~ ( 2 ) 0 0.0000 0.0288 6.8769 0.0521 1.5924 0.7122 0.2287
α~ (3 ) 0 0.2231 4.2691 0.1282 6.9700 0.1021 13.7408 0.9534
α~ ( 4 ) 0 0.0000 0.0782 0.2518 0.0521 0.5961 0.2134 6.7607
Denominator 1 4.7048 4.5886 7.2910 7.2364 2.3278 15.0221 8.4209

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 17

Table 11 – Normalized forward state metrics

α~ ( s ) k=0 k=1 k=2 k=3 k=4 k=5 k=6 k=7


α~ (1) 1 0.9526 0.0463 0.0047 0.0224 0.0160 0.0237 0.0568
α~ ( 2 ) 0 0.0063 0.9432 0.0072 0.6841 0.0474 0.0272
α~ (3 ) 0 0.0474 0.9304 0.0176 0.9632 0.0438 0.9147 0.1132
α~ ( 4 ) 0 0.0170 0.0345 0.0072 0.2561 0.0142 0.8028

3. Computing backward state metrics


~
~
∑β k ( s )γ k ( s ' , s )
β k −1 (s' ) = s
(1.18)
∑∑ α~
s s'
k −2 ( s ' )γ k −1 ( s ' , s )

We assume that the signal will end in state 1. (Tail bits are added to force it end in
state 1.) The ending state is always s1, so it gets a value of 1.0 just as the forward
state metric. The rest of the three states at k = 7 are given a backward state value of
zero. Look at how these propagate backwards.

∑ β (s)γ
s
k k (s ', s)
(1.19)

We will compute the backward state metric at state 3. There are two branches
coming into state s3 at k = 5. So we will compute their backward metrics first. For
state s2, it has two branches coming into it, 2-1 and 2-3.

β 72 = 1.0 x 0.0498 + 0.0 x 1.6487 = 0.0498

β 74 = 0 (It has two branches coming into it but both have starting backward
metrics of 0 so, it is zero as well.)

1.0 s1
.0033
.6065
(.0498)
0 s2
.0005 1.6487
(.001219) .3679
0 s3
0 .0821
(0)
.0821
0 s4
k k+1 k+2

Figure 15 – Computing backward state metrics

Compute all four as shown at k = 7. Now again, we normalize. But we normalize


with the same set of numbers as the forward metrics so they are both on the same

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 18

footing. Divide each backward state metric after computation and normalize before
proceeding to the next time tick.

Table 12 –Backward state metrics


s

~
β (s) k=1 k=2 k=3 k=4 k=5 k=6 k=7
~
β (1) 0.3634 2.7979 2.6319 2.5766 26.8557 20.0855 1
~
β (2) 2.7440 1.3419 0.0987 51.7051 0.0666 0.0498 0
~
β (3) 2.7810 0.2327 7.1484 0.0495 0.001219 0.0000 0
~
β (4) 1.4400 7.2464 7.1484 0.0237 0.0090 0.0000 0

Table 13 –Normalized Backward state metrics

~
β (s) k=1 k=2 k=3 k=4 k=5 k=6 k=7
~
β (1) 0.0772 0.6097 0.3610 0.3561 11.5369 1.3371 1
~
β (2) 0.5832 0.2924 0.0135 7.1451 0.0286 0.0033 0
~
β (3) 0.5911 0.0507 0.9804 0.0068 0.0005 0
~
β (4) 0.3061 1.5792 0.9804 0.0033 0.0039 0
Denominator 4.7048 4.5886 7.2910 7.2364 2.3278 15.0221 8.4209

4. Computing Extrinsic L-values

Now we will compute the Extrinsic L-value. These are given by the product of all
three metrics.

σ k (s) = α k −1 (s ') ⋅ γ ke (s ', s) ⋅ βk (s)

This is multiplication of the forward, backward the partial branch metric for each
branch. We get one value for each branch, or eight values per time as shown below.

Table 14 – Computation of final branch metrics

σ (s ) Branch Product of k=1 k=2 k=3 k=4 k=5 k=6 k=7


α~ (1) γ e (1,1) β (1)
σ (1) ~
1-1 0.9409 0.2137 0.0276 0.0045 0.1568 0.0582 0.0390
α ( 2 ) γ (2 , 3 ) β ( 2 )
σ (2) ~ e ~
2-3 0.0000 0.0000 0.0101 0.0175 0.0000 0.0000 0.0000
α ( 3 ) γ (3, 4 ) β ( 3 )
σ (3 ) ~ e ~
3-4 0.0000 0.2036 0.5533 0.0000 0.0061 0.0000 0.0000
α~ ( 4 ) γ e (4 , 2 ) β ( 4 )
~
σ (4) 4-2 0.0000 0.0000 0.0001 0.0908 0.0003 0.0003 0.0000
σ (1) 1-3 α~ (1) γ e (1,3 ) β~ (1) 0.0485 0.1313 0.0275 0.0000 0.0000 0.0000 0.0000
σ (2) 2-1 α~ ( 2 ) γ e (2 ,1) β~ ( 2 ) 0.0000 0.0000 0.0014 0.1235 0.1370 0.3365 0.0288
σ (3 ) 3-2 α~ ( 3 ) γ e (3, 2 ) β~ ( 3 ) 0.0000 0.0051 0.0208 0.3416 0.0167 0.0004 0.0000
σ (4) α~ ( 4 ) γ e (4 , 4 ) β ( 4 )
~
4-4 0.0000 0.0000 0.0275 0.0003 0.0000 0.0000 0.0000
Ratio (+1/-1) 0.052 0.327 0.131 4.124 0.941 5.756 0.736

Below are the metrics calculated so far.

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 19

Figure 16 – Forward and backward state metrics for the trellis

To compute the L-value we need the ratio of these numbers for the +1 and -1
branches. For that we add the top four branch metrics and divide by the sum of the
bottom four branch metrics to satisfy the following equation

∑α 
k −1 (s ') ⋅ β k (s) ⋅ γ k (s ', s)
e

u+
log (1.20)
∑α 
k −1 (s ') ⋅β k (s) ⋅ γ k (s ', s)
e

u−

Take the natural log of the ratio. This is the extrinsic value output for this
iteration. These values after interleaving go to Decoder 2 and become it’s a-priori
probabilities.

Normally the decoder at this point would stop because it has done its job of
computing the extrinsic value. But I will compute the L-value of the bit to show you
how the bit decisions can be made at any iteration.

The L-value of the bit is given by

= L _ apriori + L _ channel + Le (uk ) (1.21)

At each iteration, we have all three of these as shown in Table 17.

Table 17 – Computing L(uk) and making a decision


Interleave 1-3 2-4 3-1 4-5 5-2 6-6 7-7
L(extrinsic) -2.965 -1.118 -2.035 1.417 -0.060 1.750 -0.306
Le, interleaved -2.035 -0.060 -2.965 -1.118 1.417 1.750 -0.306
Lc ⋅ y k2 , p 2.000 1.000 3.000 -2.000 2.000 -4.000 -5.000
L ek 0.000 0.000 0.000 0.000 0.000 0.000 0.000
L(uk) -0.965 -0.118 0.965 -0.583 1.940 -2.250 -5.306
Decision -1 -1 1 -1 1 -1 -1
Transmitted 1 -1 1 -1 1 -1 -1
Error Error

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 20

If we were to make decision now, there would be one uncorrected error in bit 1.
Notice that we make independent decisions in each time tick, with no regard to the
trellis, so that makes this algorithm a bit error rate minimization algorithm, where in
Viterbi decoding the decision is made at the sequence level, so that makes Viterbi a
sequence error minimization algorithm.

From here the Le, interleaved numbers go to the DEC2 as follows.

Table 18 – Transfer of a-posteriori to DEC2 from DEC1


Values In k=1 k=2 k=3 k=4 k=5 k=6 k=7
Lc ⋅ y 1k , s 2 1 3 -2 2 -4 -5
Lc ⋅ y k2 , p 6 -1 2 -2 -5 5 -6
L ek -2.965 -1.118 -2.035 1.417 -0.060 1.750 -0.306

The Le values for this decoder are no longer 0 as they were for DEC1. They are
now giving a hint as to what was transmitted. So far the relationship is not clear, but
as iterations continue these values become more stable and hopefully have the same
sign as the transmitted bit.

Here are all the tables giving pertinent information for the four iterations. The
code makes errors in earlier iterations but then converges to the correct value. Only
the first iteration is independent, after which, each iteration is slowly modifying the
other. The extrinsic values do converge as we can see in the plot of these value for
each decoder for k = 1 to k = 4.

Table 19 – Summary of decisions after 4 iterations


DEC1 Decisions k=1 k=2 k=3 k=4 k=5 k=6 k=7
Iteration 1 0 0 1 0 1 0 0
Iteration 2 1 0 1 0 1 0 0
Iteration 3 1 0 1 0 1 0 0
Iteration 4 1 1 1 0 0 0 0

DEC2 Decisions k=1 k=2 k=3 k=4 k=5 k=6 k=7


Iteration 1 1 0 0 0 1 1 0
Iteration 2 1 1 1 0 0 0 0
Iteration 3 1 1 1 0 0 0 0
Iteration 4 1 1 1 0 0 0 0

The L-values that are passed back and forth are shown in table 20.

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 21

Table 20 – Extrinsic output and L(uk) after four iterations


DEC1 Extrinsic Output k=1 k=2 k=3 k=4 k=5 k=6 k=7
Iteration 1 -2.96 -1.12 -2.04 1.42 -0.06 1.75 -0.31
Iteration 2 -5.85 -3.09 0.30 -0.70 0.50 -3.16 -0.19
Iteration 3 -5.27 -0.56 0.56 -0.91 -1.61 -3.41 -0.74
Iteration 4 -0.36 2.42 4.64 -6.27 -5.27 -8.14 -6.08

DEC1 L(uk) k=1 k=2 k=3 k=4 k=5 k=6 k=7


Iteration 1 -0.96 -0.12 0.96 -0.58 1.94 -2.25 -5.31
Iteration 2 3.63 -2.59 1.72 -2.91 3.78 -1.69 -10.74
Iteration 3 4.58 4.31 2.13 -4.29 -3.79 -2.01 -6.28
Iteration 4 12.25 9.71 10.89 -12.20 -9.70 -10.80 -16.52

DEC2 Extrinsic Output k=1 k=2 k=3 k=4 k=5 k=6 k=7
Iteration 1 7.48 -0.50 -1.58 -0.21 1.28 5.47 -5.55
Iteration 2 7.85 3.87 -1.43 -1.38 -4.18 5.40 -0.54
Iteration 3 10.61 6.30 3.25 -3.92 -6.43 1.34 -5.43
Iteration 4 18.13 13.25 12.70 -11.72 -13.38 -7.89 -14.96

DEC2 L(uk) k=1 k=2 k=3 k=4 k=5 k=6 k=7


Iteration 1 6.51 -0.62 -0.62 -0.79 3.22 3.22 -10.86
Iteration 2 4.00 1.78 1.87 -4.09 -1.68 -1.77 -5.73
Iteration 3 7.33 6.74 6.81 -6.83 -6.04 -6.08 -11.18
Iteration 4 19.77 16.66 20.34 -19.99 -16.65 -20.03 -26.04

35.00 30.00 35.00

30.00 25.00 30.00

25.00 25.00
20.00

20.00
20.00
15.00

15.00
15.00
10.00
10.00
10.00
5.00
5.00
5.00
0.00
0.00
1 2 3 4 0.00
1 2 3 4
-5.00 -5.00 1 2 3 4

5.00 0.00
0.00 10.00
1 2 3 4
1 2 3 4
-5.00
5.00 0.00
-5.00
1 2 3 4
-10.00
0.00 -5.00
-10.00
1 2 3 4 -15.00
-5.00 -10.00
-15.00 -20.00
-10.00 -15.00
-25.00
-20.00
-15.00 -20.00
-30.00
-25.00
-20.00 -25.00
-35.00

-30.00
-25.00 -30.00 -40.00

-35.00 -30.00 -35.00 -45.00

Figure 17 – Extrinsic values for bits 1, 1, 1, 0 , 0, 0, 0 over 4 iterations, for


DEC1 and DEC2

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 22

Modifications of MAP algorithm

Serial Iterative Decoding

The algorithm just described is for a parallel Iterative decoder for a PCCC code. The
same algorithm with minor modification can be applied to Serial concatenated codes
(SCCC). The general process is shown figure below. The main difference in decoding a
SCCC code is that instead of DEC2 waiting for the initial decision from DEC1, DEC2
also computes its own Output extrinsic value independent of the estimate from DEC1. In
other words, both assume the initial extrinsic value to be 0.0. The inner decoder produces
the extrinsic value of code bits only. The outer decoder uses both the estimate from inner
decoder and produces extrinsic value based on code bits and information bits. The major
difference in the two methods is one, that of starting point for outer DEC and second the
values have to be deinterleaved instead of interleaved as would be done for PCCC.

The performance of SCCC is better in the high SNR range. PCCC suffer from a flattening
of the BER vs. Eb/N0 curve around SNR of 6-8 dB, whereas SCCC don’t seem to have
this problem.

However, keep in mind that most communication systems are power limited, so operate
in the low SNR range where PCCC work well. This is why PCCC codes are used more
often.

Outer
Encoder π Inner
Encoder

π
e
y1k, p L12 (u k )
Inner Le21 (u k )
Decoder π −1 Outer
y1k,s Decoder L(u k )
Hard decision based on
π −1 sign of L(u k )
yk2, p

Figure 18 – Serial concatenated code and its decoding process

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 23

Modifications of MAP algorithm

Log-MAP

In MAP algorithm, despite having normalized many of the values, the numbers
can still get out of hand. The length of the sequence is large so the number of multiply
operations add large delays. Since adding is a much faster machine process, we can
convert all multiplication in the log domain to addition.

In calculating the extrinsic metric, all forward, backward and transition metrics
are converted to logs. Then they can be added instead of multiplied. The following
approximation is used to add the three numbers.

Here is the operating equation for converting a multiplication to a sum.


− a −b
ln(e a + e b ) = max(a, b) + ln(1 + e ) (1.22)

In Table 9 and 14, we are multiplying a lot of exponentials and the application of the
above equation turns all of them into simpler machine process. Max function is easy and
the last term can be turned into a look-up table of say a few numbers.

For example,

If we have a = 7 and b = 5, we get

−2
ln(e 7 + e5 ) = max(7,5) + ln(1 + e ) = 7 + .127 = 7.127

In the second terms, the larger the difference between the numbers, the
smaller the last term gets. So we can just store a few values such as below and
then look them up as needed.

0.2 0.598138869
0.5 0.474076984
1 0.313261688
1.5 0.201413278
2 0.126928011
2.5 0.078889734
3 0.048587352
3.5 0.029750418
4 0.018149928

This simplifies the calculations a lot. This is called the Log-MAP version.

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 24

Max-Log-MAP

We can also ignore the last term. This reduces the number of operations, but
results in higher BER, which is obvious isn’t it. When two numbers are close, we
need to be especially careful about the decision and this method instead forces the
decision in the favor of the slightly larger number. This modification is called Max-
Log-MAP. It is overall not as accurate as the Log-MAP but is faster. Although by
doing more iteration, its accuracy can be as good as the Log-MAP.

Charan Langton
Complextoreal.com
I often make changes/corrections in these tutorials, See website for most recent version of
this and other tutorials. - February 2007

Revised June 2007

Find any typos, errors? Please email me. mntcastle@earthlink.net

Bibliography

1. On-line paper by Dr. Bernard Sklar, Maximum a Posteriori Decoding of Turbo Codes
This is from his book, which is of course well worth buying.

2. Paper by S. Benedetto, D. Divsalar, G. Montorsi, F. Pollara


http://www.hs-zigr.de/~jvogt/paper/el2.pdf

3. Tech online, Course on Turbo Codes: From Theory to Practice

4. Turbo Code Tutorial, William E. Ryan, New Mexico University, Online paper

5. Turbo Code in IS-2000 Code Division Multiple Access Communications Under Fading
by Jian Qi, Thesis, 1999, http://hometown.aol.com/jxqi/qi/thesis/ (Contains a worked out
example for one decoder.)

6. Material by Binton Cooper on Convolutional codes,


http://www.ece.jhu.edu/~cooper/courses/466/09va.pdf

7. Turbo Decoding, a brief introduction, charts by Brinton Cooper III


http://www.ece.jhu.edu/~cooper/courses/466/12turbo.pdf

8. http://www.mit.edu/~6.454/www_fall_2000/jnl/divsalar-tm.pdf

Copyright 2006 Charan Langton www.complextoreal.com


Turbo Coding and MAP Decoding - Example 25

9. http://www.ee.vt.edu/~yufei/turbo.html
Matlab code

10. Turbo Codes for Deep Space Communications, paper by D. Divalsara and F. Pollara

11. http://www.iterativesolutions.com/Matlab.htm by Matthew Valenti

12.
http://www.commsdesign.com/design_center/3gwireless/design_corner/OEG20020424S0
003

13. Paper by Hamid R. Sadjadpour


http://www.soe.ucsc.edu/~hamid/PSI00073.pdf

14. Fundamentals of Turbo Coding, paper by Bernard Sklar

Copyright 2006 Charan Langton www.complextoreal.com

Potrebbero piacerti anche