2011EE10509, Digital Electronic Circuits, Semester III, IIT Delhi

VITERBI ALGORITHM: CONVOLUTIONAL CODING AND DECODING

G. Sharma, Second year Undergraduate, Electrical Engineering, IIT Delhi
1

AbstractConvolutional codes are extensively employed in the digital communication systems owing to their easy error correction property. This paper describes and illustrates the (a) The principle of convolutional codes and realization of encoders as FSMs, (b) The basic Viterbi Algorithm and (c) The application of Viterbi algorithm to decode convolutional codes. It aims at providing a simplified understanding of concepts involved in convolutional coding and decoding. Index TermsDecoding, Viterbi Algorithm, Convolutional Codes, Encoders. I. INTRODUCTION The Viterbi algorithm is a recursive solution to the problem of finding the most probable sequence of hidden states the Viterbi path of a discrete time finite state Markov Process of a memoryless system[10]. A Viterbi decoder uses the Viterbi algorithm to decode a bitstream coded using the convolutional coder. [1]This method was invented by Andrew Viterbi in 1967 and hence, holds his name. The algorithm was proposed as a method for decoding convolutional codes over noisy digital communication links. [2]Hence the algorithm finds immense application in CDMA and GSM digital cellular, dialup modems, satellite and deep-space. Apart from its use in the digital communication system, The Viterbi algorithm also finds use to solve problems pertaining to HMM. [2]A generalization of the Viterbi algorithm, called the max-sum algorithm can be used to find the most probable assignment of variables in a large number of graphical models, e.g. Bayesian networks, Markov random fields and conditional random fields. [3]Owing to its use in dynamic programming algorithms to find a single most possible explaination to any problem, the Viterbi algorithm also aids in solving economic and other survey based problems. This paper discusses the application of the Viterbi algorithm in the coding and decoding of convolutional codes. II. CONVOLUTIONAL CODES A convolutional code is an error correcting channel code, first introduced by Elias in 1955 as an alternate to block codes[4]. [1] Channel coding aims to improve the channel capacity by

deliberately adding some redundant information to the data being transmitted through the channel. Convolutional coding and block coding are the two major forms of channel coding. Convolutional codes differ from block codes considering that block codes encoders are memoryless systems [8]. In conventional convolutional code encoders, the encoder outputs at any given time not only depend on the inputs at that time unit but also on some of the previous inputs [13]. These previous inputs form the memory order k[7]. Initially, sequential decoders proposed by Wozencraft and threshold decoders proposed by Massey were used to decode convolutional codes[4]. However, with the introduction of the Viterbi algorithm, despite of its high resource consuming nature, its maximum likelihood decoding principle, made it best suited for decoding convolutional codes. A. Convolutional encoders Convolutional codes operate on serial data; they accept a continuous stream of bits and map them into an output stream. A (m, n, k) convolutional code can be implemented with an ninput, m-output linear sequential circuit with input memory k [5]. The code rate of an encoder is defined as (m/n). A conventional encoder(fig 2(a)) is fundamentally a finite state machine that generates codes by passing the information sequence to be transmitted through a series of linear finitestate shift registers. [5] A binary convolutional encoder can be designed as a machine of shift registers and modulo-2 adders(XOR gates), where the output bits are concatenations of modular-2 additions of certain shift register states(as suggested by the generator polynomial) and present input bits. Let the state of the machine be represented by a k-tuple such that the jth entry is the output of the jth register. The initial state of all the shift registers is assumed to be 0. Now suppose, m1 is the first input bit, the state would be (0,0,0.0). As the input bits are fed into the system, the shift registers keep shifting the bits one to the right. Each modulo-2 adder is represented by a generator polynomial. Each generator polynomial is a (k+1) tuple, similar to the state tuple, where the (j + 1)th entry represents whether or not is the jth state value (corresponding to the state tuple; for j = 0, take input bit) an input of the particular adder. There are multiple possibilities for the generator polynomial of any particular adder. The concatenation of the outputs of all

2011EE10509, Digital Electronic Circuits, Semester III, IIT Delhi

the adders gives the final output sequence. B. Representation of encoders Trellis diagrams, tree diagrams and state tables can conveniently represent a convolutional encoder. This section discusses the first and the last. A Trellis is a directed graph having at most k nodes, where k is the total number of possible states of the flip-flops at any time t. Trellis diagrams are used for an easy representation of convolution codes and help in decoding as well. They basically connect, to the present state, all the possible consequent states of the present state. They also describe the output at each state transition. Note that, of all the possible states, a given state can transit in only 2 other states. Next State/output symbol, if Current State: 00 01 10 11 Input : = 1 Input: = 0

10/11 10/00 11/01 11/10

00/00 00/11 01/10 01/01

Fig 2:(b) is the state table for (a).It depicts the current state and the input and predicts the corresponding next state of the system in part (a).

C. An Illustration: The given example is that of a convolutional encoder. The encoder has been realized as a finite state machine comprising of 3 flip-flops to form a 3-bit wide shift register. The code rate of the encoder is (1/3). The parameters used are (3,1,3). [4]

Fig 1.This figure depicts a trellis diagram for a convolutional code encoder with code rate (1/3) realized using 2 flip-flops forming a 2-bit wide shift register. The bold lines represent input 0,whereas the dashed lines represent input 1.

State tables are tables depicting the next state for each present state for the appropriate input, as well as the output at each state.

Fig 3. The figure depicts a convolutional code encoder with a rate of (1/3) realized using XOR-gates and shift registers.The output bitstream would be given as y1y2y3 where y1, y2 and y3 are the outputs generated at the XORgates.

(a)

State tables and the trellis diagrams aid in decoding purposes as they are convenient and clear pictoral representations of a process otherwise complicated to understand and to keep track of.

The generators are g =[100], g =[101], and g =[111] . 1 2 3 Assuming the initial state to be zero for all the shift registers, let us try to find the code for the input bit stream {100}. As the first input bit is fed into the machine, the outputs at the XOR gates would be: y1 = 0; y2 = 0; y3 = 0; Therefore the first part of the output generated is 000. Now as the second bit is fed into the system,: y1 = 1; y2 = 1; y3 = 1; The output becomes 000 111 . The final output bit sequence (code word) is 000 111 100. This is thus, our obtained convolutional 6-bit encoded stream, with our 2-bit input stream. In the next section, we shall describe the algorithm used to decode this stream. .

2011EE10509, Digital Electronic Circuits, Semester III, IIT Delhi

III. THE DECODER The Viterbi algorithm is universally used as it provides maximum likelihood performance. Viterbi decoders are thus easy to implement in VLSI hardware and in software on CPUs with SIMD instruction sets. In the way, we worked out the encoder, we assumed that the voltages were digitized before inputting in the system. The decoding process, of this like, is termed hard decision decoding.[1] A. The Algorithm Suppose we are given a Hidden Markov Model (HMM) with state space S, initial probabilities of being in state i and transition probabilities ai,j of transitioning from state i to state j. Say we observe outputs (y1,y2,..yt). Let (x1,x2,xT) be the state sequence that produces the observations. Consider the following recurrence relations. [2]
Fig 4

The non-zero branch metrics correspond to cases when there are bit errors, whereas zero branch metrics signify absence of any. The path metric, also known, as the cumulative branch metric is a value associated with a state in the trellis. It corresponds to the Hamming distance over the path, with smallest net Hamming distance between the initial state and the current state, measured over all possible paths between the two states. The smallest Hamming distance path is the most likely path. This path minimizes the total number of bit errors. The path metric for the state s at the time i , can be represented as PM[s, i] . Now, to calculate the path metric at the time instant i + 1, PM[s,i+1] , it is important to understand that if the system is in state s at time i + 1, there are only two possible states in which the system can reside at time instant i . These two predecessor states, say ! and ", are always the same for a given state, as these only depend on the structure of the encoder. Taking the example of Fig. 4 , to arrive in state 01 at time i + 1, one of the two conditions must hold: The system was in state 10 at time i and the input message bit was a 0, making the path metric of the new state, PM[01, i + 1] is equal to PM[10, i] + 2. The other option is that the system was in state 11 and the input message bit was a 0. In which case the path metric of the new state, PM[01, i + 1] is equal to PM[11, i] + 1. We choose, the second possibility as it results in a lower Path metric for state s at time i + 1. Hence, we can conclude that PM[s, i + 1] = min(PM[!, i] + BM[! # s], PM[", i] + BM[" # s]), where ! and " are the two predecessor states. [1] It is also important to note, that not only do we need to save the path metrics of each state at each time instant i (needed for calculation of PM[s, i + 1]) but also the predecessor state of the current state which results in the mimimum path metric for the state. We, initialize another matrix PATH, such that PATH[s,i] stores the predessor state for which PM[s,i] is minimum. If the total time instants are j, then to find the most likely Encrypted output signal , we look for the state k, such that PM[k,j] is minimum. Retracing the path of (k,j) in the PATH matrix gives us the most likely value of the desired signal.

Here Vt,k is the probability of the most likely state sequence that terminates with k as its final state and which results in the given t observations. The Viterbi path can be recovered by storing pointers that remember which state x resulted in maximum probability on transition. Let Ptr (k,t) be the function that returns the value of x which results in the maximum possible value of Vt,k. Then:

In this dynamic programming problem, we construct two 2dimensional tables T1,T2. Each element of T1, T1[i,j], stores the probability of the most likely path so far and each element of T2, T2 [ i , j ], stores the previous state from which the transition in to the current state has the maximum probability. As T1[i,j] represents the probability of the path so far, T1[k,T] would represent the probability of the entire path. To find the most probable path , we look for k such that T1[k,T] is maximum. K becomes xT .Now, we go to T2 and look for the corresponding previous state for k (= T2[k,T]), say p. Hence p becomes xT-1 .We then look for the previous state of p in T2. This way we can retrace the most likely path using the Viterbi algorithm.[11] B. The Decoding using Viterbi Algorithm The decoding algorithm uses two metrics: the branch metric (BM) and the path metric (PM) [1].The branch metric is a measure of the difference between what was transmitted and what was received. It is defined for each state transition in the trellis. In hard decision decoding, the branch metric is the Hamming distance. Hamming distance is the number of bits that are different in the expected parity and the received one. An example is shown in Fig 4.

2011EE10509, Digital Electronic Circuits, Semester III, IIT Delhi

C. An Illustration: Consider the following example, where the input sequence 010 was convolutionally encoded to obtain 00 11 10 However the signal was received ,with error, as 00 11 11 .We will apply the Viterbi algorithm to obtain the original input.

use ,not only in speech recognition techniques but also for digital video broad casting and digital handwriting recognition. Hidden Markov Models along with the Viterbi Algorithm model the problem in most cases,example, Time State 00 State 01 State 10 State 11 =1 =2 =3 Time! State 00! State 01! State 10! State 11! =1! 00! =2! =3! 00! 10! 00! 10! 01! 10! 00! 10!

2 3

3 1

!
00!

Table 1: Path metrics ;

Table 2: Previous states for minimum PM.

The application of the Viterbi algorithm in handwriting detection techniques, would consider parameters such as stroke insertion, deletion and merging, to compute the similarity between the modeled and the unknown characters. Fig 5: Trellis diagrams for various time instants for the above example. Looking at table 1 , the minimum path metric after t = 3 , is obtained for state 01. Assuming the bit sequence continues only till this instant , the state path can be found by retracing the path using table 2. (assuming, the initial state was 00) State sequence in this case:00-00-10-01. Hence, the corresponding input sequence can be predicted to be 010(transition from 00->01 implies input 1, 00->00 implies 0,and 10->01 implies 0 as well), which exactly matches with the actual input! IV. HARDWARE IMPLEMENTATION A hardware Viterbi decoder for basic code usually consists of the following major blocks: Branch metric unit (BMU) : A branch metric unit calculates branch metrics. Path metric unit (PMU) : A path metric unit sums branch metrics to get minimum metrics for paths, one of which can be chosen as optimal. Trace back unit (TBU): Back-trace unit stores the maximumprobability path as suggested by the output of the PMU. [6] CONCLUSION The applications of the Viterbi Algorithm are way more diverse apart from just its use in decoding Convolutional codes. Its maximum likelihood principle puts it to immense It is evident that the applications of this algorithm are enormous and it still has potential to venture into new avenues of study . REFERENCES
*Viterbi, A.J., Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm, IEEE Transactions on Information Theory, April 1967; *Omura, J.K., On the Viterbi Decoding Algorithm, IEEE Transactions on Information Theory, January 1969 [8-9].D.Forney, Jr.,Convolutional codes 11: Maximum likelihood decoding, Stanford Electronics Labs., Stanford, Calif., Tech. Rep. 7004-1, June 1972 [1]http://web.mit.edu/6.02/www/f2010/handouts/lectures/L9.pdf [2] http://en.wikipedia.org/wiki/Viterbi_algorithm [3]http://homepages.ulb.ac.be/~dgonze/TEACHING/viterbi.pdf [4]http://wits.ice.nsysu.edu.tw/course/pdfdownload/9221/CCConvolutionalCode(II).pdf [5]http://ct.ee.ntust.edu.tw/convolutional_codes.pdf [6]http://en.wikipedia.org/wiki/Viterbi_decoder [7]http://en.wikipedia.org/wiki/Convolutional_code http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1450960 [11]http://www.cim.mcgill.ca/~latorres/Viterbi/va_alg.htm [12]http://rywei.ce.ncu.edu.tw/course/96/ACT/lect1-print.pdf Fig1,3http://wits.ice.nsysu.edu.tw/course/pdfdownload/9221/CCConvolutionalCode(II).pdf Fig2(a),5: http://www.cs.sunysb.edu/~jgao/CSE590-fall09/viterbi.pdf Fig4: http://web.mit.edu/6.02/www/f2010/handouts/lectures/L9.pdf *General refrences