It

Practical Workbook
INFORMATION THEORY
Name
: ________________________
Year
: ________________________
Batch
: ________________________
Roll No
: ________________________
Department: ________________________
4th Edition: 2014
Department of Computer & Information Systems Engineering

NED University of Engineering & Technology
INTRODUCTION
The laboratory sessions in the practical workbook of Information Theory cover the concepts,
tools, techniques and activities related to the error detection and correction and fundamental
communication process of information transfer. The workbook starts with an overview of
MATLAB followed by its application to coding theory.
The next couple of laboratory sessions cover the different types of error detecting and
correcting schemes, their encoding and decoding process and the bit error rate. The concept of
source encoding is then introduced with emphasis on Huffman Encoding. It is then followed
by the various concept of cryptography.
Next five lab sessions once again deals with encoding and decoding process but on a different
platform, their implementation is based on models or blocks. All the concepts described
throughout the workbook will be verified through implementation in MATLAB and Simulink.
CONTENTS
Lab Session No.
Object
Page No.
Application of MATLAB to coding theory
Learning basic commands of MATLAB
Encoding messages for a forward error correction system with a given Linear 9
block code and verifying through simulation
Decoding encoded words for a forward error correction system with a given Linear 14
block code and verifying through simulation
Encoding the messages for a system with a given cyclic polynomial code and 23
verifying through simulation
Decoding the messages for a system with a given cyclic polynomial code and 31
verifying through simulation
7
8
Understanding the concept of loss less data compression technique using Huffman 37
coding
Implementation of Hill cipher technique in cryptography
41
Implementation of Vigenere cipher technique in cryptography
48
10
Learning basic model designs with Simulink
51
11
Encoding the data bits using a Binary Cyclic block encoder in Simulink
53
12
Decoding the code words using a Binary Cyclic block decoder in Simulink
55
13
Encoding the data bits using a Binary Linear block encoder in Simulink
57
14
Decoding the code words using Binary Linear block decoder in Simulink
59
Information Theory
Lab Session 01
NED University of Engineering & Technology Department of Computer & Information Systems Engineering
Lab Session 01
OBJECT
Application of MATLAB to coding theory
THEORY
INTRODUCTION:
MATLAB is a high-performance language for technical computing. It integrates computation,
visualization, and programming in an easy-to-use environment where problems and solutions
are expressed in familiar mathematical notation.
Starting MATLAB
On Windows platforms, start MATLAB by double-clicking the MATLAB shortcut icon on
your Windows desktop.
Quitting MATLAB
To end your MATLAB session, select File -> Exit MATLAB in the desktop, or type quit in
the Command Window.
THE MATLAB ENVIRONMENT
The MATLAB environment consists of five main parts:
Development Environment
This is the set of tools and facilities that help you use MATLAB functions and files. Many of
these tools are graphical user interfaces. It includes the MATLAB desktop and Command
Window, a command history, an editor and debugger, and browsers for viewing help, the
workspace, files, and the search path.
GETTING STARTED
a) Entering Matrices
The best way for you to get started with MATLAB is to learn how to handle matrices. Start MATLAB
and follow along with each example. You can enter matrices into MATLAB by entering an explicit list
of elements or generating matrices using built-in functions.
You only have to follow a few basic conventions: Separate the elements of a row with blanks or
commas. Use a semicolon ; to indicate the end of each row. Surround the entire list of elements
with square brackets [ ].
Consider the following vector, x (recall that a vector is simply a matrix with only one row or column)
x = [1,3,5,7,9,11]
Information Theory
Lab Session 01
x = 1 3 5 7 9 11
Notice that a row vector is the default. We could have used spaces as the delimiter between columns
x = [1 3 5 7 9 11]
x = 1 3 5 7 9 11
b) The MATLAB Workspace
We can view the variables currently in the workspace by typing
who
Your variables are:
ans
x
y
z
leaving 621420 bytes of memory free.
More detail about the size of the matrices can be obtained by typing
whos
c) Complex variables
Both i and j represent the imaginary number, -1, by default
i
ans = 0 + 1.0000i
j
ans = 0 + 1.0000i
d) The Colon Operator

The colon : is one of the most important MATLAB operators. It occurs in several different
forms. The expression 1:10 is a row vector containing the integers from 1 to 10,
1
2
3
4
5
6
To obtain non unit spacing, specify an increment.
For example,
100:-7:50 is
100
93
and
0:pi/4:pi
is
0
0.7854
86
79
1.5708
72
65
2.3562
58
10
51
3.1416
Subscript expressions involving colons refer to portions of a matrix.

A(1:k,j) is the first k elements of the jth column of A. So sum(A(1:4,4)) computes
the sum of the fourth column. But there is a better way. The colon by itself refers to all the
elements in a row or column of a matrix and the keyword end refers to the last row or column.
So sum(A(:,end)) computes the sum of the elements in the last column of A.
ans =
34
Information Theory
Lab Session 01
e) Operators
Expressions use familiar arithmetic operators and precedence rules.
+
*
/
^
( )
Addition
Subtraction
Multiplication
Division
Power
Complex conjugate transpose
Specify evaluation order
Exercises:
Write the shortest MATLAB code for each problem, avoiding unnecessary details if at all
possible, along with the final displayed output.
1.1)
Create a session diary myfirstdiary that records the executions of all script
files.
Declare predefined variables (, i, j) along with variable X .

Where X= [1 2 3; 4 5 6].
Now answer the following questions, also show the output wherever necessary:
1.1.1) What is the difference between clear and clear X command?
_____________________________________________________________________
_____________________________________________________________________
1.1.2) What is the difference between CLC and clear command?
_____________________________________________________________________
_____________________________________________________________________
1.1.3) What is the difference between displaying the value of before and after
using the command format long?

_____________________________________________________________________
_____________________________________________________________________
1.1.4) What is the purpose of diary command?
_____________________________________________________________________
_____________________________________________________________________
Information Theory
Lab Session 01
1.2)
A supermarket conveyor belt holds an array of groceries. The price of each product
(in pounds) is [ 0.6 1.2 0.5 1.3 ] ; while the numbers of each product are [ 3 2 1 5 ].
Use MATLAB to calculate the total bill.
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
1.3)
The _________(X) function will sort a vector or matrix X into increasing row order.
Use this function to sort a list of names into alphabetical order.
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
1.4)
The identity matrix is a square matrix that has ones on the diagonal and zeros
elsewhere. You can generate one with the eye() function in MATLAB. Use
MATLAB to find a matrix B, such that when multiplied by matrix A=[ 1 2; -1 0 ] the
identity matrix I=[ 1 0; 0 1 ] is generated. That is A*B=I.
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
Information Theory
Lab Session 02
Lab Session 02
OBJECT
Learning basic commands of MATLAB
THEORY
INTRODUCTION:
MATLAB is a high-performance language for technical computing. It integrates computation,
visualization, and programming in an easy-to-use environment where problems and solutions are
expressed in familiar mathematical notation. The purpose of this lab session is to review MATLAB for
those that have used it before, and to provide a brief introduction to MATLAB for those that have not
used it before. This is a "hands-on" session of introduction. Lets begin with:
a) Entering Matrices
The best way for you to get started with MATLAB is to learn how to handle matrices. Start
MATLAB and follow along with each example. You can enter matrices into MATLAB by
entering an explicit list of elements or generating matrices using built-in functions.
You only have to follow a few basic conventions: Separate the elements of a row with blanks
or commas. Use a semicolon ; to indicate the end of each row. Surround the entire list of
elements with square brackets [ ].
To enter a matrix A, simply type in the Command Window
A = [16 3 2 13; 5 10 11 8; 9 6 7 12; 4 15 14 1]
b) Sum, Transpose, and diagonal
The first statement to try is
sum(A)
MATLAB replies with
ans =
34
34
34
34
The transpose operation is denoted by an apostrophe or single quote . It flips a matrix about
its main diagonal and it turns a row vector into a column vector.
The sum of the elements on the main diagonal is obtained with the sum and the diag
functions.
diag(A) produces
ans =
16
10
7
1
Information Theory
Lab Session 02
c) Subscripts
The element in row i and column j of A is denoted by A(i,j). For example, A(4,2) is the
number in the fourth row and second column. For our square matrix A, A(4,2) is 15. So to
compute the sum of the elements in the fourth column of A, type
A(1,4) + A(2,4) + A(3,4) + A(4,4)
This produces
ans =
34
d) Generating Matrices:
MATLAB provides functions that generate basic matrices three of which are given below.
Zeros
Ones
Rand
All zeros
All ones
Uniformly distributed random elements
Example:
Z = zeros(2,4)
Z =
0
0
0
0
0
0
0
0
e) Concatenation
Concatenation is the process of joining small matrices to make bigger ones. In fact, you made
your first matrix by concatenating its individual elements. The pair of square brackets, [], is
the concatenation operator. For an example, start with the 4-by-4 matrix A, and form
B = [A
A+32; A+48
A+16]
f) Deleting Rows and Columns

You can delete rows and columns from a matrix using just a pair of square brackets. Start with
X = A;
Then, to delete the second column of X, use
X(:,2) = []
This changes X to
X = 16
5
9
4
2
11
7
14
13
8
12
1
Information Theory
Lab Session 02
g) Scalar Expansion
Matrices and scalars can be combined in several different ways. For example, a scalar is
subtracted from a matrix by subtracting it from each element. The average value of the elements
in matrix A is 8.5, so
B = A - 8.5
forms a matrix whose column sums are zero.
B =
7.5
-5.5
-6.5
4.5
-3.5
1.5
2.5
-0.5
0.5
-2.5
-1.5
3.5
-4.5
6.5
5.5
-7.5
h) Vector Products
A row vector and a column vector of the same length can be multiplied in either order. The
result is either a scalar, the inner product, or a matrix, the outer product.
u = [3; 1; 4];
v = [2 0 -1];
x = v*u
x = 2
X = u*v
X =
i)
6
2
8
0
0
0
-3
-1
-4
Building Tables
Array operations are useful for building tables. Suppose n is the column vector
n = (0:9)';
Then
pows = [n n.^2 2.^n]
builds a table of squares and powers of 2.
pows =
0
1
2
3
4
5
6
7
8
9
0
1
4
9
16
25
36
49
64
81
1
2
4
8
16
32
64
128
256
512
Information Theory
Lab Session 02
Exercise:
2.1)
Calculate the rank, determinant and matrix inverse of the following matrices
1
2
1
A = 1 2 1
2
4
2
1 2
1
1 4 1
2 4
2
C=
1 2
1
1 4 1
2 4
5
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
2.1.1) Find CC-1; use same value of C as given in Q1
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
2.2)
Calculate
2.2.1) xTx, and
2.2.2) xxT, where
1
x= 2
3
4
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
Information Theory
Lab Session 03
Lab Session 03
OBJECT
Encoding messages for a forward error correction system with a given Linear block code
and verifying through simulation
THEORY
In mathematics and information theory, linear code is an important type of block code used in
error correction and detection schemes. Linear codes allow for more efficient encoding and
decoding algorithms than other codes.
Linear codes are applied in methods of transmitting symbols (e.g., bits) on communications
channel so that, if errors occur in the communication, some errors can be detected by the
recipient of the message block. A linear code of length n transmits blocks containing n
symbols. For example, the (7, 4) Hamming code is a binary linear code which represents 4-bit
values with 7 bits. In this way, the recipient can detect errors as severe as 2 bits per block.
In linear block codes, a block of k information bits is followed by a group of r check bits
derived from the information bits and at the receiver the check bits are used to verify the
information bits which are preceding the check bits.
No of code words = 2k
Block length = n
Code rate = k/n
Each block of k bits is encoded into block of n bits (n>k) by adding n-k = r check bits.
Where 2r k + r + 1.
-------------------------(A)
The check bits are determined by some predetermined rule.

C=DG
Where:
C = code vector
D = Data (message) vector
G = Generator matrix which is defined as [ Ik | P]; Ik is identity matrix of order k and P is the
predefined encoding rule.
MATLAB Syntax:
The encode function is used for encoding. The syntax is as follows.
code = encode(msg,n,k,'linear/fmt',genmat)
where, the codeword length is n and the message length is k.
msg represents data or message. It can be in decimal or binary format. The default value for
this parameter is binary. We will use binary format in this and foregoing lab sessions.
Information Theory
Lab Session 03
For example:
Format of msg can be a binary column vector as given below.
msg = [0 1 1 0 0 1 0 1 1 0 0 1]'. The ' symbol indicates matrix transpose.
Format of msg can also be binary matrix with k columns. In this case format of code will be
binary matrix with n columns.
msg = [0 1 1 0; 0 1 0 1; 1 0 0 1]. Here k = 4.
For Linear Block codes, encode function encodes msg using genmat as the generator
matrix. genmat, a k-by-n matrix, is required as input.
Example:
The example below illustrates two different information formats (binary vector and binary
matrix) for linear block code. The two messages have identical content in different formats.
As a result, the two codes created by encode function have identical content in
correspondingly different formats.
Here k = 11. Putting r = 4 to satisfy (A)
r = 4; % r is the number of check bits.
k = 11; % Message length
n = k + r % Codeword length = 15 using formula n-k=r
% Create 100 messages, k bits each.
msg1 = randint(100*k,1,[0,1]); % As a column vector
msg2 = vec2mat(msg1,k); % As a k-column matrix
% Create 100 codewords, n bits each.
P =[1 1 1 1;
0 1 1 1;
1 1 1 0;
1 1 0 1;
0 0 1 1;
0 1 0 1;
0 1 1 0;
1 0 0 1;
1 0 1 0;
1 0 1 1;
1 1 0 0]
genmat=[eye(11) P]; % concatenate P submatrix or predefined
rule with Identity matrix.
code1 = encode(msg1,n,k,'linear/binary',genmat);
code2 = encode(msg2,n,k,'linear/binary',genmat);
if ( vec2mat(code1,n)==code2 )
disp('All two formats produced the same content.')
end
Instead of randomly generating data words, you can create a data matrix of your own
containing the data words that you want to encode.
10
Information Theory
Lab Session 03
Exercise:
3.1)
Given (6,3) linear block code generated by the predefined matrix [0 1 1; 1 0 1; 1 1 0].
3.1.1) Encode the messages [1 1 1] and [1 0 1] manually and verify through MATLAB.
111 = __________.
101 = _________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
3.1.2) Encode all the possible messages in MATLAB and write the encoded words.
Message Code Vector

000
001
010
011
100
101
110
111
3.1.3) The minimum hamming distance = _____________.
3.1.4) The no. of errors this code can detect is ____________
3.1.5) The no. of errors this code can correct is ____________
3.1.6) The code rate = ___________________.
3.1.7) Is this code a systematic code? Justify your answer. (HINT: see the tabulated code
words obtained in Q1: b)
11
Information Theory
Lab Session 03
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
3.2)
Consider a systematic (6, 3) block code generated by the submatrix [1 1 0; 0 1 1; 1 0 1].
3.2.1) Write the value of n and k.

________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
3.2.2) Tabulate the code words through simulation and write their corresponding weights.
Message Code Vector Weight dmin
000
001
010
011
100
101
110
111
3.2.3) What is the value of dmin ?
________________________________________________________________________
3.3)
Write MATLAB code to create 10 messages randomly having size of 4 bits each.
Design appropriate encoding rule and use encode function to encode the generated
messages.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
12
Information Theory
Lab Session 03
3.4)
Write a program in any language(C or C++) to implement ENCODE function that

accepts n, k, generator matrix and message bits from the use as input and produces the
encoded word.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
13
Information Theory
Lab Session 04
Lab Session 04
OBJECT
Decoding encoded words for a forward error correction system with a given Linear Block
code and verifying through simulation
THEORY
Let Y stands for the received vector when a particular code vector X has been transmitted. Any
transmission errors will result in Y X. The decoder detects or corrects errors in Y using stored
information about the code.
A direct way of performing error detection would be to compare Y with every vector in the
code. This method requires storing all 2k code vectors at the receiver and performing up to 2k
comparisons. But efficient codes generally have large values of k, which implies rather
extensive and expensive decoding hardware.
More practical decoding methods for codes with large k involve parity check information
derived from the codes P submatrix. Associated with any systematic linear (n,k) block code
is a (n-k) n matrix called the parity check matrix H. This matrix is defined by
H = [P | Ir]
Where Ir is the r r identity matrix and n - k = r. The parity check matrix has a crucial
property for error detection which is
XHT = (0 0 0 0 ..0)
provided that X belongs to the set of code vectors. However, when Y is not a code vector, the
product YHT contains at least one nonzero element.
Therefore, given HT and a received vector Y, error detection can be based on
S = Y HT
an r-bit vector called the syndrome. If all elements of S equal zero, then either Y equals the
transmitted vector X and there are no transmission errors, or Y equals some other code vector
and the transmission errors are undetectable. Otherwise, errors are indicated by the presence of
nonzero elements in S. Thus a decoder for error detection simply takes the form of a syndrome
calculator.
We develop the decoding method by introducing an n-bit error vector E whose non zero
elements mark the positions of transmission errors in Y. For instance, if X = (1 0 1 1 0) and Y
= (1 0 0 1 1) then E = (0 0 1 0 1). In general,
Y=X E
And conversely,
X=Y E
T
Substituting Y = X + E into S = YH , we obtain
S = (X E)HT
S =XHT EHT
S = EHT
which reveals that the syndrome depends entirely on the error pattern, not the specific
transmitted vector.
14
Information Theory
Lab Session 04
Syndrome decoding: example

An (8, 4) binary linear block code C is defined by systematic matrices:
G=
1000|0111
0100|1011
0010|1101
0001|1110
H=
Consider two possible messages:

m1 = [0 1 1 0]
C = DG or MG
0
C = [0 1 1 0] [0 1
0 0
0 0
1
C =
m2 = [1 0 1 1]
1
0
1 0 1 1
0 1 1 1
0
0
0111|1000
1011|0100
1101|0010
1110|0001
0
0
0
1
1
1
0
1
1
1]
1
0
[[(0 1) (1 0) (1 0) (0 0)] . . .
. . . [(0 1) (1 1) (1 1) (0 0)]]
First and last elements of the matrix are shown above

C =
[(0 0 0 0)
(0 1 0 0) (0 0 1 0) (0 0 0 0) (0 1 1 0)]
(0 0 1 0)
(0 1 0 0)
(0 1 1 0)]
C = [0 1 1 0 0 1 1 0]
Hence,
c1 = [0 1 1 0 0 1 1 0]
c2 = [1 0 1 1 0 1 0 0]
Suppose error pattern e = [0 0 0 0 0 1 0 0] is added to both code words.

r1 = [0 1 1 0 0 0 1 0]
r2 = [0 1 0 0 1 1 1 1]
Calculating the syndrome using S = YHT. Here Y is the received erroneous code word.
s1 = [0 1 0 0]
s2 = [0 1 0 0 ]
The syndromes are the same and equal column 6 of H, so decoder corrects bit 6.
Bit Error Rate:
Bit error rate = average no. of erroneous bits per block / total no. of bits per block
MATLAB FUNCTIONS:
In order to develop MATLAB code for decoding and calculating the bit error rate, the following
functions may be used.
1) DECODE:
Function: Block decoder
Syntax
msg = decode(code,n,k,'linear/fmt',genmat,trt)
[msg,err] = decode(...)
15
Information Theory
Lab Session 04
trt
Uses syndtable to create the syndrome decoding table associated with the method's
parity-check matrix. Syndtable function is described later in this lab session.
Description
The decode function aims to recover messages that were encoded using an error-correction
coding technique. The technique and the defining parameters must match those that were used
to encode the original signal.
Decode function decodes code, which is a linear block code determined by the k-by-n
generator matrix genmat. genmat is required as input. decode tries to correct errors using
the decoding table trt, where trt is a 2^(n-k)-by-n matrix.
2) SYNDTABLE:
Function: Produces syndrome decoding table
Syntax:
t = syndtable(h)
Description
t = syndtable(h) returns a decoding table for an error-correcting binary code having
codeword length n and message length k. h is an (n-k)-by-n parity-check matrix for the
code. t is a 2n-k-by-n binary matrix. The rth row of t is an error pattern for a received binary
codeword whose syndrome has decimal integer value r-1. (The syndrome of a received
codeword is its product with the transpose of the parity-check matrix.).
3) BITERR:
Function: Computes number of bit errors and bit error rate.
Syntax
[number,ratio] = biterr(x,y)
Description
The biterr function compares unsigned binary representations of elements in vector x with those
in vector y.
Each element of x and y must be a nonnegative decimal integer; biterr converts each
element into its natural unsigned binary representation. number is a scalar or vector that
indicates the number of bits that differ. ratio is number divided by the total number of bits.
The total number of bits, the size of number, and the elements that biterr compares are
determined by the dimensions of x and y.
4) GEN2PAR
Function: Convert between parity-check and generator matrices
Syntax
parmat = gen2par(genmat)
genmat = gen2par(parmat)
Description
parmat = gen2par(genmat) converts the standard-form binary generator matrix
genmat into the corresponding parity-check matrix parmat.
genmat = gen2par(parmat) converts the standard-form binary parity-check matrix
parmat into the corresponding generator matrix genmat.
16
Information Theory
Lab Session 04
Example 1
The command below compares the column vector [0; 0; 0] to each column of a random binary
matrix. The output is the number and proportion of 1s in the matrix.
[number,ratio] = biterr([0;0;0],randint(3,5))
The output is
number =
2
ratio = 0.6667
1
0.3333
1
0.3333
0
0
1
0.3333
Example 2
The script below adds errors to 10% of the elements in a matrix. Each entry in the matrix is a
two-bit number in decimal form. The script computes the bit error rate using biterr.
x = randint(100,100,4);
% Original signal
% Create errors to add to ten percent of the elements of x.
% Errors can be either 1, 2, or 3 (not zero).
errorplace = (rand(100,100) > .9); % Where to put errors
errorvalue = randint(100,100,[1,3]); % Value of the errors
errors = errorplace.*errorvalue;
y = rem(x+errors,4); % Signal with errors added, mod 4
[num_bit,ratio_bit] = biterr(x,y,2)
The output is
num_bit =
ratio_bit =
1304
0.0652
Example 3
The commands below convert the parity-check matrix for a Hamming code into the
corresponding generator matrix and back again.
parmat = hammgen(3)
genmat = gen2par(parmat)
parmat2 = gen2par(genmat) % Ans should be the same as parmat
above
The output
parmat =
1
0
0
genmat =
1
0
1
1
parmat2 =
1
0
0
is
0
1
0
0
0
1
1
1
0
0
1
1
1
1
1
1
0
1
1
1
1
0
0
1
1
1
1
0
0
0
0
1
0
0
0
0
1
0
0
0
0
1
0
1
0
0
0
1
1
1
0
0
1
1
1
1
1
1
0
1
17
Information Theory
Lab Session 04
The above example uses hammgen function whose explanation is as follows.

h = hammgen(m)
The codeword length is n. n has the form 2m-1 for some positive integer m greater than or equal
to 3. The message length, k, has the form n-m. hammgen function produces an m-by-n paritycheck matrix for a Hamming code having codeword length n = 2m-1. The input m is a positive
integer greater than or equal to 3. The message length of the code is n-m.
You can also provide your own generator matrix as input.
Example 4
The next example creates a linear code, adds noise, and then decodes the noisy code. It uses the
decode function.
n = 3; k = 2; % A (3,2) linear code
msg = randint(100,k,[0,1]); % 100 messages, k bits each
code = encode(msg,n,k,'linear/binary');% Add noise.
noisycode = rem(code + randerr(100,n,[0 1;.7 .3]), 2);
newmsg = decode(noisycode,n,k,'linear'); % Try to decode.
% Compute error rate for decoding the noisy code.
[number,ratio] = biterr(newmsg,msg);
disp(['The bit error rate is ',num2str(ratio)])
The output is below.
Your error rate results might vary because the noise is random.
The bit error rate is 0.08
Exercises:
4.1) Apply table lookup decoding to (7,4) hamming code. The P submatrix is given as
1
1
1
0
0
1
1
1
1
1
0
1
4.1.1) Write the value of n and k.

________________________________________________________________________
4.1.2) Create G and H matrices.
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
18
Information Theory
Lab Session 04
4.1.3) Tabulate the code words through simulation and write their corresponding
weights.
Message Code Vector Weights

0000
0001
0010
0011
0100
0101
0110
0111
1000
1001
1010
1011
1100
1101
1110
1111
4.1.4) The minimum hamming distance = _____________.
4.1.5) The no. of errors this code can detect is
________________________________________________________________________
________________________________________________________________________
4.1.6) The no. of errors this code can correct is
________________________________________________________________________
________________________________________________________________________
4.1.7) Find syndromes for all possible single error patterns.
S
000
101
E
0000000
1000000
19
Information Theory
Lab Session 04
4.1.8) Suppose a received word happens to have two errors, such that E = (1 0 0 0 0 1 0).
Calculate the syndrome.
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
4.1.9) What will be the error pattern (write form table).
________________________________________________________________________
4.1.10) Write the corrected word.
________________________________________________________________________
4.1.11) What observations do you make? How many errors the decoded output word will
contain.
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
4.2)
Write MATLAB code to construct lookup table for the (6,3) block code generated by
the submatrix
1 1 0
0 1 1
1 0 1
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
20
Information Theory
Lab Session 04
4.3)
Consider a (5,3) block code obtained by deleting the last column of the P submatrix in
above question. Construct the lookup table and show that this code could be used for
error detection and not correction. Also write MATLAB code to implement the given
functionality.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
4.4)
A (7,4) LBC has generator matrix G given by

1
0
0
0
0
1
0
0
0
0
1
0
0
0
0
1
21
1
1
0
1
1
0
1
1
0
1
1
1
Information Theory
Lab Session 04
4.4.1) Determine the PCM for above matrix.

________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
4.4.2) Write MATLAB code to encode all possible data words and then decode all the
encoded words.
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
4.4.3) Insert double error in any code block. Decode manually and through simulation.
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
22
Information Theory
Lab Session 04
4.4.4) The bit error rate for single error in first two code blocks is
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
4.4.5) Insert single error in every code block. Write MATLAB code to find bit error rate.
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
4.4.6) Write a program (C or C++) to implement DECODE function.
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
__________________________________________________________
23
Information Theory
Lab Session 05
Lab Session 05
OBJECT
Encoding the messages for a system with a given cyclic polynomial code and verifying
through simulation
THEORY
Cyclic codes are a subclass of linear block codes with a cyclic structure that leads to more
practical implementation. Block codes used in forward error correction systems are almost
always cyclic codes.
In cyclic coding, data is sent with a checksum. When arrives, checksum is recalculated. It
should match the one that was sent.
To describe a cyclic code, we express an arbitrary n-bit vector in the form
X = (xn-1 xn-2 x1 x0)
Now suppose that X has been loaded into a shift register with feedback connection from the
first to last stage. Shifting all bits one position to the left yields the cyclic shift of X, written as
X' = (xn-2 xn-3 x1 x0 xn-1)
A second shift produces
X = (xn-3 x1 x0 xn-1 xn-2) and so forth.
A linear code is cyclic if every cyclic shift of a code vector X is another vector in the code.
Since cyclic codes are linear block codes, all the properties of linear block codes apply to
cyclic codes.
Cyclic codes are used in applications where burst errors can occur. Burst error is an error in
which a group of adjacent bits is affected.
Encoding of Cyclic Codes
Direct Approach:
Let M(x) is the message polynomial and G(x) is the generator polynomial. Then the code
vector V(x) can be encoded as
V(x) = M(x). G(x)
24
Information Theory
Lab Session 05
Example 1
The (6, 2) repetition code having valid code C = {[000000], [010101], [101010], [111111]} is
cyclic, since a cyclic shift of any of its code vectors results in a vector that is an element of C.
Example 2
The (5, 2) LBC is defined by the generator matrix G = [1 0 1 1 1; 0 1 1 0 1] is a single error
correcting code. We need to determine if it is cyclic.
The valid code words are
C = {[00000], [01101], [10111], [11010]}
The given code is NOT a cyclic code because cyclic shift of (11010) results in (10101) which
is not a valid code word. Also cyclic shift of (10111) results in (01111) which is not a member
of C.
It is convenient to think of words as polynomials rather than vectors. Since the code is binary,
the coefficients will be 0 or 1. The degree of polynomial equals to its highest exponent. For
example the degree of the polynomial obtained from (1 0 1 1) is 3
Example 3
Given (6, 3) code with G(x) = (1 1 0 1). Encode M(x) = (101)
V(x) = M(x). G(x)
V(x) = (x3 + x2 + 1). (x2 + 1)
V(x) = x5 + x3 + x4 + x2 + x2 + 1
V(x) = 1.x5 + 1.x4 + 1.x3 + (1 1) x2 + 0.x + 1
V(x) = 1.x5 + 1.x4 + 1.x3 + 0.x2 + 0.x + 1
V(x) = (1 1 1 0 0 1)
The degree of polynomial is defined to be the highest power of x in the polynomial for which
the coefficient of x is not zero. The degree of polynomial M(x) is denoted by deg
[M(x)]. If M is our set of k bit source vectors, the highest degree polynomial in the set of
polynomials M(x) that describes our source vectors is k-1. If deg [G(x)] = r, then the
highest degree of our set of code polynomials V(x) is n-1 or k-1+r. Therefore deg
[G(x)] tells us the number of check bits in the code.
Property of generator Polynomial
G(x) is the generator polynomial for a linear cyclic code of length n if and only if G(x)
divides 1+xn without a remainder.
Example 4
Given (7, 4) cyclic code. Find a generator polynomial for a code of length n = 7 for encoding
of data of length k = 4.
25
Information Theory
Lab Session 05
Solution:
G(x) should be a degree 7 4 = 3 and should divide 1 + xn without a remainder.
1 + x7 can be factorized as
1 + x7 = (1+x+x3) (1+x2+x3) (1+x)
So we can choose for G(x) either (1+x+x3) or (1+x2+x3).
Systematic form:
Let
U(x) = uk-1 xk-1 + uk-2 xk-2 + ... + u1 x+ u0 be an information polynomial and
G (x) = g n - k x n - k + g n - k - 1xn -k-1 +... + g1 x + g0 be a generator polynomial of an (n, k) cyclic
code. The code polynomial is given by
V(x) = xn-k U(x) / G(x)
= Vs(x) / G(x)
Where
Vs(x) = x n -k U(x)
MATLAB FUNCTIONS
1) Encode
Function: Block encoder
Syntax
Code = encode (msg,n,k,'cyclic/fmt',genpoly)
Description
Encode function encodes msg and creates a systematic cyclic code. genpoly is a row
vector that gives the coefficients, in order of ascending powers, of the binary generator
polynomial. The default value of genpoly is cyclpoly (n,k). This function is
explained below. By definition, the generator polynomial for an [n, k] cyclic code must have
degree n-k and must divide xn-1. You are already familiar with the input arguments msg,
n and k.
2) Cyclpoly
Function: Produce generator polynomials for a cyclic code.
Syntax
pol = cyclpoly (n, k)
Description
In MATLAB a polynomial is represented as a row containing the coefficients in order of
ascending powers. The results you would get will be in ascending order of coefficients while
in manual calculation the results will be in descending order. It does not matter what
convention you follow but it should be consistent. Cyclpoly function returns the row
vector representing one nontrivial generator polynomial for a cyclic code having codeword
length n and message length k.
26
Information Theory
Lab Session 05
The Galois Field Computation in MATLAB

A Galois field is an algebraic field that has a finite number of members. Galois fields having
2^m members are used in error-control coding and are denoted GF (2^m) where m is an
integer between 1 and 16. This portion describes how to work with fields that have 2^m
members with m = 0.
3) gf
Function: Creates a Galois field array.
Syntax
x_gf = gf(x,m)
x_gf = gf(x)
Description
x_gf = gf(x,m) creates a Galois field array from the matrix x. The Galois field has 2^m
elements, where m is an integer between 1 and 16. The elements of x must be integers
between 0 and 2^m-1. The output x_gf is a variable that MATLAB recognizes as a Galois
field array, rather than an array of integers. As a result, when you manipulate x_gf using
operators or functions such as + or det, MATLAB works within the Galois field you have
specified.
X_gf = gf(x) creates a GF (2) array from the matrix x. Each element of x must be 0 or
1.
4) conv
Function: Convolution and polynomial multiplication
Syntax:
w = conv (u,v)
Description:
conv function convolves vectors u and v. Algebraically, convolution is the same operation as
multiplying the polynomials whose coefficients are the elements of u and v.
5) deconv
Function: Deconvolution and polynomial division.
Syntax:
[q,r] = deconv (v,u)
Description:
Deconv function deconvolves vector u out of vector v, using long division.
The quotient is returned in vector q and the remainder in vector r such that
v = conv (u,q)+r.
If u and v are vectors of polynomial coefficients, convolving them is equivalent to
multiplying the two polynomials, and deconvolution is polynomial division.
The result of dividing v by u is quotient q and remainder r.
27
Information Theory
Lab Session 05
Exercises:
5.1)
The generator polynomial of a (7, 4) cyclic code is given by

G(x) = 1 + x2 + x3
Find V by direct method as well as the systematic approach if M(x) = (1 0 1 0).

Write code to implement the above functionality in MATLAB using encode function and
then conv function.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
5.2) Consider the cyclic (7, 4) cyclic code generated by G(x) = x3 + x2 + 1.
5.2.1) Use long division to calculate the check bit polynomial C(x) when M = (1 1 0 0).
Also write the complete code polynomial.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
28
Information Theory
Lab Session 05
___________________________________________________________________________
___________________________________________________________________________
5.2.2) Write MATLAB code to perform the above functionality. Use encode function
and then conv/deconv functions.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
The generator polynomial of (7, 4) cyclic codes is given by G(x) = 1 + x2 + x3. Find V
by the systematic approach if M(x) = (1 0 0 1).
___________________________________________________________________________
5.3)
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
5.4) Consider the cyclic (7, 4) hamming code generated by G(x) = x3 + x + 1.
5.4.1) Use systematic approach to calculate the check bit polynomial C(x) when M = (1 1
0 1). Also write the complete code polynomial.
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
29
Information Theory
Lab Session 05
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
5.5) Consider (6, 3) cyclic code. Create MATLAB code to perform the following tasks.
5.5.1) Create generator polynomial.
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
5.6) Write a program (C or C++) to implement ENCODE function for Cyclic codes
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
30
Information Theory
Lab Session 06
Lab Session 06
OBJECT
Decoding the messages for a system with a given cyclic polynomial code and verifying
through simulation
THEORY
In cyclic coding, data is sent with a checksum. When arrives, checksum is recalculated. It
should match the one that was sent.
To describe a cyclic code, we express an arbitrary n-bit vector in the form
X = (xn-1 xn-2 x1 x0)
Now suppose that X has been loaded into a shift register with feedback connection from the
first to last stage. Shifting all bits one position to the left yields the cyclic shift of X, written as
X' = (xn-2 xn-3 x1 x0 xn-1)
A second shift produces
X = (xn-3 x1 x0 xn-1 xn-2) and so forth.
Since cyclic codes are linear block codes, all the properties of linear block codes apply to
cyclic codes.
Cyclic codes are used in applications where burst errors can occur. Burst error is an error in
which a group of adjacent bits is affected.
Decoding of Cyclic codes:
Let T(x) and E(x) be the code polynomial of a systematic cyclic code and an error polynomial
of degree n - 1 or less, respectively. An error is the same as adding some E(x) to T(x) for
e.g. adding 1010011000001110000 will flip the bits at the locations where "1" is in the error
bit string. Instead of T(x) arriving, T(x) + E(x) arrives.
So R(x) = T(x) E(x)
In general, each 1 bit in E(x) corresponds to a bit that has been flipped in the message.
If there are k 1 bits in E(x), k single-bit errors have occurred. A burst error looks like 1....1
Far end receives T(x) + E(x) = R(x). T(x) is multiple of G(x) (remainder zero). Hence
remainder when you divide (T(x) + E(x)) by G(x) = remainder when you divide E(x) by
G(x). For e.g. remainder when divide (1000+n) by 10 = remainder when you divide n by 10
Here the remainder of E(x) / G(x) is called syndrome vector denoted by S(x). If remainder
when you divide E(x) by G(x) is zero, the error will not be detected.
In general, if E(x) is a multiple of G(x), the error will not be detected.
Otherwise, it will. All other error patterns will be caught.
31
Information Theory
Lab Session 06
MATLAB FUNCTIONS
1) Decode
Function: Block decoder
Syntax
msg = decode (code, n ,k, 'cyclic/fmt', genpoly, trt)
[msg, err] = decode (...)
[msg, err, ccode] = decode (...)
Description
decode function decodes the cyclic code code and tries to correct errors using the
decoding table trt, where trt is a 2^(n-k)-by-n matrix. genpoly is a row vector that
gives the coefficients, in order of ascending powers, of the binary generator polynomial of the
code. The default value of genpoly is cyclpoly (n, k). By definition, the generator
polynomial for an [n, k] cyclic code must have degree n-k and must divide xn-1.
2) Cyclpoly
Function: Produce generator polynomials for a cyclic code.
Syntax
pol = cyclpoly (n, k)
Description
In MATLAB a polynomial is represented as a row containing the coefficients in order of
ascending powers. The results you would get will be in ascending order of coefficients while
in manual calculation the results will be in descending order. It does not matter what
convention you follow but it should be consistent. Cyclpoly function returns the row
vector representing one nontrivial generator polynomial for a cyclic code having codeword
length n and message length k.
3) conv
Function: Convolution and polynomial multiplication
Syntax:
w = conv (u,v)
Description:
conv function convolves vectors u and v. Algebraically, convolution is the same operation as
multiplying the polynomials whose coefficients are the elements of u and v.
4) deconv
Function: Deconvolution and polynomial division.
Syntax:
[q,r] = deconv (v,u)
Description:
Deconv function deconvolves vector u out of vector v, using long division.
The quotient is returned in vector q and the remainder in vector r such that
v = conv (u,q)+r.
If u and v are vectors of polynomial coefficients, convolving them is equivalent to
multiplying the two polynomials, and deconvolution is polynomial division.
The result of dividing v by u is quotient q and remainder r.
32
Information Theory
Lab Session 06
Exercises:
6.1) Consider the cyclic (7, 4) cyclic code generated by G(x) = x3 + x + 1.
6.1.1) Use long division to calculate the check bit polynomial C(x) when M = (1 1 0 0).
Also write the complete code polynomial.
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
6.1.2) Write MATLAB code to perform the above functionality. Use decode functions
and then conv/deconv functions.
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
The generator polynomial of (7, 4) cyclic codes is given by G(x) = 1 + x2 + x3. Find V
by the systematic approach if M(x) = (1 0 0 1).Let the received word is [1 1 1 0 0 1 1].
Find the syndrome vector.
________________________________________________________________________
6.2)
________________________________________________________________________
________________________________________________________________________
33
Information Theory
Lab Session 06
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
6.3) Consider the cyclic (7, 4) hamming code generated by G(x) = x3 + x + 1.
6.3.1) Use systematic approach to calculate the check bit polynomial C(x) when M = (1 1
0 1). Also write the complete code polynomial.
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
6.3.2)
Write MATLAB code to implement the functionality explained in part (a).
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
34
Information Theory
Lab Session 06
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
6.3.3)
Find the syndrome vectors for the following erroneous received words.
i.
1010101
ii.
1000001
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
6.4) Consider (6, 3) cyclic code. Create MATLAB code to perform the following tasks.
6.4.1) Create generator polynomial.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
6.4.2) Write all possible code words and encode using encode function.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
6.4.3) Use decode function to decode all the encoded words.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
35
Information Theory
Lab Session 06
6.4.4)
Induce single error in all the encoded words and then decode.
Record the output that you get.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
6.5) Write a program (C or C++) to implement DECODE function based on Cyclic Codes
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
36
Information Theory
Lab Session 07
Lab Session 07
OBJECT
Understanding the concept of loss-less data compression technique using Huffman coding
THEORY
Huffman coding
Huffman coding is a compression method that converts characters into variable length bit
strings. Most-frequently occurring characters are converted to shortest bit strings; least
frequent, the longest.
Huffman coding uses a specific method for choosing the representation for each symbol,
resulting in a prefix-free code (sometimes called "prefix codes") (that is, the bit string
representing some particular symbol is never a prefix of the bit string representing any other
symbol).
The Prefix Property
Data encoded using Huffman coding is uniquely decodable. This is because Huffman codes
satisfy an important property called the prefix property. This property guarantees that no
codeword is a prefix of another Huffman codeword. For example, 10 and 101 cannot
simultaneously be valid Huffman code words because the first is a prefix of the second. Thus,
any bit stream is uniquely decodable with a given Huffman code. We can see by inspection
that the code words generated below are valid Huffman code words. To see why the prefix
property is essential, consider the code words given below in which e is encoded with 110
which is a prefix of f.
character
codeword
a
0
b
101
c
100
d
111
e
110
f
1100
The decoding of 11000100110 is ambiguous:

11000100110 => face
11000100110 => eaace
Huffman Coding in MATLAB

In MATLAB Huffman code dictionary associates each data symbol with a codeword. It has
the property that no codeword in the dictionary is a prefix of any other codeword in the
dictionary.
The huffmandict, huffmanenco, and huffmandeco functions support Huffman
coding and decoding.
Creating a Huffman Code Dictionary
dict = huffmandict(symbols, p)
Huffman coding requires statistical information about the source of the data being encoded. In
particular, the p input argument in the huffmandict function lists the probability with
which the source produces each symbol in its alphabet.
37
Information Theory
Lab Session 07
For example, consider a data source that produces 1s with probability 0.1, 2s with probability
0.1, and 3s with probability 0.8. The main computational step in encoding data from this
source using a Huffman code is to create a dictionary that associates each data symbol with a
codeword. The commands below create such a dictionary and then show the codeword vector
associated with a particular value from the data source.
symbols = [1 2 3]; % Data symbols
p = [0.1 0.1 0.8]; % Probability of each data symbol
dict = huffmandict(symbols, p) % Create the dictionary.
dict {1,:} % Show first row of the dictionary.
dict {2,:} % Show second row of the dictionary.
dict {3,:} % Show third row of the dictionary.
The output below shows that the most probable data symbol, 3, is associated with a one-digit
codeword, while less probable data symbols are associated with two-digit code words. The
output also shows, for example, that a Huffman encoder receiving the data symbol 1 should
substitute the sequence 11.
dict =
ans
ans
ans
ans
ans
ans
=
=
=
=
=
=
[1]
[2]
[3]
1
1
2
1
3
0
[1x2 double]
[1x2 double]
[
0]
1
0
Creating and Decoding a Huffman Code

The example below performs Huffman encoding and decoding using huffmanenco and
huffmandeco functions. A source is used whose alphabet has three symbols. Notice that
the huffmanenco and huffmandeco functions use the dictionary that huffmandict
created.
sig = repmat([3 3 1 3 3 3 3 3 2 3],1,50); % Data to encode
symbols = [1 2 3]; % Distinct data symbols appearing in sig
p = [0.1 0.1 0.8]; % Probability of each data symbol
dict = huffmandict(symbols,p); % Create the dictionary.
hcode = huffmanenco(sig,dict); % Encode the data.
dhsig = huffmandeco(hcode,dict); % Decode the code.
38
Information Theory
Lab Session 07
Exercises:
7.1)
Five source symbols of the alphabet of a source and their probabilities are shown
Symbol
S0
S1
S2
S3
S4
Probability
0.4
0.2
0.2
0.1
0.1
7.1.1) Build the Huffman code tree for the message and find the codeword for each character.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
7.1.2) Write MATLAB code to obtain Huffman coding for the given characters.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
39
Information Theory
Lab Session 07
7.2)
Construct a Huffman code for the source X emitting symbols with probabilities as
shown below.
Symbol
A
C
E
Probability
0.4
0.1
0.06
Symbol
B
D
F
Probability
0.3
0.1
0.04
Determine:
7.2.1) The code word for each symbol.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
7.2.2) The average number of transmitted binary digits per code word.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
7.2.3) Write MATLAB code to obtain Huffman code for the given source.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
40
Information Theory
Lab Session 08
Lab Session 08
OBJECT
Implementation of Hill cipher technique in cryptography
THEORY
Cryptography:
Cryptography is the science of keeping secrets secret. Assume a sender referred to here as Jack
wants to send a message m to a receiver referred to as Jill. He uses an insecure communication
channel. For example, the channel could be a computer network or a telephone line. There is a
problem if the message contains confidential information. The message could be intercepted
and read by an eavesdropper. Or, even worse, the adversary might be able to modify the message
during transmission in such a way that the legitimate recipient Jill does not detect the
manipulation.
One objective of cryptography is to provide methods for preventing such attacks.
The fundamental and classical task of cryptography is to provide confidentiality by encryption
methods. The message to be transmitted can be some text, numerical data. An executable
program or any other kind of information called the plaintext.
Providing confidentiality is not the only objective of cryptography. Cryptography is also used
to provide solutions for other problems:
1. Data integrity. The receiver of a message should be able to check whether the message
was modified during transmission, either accidently or deliberately. No one should be
able to substitute a false message for the original message, or for parts of it.
2. Authentication. The receiver of a message should be able to verify its origin. No one
should be able to send a message to Jack and pretend to be Jill (data origin
authentication). When initiating a communication, Jack and Jill should be able to
identify each other (entity authentication).
3. Non-repudiation. The sender should not be able to later deny that she sent a message.
Few terminologies:
Cryptanalysis is the (art and) science of breaking a code
Cryptology is the branch of math needed for cryptography and cryptanalysis
The original message is plaintext (sometimes also called clear text)
Disguising the content of a message is called encryption. This results in cipher text,
the encrypted message.
Turning cipher text back into plaintext is called decryption
Flow of Cryptographic Algorithm:
A cryptographic algorithm (also called a cipher) is the mathematical function used for
encryption and decryption. Usually there are two functions, one for encryption and one for
decryption.
If security is based on keeping the algorithm secret, then its a restricted algorithm.
Restricted algorithms are not a good idea because every time a user leaves a group; the
algorithm has to be changed. A group must develop their own algorithm; if they dont have the
expertise, then it will be subpar.
41
Information Theory
Lab Session 08
Modern algorithms use a key, denoted by K. A key is taken from a large range of possible
values, the keyspace. A key is used as additional input for the en-/decryption function to do
the en-/decrypting.
Cryptography in MATLAB
The cryptographic technique, which is known as a Hill cipher, involves arithmetic in a finite
field.
Almost all modern computers use the ASCII character set to store basic text. ASCII stands for
American Standard Code for Information Interchange. The character set uses 7 of the 8 bits in
a byte to encode 128 characters. The first 32 characters are nonprinting control characters, such
as tab, backspace, and end-of-line. The 128th character is another nonprinting character that
corresponds to the Delete key on your keyboard. In between these control characters are 95
printable characters, including a space, 10 digits, 26 lowercase letters, 26 uppercase letters, and
32 punctuation marks. MATLAB can easily display all the printable characters in the order
determined by their ASCII encoding. Start with
x = reshape (32:127, 32, 3)'
This produces a 3-by-32 matrix.
32 33 34 61
62
63
x =[64 65 66
93
94
95 ]
96 97 98 125 126 127
The char function converts numbers to characters. The statement
c = char(x)
produces
c=
!"#$%&'()*+,-./0123456789:;<=>?
@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_
`abcdefghijklmnopqrstuvwxyz{|}~
Line1
Line2
Line3
Count all the characters that are displayed. Line1 has 31 characters, line2 has 32 characters and
line3 has 31 characters. According to the size of matrix x there should be 96 characters in total
but c has produced just 94 characters. Actually c has produced 96 characters but 2 characters
are nonprinting characters.
char(32) is (blank)
char(127) is delete
42
Information Theory
Lab Session 08
The last printable character in c is the tilde, indicating that

char(126)
is the same as
'~'
The characters representing digits are in the first line of c. In fact,
d = char(48:57)
displays a 10-character string
d = 0123456789
This string can be converted to the corresponding numerical values with double or real.
The statement
double(d) - '0'
produces
0
Comparing the second and third lines of c, we see that the ASCII encoding of the lowercase
letters is obtained by adding 32 to the ASCII encoding of the uppercase letters. Understanding
this encoding allows us to use vector and matrix operations in MATLAB to manipulate text.
The ASCII standard is often extended to make use of all eight bits in a byte, but the characters
that are displayed depend on the computer and operating system you are using, the font you
have chosen, and even the country you live in.
Try
char(reshape(160:255,32,3)')
Our encryption technique involves modular arithmetic. All the quantities involved are integers
and the result of any arithmetic operation is reduced by taking the remainder or modulus with
respect to a prime number p. The functions rem(x,y) and mod(x,y) both compute the
remainder if x is divided by y. They produce the same result if x and y have the same sign; the
result also has that sign. But if x and y have opposite signs, then rem(x,y) has the same sign
as x, while mod(x,y) has the same sign as y. Here is a table:
x = [37 -37 37 -37]';
y = [10 10 -10 -10]';
r = [ x y rem(x,y) mod(x,y)]
37
37
produces
37
37
10
10
10
10
7
7
7
7
7
3
3
7
We have chosen to encrypt text that uses the entire ASCII character set, not just the letters.
There are 95 such characters (including space). The next larger prime number is p = 97, so we
represent the p characters by the integers 0: p-1 and do arithmetic mod p.
The characters are encoded two at a time. Each pair of characters is represented by a 2-vector,
x. For example, suppose the text contains the pair of letters 'TV'. The ASCII values for this pair
of letters are 84 and 86. Subtracting 32 to make the representation start at 0 produces the column
vector
43
Information Theory
Lab Session 08
x = (52
)
54
The encryption is done with a 2-by-2 matrix-vector multiplication over the integers mod p. The
symbol is used to indicate that two integers have the same remainder, modulo the specified
prime:
y Ax, mod p,
where A is the matrix
71 2
= (
)
2 26
For our example, the product Ax is
3800
= (
)
1508
If this is reduced mod p, the result is
17
= ( )
53
Converting this back to characters by adding 32 produces 1U.
Now comes the interesting part. Over the integers modulo p, the matrix A is its own inverse. If
y Ax, mod p,
then
x Ay, mod p.
In other words, in arithmetic mod p, A2 is the identity matrix. You can check this with
MATLAB.
p = 97;
A = [71 2; 2 26]
I = mod(A^2,p)
produces
=
I=
71
2
2
26
1 0
0 1
This means that the encryption process is its own inverse. The same function can be used to
both encrypt and decrypt a message.
The M-file crypto.m begins with a preamble.
function y = crypto(x)
% CRYPTO Cryptography example.
% y = crypto(x) converts an ASCII text string into another
% coded string. The function is its own inverse, so
% crypto(crypto(x)) gives x back.
% See also: ENCRYPT.
A comment precedes the statement that assigns the prime p.
% Use a two-character Hill cipher with arithmetic
% modulo 97, a prime.
p = 97;
44
Information Theory
Lab Session 08
Choose two characters above ASCII 128 to expand the size of the character set from 95 to 97.
c1 = char(169);
c2 = char(174);
x(x==c1) = 127;
x(x==c2) = 128;
The conversion from characters to numerical values is done by
x = mod(real(x-32),p);
Prepare for the matrix-vector product by forming a matrix with two rows and lots of columns.
n = 2*floor(length(x)/2);
X = reshape(x(1:n),2,n/2);
All this preparation has been so that we can do the actual finite field arithmetic quickly and
easily.
% Encode with matrix multiplication modulo p.
A = [71 2; 2 26];
Y = mod(A*X,p);
Reshape into a single row.
y = reshape(Y,1,n);
If length(x) is odd, encode the last character
if length(x) > n
y(n+1) = mod((p-1)*x(n+1),p);
end
Finally, convert the numbers back to characters.
y = char(y+32);
y(y==127) = c1;
y(y==128) = c2;
Let's follow the computation of y = crypto('Hello world'). We begin with a
character string.
x = 'Hello world'
This is converted to an integer vector.
x = 40 69 76 76 79 0 87 79 82 76 68
length(x) is odd, so the reshaping temporarily ignores the last element
40 76 79 87 82
X=
69 76 0 79 76
A conventional matrix-vector multiplication A*X produces an intermediate matrix.
2978 5548 5609 6335 5974
1874 2128
158 2228 2140
Then the mod(.,p) operation produces
68 19 80 30 57
Y=
31 91 61 94 6
This is rearranged to a row vector.
y = 68 31 19 91 80 61 30 94 57 6
Now the last element of x is encoded by itself and attached to the end of y.
45
Information Theory
Lab Session 08
y = 68 31 19 91 80 61 30 94 57 6 29
Finally, y is converted back to a character string to produce the encrypted result.
y = 'd?3{p]>~Y&='
If we now compute crypto(y), we get back our original 'Hello world'.
Exercises:
8.1) If x is the character string consisting of just two blanks, x = ' ', then crypto(x) is actually
equal to x. Why does this happen? Are there any other two-character strings that crypto
do not change?
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
8.2) Find another 2-by-2 integer matrix A for which mod(A*A,97) is the identity matrix.
Replace the matrix in crypto.m with your matrix and verify that the function still works
correctly.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
46
Information Theory
Lab Session 08
8.3) The function crypto works with 97 characters instead of 95. It can produce output, and
correctly handle input, that contains two characters with ASCII values greater than 127.
What are these characters? Why are they necessary? What happens to other characters
with ASCII values greater than 127?
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
8.4)
Create a new crypto function that works with just 29 characters: the 26 lowercase letters,
plus blank, period, and comma. You will need to find a 2-by-2 integer matrix A for
which mod(A*A,29) is the identity matrix.
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
47
Information Theory
Lab Session 09
Lab Session 09
OBJECT
Implementation of Vigenere cipher technique in cryptography
THEORY
This cipher produces encrypted cipher text from an input plaintext message using a key and a
matrix of substitution alphabets. Recovery of the plaintext from the cipher text requires the key.
By modern standards, this isn't a terribly secure cryptosystem, but it is easily implemented, and
difficult enough that encrypted messages can't be casually deciphered.
The Vigenere cipher encrypts its inputs by using the key and the plaintext as indices into a fixed
lookup table: the Vigenere square. For ease of computation, the algorithm first maps letters to
numbers: A=1, B=2... SPACE=27. As shown in the matrix below, each row in the square is
derived from the row above by circularly shifting it one place to the left:
1 2
2 3
3 4
...
27 1
3 4 5 ... 26 27
4 5 6 ... 27 1
5 6 7 ... 1 2
2 3 4 ... 25 26
To encrypt, replicate the letters in the key so the key and plaintext are the same length. Then,
derive each cipher text letter by lookup in the Vigenere square: use the key letter as the row
index and the plaintext letter as the column index. If the key K and the plaintext P are n letters
long, form the cipher text result C by indexing into the Vigenere square V, as follows:
C(n) = V(K(n), P(n))
Decryption simply reverses the processs, using the key letter to determine the row
index and the ciphertext letter to determine the colum index, which is the plaintext
letter. For example:
P(n) = FIND(V(K(n,:)) == C(n))
Implementing the Vigenere Cipher in MATLAB

To implement the Vigenere cipher in MATLAB, I wrote three functions:
vigenere - Generates the Vigenere square.

encrypt - Transforms a plaintext message into ciphertext.
The vigenere function uses three additional MATLAB functions:

circshift - Creates each row of the Vigenere square.
arrayfun - Iterates over the row indices, calling circshift on each; returns a cell array of the
rows.
48
Information Theory
Lab Session 09
reshape - Builds a matrix from the cell array.

function v = vigenere
% VIGENERE Return a Vigenere square in a 27x27 matrix.
% Square 27 on a side
count = 27;
% Map letters to numbers, A=1, B=2, ... SPACE=27
alpha = 1:count;
% Create a matrix with 27 shifted substitution alphabets
%
1 2 3 4 5 ... 26 27
%
2 3 4 5 6 ... 27 1
%
3 4 5 6 7 ... 1 2
% etc.
v = arrayfun(@(n) circshift(alpha, [0, -n]), 0:count-1,...
'UniformOutput', false);
v = reshape([v{:}], count, count);
encrypt creates a Vigenere square and transforms the key and plaintext from strings into vectors
of numbers. It encrypts the plaintext and then translates the encrypted message from a vector of
numbers into a character string. encrypt calls arrayfun to perform the encryption lookup instead
of using a for-loop. arrayfun is faster and it won't ever get the length of the plaintext wrong, as
a hand-written for-loop might.
function ciphertext = encrypt(plaintext, key)
v = vigenere;
% Squeeze out everything except letters and the space
character
exclude = regexp(plaintext, '[^a-zA-Z ]');
plaintext(exclude) = [];
% Make the key and the plaintext lower case, and convert to
% numeric values.
key = lower(key) - double('a') + 1;
key(key < 0) = 27;
plaintext = lower(plaintext) - double('a') + 1;
plaintext(plaintext < 0) = 27;
% Replicate the key so that it is as long as the plaintext.
keyIndex = mod(0:(numel(plaintext)-1), numel(key))+1;
k = key(keyIndex);
% Encrypt: C(n) = V(k(n), plaintext(n))
ciphertext = arrayfun(@(m,n) v(m,n), k, plaintext) - 1;
ciphertext(ciphertext == 26) = double(' ') - double('a');
ciphertext = upper(char(ciphertext + double('a')));
decrypt reverses the encryption process by searching for the ciphertext letter in the key letter
row. The column index of the ciphertext letter maps back (1=A, 2=B, ..., 27=SPACE) to the
49
Information Theory
Lab Session 09
plaintext letter. Like encrypt, decrypt loops over the cipertext using arrayfun instead of a forloop.
function plaintext = decrypt(ciphertext,key)
v = vigenere;
% Convert the key and the ciphertext to one-based numeric
values;
% A=1, B=2, etc. SPACE=27
key = lower(key) - double('a') + 1;
key(key < 0) = 27;
ciphertext = lower(ciphertext) - double('a') + 1;
ciphertext(ciphertext < 0) = 27;
% Replicate the key so that it is as long as the ciphertext.
keyIndex = mod(0:(numel(ciphertext)-1), numel(key))+1;
k = key(keyIndex);
% Decrypt. Each letter of the key determines a row in the Vigenere
% square. In that row, find the column index of the corresponding
% ciphertext letter. Convert the index back to a letter to
determine
% the decrypted character (1=A, 2=B, etc.).
plaintext = arrayfun(@(m,n) find(v(m,:) == n), k, ciphertext) 1;
plaintext(plaintext == 26) = double(' ') - double('a');
plaintext = upper(char(plaintext + double('a')));
Most of the complexity in both encrypt and decrypt stems from extending the algorithm to allow
spaces in the plaintext and the key. Without spaces, reading decrypted messages is difficult and
error-prone.
Look how easy it is to implement a Caesar cipher with shift 5 in MATLAB:
>> myword='supercalifragilisticexpialidocious'
myword = supercalifragilisticexpialidocious
>> secretword=setstr(rem(myword+5-97,26)+97)
secretword =
xzujwhfqnkwflnqnxynhjcunfqnithntzx
>> undecodedword=setstr(rem(secretword-5-97+26,26)+97)
undecodedword =
supercalifragilisticexpialidocious
Explanation: MATLAB automatically converts the letters in a string to their ASCII decimal
equivalents if the string is used in a numeric context. Since the letters "a" to "z" are in ASCII
positions 97 through 122, subtracting 97 translates to the range 0 to 25. Adding 5 applies the
shift, and taking the remainder upon division by 26 via the function rem reduces modulo26.
Adding 97 translate back to the range 97 to 122, and applying the setstr function converts back
to an alphabetic string. Reversing the operations decodes the message.
50
Information Theory
Lab Session 10
Lab Session 10
OBJECT
Learning basic model designs with Simulink
To help students become familiar with the Simulink environment.
THEORY
Simulink is a program for simulating signals and dynamic systems. As an extension of
MATLAB, Simulink adds many features specific to the simulation of dynamic systems while
retaining all of MATLABs general purpose functionality.
Simulink has two phases of use: model definition and model analysis. A typical session starts
by either defining a new model or by recalling a previously defined model, and then proceeds
to analyze that model. In order to facilitate the model definition, Simulink has a large class of
windows called block diagram windows. In these windows, models are created and edited
principally using mouse-driven commands. An important part of mastering Simulink is to
become familiar with manipulations of various model components in these windows.
After you define a model, you can analyze it either by choosing options from the Simulink
menus or by entering commands in the MATLAB command window. The progress of an
ongoing simulation can be viewed while it is running, and the final results can be made available
in the MATLAB workspace when the simulation is complete.
Examples:
1. Product:
Multiply and divide scalars and non-scalars or multiply and invert matrices
Library
Math Operations
Description
Default Product Block Use

By default, the Product block outputs the result of multiplying two inputs: two scalars, a scalar
and a non-scalar, or two non-scalars that have the same dimensions. The default parameter
values that specify this behavior are:
Multiplication: Element-wise (.*)
Number of inputs: 2
Setting non-default values for either of these parameters can change a Product block to
function as a Divide block or a Product of Elements block.
2. Logical operations:
Perform specified logical operation on input
51
Information Theory
Lab Session 10
Library
Logic and Bit Operations
Description
The Logical Operator block performs the specified logical operation on its inputs. An input
value is TRUE (1) if it is nonzero and FALSE (0) if it is zero.
Exercises:
10.1) Design a model that compares (>, <, >=, <=, = = ) the two positive real numbers and
produces the output either TRUE (1) or FALSE (0)
10.2) Design a model that:
10.2.1) Identifies the minimum absolute value from the list of numbers available.
10.2.2) Identifies the maximum absolute value from the list of numbers available.
10.3) Design a model that performs the summation of any four elements (say 20,10,30,45)
10.4) Design a model that performs the following operation: -20+10-30+45.
10.5) Design a model that sorts (ascending and descending both) at least 10 random numbers.
10.6) Design a model that concatenates three column vectors with three elements in each
column.
Attach printout of simulated models here.
52
Information Theory
Lab Session 11
Lab Session 11
OBJECT
Encoding the data bits using a Binary Cyclic block encoder in Simulink
THEORY
In cyclic coding, data is sent with a checksum. When arrives, checksum is recalculated. It should
match the one that was sent.
Refer lab session #5 for further description.
SIMULINK FUNCTIONS
1. Binary Cyclic Encoder
Create systematic cyclic code from binary vector data
Library
Block sublibrary of Channel Coding
Description
The Binary Cyclic Encoder block creates a systematic cyclic code with message length K and
codeword length N. The number N must have the form 2M-1, where M is an integer greater than
or equal to 3.
The input must contain exactly K elements. If it is frame-based, then it must be a column vector.
The output is a vector of length N.
You can determine the systematic cyclic coding scheme in one of two ways:
To create an [N, K] code, enter N and K as the first and second dialog parameters,
respectively. The block computes an appropriate generator polynomial, namely,
cyclpoly(N,K,'min').
To create a code with codeword length N and a particular degree-(N-K) binary
generator polynomial, enter N as the first parameter and a binary vector as the second
parameter. The vector represents the generator polynomial by listing its coefficients in
order of ascending exponents. You can create cyclic generator polynomials using the
Communications Toolbox cyclpoly function.
This block supports double and boolean data types.
Codeword length N
The codeword length, which is also the output vector length.
Message length K, or generator polynomial
53
Information Theory
Lab Session 11
Either the message length, which is also the input vector length; or a binary vector that
represents the generator polynomial for the code.
Exercises:
11.1 Create a model that implements the functionality of Binary Cyclic Encoder.
Attach printout of simulated model here.
54
Information Theory
Lab Session 12
Lab Session 12
OBJECT
Decoding the code words using a Binary Cyclic block decoder in Simulink
THEORY
Decoding of Cyclic codes:
Refer lab session#6 for description.
SIMULINK FUNCTIONS
1. Binary Cyclic Decoder
Decode systematic cyclic code to recover binary vector data
Library
Description
The Binary Cyclic Decoder block recovers a message vector from a codeword vector of a binary
systematic cyclic code. For proper decoding, the parameter values in this block should match
those in the corresponding Binary Cyclic Encoder block.
If the cyclic code has message length K and codeword length N, then N must have the form 2M1 for some integer M greater than or equal to 3.
The input must contain exactly N elements. If it is frame-based, then it must be a column vector.
The output is a vector of length K.
You can determine the systematic cyclic coding scheme in one of two ways:
To create an [N,K] code, enter N and K as the first and second dialog parameters,
respectively. The block computes an appropriate generator polynomial, namely,
cyclpoly(N,K,'min').
To create a code with codeword length N and a particular degree-(N-K) binary generator
polynomial, enter N as the first parameter and a binary vector as the second parameter. The
vector represents the generator polynomial by listing its coefficients in order of ascending
exponents. You can create cyclic generator polynomials using the Communications
Toolbox cyclpoly function.
55
Information Theory
Lab Session 12
Codeword length N
The codeword length N, which is also the input vector length.
Message length K, or generator polynomial
Either the message length, which is also the output vector length; or a binary vector that
represents the generator polynomial for the code.
Exercise:
12.1 Create a model that implements the functionality of Binary Cyclic Decoder.
56
Information Theory
Lab Session 13
Lab Session 13
OBJECT
Encoding the data bits using a Binary Linear block encoder in Simulink
THEORY
In mathematics and information theory, linear code is an important type of block code used in
error correction and detection schemes. Linear codes allow for more efficient encoding and
decoding algorithms than other codes.
Linear codes are applied in methods of transmitting symbols (e.g., bits) on communications
channel so that, if errors occur in the communication, some errors can be detected by the
recipient of the message block. A linear code of length n transmits blocks containing n symbols.
For example, the (7, 4) Hamming code is a binary linear code which represents 4-bit values
with 7 bits. In this way, the recipient can detect errors as severe as 2 bits per block.
In linear block codes, a block of k information bits is followed by a group of r check bits
derived from the information bits and at the receiver the check bits are used to verify the
information bits which are preceding the check bits.
For further information refer lab session# 3
SIMULINK FUNCTIONS:
1. Binary Linear Encoder
Create linear block code from binary vector data
Library
Block sub library of Channel Coding
Description
The Binary Linear Encoder block creates a binary linear block code using a generator matrix
that you specify. If K is the message length of the code, then the Generator matrix parameter
must have K rows. If N is the codeword length of the code, then Generator matrix must have
N columns.
The input must contain exactly K elements. If it is frame-based, then it must be a column
vector. The output is a vector of length N.
This block supports double and Boolean data types.
57
Information Theory
Lab Session 13
Generator matrix
A K-by-N matrix, where K is the message length and N is the codeword length.
Exercises:
13.1) Create a model that implements the functionality of Binary Linear Block Encoder
Attach printout of simulated models here.
58
Information Theory
Lab Session 14
Lab Session 14
OBJECT
Decoding the code words using a Binary Linear block decoder in Simulink
THEORY
Refer lab session# 4 for description.
SIMULINK FUNCTIONS:
In order to develop models in Simulink for decoding following functions are to be used.
1. Binary Linear Decoder
Decode linear block code to recover binary vector data
Library
Description
The Binary Linear Decoder block recovers a binary message vector from a binary code word
vector of a linear block code.
The Generator matrix parameter is the generator matrix for the block code. For proper
decoding, this should match the Generator matrix parameter in the corresponding Binary
Linear Encoder block. If N is the code word length of the code, then Generator matrix must
have N columns. If K is the message length of the code, then the Generator matrix parameter
must have K rows.
The input must contain exactly N elements. If it is frame-based, then it must be a column vector.
The output is a vector of length K.
The decoder tries to correct errors, using the Decoding table parameter. If Decoding table is
the scalar 0, then the block defaults to the table produced by the Communications Toolbox
function syndtable. Otherwise, Decoding table must be a 2N-K-by-N binary matrix. The rth row
of this matrix is the correction vector for a received binary code word whose syndrome has
decimal integer value r-1. The syndrome of a received code word is its product with the
transpose of the parity-check matrix.
Generator matrix
59
Information Theory
Lab Session 14
Generator matrix for the code; same as in Binary Linear Encoder block.
Decoding table
Either a 2N-K-by-N matrix that lists correction vectors for each code word's syndrome;
or the scalar 0, in which case the block defaults to the table corresponding to the
Generator matrix parameter.
2. Display
Refer lab session #11
3. Constant
Refer lab session #11
Exercise:
14.1) Create a model that implements the functionality of Binary Linear Block Decoder
60

It

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

It

Caricato da

Copyright:

Formati disponibili

Practical Workbook

4th Edition: 2014

Department of Computer & Information Systems Engineering

Lab Session No.

Application of MATLAB to coding theory

Learning basic commands of MATLAB

Implementation of Vigenere cipher technique in cryptography

Learning basic model designs with Simulink

d) The Colon Operator

Subscript expressions involving colons refer to portions of a matrix.

Declare predefined variables (, i, j) along with variable X .

using the command format long?

f) Deleting Rows and Columns

The check bits are determined by some predetermined rule.

Message Code Vector

Consider a systematic (6, 3) block code generated by the submatrix [1 1 0; 0 1 1; 1 0 1].

3.2.1) Write the value of n and k.

Write a program in any language(C or C++) to implement ENCODE function that

Syndrome decoding: example

Consider two possible messages:

First and last elements of the matrix are shown above

Suppose error pattern e = [0 0 0 0 0 1 0 0] is added to both code words.

The above example uses hammgen function whose explanation is as follows.

4.1.1) Write the value of n and k.

Message Code Vector Weights

A (7,4) LBC has generator matrix G given by

4.4.1) Determine the PCM for above matrix.

The Galois Field Computation in MATLAB

The generator polynomial of a (7, 4) cyclic code is given by

Find V by direct method as well as the systematic approach if M(x) = (1 0 1 0).

Write MATLAB code to implement the functionality explained in part (a).

The decoding of 11000100110 is ambiguous:

Huffman Coding in MATLAB

Creating and Decoding a Huffman Code

The last printable character in c is the tilde, indicating that

Implementing the Vigenere Cipher in MATLAB

vigenere - Generates the Vigenere square.

The vigenere function uses three additional MATLAB functions:

reshape - Builds a matrix from the cell array.

Default Product Block Use

Attach printout of simulated models here.

Potrebbero piacerti anche