All Matlab Codes PDF

Jawaharlal Nehru Engineering College
Laboratory Manual
Information Theory and Coding

For
Third Year Students
Manual made by
Dr. V.B.MALODE
Prof. V. A.KULKARNI
” Author JNEC, Aurangabad

ETC 324: Information Theory and coding 2018
MGM’S
N-6, CIDCO, Aurangabad
Department of Electronics &Telecommunication
Vision of the Department:

To develop GREAT technocrats and to establish centre of excellence in the field of Electronics
and Telecommunications.
Global technocrats with human values ”
“ Research and lifelong learning attitude,
Excellent ability to tackle challenges
Awareness of the needs of society
Technical expertise

Mission of the Department:
1. To provide good technical education and enhance technical competency by providing good
infrastructure, resources, effective teaching learning process and competent, caring and
committed faculty.
2. To provide various platforms to students for cultivating professional attitude and ethical
values.
3. Creating a strong foundation among students which will enable them to pursue their career
choice.
2
Technical Document
This technical document is a series of Laboratory manuals of Electronics & Telecommunication

and is a certified document of Jawaharlal Nehru Engineering College. The care has been taken to
make the document error free but still if any error is found kindly bring it to the notice of subject
teacher and HOD.
Recommended by,
HOD
Approved by,
Principal
Copies:
• Departmental Library
• Laboratory
• HOD
• Principal
3
FOREWORD
It is my great pleasure to present this laboratory manual for Third year engineering students for
the subject of Information Theory and coding, keeping in view the vast coverage required for
visualization of concepts of ITC.
As a student, many of you may be wondering with some of the questions in your mind regarding
the subject and exactly that has been tried to answer through this manual.
Faculty members are also advised to cover these aspects in initial stage itself, It will greatly
relieve them in future, as much of the load will be taken care by the enthusiasm of the students
once they are conceptually clear. Students are advised to thoroughly go through this manual
rather than only the topics mentioned in the syllabus, as practical aspects are the key to
understanding and conceptual visualization of theoretical aspects covered in the books.
Good Luck for your Enjoyable Laboratory Sessions.
H.O.D.
4
SUBJECT INDEX
Sr. Title Page No.

No.
1 Dos and Don’ts, Instruction to Lab teachers 6
2 Lab course objectives 7
3 Lab Exercises
I Pre-Requisite 1
II Pre-Requisite 2: Introduction to Matlab 8
1 Determination of entropy of a given source 14
2 Determination of various entropies and mutual information of
15
a given channel (Noise free channel)
3 Determination of various entropies and mutual information of
18
a given channel (Binary symmetric channel)
4 Generation and evaluation of variable length source coding
21
using MATLAB (Huffman Coding and decoding)
5 Coding & decoding of Linear block codes 24
6 Coding & decoding of Cyclic codes 26
7 Coding and decoding of convolutional codes 28
8 Coding and decoding of BCH codes. 30
I Postrequisite 33
2 Questions based on the subject 39
3 Conduction of viva voce examination 44
4 Evaluation and marking scheme 44
5
1. DOs and DON’Ts in Laboratory:
1. Do not handle software and PC without reading the instructions/

Instruction manuals.
2. Refer Help for debugging the program.
3. Go through Demos of communication tool box in Matlab.
4. Strictly observe the instructions given by the teacher/Lab Instructor.
2 Instruction for Laboratory Teachers:
1. Lab work completed during prior session should be corrected during the
next lab session.
2. Students should be guided and helped whenever they face difficulties.
3. The promptness of submission should be encouraged by way of marking
and evaluation patterns that will benefit the sincere students.
6
Laboratory Course Objectives:
1. To expose students to knowledge about information and various entropies.

2. To make students understand working of various codes like linear block, cyclic,
convolution and BCH codes.
3. To explore source coding for text, audio and speech.
Course Outcomes:
Students will be able to
1. Demonstrate various entropies and information.
2. Apply source coding techniques.
3. Construct codes using different coding techniques.
4. Explain various coding schemes for text ,speech and audio.
7
INTRODUCTION TO MATLAB- PREREQUISITE

Q. Solve the following using Matlab commands:
1) Generate and Display a sequence of 10 numbers.

COMMAND: V = 1:1:10
OUTPUT: V =
1 2 3 4 5 6 7 8 9 10
2) Display a sequene of 20 numbers with an interval of 2
COMMAND: B = 1:2:40
OUTPUT: B =
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39
3) Write a row vector
COMMAND: C = [1,2,3,4]
OUTPUT: C=
1 2 3 4
4) Write a Column vector
COMMAND: D = [1;2;3;4]
OUTPUT: D = 1
2
3
4
5) Write a matrix 3*3
COMMAND: E = [1,2,3;4,5,6;7,8,9]
OUTPUT: E =
1 2 3
4 5 6
7 8 9
6) Add 2 Matrix.
COMMAND: X = [1,2,3;4,5,6] ; Y = [6,5,4;3,2,1]
F = X+Y
8
OUTPUT: X =
1 2 3
4 5 6
Y=
6 5 4
3 2 1
F=
7 7 7
7 7 7
7) X is row vector, change it to column vector.
COMMAND: Z = X'
OUTPUT: Z =
1 4
2 5
3 6
8) Generate the following matrix
COMMAND: A=[1,2,3;4,5,6;7,8,9]
C=[1^3,2+sqrt(3),3*sin(1);exp(2),17/3,pi+3;1/3,2-sqrt(3),-7*cos(pi/7)]
x=[1 2 3 4 5 6]'
OUTPUT: A=
1 2 3
4 5 6
7 8 9
9
C=
1.0000 3.7321 2.5244
7.3891 5.6667 6.1416
0.3333 0.2679 -6.3068
x=
1
2
3
4
5
6
9) Generate 5x3 matrix whose first three rows are rows of A and last two rows are all ones.
COMMAND: w=[A;ones(2,3)]
OUTPUT: w=
1 2 3
4 5 6
7 8 9
1 1 1
1 1 1
(3) What happens: (1) B = A; B(3,3) = 10; (2) x = [1:50] ’; x(length(x))

(3) D = [1 1; 2 2]
E = [5 10; 5 10]
F = [D E D; E D E]
G = [F, rand(4,1) ]
COMMAND: 1 ) B = A; B(3,3) = 10; (2) x = [1:50] ’; x(length(x))
(3) D = [1 1; 2 2]
E = [5 10; 5 10]
F = [D E D; E D E]
G = [F, rand(4,1) ]
OUTPUT: (1) B =
1 2 3
4 5 6
7 8 10
(2) ans =
50
(3) D =
1 1
2 2
E=
5 10
5 10
10
F=
1 1 5 10 1 1
2 2 5 10 2 2
5 10 1 1 5 10
5 10 2 2 5 10
G=
1.0000 1.0000 5.0000 10.0000 1.0000 1.0000 0.8147
2.0000 2.0000 5.0000 10.0000 2.0000 2.0000 0.9058
5.0000 10.0000 1.0000 1.0000 5.0000 10.0000 0.1270
5.0000 10.0000 2.0000 2.0000 5.0000 10.0000 0.9134
(10) Using matrix A, find 1.A(1) 2. A(2) 3. A(9) 4. A(1,1) 5. A(2,1) 6. A(1,2) 7. A(1:3, 2)
8. A([1 3 4], [1 3]) 9. a = A(:, 2) 10. a = A(:) 11. A(2,end)
COMMAND: 1.A(1) 2. A(2) 3. A(9) 4. A(1,1) 5. A(2,1) 6. A(1,2) 7. A(1:3, 2)

8. A([1 3 ], [1 3]) 9. a = A(:, 2) 10. a = A(:) 11. A(2,end)
OUTPUT:
1 ans =
1 9. a =
2.ans = 2
4 5
3.ans = 8
9
4.ans = 10. a =
1 1
5.ans = 4
2 7
6.ans = 2
4 5
7.ans = 8
2 3
5 6
8 9
8.ans = 11. ans =
1 3 6
7 9
11
11) Create a 3x4 matrix for the following

1. random matrix 2. zero matrix 3. ones matrix 4. identity matrix
COMMAND: 1. rand(3,4) 2.zeros(3,4) 3.ones(3,4) 4. eye(3,4)
OUTPUT: 1. ans =
0.6324 0.5469 0.1576 0.4854
0.0975 0.9575 0.9706 0.8003
0.2785 0.9649 0.9572 0.1419
2.ans =
0 0 0 0
0 0 0 0
0 0 0 0
3.ans =
1 1 1 1
1 1 1 1
1 1 1 1
4.ans =
1 0 0 0
0 1 0 0
0 0 1 0
12) Create a magic matrix of 3x3.

COMMAND: MAGIC(3)
OUTPUT: 8 1 6
3 5 7
4 9 2
13) Create 3D 3x3 ones matrix.

COMMAND: y=ones(3,3,3)
OUTPUT: y(:,:,1) =
1 1 1
1 1 1
1 1 1
y(:,:,2) =
1 1 1
1 1 1
1 1 1
y(:,:,3) =
1 1 1
1 1 1
1 1 1
14) Write command to delete second row of A.
COMMAND: A(2,:)=[]
OUTPUT: A=
1 2 3
7 8 9
15) Write a command to delete 3rd through 5th column of 6x6 matrix. For this create a 6x6
matrix.
COMMAND:
Z=[1,2,3,4,5,6;7,8,9,10,11,12;13,14,15,16,17,18;19,20,21,22,23,24;25,26,27,28,29,30;31,32,33,
34,35,36]
Z(:,3:5)=[]
12
OUTPUT: Z=
1 2 3 4 5 6
7 8 9 10 11 12
13 14 15 16 17 18
19 20 21 22 23 24
25 26 27 28 29 30
31 32 33 34 35 36
Z=
1 2 6
7 8 12
13 14 18
19 20 24
25 26 30
31 32 36
16) Write the commands/programs and calculations wherever applicable:

1. Area=pi*r^2, (where r=(pi^(1/3 )-1))
2.Calculate and compare for:1. exp((pi/2)*i) 2. exp(pi/2i)
3. e^3 4. ln(e^3) 5. log10(e^3) 6. log10(10^5) 7. e^(pi*sqrt(163))
8.solve for 3^x=17
COMMAND: 1. Area= pi*((pi^(1/3))-1)^2 2.1 exp((pi/2)*i) 2.2 . exp(pi/2i)
3.exp(3) 4. Log(exp(3)) 5. Log10(exp(3)) 6. Log10(10^5) 7.
exp(pi*sqrt(163))
8. x=log(17)/log(3)
OUTPUT: 1. Area = 2.1. ans = 2.2 ans=

0.6781 0.0000 + 1.0000i 0.0000 - 1.0000i
3. ans = 4. ans = 5. ans = 6. ans= 7. ans=
20.0855 3 1.3029 5 2.6254e+017
8. x=
2.5789
17) Create a vector x whose entries are the square root of natural numbers 1 to 10 (use’ for’
statement)
COMMAND: for i=1:10
x(i)=i^2
end
OUTPUT: x=
1 4 9 16 25 36 49 64 81 100
13
INFORMATION THEORY AND CODING TECHNIQUES
EXPERIMENT NO. 1
Determination of Entropy
Aim: To find information and entropy of a given source.

Apparatus: PC, Matlab s/w
Theory: (1) What is discrete memory less source?
(2) Write definition, formula and units for the following
i) Information
ii) Entropy
iii) Information rate.
(3).What are the different types of entropies?
Algorithm:
1. Enter no. of symbols.
2. Input the probabilities of symbols resp.
3. Calculate the entropy of the channel input. i.e. H(x) using the formula:
H(x)=
Conclusion:
Program:
(1)% Find entropy of the source
clc;
clear all;
close all;
i=input('Enter no. of elements=');
p=input('Enter probabilities=');
sum=0;
for n=1:i
H=sum+(p(n)*log2(1/p(n)));
sum=H;
end
disp('H(x): ');
disp(H);
output:
Enter no. of elements=2
Enter probabilities=[2/3,4/5]
H(x):
0.6475
14
Experiment no.2
Determination of various entropies and mutual information of the given channel.
Aim: Write a program for determination of various entropies and mutual information of a
given
channel. Test various types of channel such as
a) Noise free channel. b) Error free channel
c) Binary symmetric channel
Compare channel capacity of above channels.
Apparatus: PC,MATAB/C
Theory:
1. Explain discrete memoryless channel.
2. Explain various types of channels with their equations and neat diagrams.
3. Explain in detail mutual information with its equation.
4. Explain relations between various entropies and mutual information, giving their
equations
Algorithm:
I) Entropies:
1. Input the no. of inputs of a channel.
2. Input the no. of outputs of a channel.
3. Input the channel matrix. Test the condition that sum of all the entries in each row
should be equal to 1.
4. Input the channel input probabilities. i.e. P[X].
5. Calculate the entropy of the channel input. i.e. H(X)
6. Calculate output probability matrix P[Y], by multiplying input probability matrix by
channel matrix.
7. Also calculate entropy of channel output. i.e. H(Y).
8. Convert input probability matrix into diagonal matrix.i.e. P[X]d
9. Calculate the joint probability matrix by multiplying input probability matrix in
diagonal form by channel matrix.
10. Calculate joint entropy with the help of formula
11. Calculate conditional entropies H(Y/X)&H(X/Y).
12. Also we can calculate mutual information as
I(X;Y)=H(X)-H(X/Y) or
I(X;Y)=H(Y)-H(Y/X)
II) Various channels

1) Noise free channel
1. For noise free channel enter only diagonal elements of the joint probability matrix.
Condition mentioned on step no.3 should be satisfied.
2. Repeat steps from 4 to 10.
2)Error free channel

1. A channel is said to be error free if capacity of the channel is greater than entropy of the
channel. So at first calculate the capacity of the channel using the formula
Capacity C= log M bits/symbol
Where M:- No. of inputs of the channel.
15
2. Calculate entropy of the input.

3.Compare capacity of the channel with channel input entropy.
3) Binary Symmetric Channel

1.BSC channel is characterized by No.of input = No. of output = 2
2.Conditional probability matrix is as follows
P(Y/X)=
3. Derive the joint probability matrix from this matrix by multiplying it by
P(X) =[ 0 1]
So the matrix which we take input from user is
P(X,Y)=
Where p should be entered by the user.
4.Then repeat steps 4 to 8 to calculate all the required quantities.
Conclusion:
Program:
% %prgm for entropy and MI for noise free channel
clc;
clear all;
close all;
q=input('Enter joint probabilities matrix=');
sum=0;
%probability P(x)
for n=1:i
w=0;
for m=1:i
p(n)=w+q(n,m)
w=p(n);
end
end
disp('P(x):');
disp(p);
% entropy H(x)
for n=1:i
sum=H;
end
disp('H(x): ');
disp(H);
%conditional probability matrix
for n=1:i
for m=1:i
a(n,m)=q(n,m)/p(n);
end
end
disp('P(Y/X):');
disp(a);
% entropy H(Y/X)
d=0;
for n=1:i
16
for m=1:i
if(a(n,m)>0)
H1=d+(q(n,m)*log2(1/a(n,m)));
d=H1
end
end
end
disp('H(Y/X):');
disp(H1);
% MI
m=H-H1;
disp('MI=');
disp(m);
% probability P(Y)
for n=1:i
w=0;
for m=1:i
s(n)=w+q(m,n);
w=s(n);
end
end
disp('P(Y):');
disp(s);
% entropy H(Y)
k=0;
for n=1:i
H2=k+(s(n)*log2(1/s(n)));
k=H2;
end
disp('H(Y): ');
disp(H2);
Output
: Enter no. of elements=3 Enter joint probabilities matrix=[.2 0 0;0 .4
0;0 0 .4]
p= H(Y/X):
0
0.2000 0.4000 0.4000
P(x): MI=
0.2000 0.4000 0.4000 1.5219
H(x):
1.5219 P(Y):
0.2000 0.4000 0.4000
P(Y/X):
1 0 0 H(Y):
0 1 0 1.5219
0 0 1
17
Experiment no.3
Determination of various entropies and mutual information of the given BSC channel.
Aim: Write a program for determination of various entropies and mutual information of a given
channel. (Binary symmetric channel).
Apparatus: PC,MATAB/C
Theory:
1. Explain in detail BSC with neat diagram.
2. Find capacity of BSC channel.
Algorithm:
I) Entropies:
1. Input the no. of inputs of a channel.
2. Input the no. of outputs of a channel.
3. Input the channel matrix. Test the condition that sum of all the entries in each row should
be equal to 1.
4. Input the channel input probabilities. i.e. P[X].
5. Calculate the entropy of the channel input. i.e. H(X)
6. Calculate output probability matrix P[Y], by multiplying input probability matrix by
channel matrix.
7. Also calculate entropy of channel output. i.e. H(Y).
8. Convert input probability matrix into diagonal matrix.i.e. P[X]d
9. Calculate the joint probability matrix by multiplying input probability matrix in diagonal
form by channel matrix.
10. Calculate joint entropy with the help of formula
11. Calculate conditional entropies H(Y/X)&H(X/Y).
12. Also we can calculate mutual information as
I(X;Y)=H(X)-H(X/Y) or
I(X;Y)=H(Y)-H(Y/X)
Binary Symmetric Channel

1.BSC channel is characterized by No.of input = No. of output = 2
2.Conditional probability matrix is as follows
P(Y/X)=
3. Derive the joint probability matrix from this matrix by multiplying it by
P(X) =[ 0 1]
So the matrix which we take input from user is
P(X,Y)=
Where p should be entered by the user.
4.Then repeat steps 4 to 8 to calculate all the required quantities.
Conclusion:
18
% %prgm for entropy and MI for BSC channel
clc;
clear all;
close all;
p=input('Enter probability=');
q=input('Enter conditional probabilities matrix=');
sum=0;
% entropy H(x)
for n=1:i
sum=H;
end
disp('H(x): ');
disp(H);
%Joint probability matrix

for n=1:i
for m=1:i
a(n,m)=q(n,m)*p(n);
end
end
disp('P(X,Y):');
disp(a);
% entropy H(Y/X)
d=0;
for n=1:i
for m=1:i
H1=d+(a(n,m)*log2(1/q(n,m)));
d=H1;
end
end
disp('H(Y/X):');
disp(H1);
% probability P(Y)
for n=1:i
w=0;
for m=1:i
s(n)=w+a(m,n);
w=s(n);
end
end
disp('P(Y):');
disp(s);
% entropy H(Y)
k=0;
for n=1:i
H2=k+(s(n)*log2(1/s(n)));
k=H2;
end
disp('H(Y): ');
disp(H2);
% Find Mutual Information

m=H2-H1;
disp('MI=');
disp(m);
19
Output:
Enter no. of elements=2
Enter probability= [3/4 1/4]
Enter conditional probabilities matrix=[1/3 2/3;2/3 1/3]
H(x):
0.8113
P(X,Y):
0.2500 0.5000
0.1667 0.0833
H(Y/X):
0.9183
P(Y):
0.4167 0.5833
H(Y):
0.9799
MI=
0.0616
20
INFORMATION THEORY AND CODING TECHNIQUES

Experiment. No.4
Encoding and decoding of Huffman code
(Variable length source coding )
Aim: Write a program for generation and evaluation of variable length source coding using
Huffman Coding and decoding.
Calculate the entropy, average length and efficiency of Huffman Coding.
Apparatus: Matlab/C
Theory:
1. Explain variable length coding.
2. Explain Huffman coding techniques.
3. Solve theoretically and verify using matlab program the given example.
4. Explain the commands:1. Huffmandict, 2. Huffmanenco 3. huffmandeco
Algorithm:
1. Start.
2. Input the total number of probabilities.
3. Arrange the messages in decreasing order of probabilities.
4. Add last two probabilities.
5. Assign them ‘0’ and ‘1’.
6. With addition & other probabilities again sort out the total probabilities.
7. If the addition result is equal to probability of an symbol then put it on the top
8. Repeat the program from step 4 until addition is 1.
9. To find code for particular symbol take the path of probability of symbol and write cod
in reverse fashion.
10. Find out entropy, avg. code word length and efficiency.
11. Stop .
Conclusion:
21
INFORMATION THEORY AND CODING
Program:
1. %Write a MATLAB based program for encoding and decoding of Huffman code
%(variable length source coding )
clc;
clear all;
close all;
symbol =[1:5]; % Distinct data symbols appearing in sig
p = [0.1 0.1 0.4 .3 .1]; % Probability of each data symbol
[dict,avglen]=huffmandict(symbol,p)
samplecode = dict{5,2} % Codeword for fifth signal value
dict{1,:}
dict{2,:}
dict{3,:}
dict{4,:}
dict{5,:}
hcode = huffmanenco(symbol,dict); % Encode the data.
dhsig = huffmandeco(hcode,dict); % Decode the code.
disp('encoded msg:');
disp(hcode);
disp('decoded msg:');
disp(dhsig);
code_length=length(hcode)
for m=1:5
H=Hx+(p(m)*log2(1/p(m)));
Hx=H;
end
disp('Hx=');
disp(H);
Efficiency=(Hx/avglen)*100
Output:
dict =
[1] [1x4 double]
[2] [1x4 double]
[3] [ 1]
[4] [1x2 double]
[5] [1x3 double]
avglen =
2.1000
samplecode = 3 encoded msg:

0 0 1 ans = 0 0 0 1 0 0 0
ans = 1 0 1 0 1 0 0 1
1 ans = decoded msg:
ans = 4 1 2 3 4 5
0 0 0 1 ans = code_length =
ans = 0 1 14
2 ans = Hx=
ans = 5 2.0464
0 0 0 0 ans = Efficiency =
ans = 0 0 1 97.4495
22
II: Huffman coding:string

%Write a MATLAB based program for encoding and decoding of Huffman code
clc;
clear all;
close all;
msg='TEECT'
symbol ={'T' 'E' 'C'}; % Distinct data symbols appearing in sig
p = [2/5 2/5 1/5]; % Probability of each data symbol
[dict,avglen]=huffmandict(symbol,p)
dict{1,:}
dict{2,:}
dict{3,:}
hcode = huffmanenco(msg,dict); % Encode the data.
dhsig = huffmandeco(hcode,dict); % Decode the code.
disp('encoded msg:');
disp(hcode);
disp('decoded msg:');
disp(dhsig);
Output:
msg =
TEECT
dict =
'T' [1x2 double]
'E' [ 1]
'C' [1x2 double]
avglen =
1.6000
ans =
T
ans =
0 0
ans =
E
ans =
1
ans =
C
ans =
0 1
encoded msg:
0 0 1 1 0 1 0 0
decoded msg:
'T' 'E' 'E' 'C' 'T'
23
Expt no. 5
Title: Write a Program for coding of Linear block codes.

Aim: Error detecting and correcting using liner block code.
Apparatus: MATAB/C
THEORY: (1)Explain Linear block codes in detail

(2) Explain Generator matrix and Parity check matrix giving their relation.
(3) Explain syndrome in LBC.
(4) Explain with example coding and decoding in LBC.
Algorithm:
1. Start
2. Accept size of LBC block code in terms n and k
3. Accept parity p matrix of size k x (n-k)
4. Generate generator matrix such that G = [Ik | P] Of size k x n in which Ik is an identity
matrix.
5. Generate parity check matrix such that H= [PT | In-k] Of size (n-k) x n in which PT is an
transpose of P matrix.
6. Generate msg. vector
7. Generate code vector by formula, C = MG
8. Display it
9. Also calculate hamming weight of each code word and that is done by calculating total no.
of ones in the code vector. Display it.
10. Calculate detecting capability by Td =dmin- 1 , where dmin is minimum hamming distance.
11. Calculate error correcting capability tc by, tc= (dmin-1) /2
12. Display parity matrix H
13. Calculate syndrome vector for different error pattern ‘E’. S= E.HT
14. Compare this S with each S created for different error pattern where S is matched error is in
that respective bit w.r.t. the error pattern. Display no. of bits where is there. If S of received
vector is ‘0’ then display received vector is correct.
Conclusion:
24
Program % LINEAR BLOCK CODE

%Generate the codeword using LBC
clc;
close all;
n=6;
k=3;
p=[0 1 1 ; 1 0 1; 1 1 0]; % Parity Matrix
d=input('enter three bit message=');
ik=eye(k);
g=cat(2,ik,p);
disp('Generator Matrix:');
disp(g);
c1=mtimes(d,g);
c=mod(c1,2);
disp('The codeword for given message is:');
disp(c);
OUTPUT:
enter three bit message[1 0 0]
Generator Matrix:
1 0 0 0 1 1
0 1 0 1 0 1
0 0 1 1 1 0
The codeword for given message is:
1 0 0 0 1 1
25
Expt. No. 6
Aim: Write a Program for coding & decoding of Cyclic codes.
Objective:
Error detecting and correcting using Cyclic code.
Software Requirement: MATAB/C
THEORY: (1)Explain coding and decoding of cyclic codes in detail
Cyclic Code:
Cyclic code are the sub-class of linear block codes. They have a property that a cyclic shift
of one code word produces another code word. Suppose there is an n-bit code vector.
X = (xn-1, xn-2, ……………………x1, x0)
Here xn-1, xn-2, ……………………x1, x0 represent the individual bits of the code vector X.
if the code vector is shifted cyclically , then another code vector X is obtained
X1 = (xn-2, xn-3, ……………………x1, x0, xn-1)
Algorithm:
1. Start
2. Get the values of n & k.
3. Get the generator polynomial i.e. its coefficient from user.
4. Get the message vector.
5. Get the message generator matrix.
6. Multiply message polynomial with msg bit shifted by n-k
7. Divide this term i.e. Xn-k d(x) by g(x)
8. To get code word polynomial add Xn-k d(x) with reminder of division.
9. Display the code word.
10. Generate the error pattern & corresponding syndrome with displaying.
11. Enter the received code vector.
12. Divide the received code vector polynomial by generator polynomial.
13. The reminder of division will be the syndrome polynomial.
14. From syndrome detect the corresponding error pattern.
15. Stop.
Conclusion:
26
Program:
%Encoding for (7,4) Cyclic code
clc;
clear all;
%Encoding
n=7; k=4;
p=[1 1 0 ; 1 1 1; 0 0 1 ; 1 0 1]; % Parity Matrix
d=[1 1 0 1]; % Message word
ik=eye(k);
g=cat(2,ik,p);
disp ('Generator Matrix:');
disp(g);
g1=cyclpoly(n,k,'max');
disp(‘g1=’);
disp(g1);
gp=poly2sym(g1);
disp('Generator Polynomial:');
disp(gp);
c1=mtimes(d,g);
c=mod(c1,2);
disp('The codeword for given message is:');
disp(c);
OUTPUT:
g1 =
1 1 0 1
Generator Polynomial:
1 0 0 1 0 1 1
0 1 0 1 1 1 0
0 0 1 0 1 1 1
The codeword for given message is:
1 1 0 1 1 0 0
27
Expt no.7
Title: Write a program for coding and decoding of convolutional codes.
Objective:
Error detecting and correcting using convolutional code.
THEORY:
Explain in detail convolution code
Convolution Code:
Convolution coding is an alternative to block codes. They differ from block codes in that the
encoder contains memory. It means encoder output at any given time is dependent on
presentas well as past inputs.
Convolution codes are commonly specified by three parameter (n , k, m ) where n is no.
of outputs bits(coded), k is no. of inputs bits(msg.), m is memory order.
Algorithm:
1. Start
2. Get the values of n & k.
3. Get the generator polynomial i.e. its coefficient from user.
4. Get the message vector.
5. Get the message generator matrix.
Program:
%cyclic convolution
clc;
clear all;
k=input('ennter the no. of message bits k=');
q=input('given data q=');
fprintf('data polynomial is')
d=poly2sym(q)
n=input('enter the no. of information bits n=');
w=[1,0,0,0];
x=poly2sym(w);
28
e=cyclpoly(n,k);
fprintf('generator polynomial ')
g=poly2sym(e)
z=conv(w,q);
r=poly2sym(z)
[m,v]=gfdeconv(z,e);
fprintf('polynomial')
p=poly2sym(v)
b=r+p;
fprintf('codeword is=')
c=sym2poly(b)
OUTPUT:
29
Expt. No.8
Title: Write a program for coding and decoding of BCH codes.
Objective:
Error detecting and correcting using BCH codes.
THEORY: Explain BCH codes
BCH Code:
The BCH code are the most powerful and widely used random error correcting cyclic codes.
These code were discovered by Hocquenghem in 1959 and independently by Bose
and Choudhary in 1960.
An (n,k) binary BCH code is specified as,
Block length n= 2m – 1
Parity check bits n-k = mtc
Minimum distance dmin ≥ 2 tc +1
Where m ≥ 3 is any integer and tc is no. of error the code is capable of correcting.
Procedure :
1. Given code length n and error correcting capability tc
2. Find m = log2 (n + 1) where n= 2m – 1

3. Find k = 2m – 1 – mtc = n - mtc
4. Take a primitive polynomial of degree m
5. Construct the finite field GF(2m) using the primitive polynomial p(x)
6. Find the minimal polynomial of each element in GF(2m)
7. Find g(x) = LCM [ m1(x).m3(x). m5(x)……………….m2tc-1(x)]
8. Find the code word using c(x) = d(x) . g(x)
Conclusion:
30
Program:
% Program for encoding and decoding BCH code
clc;
clear all;
close all;
m=4;
n=2^m-1;%codeword length
k=5;%message length
m=input('enter msg of length 5=');
%m=[1 1 1 0 1];
msg=gf(m);
disp('message=');
disp(msg);
%Find t,error correction cap[ability
[genpoly,t]=bchgenpoly(n,k);
disp('Error correction capability=');
disp(t);
% Encode the message
code=bchenc(msg,n,k);
disp('Encoded message=');
c=gf(code);
disp(c);
noisycode=code+randerr(1,n,1:t);
disp('received codeword =');
disp(noisycode);
% decode noisy code
[newmsg,err,ccode]=bchdec(noisycode,n,k);
disp('decoded message=');
disp(newmsg);
if msg==newmsg
disp('message recovered perfectly')
else
disp('Error in message recovered');
end
Output:
enter msg of length 5=[1 0 0 0 1]
message=
gf object: 1-by-5
Error correction capability=
3
Encoded message=
gf object: 1-by-15
received codeword =
gf object: 1-by-15
decoded message=
gf object: 1-by-5
31
% Implementation of algorithms for RS Coding & Decoding

%RS CODING
clear all;
clc;
n=input('accept n=');
k=input('accept k=');
m=input('accept message=');
msg=gf([m],k);
c = rsenc(msg,n,k); % Code will be a Galois array.
disp(c)
OUTPUT:
accept n=6
accept k=4
accept message=[1 1 1 1]
gf object: 1-by-6
32
Post Requisite
% Create Integrator model using Simulink Matlab
Create a New Model
Before creating a model, you need to start MATLAB® and then start Simulink.
1. Start the Simulink software. In the MATLAB Command

Window, enter:Simulink,The Simulink Library Browser opens.
2.From the Simulink Library Browser menu, select File > New > Model.
A Simulink Editor window opens with an empty canvas in the right-hand pane
 Select File > Save as. The Save As dialog box opens.
 In the File name box, enter a name for your model. For example, enter simple_model. Then
click Save.
Your model is saved with the file name simple_model.
To create this simple model, you need four Simulink blocks:

• Sine Wave — Generates an input signal for the model.
• Integrator — Processes the input signal.
• Bus Creator — Combines the input signal and processed signal into one signal.
• Scope — Visualizes the signals.
33
Simulating this model integrates a sine wave signal to a cosine signal and then displays
the result, along with the original signal, in a scope window.
Open the Simulink Library Browser
Browse or Search for Specific Blocks

1 Search for a Sine Wave block. In the search box on the browser toolbar, enter sine, and
then press the Enter key. Simulink searches the libraries for blocks with sine in their
name, and then displays the blocks.
34
2 Get detailed information about a block. Right-click a block, and then select Help for the
<block name>. The Help browser opens with the reference page for the block.
3 View block parameters. Right-click a block, and then select Block Parameters. The block
parameters dialog box opens.
Add Blocks to a Model
To build a model, begin by copying blocks from the Simulink Library Browser to the
Simulink Editor.
1 In the Simulink Library Browser, select the Sources library.
2 From the right pane, select the Sine Wave block.
35
3 Drag the Sine Wave block to the Simulink Editor. A copy of the Sine Wave block
appears in your model.
4 Similarly Add Integrator, Bus connector and Scope block in model.
5 Draw Signal Lines Between Block
36
Define Simulation Parameters

Before you simulate the behavior of a model, define the simulation parameters. Simulation
parameters include the type of numerical solver, start, and stop times, and maximum step
size.
clicking the parameters button on the Simulink Editor toolbar
1 From the Simulink Editor menu, select Simulation > Model Configuration Parameters.
The Configuration Parameters dialog box opens to the Solver pane.
2 In the Stop time field, enter 20. In the Max step size field, enter 0.2.
3 Click OK.
Run Simulation
After you define the Model Configuration Parameters, you are ready to simulate your
model.
1 From the Simulink Editor menu bar, select Simulation > Run.
The simulation runs, and then stops when it reaches the stop time specified in the
Model Configuration Parameters dialog box.
Observe Simulation Results

After simulating a model you can view the simulation results in a Scope window.
1 Double-click the Scope block.
The Scope window opens and displays the simulation results. The plot shows a sine wave signal
with the resulting cosine wave signal
37
3 Change the appearance of the display. For example, select white for the display color
and axes background color (icons with a pitcher).
4 Select black for the ticks, labels, and grid colors (icon with a paintbrush).
5 Change signal line colors for the Sine Wave to blue and the Integrator to red. To see
your changes, click OK or Apply.
38
Questions based on Syllabus

1. What is information?
Ans: Information can be defined as the inverse of probability of occurrence
= log2(1/ pk)
2. What are different units of information?
Ans: Bits, Nats, Decit
3. What is entropy?
Ans: Entropy can be defined as the average amount of information per source symbol.
4. What is discrete source?
Ans: If a source emits symbols  = {s0, s1, s2 ,… s k-1} from a fixed finite alphabet then
the source is said to be discrete source.
5. State Shannon’s first theorem.
Ans: A distortion less coding occurs when L  H(x) where L represents the average codeword
length and H(x) represents entropy.
6. What is data compaction?
Ans: Data compaction is used to remove redundant information so that the decoder
reconstructs the original data with no loss of information.
7. What is decision tree? Where it is used?
Ans: The decision tree is a tree that has an initial state and terminal states corresponding to
source symbols s0, s1, s2 ,… s k-1. Once each terminal state emits its symbol, the
decoder is reset to its initial state.
Decision tree is used for decoding operation of prefix codes.
8. What is instantaneous code?
Ans: If a codeword is not a prefix of any other code word then it is said to be instantaneous
code.
9. What is discrete channel?
Ans: The channel is said to be discrete when both the alphabets and have finite sizes.
10. What is memory less channel?
Ans: The channel is said to be memory less when the current output symbol depends only on
the current input symbol and not any of the previous choices.
11. What is the important property while using the joint probability (xj, yk)?
Ans: The sum of all the elements in a matrix is equal to 1.
12. What is the important property while using the conditional probability (xj / yk)?
Ans: The sum of all the elements along the column side should be equal to 1.
39
13. What is the important property while using the conditional probability (yk / xj)?
Ans: The sum of all the elements along the row side should be equal to 1.
14. What is prefix coding?
Ans: Prefix coding is variable length coding algorithm. It assigns binary digits to the messages
as per their probabilities of occurrence. In prefix code, no codeword is the prefix of any
other codeword.
15. State the channel coding theorem for a discrete memoryless channel.
Ans: Given a source of „M‟ equally likely messages, with M>>1, which is generating
information t a rate R. Given channel with capacity C. Then if, R ≤ C, there exists a
coding technique such that the output of the source may be transmitted over the
channel with probability of error in the received message which may be made
arbitrarily small.
16. Explain channel capacity theorem.
Ans: The channel capacity of the discrete memory less channel is given as maximum
average mutual information. The maximization is taken with respect to input
probabilities P(xi). C = B log2(1+S/N) bits/sec, where B is channel bandwidth.
17. Define mutual information?
Ans: Mutual information of the channel is the average amount of information gained by the
transmitter when the state of the receiver is known.
I(X ;Y ) = H(Y) – H(Y /X ) or = H(X) – H(X /Y )
18. Define channel capacity?
Ans: Channel capacity of a discrete memory less channel can be defined as the maximum
value of the mutual information I ( X;Y ) , Where the maximization is carried out for all
input probabilities {p(xj)} when the symbols whose input probabilities {p(xj)} are
equiprobable.
19. What is the use of error control coding?
Ans: The main use of error control coding is to reduce the overall probability of error, which
is also known as channel coding.
20. What is the difference between systematic code and non-systematic code?
Ans: • If the parity bits are followed by message bits then it is said to be systematic codes.
• If the message bits and parity check bits are randomly arranged then it is said to be non-
systematic codes.
21. What is a Repetition code?
Ans: A single message bit is encoded in to a block of ‘n’ identical bits producing a
(n, 1) block code. There are only two code words in the code. all-zero code word
and all-one code word.
40
22. What is forward acting error correction method?

Ans: The method of controlling errors at the receiver through attempts to correct noise-
induced errors is called forward acting error correction method.
23. What is error detection?
Ans: The decoder accepts the received sequence and checks whether it matches a valid
message sequence. If not, the decoder discards the received sequence and notifies the
transmitter that errors have occurred and the received message must be retransmitted.
This method of error control is called error detection.
24. Give the properties of syndrome in linear block code.
Ans: • The syndrome depends only on the error patterns and not on the transmitted code word.
• All error patterns that differ by a code word have the same syndrome.
25. What is Hamming code?
Ans: This is a family of (n,k) linear block code.Block length : n= 2m – 1.
Number of message bits : k = 2m – m-1. Number of parity bits : n – k = m
Where m ≥ 3 and m should be any positive integer.
26. What is the linear property of a code word?
Ans Linearity property: The sum of any two code words in the code is also a code word.
27. What is the cyclic property of a code word?
Ans: Cyclic property:Any cyclic shift of a code word in the code is also a code word.
28. Give the difference between linear block code and cyclic code.
Ans: • Linear block code can be simply represented in matrix form
• Cyclic code can be represented by polynomial form.
29. Define Hamming distance (HD)?
Ans: The number of bit position in which two adjacent code vectors differs is known as
Hamming distance.(e.g) if c1 = 1 0 0 1 0 1 1 0 and c2 = 1 1 0 0 1 1 0 1,then HD = 5
30. Define Weight of a code vector?
Ans: The number of non-zero components in a code vector is known as weight of a code
vector.(e.g) if c1 = 1 0 0 1 0 1 1 0, then W(c1) = 4
31. Define minimum distance?
Ans: The minimum distance of a linear block code is the smallest hamming distance between
any pairs of code words in a code.(e.g) if c1 = 0 0 1 1 1 0, c2 = 0 1 1 0 1 1,
c3 = 1 1 0 1 1 0, then d min = 3.
32. What is convolutional code?
Ans: A convolutional code is the code in which parity bits are continuously interleaved by
information (or) message bits.
41
33. .How compression is taken place in text and audio?

Ans: In text the large volume of information is reduced, where as in audio the bandwidth is
reduced.
34. Specify the various compression principles?
Ans: • Source encoders and destination decoders • Loss less and lossy compression
• Entropy encoding • Source encoding
35. Define run-length encoding?
Ans: This can be used for long sub strings of the same character or binary digits.
(e.g) 000000011111111110000011……..This can be represented in run-length as:
0,7,1,10,0,5,1,2……….
36. Define transform coding?
Ans: This is used to transform the source information from spatial time domain representation
into frequency domain representation.
37. Define code redundancy.
Ans: It is the measure of redundancy of bits in the encoded message sequence. It is given as,
Redundancy = 1 – code efficiency, i.e. 1 – ή .It should be as low as possible.
38. What is the capacity of the channel having infinite bandwidth?
Ans: The capacity of such channel is given as, C = 1.44 (S/N0)
39. Give differences between Arithmetic coding and Huffman coding.
Ans: • In arithmetic coding a single code word is used for each encoded string of characters.
• In Huffman coding a separate codeword is used for each character.
40. An alphabet set contains 3 letters A,B, C transmitted with probabilities of 1/3, 1/4
1/4. Find entropy.
Ans: p1 = 1/3, p2 = 1/4, p3 = 1/4.
H =∑pk log 2 (1/pk) = 1/3 log23 + 1/4 log2 4 +1/4 log24 = 1.52832 bits/symbol
41. What is meant by linear code?
Ans: A code is linear if modulo-2 sum of any two code vectors produces another code vector.
This means any code vector can be expressed as linear combination of other code
vectors.
42. For M equally likely messages, the average amount of information H is____.
Ans: log2M
43. The capacity of a binary symmetric channel, given H(P) is binary entropy function
is _________
Ans: 1-H(P)
42
44. For M equally likely messages, M>>1, if the rate of information R ≤ C, the
probability of error is ____.
Ans: very small
45. Code rate r, k information bits and n as total bits, is defined as
Ans: r=k/n
46. The information rate R for given average information H= 2.0 for analog signal
band limited to B Hz is
Ans: 4 bps
47. The expected information contained in a message is called
Ans: Entropy
48. The capacity of Gaussian channel is
Ans: C = B(1+S/N) bits/s
49. According to Shannon Hartley theorem,
Ans: The channel capacity does not become infinite with infinite bandwidth. And
Has a tradeoff between bandwidth and Signal to noise ratio.
50. The negative statement for Shannon's theorem states that
Ans: If R > C, the error probability increases towards Unity
43
Conduction of Viva-Voce Examinations:

Teacher should conduct oral exams of the students with full preparation. Normally, the objective
questions with guess are to be avoided. To make it meaningful, the questions should be such that
depth of the students in the subject is tested. Oral examinations are to be conducted in co-cordial
environment amongst the teachers taking the examination. Teachers taking such examinations
should not have ill thoughts about each other and courtesies should be offered to each other in
case of difference of opinion, which should be critically suppressed in front of the students.
4. Evaluation and marking system:
Basic honesty in the evaluation and marking system is absolutely essential and in the process,
impartial nature of the evaluator is required in the examination system. It is a wrong approach or
concept to award the students by way of easy marking to get cheap popularity among the
students, which they do not deserve. It is a primary responsibility of the teacher that right
students who are really putting up lot of hard work with right kind of intelligence are correctly
awarded.
The marking patterns should be justifiable to the students without any ambiguity and teacher
should see that students are faced with just circumstances.
44

All Matlab Codes PDF

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

All Matlab Codes PDF

Caricato da

Copyright:

Formati disponibili

Jawaharlal Nehru Engineering College

Information Theory and Coding

Third Year Students

” Author JNEC, Aurangabad

Vision of the Department:

Jawaharlal Nehru Engineering College

This technical document is a series of Laboratory manuals of Electronics & Telecommunication

Good Luck for your Enjoyable Laboratory Sessions.

Sr. Title Page No.

2 Lab course objectives 7

1. DOs and DON’Ts in Laboratory:

1. Do not handle software and PC without reading the instructions/

2 Instruction for Laboratory Teachers:

Laboratory Course Objectives:

1. To expose students to knowledge about information and various entropies.

INTRODUCTION TO MATLAB- PREREQUISITE

1) Generate and Display a sequence of 10 numbers.

(3) What happens: (1) B = A; B(3,3) = 10; (2) x = [1:50] ’; x(length(x))

COMMAND: 1.A(1) 2. A(2) 3. A(9) 4. A(1,1) 5. A(2,1) 6. A(1,2) 7. A(1:3, 2)

11) Create a 3x4 matrix for the following

12) Create a magic matrix of 3x3.

13) Create 3D 3x3 ones matrix.

16) Write the commands/programs and calculations wherever applicable:

OUTPUT: 1. Area = 2.1. ans = 2.2 ans=

INFORMATION THEORY AND CODING TECHNIQUES

Aim: To find information and entropy of a given source.

II) Various channels

2)Error free channel

2. Calculate entropy of the input.

3) Binary Symmetric Channel

Binary Symmetric Channel

% %prgm for entropy and MI for BSC channel

%Joint probability matrix

% Find Mutual Information

INFORMATION THEORY AND CODING TECHNIQUES

INFORMATION THEORY AND CODING

samplecode = 3 encoded msg:

II: Huffman coding:string

Title: Write a Program for coding of Linear block codes.

THEORY: (1)Explain Linear block codes in detail

Program % LINEAR BLOCK CODE

Aim: Write a Program for coding & decoding of Cyclic codes.

Title: Write a program for coding and decoding of convolutional codes.

THEORY: Explain BCH codes

2. Find m = log2 (n + 1) where n= 2m – 1

% Implementation of algorithms for RS Coding & Decoding

Create a New Model

1. Start the Simulink software. In the MATLAB Command

Your model is saved with the file name simple_model.

To create this simple model, you need four Simulink blocks:

Open the Simulink Library Browser

Browse or Search for Specific Blocks

Add Blocks to a Model

2 From the right pane, select the Sine Wave block.

4 Similarly Add Integrator, Bus connector and Scope block in model.

5 Draw Signal Lines Between Block

Define Simulation Parameters

Observe Simulation Results

1 Double-click the Scope block.

Questions based on Syllabus

22. What is forward acting error correction method?

33. .How compression is taken place in text and audio?

Conduction of Viva-Voce Examinations: