11/17/2014

Information and Coding Theory

ECE533

Overview
Codes?
Digital Communication
Linear Block Codes Signal Detection
Channel
Hamming Codes
Nikesh Bajaj Shannon Theorem
nikesh.14730@lpu.co.in
Digital Signal Processing
Lovely Professional University 2 By Nikesh Bajaj

Channel Coding Channel Coding

Words and Codeword
Code(n,k) or code(n,k,d)
Code rate k/n
Code C = {000 111 }
Hamming distance d
Hamming weight w
Error detection capability ed
Error Correction Capability ec

3 By Nikesh Bajaj 4 By Nikesh Bajaj

Block Codes Linear Block Codes

Words, Codeword All Zeros
Code rate Sum of any two codewords is a codeword
Hamming distance/Min d* d* = w* Proof
Non binary Example
Hamming weight/ Min w* C ={00000 10100 11110 11001}
C={0100 1111}
d(c1,c2)=w(c1-c2)
C ={0000 1010 0101 1111}
Block length (n) C={12, 21} over GF(3)
Code(n,k) M=qk
5 By Nikesh Bajaj 6 By Nikesh Bajaj

1
 11/17/2014

Encoding Example : Repetition Code

C ={0000 1010 0101 1111} Code(n,1) n is Odd
Code(3,1)
0 000
1 111
Perform following for this code
Encode Msg = 11010100110
Decode Rx = 111101000010001001001111

7 By Nikesh Bajaj 8 By Nikesh Bajaj

Encoding: Matrix form G Linear Codes

1 0 1 Encoding
G
0 1 0

9 By Nikesh Bajaj 10 By Nikesh Bajaj

Generator Matrix (Properties) Find linear code

Size kxn, Q: S={1100 0100 0011} over GF(2)
rank k Q: S={12 21} over GF(3)
Rows are linearly independent and basis vector
Any set of basis vector can be used to generate code span
Generator matrix is not unique for given linear code
Saves memory space
Sufficiency

11 By Nikesh Bajaj 12 By Nikesh Bajaj

2
 11/17/2014

Systematic Codes Find systematic G

Code(5,2) over GF(3) . 0 1 2 1
G= 1 0 1 0
1 2 2 1

13 By Nikesh Bajaj 14 By Nikesh Bajaj

Linear Codes Parity check matrix H

Size==??
Encoding
purpose
Limitation
Uniqueness
Check matrix or Parity Check Matrix H Sufficiency
G =[I | P] or G = [P | I] d* =min rows in HT or column of H whose sum is
H = [-PT | I ] or H = [ I | -PT ] zero (in linear code) and binary

15 By Nikesh Bajaj 16 By Nikesh Bajaj

Hamming Code Parity check matrix H

1 0 0 0 1 0 1 Q: find for H
0 1 0 0 1 1 1 . 0 1 2 1
G

0 0 1 0 0 1 0 G= 1 0 1 0

0 0 0 1 0 1 0
1 2 2 1

17 By Nikesh Bajaj 18 By Nikesh Bajaj

3
 11/17/2014

What you can find? Error Correction and Detection

19 By Nikesh Bajaj 20 By Nikesh Bajaj

Singleton Bound Error detecting capability

Singleton Bound For code(n,k) Code (n, k, d*)
d* n k + 1 d*-1
Hamming Sphere C={000 111}
C={001 110 101}
Maximum Distance Separable (MDS) code
d* =n-k+1

21 By Nikesh Bajaj 22 By Nikesh Bajaj

Decoding Decoding
Spheres of t radius, t errors
Vector space, q-ary, n-tuples
d* 2t+1 Coders
Worst case Incomplete Coder
Nearest neighborhood decoding Complete Coder
C={00000 01010 10101 11111}
Tx=11111--- Rx =11110
Tx=00000 --- Rx =01000

23 By Nikesh Bajaj 24 By Nikesh Bajaj

4
 11/17/2014

Coset Coset
For C (n, k) over GF(q) and a be any vector of Coset Leader: Vector of minimum weight
length n then the set If two, choose any one
Coset a + C ={c1+a, c2+a } c1, c2 C Find Cosets for G=[1 0 1; 0 1 0]
Theorems (Prove) Ref: 106 in RB GF(qn) = C (a1+C) (a2+C) (at+C)
Every vector b of length n is in some Coset t =qn-k -1
Each Coset contain exactly qk vector
Two Cosets are either disjoint or Concide
If a+C is Coset of C and b a+C then b+C =a+C
a and b are said to be in same Coset if (a-b) C

25 By Nikesh Bajaj 26 By Nikesh Bajaj

Standard Array Standard array: Decoding

Standard Array: for Code C (n,k), qn-k x qk array of all STEPS
.
vector In GF(qn) C={0000 1011 0101 1111}

If Rx v = 1101
c= v - e

27 By Nikesh Bajaj 28 By Nikesh Bajaj

Standard array: Decoding Example

29 By Nikesh Bajaj 30 By Nikesh Bajaj

5
 11/17/2014

Syndrome: Decoding Syndrome: Decoding

Why syndrome?

x and y are in same coset iff both have same

syndrome

31 By Nikesh Bajaj 32 By Nikesh Bajaj

Important skills that you should have is to solve such problems

Example: Syndrome Decoding Problems

Decode Rx sequence and compute Transmitted Tx and Original
massage Mx if LBC with generator matrix G is used over GF(2)
Rx =110101000101010100101000111010

Consider different G as separate case

1 0 1
G
0 1 0

1 0 0 0 1 0 1
0 1 0 0 1 1 1
G
0 0 1 0 0 1 0

0 0 0 1 0 1 0
33 By Nikesh Bajaj 34 By Nikesh Bajaj

Important skills that you should have is to solve such problems

Problems Bounds and Codes

Decode Rx sequence and compute Transmitted Tx and Singleton Bound
Original massage Mx if LBC with generator matrix G is used
over GF(3) Hamming Bound
Rx =0212010202000111102021 Perfect Code
Consider different G as separate case Hamming Code
Optimal Code
MDS Code

35 By Nikesh Bajaj 36 By Nikesh Bajaj

6
 11/17/2014

Sphere and # vector Hamming sphere

A sphere of radius t (0<=t<=n) contains vectors

Example: # of vector of distance 2 from a vector of length

4 in GF(2)
Hamming Bound
MV <= qn

37 By Nikesh Bajaj 38 By Nikesh Bajaj

Perfect Codes Equivalent Codes

Two q-ary codes are called equivalent codes if one
Perfect Code is one which achieves the Hamming
can be obtained from other by one or both
Bound
Permutation of components
MV = qn Permutation of position of codewords

2 0 1 2 2 2

C 1 2 0 C 1 1 1
0 1 2 0 0 0

39 By Nikesh Bajaj 40 By Nikesh Bajaj

Equivalent Linear Codes Hamming Codes (binary*)

Two Linear q-ary codes are called equivalent codes n = 2m 1
if one can be obtained from other by one or both
k = 2m m 1
Multiplication of components by any non-zero scalar
Permutation of position of codewords (n, k) =(2m 1, 2m m 1)

1 0 0 0 1 0 1
0 1 0 0 1 1 1
G
0 0 1 0 0 1 0

0 0 0 1 0 1 0

41 By Nikesh Bajaj 42 By Nikesh Bajaj

7
 11/17/2014

Optimal Linear Codes MDS Codes

For optimal code (n, k, d*) (n, n-r, d*) code satisfy d* =r+1
No (n-1, k, d*),
(n+1, k+1, d*) or
(n+1, k, d*+1) exist
(n, n-r, r+1) is MDS Codes

(24, 12, 8)
23, 12, 8 X 23 12 7
25, 13, 8 X
25, `12, 9 X
43 By Nikesh Bajaj 44 By Nikesh Bajaj

Linear Codes Visualization of Code space

.
Set S,
Extended field GF(qn ) linear span <S>
linear span <S> is sub space of GF(qn )
Linear code C= <S>

45 By Nikesh Bajaj 46 By Nikesh Bajaj

Channel Coding Repetition Code

0000
1111
Code rate =1/n
Example 2.4 from Ranjan boss

47 By Nikesh Bajaj 48 By Nikesh Bajaj

8
 11/17/2014

Coding Gain

49 By Nikesh Bajaj