Error Control Coding week 1: Binary field algebra and linear block codes

Solutions to selected problems from Chapter 3

3.1 The information sequences u = [u0 u1 u2 u3 ] are encoded into codewords v of
length n = 8, using a systematic encoder. If we assume that the systematic
positions in a codeword are the last k = 4 positions, then codewords have the
form
v = [v0 v1 v2 v3 u0 u1 u2 u3 ],
and the systematic encoding is specified by
v = uGsys ; Gsys = [ P |I ].
From the given set of parity-check equations we immediately obtain the generator and the parity check matrices. For example, we can start with the
parity check matrix H and recall that every row in H represents one paritycheck equation, and it has ones on the positions corresponding to the symbols
involved in that equation. Thus, we have

1 0 0 0| 0 1 1 1
0 1 0 0 | 1 1 1 0

H sys = [ I |P T ] =
(1)
0 0 1 0 | 1 1 0 1
0 0 0 1| 1 0 1 1

0 1 1 1| 1 0 0 0
1 1 1 0 | 0 1 0 0

Gsys = [ P |I ] =
(2)
1 1 0 1 | 0 0 1 0
1 0 1 1| 0 0 0 1
To find the minimum distance analytically, we use the property that the minimum distance of a binary linear code is equal to the smallest number of
columns of the parity-check matrix H that sum up to zero (see Corollary
3.2.2 in the book). Hence, we find that:
there are no two identical columns in H sys dmin > 2

there are no groups of 3 columns that sum up to 0 dmin > 3

there exists a group of 4 columns (for example, columns 1,2,3,6) that

sum up to 0 dmin = 4.

3.2 General structure of a systematic encoder for a linear (n, k) block code is
shown in Figure 3.2 in the book. Based on the parity check equations from the
previous problem, the systematic encoder for the (8, 4) code has the structure
as shown in Figure 1.
3.3 Syndrome vector is obtained from the received sequence r = [r0 r1 . . . r7 ]
using the parity check matrix of the code:
s = [s0 s1 s2 s3 ] = rH T .

Error Control Coding week 1: Binary field algebra and linear block codes

u0

u1

v0

u2

v1

u3

v3

v2

Figure 1: Systematic encoder for the (8, 4) code.

By substituting the parity check matrix (1), we obtain that the syndrome
digits are
s0
s1
s2
s3

=
=
=
=

r0 + r5 + r6 + r7
r1 + r4 + r5 + r6
r2 + r4 + r5 + r7
r3 + r4 + r6 + r7 .

Using these equations the syndrome circuit is easily constructed following the
general structure shown in Figure 3.4 in the book for any linear (n, k) block
code.
3.4 (a) The parity check matrix H of a linear (n, k) code C has dimensions
(n k) n and its rank is n k (all rows are linearly independent).
The matrix H 1 has dimensions (n k + 1) (n + 1) and its rank is
thus n k + 1. Since n k rows of H are linearly independent, then,
after adding a leading 0, these rows (the first n k rows of H 1 ) are still
independent. The last row of H 1 begins with a 1, while the others begin
with 0. Hence, we conclude that all rows of H 1 are independent, and
the rank of H 1 is exactly n k + 1. Therefore, H 1 is a parity check
matrix of a code C1 whose dimension is the dimension of the null space
of H 1 , that is, dim(C1 ) = (n + 1) (n k + 1) = k.

(b)+(c) Extending the parity-check matrix H with a zero-column to the left and
adding all-one row at the bottom is equivalent to adding the digit v to
the left of each codeword of the original code C, which is involved only
in the last parity check equation specified by the all-one row, that is,
v + v0 + v1 + ... + vn1 = 0.

This parity check equation implies that the weight, that is, the total
number of ones in each codeword [v v0 v1 ... vn1 ] of the extended code
C1 must be even.
From the above parity check equation, it follows that the added digit v
equals
n1
X
v =
vi .
i=0

Error Control Coding week 1: Binary field algebra and linear block codes

Therefore, v is called the overall parity check digit: it equals 0 if the

number of ones among the n symbols is even, otherwise it is equal to 1
(thus the total number of ones in the extended codeword is always even).
3.5 Let Ce and Co be the subsets of a linear code C, containing all the codewords
with even and odd weight, respectively. Thus, Ce Co = and Ce Co = C.

Let v be any odd-weight codeword from Co . Then, if we add v to each

codeword from Co , we obtain a set Ce0 of even-weight words, which fulfills
|Ce0 | = |Co | (the number of elements in the two sets is the same). Also, it holds
that Ce0 Ce , and thus
|Ce0 | = |Co | |Ce |.
(3)
Similarly, if we add v to each codeword from Ce , we obtain a set Co0 of oddweight words, which fulfills |Co0 | = |Ce |. Also, it holds that Co0 Co , and
thus
|Co0 | = |Ce | |Co |.
(4)
From (3) and (4) we conclude that
|Ce | = |Co |,
and, since |C| = 2k , it follows that the number of odd-weight and even-weight
codewords in a binary linear (n, k) code is
|Ce | = |Co | = 2k1 .

3.6 (a) The given condition on G ensures that, for any symbol position i, 0
i < n, there is a row in G with a non-zero symbol on that position. Since
rows of the generator matrix are codewords of the code C, we conclude
that in the code array each column must contain at least one non-zero
entry. Therefore, there are no all-zero columns in the code array.
(b) Consider an arbitrary ith column of the code array, 0 i < n, and let
S0 and S1 be the sets of codewords that have a 0 and a 1 on the ith
position, respectively. From part a) it follows that there is at least one
codeword with a 1 on the ith position, that is, |S1 | 1. Now we follow
an approach similar to the solution of Problem 3.5.
Let v be a codeword from S1 . Then, if we add v to each codeword from
S1 , we obtain a set S00 of codewords with a 0 on the ith position, and
|S00 | = |S1 |. Also, it holds that S00 S0 , and thus
|S00 | = |S1 | |S0 |

(5)

Similarly, if we add v to each codeword from S0 , we obtain a set S10 of

words with a 1 on the ith position, which fulfills |S10 | = |S0 |. Also, it
holds that S10 S1 , and thus
|S10 | = |S0 | |S1 |

(6)

Error Control Coding week 1: Binary field algebra and linear block codes

From (5) and (6) we conclude that

|S0 | = |S1 |.
Moreover, since C = S0 S1 , S0 S1 = , and |C| = 2k , we conclude that
|S0 | = |S1 | = 2k1 ,
that is, for any binary linear (n, k) code, exactly a half of codewords
have a 0 and a half have a 1 on each position i, 0 i < n. Hence, each
column in a code array has 2k1 zeros and 2k1 ones.
(c) For any two codewords x and y with a 0 on the ith position it holds that
their sum is also a codeword with a 0 on the ith position, that is,
( x, y S0 ) x + y = z S0 .
Hence, S0 is a subspace (that is, a subcode) of the code C. From part b)
it follows that the dimension of this subspace is k 1.
3.9 Let Ai denote the number of codewords of weight i in an (n, k) code C. Then
the numbers A0 , A1 , . . . , An are called the weight distribution of the code. For
any linear code A0 = 1 (every linear code must contain the all-zero codeword).
The first next non-zero element of the weight distribution of a linear code is
Admin , corresponding to the number of minimal-weight codewords.
For the (8, 4) code with the minimum distance dmin = 4, all the codewords
except the all-zero and the all-one codeword have minimum weight, that is,
the weight distribution is
A0 = 1; A4 = 14; A8 = 1.
When transmitting over binary symmetric channel (BSC), an undetected error
event occurs when the error pattern is equal to some non-zero codeword. Thus,
the probability of undetected error is (cf. expression (3.19) in the book)
Pu (E) =

n
X
i=1

Ai pi (1 p)ni

where p is the crossover probability of the BSC. For the (8, 4) code we have
Pu (E) = A4 p4 (1 p)84 + A8 p8 (1 p)88 = 14p4 (1 p)4 + p8
which, for p = 102 = 0.01 yields
Pu (E) = 14 108 0.994 + 1016 = 1.3448 107 .
3.10 The optimum (maximum likelihood) decoder for the BSC is the minimumdistance decoder, which can be realized as a standard array decoder, or, more
efficiently, as a syndrome decoder. For the (8, 4) code, there are 2nk = 24 =

Error Control Coding week 1: Binary field algebra and linear block codes

16 different syndrome vectors. All-zero syndrome vector indicates that the

transmission was either error-free, or an undetectable error has occurred (when
the error pattern equals a codeword). Each syndrome corresponds to a coset of
the code. Each coset has 2k = 16 sequences. The coset leader is the sequence
with the smallest weight within the coset. The code itself is a coset with
all-zero sequence as a coset leader and zero corresponding syndrome. Since
the minimum distance of the (8, 4) code is dmin = 4, all single-error patterns
are correctable, and simultaneously, all

double-error events are detectable
8
by a syndrome decoder. There are 1 = 8 single-error patterns and they
are chosen as coset leaders of 8 cosets. These 8 cosets contain sequences of
weights 1,3,5, and 7 (verify this!). The remaining
16 1 8 = 7 cosets


8
contain even-weight sequences. Thus, all 2 = 28 double-error patterns are
in these 7 cosets and they are detectable via the corresponding 7 possible
syndrome values. Double-error patterns chosen as coset leaders of these 7
cosests will be correctable, but it cannot be guaranteed that all double errors
will be corrected. However, since all double error patterns belong to the cosets
distinct from cosets whose leaders are single-error patterns, it is guaranteed
that the decoder will simultaneously detect double errors and correct single
errors.
The decoder operates as follows: first, it computes the syndrome s = rH T .
Then, a coset leader whose syndrome is equal to the obtained value is located
(for example, a look-up table can be used for this mapping). If the syndrome
= r.
is zero, it is assumed that no errors occurred and the decoder outputs v
If the syndrome is non-zero and the corresponding coset leader is a weightone sequence e, this sequence is chosen as the assumed error pattern that
= r + e. If the
occurred on the channel and the decoded codeword is v
computed syndrome corresponds to the coset leader of weight larger than 1,
this is an indicator that at least 2 errors occurred on the channel.
3.12 See solution of Problem 3.10. The syndrome decoder explained in the previous
solution is the optimum decoder. It will correct all 8 single errors and 7 double
errors that are chosen as coset leaders (the choice is not unique!). The all-zero
syndrome is interpreted as error-free transmission.
3.15 Hamming bound: For any binary (n, k) linear block code with minimum
distance of at least 2t + 1 (for some positive integer t), it holds
 

   
n
n
n
+
+ +
.
n k log2 1 +
1
2
t
Proof: According to Theorem 3.5 from the book, an (n, k) code with minimum distance 2t + 1 or greater can correct all errors of weight up to t with
the syndrome decoder whose coset leaders are the sequences of up to t ones.
There are
   
 
n
n
n
+
+ +
0
1
t

Error Control Coding week 1: Binary field algebra and linear block codes

such sequences. On the other hand, there are 2nk diferrent cosets, and this
number cannot be smaller than the number of coset leaders (correctable error
patterns). Thus we have
   
 
n
n
n
nk
2

+
+ +
.
0
1
t
Taking the logarithm log2 of both sides of the above inequality yields the
Hamming bound, which completes the proof.
3.16 Plotkin bound: The minimum distance of an (n, k) linear code satisfies
dmin

n 2k1
.
2k 1

Proof: Consider the 2k n code array whose rows are codewords of the
(n, k) code C. Each column of this array contains exactly 2k1 zeros and 2k1
ones (see Problem 3.6). Hence, the total number of ones in the code array is
n 2k1 . On the other hand, each of the 2k 1 non-zero codewords has weight
of at least dmin . Hence, the total number of ones in the code array is at least
(2k 1)dmin . By combining these two results we obtain
n 2k1 (2k 1)dmin ,
which yields the Plotkin bound
dmin

n 2k1
.
2k 1

3.17 Theorem: There exists an (n, k) linear code with a minimum distance of at
least d if
d1  
X
n
< 2nk .
i
i=1
Proof:
The number of non-zero vectors of length n and weight d 1 or less is
d1  
X
n
.
i
i=1

From the result of Problem 3.11, each of these vectors is contained in 2(k1)(nk)
d1
P n

linear codes. Therefore, there are at most M = 2(k1)(nk)
linear codes
i
that contain nonzero codewords of weight d 1 or less.

i=1

On the other hand, the number of different k n binary generator matrices in

systematic form is 2k(nk) , which means that N = 2k(nk) is the total number

Error Control Coding week 1: Binary field algebra and linear block codes

of binary linear codes. If M < N , there exists at least one code with minimum
codeword weight at least d. Condition M < N is equivalent to
(k1)(nk)

d1  
X
n
i=1

< 2k(nk)

which yields
d1  
X
n
i=1

< 2nk .

(7)

3.18 Gilbert-Varshamov bound:

There exists an (n, k) linear code with a minimum distance of at least dmin
that satisfies the inequality
dmin
X1 
i=1


dmin  
X
n
n
nk
<2

.
i
i
i=1

(The right-hand side of this inequality is known as Gilbert-Varshamov bound

and provides lower bound on the minimum distance attainable with an (n, k)
linear code).
Proof: The proof follows directly from Problem 3.17. Namely, let dmin be
the largest positive integer such that the inequality (7) holds. Then, for this
dmin it holds:
dmin
dmin  
X1 n
X
n
nk
<2

.
i
i
i=1
i=1
According to Problem 3.17, if the the left part of the above inequality is
fulfilled, there exists a linear code of minimum distance at least dmin .
3.20 A codeword of an (n, n 1) single parity check code consists of n P
1 = k
information symbols followed by an overall parity bit vn1 = p = k1
i=0 ui .
The encoder can be realized using a single memory element, as shown in Figure
2. The memory element is initially in the zero state. During the encoding it
contains the current parity of the information sequence. After k clocks, at the
output of the memory element appears the overall parity bit p which is sent
after the information block.
u = u0u1...uk1
p

Figure 2: Encoder for the (n, n 1) single parity check (SPC) code.