8b Error Correction

DIGITAL COMMUNICATION
Error - Correction
November 15, 2011
A.J. Han Vinck
Error correction is needed in high tech devices
A.J. Han Vinck, rondheim, 2011
2
fundamental limits set by Shannon (1948)
Bell Labs, 1955.
!
Fundamental problem: reproducing at the receiver a message selected at the
transmitter
Shannons contribution: - bound on the efficiency (capacity)
- ho to achieve the bound

The Bell System Technical Journal, !ol. "#, pp. $#9%&"$, '"$%'5',
(ctober, 19&).
A.J. Han Vinck 4
"vervie# o$ the %ect&re
' he (osition o$ channe% coding in a comm&nication chain
' )ome channe% mode%s
' *&rst error mode%s
' +etection r&%es
' ,ntro to the coding (rob%em
' (er$ormance
' he c%ass o$ %inear codes
' *o&nds on the minim&m distance
' E-am(%es o$ codes
' .h/ error correction0
A.J. Han Vinck 5
' +etection r&%es
' (er$ormance
' E-am(%es o$ codes
A.J. Han Vinck 6
he comm&nication mode%
so&rce
data
red&ction1
com(ression
data
(rotection
sink
2essage
constr&ction
decoder
k
n
k
34
34
A.J. Han Vinck 7
5oint-to-(oint
transmitter receiver channe%
(h/sica% modem modem
message
bits
)igna%
generator
)igna%
(rocessor
bits
message
A.J. Han Vinck 8
5osition o$ Error Contro% Coding
signal
genera!r
c"annel
#eec!r
k in$%
&is
k !%$%
&is
c"annel
k
c"annel
k in$%
&is
k !%$%
&is
k in$%
&is
signal
genera!r
c!#e# signal genera!r
#eec!r
#eec!r'#ec!#er
n in$%
&is
n
(CC (CC
c!#ing
c!#e# )!#%lai!n
A.J. Han Vinck *
transmission model (OSI)
Data Link
Control
Data Link
Control
Physial Physial
link
6nre%iab%e trans-
mission o$ bits
ransmission o$
re%iab%e (ackets
A.J. Han Vinck +,
)omething to think abo&t
message
bits
bits
message
message
com(ression (rotection o$ bits
correction o$ incorrect bits
decom(ression message
Error correction 25E7, J5E7, etc
Com(ression red&ces bit rate
5rotection increases bit rate
A.J. Han Vinck ++
' Some hannel models
' +etection r&%es
' E-am(%es o$ codes
A.J. Han Vinck +-
2emor/%ess channe%
In$% . /01234 !%$% 5
ransii!n $r!&a&iliies
memor/%ess8
- o&t(&t at time i de(ends on%/ on in(&t at time i
- in(&t and o&t(&t a%(habet $inite
A.J. Han Vinck +6
E-am(%e8 binar/ s/mmetric channe%
9*)C:
(rr!r 7!%rce
8
E
X
O%$%
In$%
E X Y
E is the binar/ error se;&ence s.t. 591: < 1-590: < (
= is the binar/ in$ormation se;&ence
> is the binar/ o&t(&t se;&ence
+9$
, ,
$
:
+
+9$
A.J. Han Vinck +4
$rom A.7N to *)C
{ } E X + '
-
-
-
-
-
+
4 0

n
N
e n p

Home#ork8 ca%c&%ate the ca(acit/ as a $&nction o$ A and
2
(
7a&ssian noise #ith (rob. densit/ $&nction
= < ?1- A > < = ? N
+ecide8 ? or -
-
-
-
-
-
-
+

A
A
e dx e p
x

A.J. Han Vinck +5

"ther mode%s
,
+
, 0lig" !n4
+ 0lig" !;;4
$
+9$
. 5
/0.<,4 < /
,

,
+
,
(
+
+9e
e
e
+9e
/0.<,4 < /
,

@-channe% 9o(tica%: Eras&re channe%
92AC:
A.J. Han Vinck +6
the eras&re channe%
A((%ication8 cdma detection, disk arra/s
,
+
,
(
+
+9e
e
e
+9e
3 1
/0 3 < ,4 < + = /0 3 < + 4 < /
,

+isk 1
+isk 2
+isk !
+isk A
+isk 5
3no#n
(osition o$
error
A.J. Han Vinck +7
Brom 7a&ssian to binar/ to eras&re
8
e
x
i
= +/-
O%$%
In$%
y
i
= x
i
+ e
?
- -
?
?
-
?
- -
?
?
-
E
E
E
A.J. Han Vinck +8
Eras&re #ith errors
,
+
,
(
+
(
(
e
e
1-(-e
1-(-e
A.J. Han Vinck +*
' !urst error models
' +etection r&%es
' (er$ormance
' E-am(%es o$ codes
A.J. Han Vinck -,
2ode%ing8 net#orking
Ack1Nack
- 1-error ca&ses retransmission
- %ong (ackets a%#a/s have an error
- short (ackets #ith ECC give %o#er e$$icienc/
(acket
)&((ose that a (acket arrives correct%/ #ith (robabi%it/ C.
.hat is then the thro&gh(&t as a $&ntion o$ C0
A.J. Han Vinck -+
A )im(%e code
' Bor %o# (acket %oss rates 9e.g. 5D:, sending d&(%icates is
e-(ensive 9#astes band#idth:
' ="E code
F ="E a gro&( o$ data (kts together to (rod&ce re(air (kt
F ransmit data ? ="E8 can recover 1 %ost (kt
10101

11100 00111 11000 10110

10101

11100 00111 11000 10110

A.J. Han Vinck --
CD DVD
Blue Laser
Density inreases sensiti"ity
A.J. Han Vinck -6
)!#eling
Ho# to mode%
scratches on a
C+0
Ans#er is
im(ortant $or
the design o$
ECC
A.J. Han Vinck -4
M!#eling> &inar1 rans)issi!n
channe%
test
se;&ence
error
se;&ence
5rob%em8 0 1 0 0 0 1 0 1 0 0 0 0 1 1 1 '''
+etermination o$ b&rst and g&ard s(ace
b&rst g&ard b&rst
A.J. Han Vinck -5
A sim(%e error detection method
r!? $ari1
@ill r!? ?ise
Trans)i c!l%)n ?ise
RESULT: RESULT: an1 &%rs !; leng"
L can &e #eece#
L
, + + , , + + , , + +
, , , + + , , + + , ,
, , , , , , , , , , ,
, + + , , + + , , , ,
, , , + + , , + + + +
, , , , , , , , , + +
.hat ha((ens #ith b&rsts o$ %ength %arger than G0
A.J. Han Vinck -6
b&rst error mode% 97i%bert-E%%iot:
(rr!r 7!%rce
#andom
#andom error channe%H o&t(&ts inde(endent
/0,4 < +9 /0+4A
!urst
!urst error channe%H o&t(&ts de(endent
(rr!r 7!%rce
/0, 2 sae < &a# 4 < /0+2sae < &a# 4 < +'-A
/0, 2 sae < g!!# 4 < + 9 /0+2sae < g!!# 4 < ,.***
)tate in$o8 good or bad
g!!# &a#
transition (robabi%it/

/
g&
/
&g
/
gg
/
&&
A.J. Han Vinck -7
;&estion
g!!# &a#
/
g&
/
&g
/
gg
/
&&
/0, 2 sae < &a# 4 < /0+2sae < &a# 4 < +'-A
/0, 2 sae < g!!# 4 < + 9 /0+2sae < g!!# 4 < ,.**
B"a is "e aCerage ;!r /0,4 ;!r> /
gg
< ,.*D /
g&
< ,.+A /
&g
< ,.**D /
&&
< ,.,+ E
In#icae "!? 1!% can 1!% e3en# "e )!#elE
A.J. Han Vinck -8
Britchman mode% $or mobi%e radio channe%s
multi$le states % and only one state !
Closer to an atual real&'orld hannel
G
n
F
+9$
(rr!r $r!&a&ili1 , (rr!r $r!&a&ili1 h
G
G
+
A.J. Han Vinck -*
E-am(%e 9$rom imo 3orhonen, He%sinki:
' ,n $ading channe%s received data can e-(erience b&rst errors that destro/
%arge n&mber o$ consec&tive bits. his is harm$&% $or channe% coding
' ,nter%eaving distrib&tes burst errors a%ong data stream
' A (rob%em o$ inter%eaving is
introd&ced e-tra de%a/
' E-am(%e be%o# sho#s b%ock
inter%eaving8
time
received
power
Reception after
fading channel
1 0 0 0 1 1 1
0 1 0 1 1 1 0
0 0 1 1 0 0 1
1 0 0 0 1 1 1 0 1 0 1 1 1 0 0 0 1 1 0 0 1
1 0 0 0 1 0 0 0 1 0 1 1 1 1 0 1 1 0 1 0 1
Received interleaved data:
Block deinterleaving :
Recovered data:
A.J. Han Vinck 6,
e-am(%e
F Consider the code C < I 000, 111 J
F A b&rst error o$ %ength ! can not be corrected.
F Get &s &se a b%ock inter%eaver !=!
A+ A- A6 F+ F- F6 C+ C- C6
2 errors
A+ A- A6 F+ F- F6 C+ C- C6
Interleaver
A+ F+ C+ A- F- C- A6 F6 C6
A+ F+ C+ A- F- C- A6 F6 C6
Deinterleaver
A+ A- A6 F+ F- F6 C+ C- C6
1 error 1 error 1 error
A.J. Han Vinck 6+
2idd%eton t/(e o$ b&rst channe% mode%
)e%ect channe% k
#ith (robabi%it/
C9k:
ransition
(robabi%it/ 590:
0
1
0
1
G
channe% 1
channe% 2
channe% k has
transition
(robabi%it/ (9k:
A.J. Han Vinck 6-
,m(&%sive Noise C%assi$ication
0a4 7ingle ransien )!#el
5arameters o$ sing%e transient 8
- (eak am(%it&de
- (se&do $re;&enc/ $
0
<11
0
- dam(ing $actor
- d&ration
- ,nterarriva% ime
2eas&rements carried o&t b/ Brance e%ecom in a ho&se d&ring A0 h
2 c%asses o$ (&%ses 9on 1KAA (&%ses: 8 sing%e transient and b&rst
A.J. Han Vinck 66
,nter%eaving8 $rom b&rst/ to random
2essage inter%eaver channe% inter%eaver
-1
message
encoder decoder
b&rst/
Lrandom errorM
(ote) inter%eaving brings encoding and decoding de%a/
Home#ork8 com(are the b%ock and convo%&tiona% inter%eaving #.r.t. de%a/
A.J. Han Vinck 64
,nter%eaving8 b%ock
Channe% mode%s are di$$ic&%t to derive8
- b&rst de$inition 0
- random and b&rst errors 0
$or (ractica% reasons8 convert b&rst into random error
read in ro# #ise

transmit
co%&mn #ise
1
0
0
1
1
0
1
0
0
1
1
0
0
0
0
0
0
1
1
0
1
0
0
1
1
A.J. Han Vinck 65
+e-,nter%eaving8 b%ock
read in co%&mn
#ise
this ro# contains 1 error
1
0
0
1
1
0
1
0
0
1
1
e
e
e
e
e
e
1
1
0
1
0
0
1
1
read o&t
ro# #ise
A.J. Han Vinck 66
,nter%eaving8 convo%&tiona%
in(&t se;&ence 0
in(&t se;&ence 1 de%a/ o$ b e%ements
in(&t se;&ence m-1 de%a/ o$ 9m-1:b e%ements

E-am(%e8 b < 5, m < !
in
o&t
A.J. Han Vinck 67
(3a)$leUMT7 T%r&! (nc!#er
' Brom E), ) 125 212 v!.A.0 92000-0N:, 62) 2&%ti(%e-ing and channe% coding
' +ata is segmented into b%ocks o$ G bits, #here A0 G 511A
HU$$erI
J7C
(nc!#er
HL!?erI
J7C
(nc!#er
InerleaCer
71se)aic
O%$%
.
k

UninerleaCe#
/ari1
K
k

InerleaCe#
/ari1
KL
k

In$%
.
k

InerleaCe#
In$%
.L
k

O%$%
A.J. Han Vinck 68
62) ,nter%eaver8
,nserting +ata into 2atri-
' +ata is $ed ro#-#ise into a E b/ C matri-. E < 5, 10, or 20, O C 25K
F ,$ G P EC then matri- is (added #ith d&mm/ characters.
.
+
.
-
.
6
.
4
.
5
.
6
.
7
.
8
.
*
.
+,
.
++
.
+-
.
+6
.
+4
.
+5
.
+6
.
+7
.
+8
.
+*
.
-,
.
-+
.
--
.
-6
.
-4
.
-5
.
-6
.
-7
.
-8
.
-*
.
6,
.
6+
.
6-
.
66
.
64
.
65
.
66
.
67
.
68
.
6*
.
4,
.
-
.
6
.
5
.
7
.
6
.
4
.
+
.
8
.
+,
.
+-
.
++
.
+5
.
+6
.
+4
.
*
.
+6
.
+8
.
--
.
-+
.
-6
.
+*
.
-,
.
+7
.
-4
.
-6
.
-8
.
-7
.
6+
.
-*
.
6,
.
-5
.
6-
.
4,
.
66
.
65
.
6*
.
67
.
68
.
66
.
64
Daa is $er)%e# within eac" r!?.
.
4,
.
66
.
65
.
6*
.
67
.
68
.
66
.
64
.
-6
.
-8
.
-7
.
6+
.
-*
.
6,
.
-5
.
6-
.
+8
.
--
.
-+
.
-6
.
+*
.
-,
.
+7
.
-4
.
+,
.
+-
.
++
.
+5
.
+6
.
+4
.
*
.
+6
.
-
.
6
.
5
.
7
.
6
.
4
.
+
.
8
J!?s are $er)%e#
.
4,
.
66
.
65
.
6*
.
67
.
68
.
66
.
64
.
-6
.
-8
.
-7
.
6+
.
-*
.
6,
.
-5
.
6-
.
+8
.
--
.
-+
.
-6
.
+*
.
-,
.
+7
.
-4
.
+,
.
+-
.
++
.
+5
.
+6
.
+4
.
*
.
+6
.
-
.
6
.
5
.
7
.
6
.
4
.
+
.
8
Daa is rea# ;r!) )ari3 c!l%)n9?ise
A.J. Han Vinck 6*
' Detetion rules
' (er$ormance
' E-am(%es o$ codes
A channe% transition mode%
A.J. Han Vinck 4,
,n(&t = channe% transition "&t(&t >

-
1
/
1
-
*
/
*
+++
-
,
/
(
59/
1
Q-
1
:
59/
*
Q-
1
:
59/
(
Q-
1
:
59/
(
Q-
,
:
59/
1
Q-
,
:
A.J. Han Vinck 4+
-rror $robability (,.P)
)&((ose $or a received vector / #e assign the message i as being transmitted.
hen, the (robabi%it/ o$ a correct decision < 59 -
i
transmitted Q / received :
Bor the assignment R #e have 59R is correctQ/: < 59 -
/
transmitted Q / rec.:
Hence, in order to ma-imiSe the (robabi%it/ o$ being correct #e assign the i
to the received vector / that ma-imiSes 59 -
i
transmitted Q / received :
92a-im&m A(osteriori 5robabi%it/, 2A5:
A.J. Han Vinck 4-
2a-im&m Gike%ihood 92G: receiver
$ind i that ma-imiSes

59 -
i
Q /: < 59 -
i
, / : 1 59 / :
< 59 / Q-
i
: 5 9 -
i
: 1 59 / :
$or e;&a%%/ %ike%/ -
i
this is e;&iva%ent to $ind
ma-im&m 59 / Q -
i
:
A.J. Han Vinck 46
e-am(%e
Bor ( < 0.1 and
=
1
< 9 0 0 :H 59 =
1
< 11! :
=
2
< 9 1 1 :H 59 =
2
< 21!:
7ive /o&r 2A5 and 2G decision $or > < 9 0 1 :
A.J. Han Vinck 44
' +etection r&%es
' Intro to the odin0 $roblem
' (er$ormance
' E-am(%es o$ codes
A.J. Han Vinck 45
*it (rotection
' "btained b/ Error Contro% Codes 9ECC:
F Bor#ard Error Correction 9BEC:
F Error +etection and $eedback 9AEC:
' 5er$ormance de(ends on error statisticsT
F Error mode%s are ver/ im(ortant
A.J. Han Vinck 46
e-am(%e
ransmit8 0 0 0 or 1 1 1
Ho# man/ errors can #e correct0 Ho# man/ errors can #e detect0
ransmit8
A < 00000H * < 01011H C < 10101H + < 11110
Ho# man/ errors can #e correct0 Ho# man/ errors can #e detect0
.hat is the di$$erence0
5ractica% comm&nication s/stem design
message
estimate
channe%
decoder
n
Code
#ord in
receive
here are 2
k
code #ords o$ %ength n
k is the n&mber o$ in$ormation bits transmitted in n channe% &ses
2
k
Code book
Code book
#ith errors
Channe% ca(acit/
+e$inition8
he rate E o$ a code is the ratio k1n, #here
k is the n&mber o$ in$ormation bits transmitted in n channe% &ses
)hannon sho#ed that8 8
$or E Ca(acit/
an encoding methods e-ist
#ith decoding error (robabi%it/ 0
Encoding and decoding according to )hannon
Code8 2
k
binar/ code#ords #here (90: < 591: < U
Channe% errors8 590 1: < 591 0: < (
i.e. V error se;&ences 2
nh9(:

+ecoder8 search aro&nd received se;&ence $or code#ord
#ith n( di$$erences
s(ace o$ 2
n
binar/ se;&ences
decoding error (robabi%it/
1. Bor t errors8 Qt1n-(QW X
0 $or n
9%a# o$ %arge n&mbers:
2. W 1 code #ord in region
9code#ords random:

<

>

n an#
4 $ 0 " +
n
k
J ;!r
, - -
-
-
4 + - 0 4 + 0 /
4 J
F7C
C 0 n 4 J 4 $ 0 " + 0 n
n
4 $ 0 n"
k
A.J. Han Vinck 5+
A $ic!rial Cie?
-
n
Cec!rs -
k

c!#e ?!r#s

A.J. Han Vinck 5-
decoder
' Com(are received #ord #ith a%% (ossib%e code#ords

code #ords
received
: Dec!#e "e c!#e ?!r# ?i" )ini)%) M !; #i;;erences 0NM!s Likel1H4

A.J. Han Vinck 56
e3a)$le
code #ords8 0 0 0 0 0 0 1 0 1 1 1 0 1 0 1 1 1 1 1 0
received8 0 0 0 1 1
di$$erence8 0 0 0 1 1 0 1 0 0 0 1 0 1 1 0 1 1 1 0 1
best g&ess8 0 1 0 1 1
!nl1 + #i;;erence
A.J. Han Vinck 54
?e "aCe s!)e $r!&le)s
' 2a((ing $rom in$ormation to code #ords
F generation o$ code #ords 9m&t&a%%/ $ar a(art:
F storing o$ code book 92
k
code #ords, %ength n:
' +ecoding
F Com(are a received #ord #ith a%% (ossib%e code #ords
55
.hat are the c%assica% research (rob%ems
*odeboo+, ma-imi.e - # of codewords of
length n
- minimum distance
d
min
*hannel characteri.ation,
-/ypes of errors0 memory in the noise behavior 1
-*hannel *apacity1
2ecoder,
-2esign of a decoding3decision algorithm
-4inimi.e complexity
A.J. Han Vinck 56
' +etection r&%es
' $erformane
' E-am(%es o$ codes
A.J. Han Vinck 57
Definitions
:
Ha))ing #isance &e?een 3 an# 1 is
#
H
>< #03D 14 is "e M !; $!sii!ns ?"ere 3
i
1
i
:
T"e )ini)%) #isance !; a c!#e C is
=
#
)in

< )in O #03D 14 2 3 CD 1 CD 3 1P
:
Ha))ing ?eig" !; a Cec!r 3 is
9 ?034

>< #03D , 4 is "e M !; $!sii!ns ?"ere 3
i
,
A.J. Han Vinck 58
e3a)$le
:
Hamming distance #0 +,,+D ,+++4 < 6
:
Minimum distance 0+,+D ,++D ++,4 < -
:
Hamming weight ?0,++,+,+4 < 4
Ha))ing ?as a ;a)!%s scienis ;r!) Fell9la& an#
inCener !; "e Ha))ing c!#e.
A.J. Han Vinck 5*
Performance
A c!#e ?i" )ini)%) #isance #
)in
is ca$a&le !;
c!rrecing err!rs i;

#
)in
- 8 +.

/r!!;> I; err!rs !cc%rD "en since #
)in
- 8 +
an inc!rrec c!#e ?!r# "as a leas 8+ #i;;erences
?i" "e receiCe# ?!r#.

A.J. Han Vinck 6,
$ic%re
-8+ #i;;erences
A F
#i;;erences ;r!) A
#i;;erences ;r!) F
A.J. Han Vinck 6+
Performane
A code #ith minim&m distance d
min
is ca(ab%e o$ correcting E eras&res i$
d
min
W E.

5roo$8

,$ E P d
min
eras&res occ&r, then at %east 1 (osition
is %e$t to disting&ish bet#een an/ t#o code#ords.
Note8 an eras&re is a (osition #here the receiver kno#s that an error occ&red.

A.J. Han Vinck 6-

Performane
A code #ith minim&m distance d
min
is ca(ab%e o$ correcting E eras&res
and t errors i$
d
min
W 2t ?E.

5roo$8

he minim&m distance is red&ced at ma-im&m b/ the va%&e E.
Hence, i$
d
min
- E W 2t

#e can sti%% correct t errors
A.J. Han Vinck 66
Performane for the %ausian hannel
?
-
?
- -
?
E
59eras&re:
59error:
(E
(E
'
Bor an error correcting code8 2t < 2 n 9(
2
?(
!
: error correcting
'
.ith eras&res8 d
min
W n 9(
2
?(
1
: ? 2 n (
!
'
)ince (
2
P (
1
error correction #itho&t eras&res is a%#a/s betterT
(
!
(
!
(
1
(
2
(
!
A.J. Han Vinck 64
' +etection r&%es
' (er$ormance
' 1he lass of linear odes
' E-am(%es o$ codes
A.J. Han Vinck 65
G,NEAE C"+E)
*inar/ codes are ca%%ed %inear i$$
Zthe com(onent #ise mod&%o-2 s&m o$ t#o code

#ords is again a code #ord.
Conse;&ent%/, the all 2ero 'ord is a code #ord.

A.J. Han Vinck 66
G,NEAE C"+E 7ENEEA"E
T"e c!#e ?!r#s are
9 linear c!)&inai!ns !; "e r!?s !; a &inar1 genera!r )ari3 G ?i"
#i)ensi!ns kD n
9 G )%s "aCe rank kQ
(3a)$le> C!nsi#er k < 6D n < 6.
+ , , + + ,
genera!r )ari3 G = + + , , + +
+ , + + , +
0+D,D+4G < 0 ,D ,D +D ,D +D +4
A.J. Han Vinck 67
71se)aic c!#es
Le in general "e )ari3 G &e ?rien as

+ , , 2 + + ,
G < R I
k
/ SA G< , + , 2 + , +
, , + 2 , + +
k < 6D n < 6
T"e c!#e generae# is
=
linearD s1se)aic
=
"as )ini)%) #isance 6.
=
"e e;;icienc1 !; "e c!#e is 6'6.
A.J. Han Vinck 68
E-am(%e 9o(tim&m:
'
)ing%e 5arit/ check code d
min
< 2, k < n-1
100 0 1
0100 0 1
7 < R I
n9+
/ S<
00 01 1
A%% code#ords have even #eightT
A.J. Han Vinck 6*
E-am(%e 9o(tim&m:
'
Ee(etition code8 d
min
< n, k < 1
' 7 < [ 1 1 1 \
A.J. Han Vinck 7,
E;&iva%ent codes
' An/ %inear code generator can be bro&ght in Ms/stematic $orm]
'
7
s/s
<
k
n

k n

n
Note8 the e%ementar/ o(eration have an inverse.
Home#ork8 give an e-am(%e $or k < A and n < ^
(le)enar1
r!?
!$erai!ns
(le)enar1
c!l%)n
!$erai!ns
N!n9
s1se)aic
;!r)
A.J. Han Vinck 7+
Property
T"e set of distances
;r!) all c!#e ?!r#s ! "e all Ter! c!#e ?!r#
is "e sa)e as ! an1 !"er c!#e ?!r#.
Proof:
#0 3D 1 4 < #0 3 3D T < 1 3 4 < #0 ,D T 4D
&1 lineari1 T is als! a c!#e ?!r#.
A.J. Han Vinck 7-
T"%sQ
"e determination !; "e )ini)%) #isance !; a
c!#e is eU%iCalen !

"e determination !; "e )ini)%) Ha))ing
?eig" !; "e c!#e ?!r#s.

T"e c!)$le3i1 !; "is !$erai!n is $r!$!ri!nal ! M
!; c!#e ?!r#s

A.J. Han Vinck 76
e3a)$le
:
C!nsi#er "e c!#e ?!r#s
=
,,,,,
=
,++,+
=
+,,++
=
++++,
H!)e?!rk> Deer)ine "e )ini)%) #isance
A.J. Han Vinck 74
Linear c!#e generator
,9=: re(resents the k bit in$o vector 9 i
0
, i
1
, ..., i
k-1
:
g9=: is a binar/ (o%/nomia% o$ degree 9 n-k :

13-()
the code vector C o$ %ength n can be described b/
C9=: < ,9=: g9=: a%% o(erations mod&%o-2.

A.J. Han Vinck 75
EX: ! "# n ! $ and %&X' ! 1 ( X ( X
)
:
@!r "e in;!r)ai!n Cec!r 0+D,D+D,4

C0.4 < 0+ 8 .
-
4 0

+ 8 . 8 .
6
4
< + 8 . 8 .
-
8 .
5
0+D+D+D ,D ,D+D ,4.
:
"e enc!#ing $r!ce#%re in 0k 3 n4 )ari3 ;!r)>
+ + , + , , ,
G < , + + , + , , c < I V G
, , + + , + ,
, , , + + , +
A.J. Han Vinck 76
I)$le)enai!n ?i" a s"i;9regiser
T"e ;!ll!?ing s"i; regiser can &e %se#> g0.4 < 0+ 8 . 8 .
6
4
i
k9+
...

i
-
i
+
i
,
H!)e?!rk>
giCe a #escri$i!n !; "e s"i; c!nr!l ! !&ain "e res%l
A.J. Han Vinck 77
)ome remarks
' 7enerators $or di$$erent k and n
F are constr&cted &sing mathematics
F %isted in man/ te-t books
.hat remains is the decodingT
A.J. Han Vinck 78
' +etection r&%es
' (er$ormance
' !ounds on the minimum distane
' E-am(%es o$ codes
A.J. Han Vinck 7*
*o&nds on minim&m distance 9Hamming:
' Ginear codes have a s/stematic e;&iva%ent 7
F 2inim&m Hamming #eight n F k ? 1
' 9)ing%eton bo&nd:
'
V code #ords _ V correctab%e error (atterns 2
n
' 3ome'ork) sho' that 3ammin0 odes satisfy the bound 'ith e4uality5
,
-
- A ' -
+
l ! g + 0 4
t p n
n
i
n
M t n
i
R M h p
n
_

,

A.J. Han Vinck 8,

Hamming bo&nd e-am(%e
' 5rob%em8
7ive the &((er bo&nd o$ the siSe o$ a %inear code C o$ %ength n < K and d
min
< !
' )o%&tion8
the code corrects &( to one error and th&s QCQ 2
K
1 9 1 ? K: < KA 1 ^ QCQ N
)ince $or a %inear code QCQ < 2
k
#e have QCQ O
e-am(%e8 1 0 0 0 1 1
7 < 0 1 0 1 0 1
0 0 1 1 1 0
A.J. Han Vinck 8+
*o&nds on minim&m distance 95%otkin:
- Gist a%% code#ords o$ a binar/ %inar code n
2
k
code#ords
- Ever/ co%&mn m&st have 2
k
12 ones 9(roo$:
9no a%% Sero co%&mn e-ists:
Conc%&sion8
he tota% n&mber o$ ones in the code#ords is n - 2
k
12
<W d
min
m&st be average, and th&s d
min
n - 2
k-1
1 92
k
-1:
A.J. Han Vinck 8-
*o&nds on minim&m distance 97i%bert:
' )tart8 )e%ect code#ord $rom 2
n
(ossib%e #ords
'
1. Eemove a%% #ords at distance P d
min
$rom se%ected code#ord
' 2. )e%ect one o$ the remaining as ne-t code#ord
' !. 7oto 1. &n%ess no (ossibi%ities %e$t.
' EE)6G8
'
home#ork8 sho# that %og21n 1 F h92(: $or d
min
-1 < 2t 2(nH ( P `
) i n
+
,
-
n
d
i
M
n
i
1
1
1
1
_
1

,
1
A.J. Han Vinck 86

(%ot
1
0.5
E < %og
2
21n
( t1n
1-h9(:
1-h92(:
0
)ing%eton
5%otkin
A.J. Han Vinck 84
Bor Additive .hite 7a&ssian Noise Channe%s
;&antiSed as binar/ s/mmetric channe%
' Error (robabi%it/ ( e
-Es1No

F #here Es is energ/ (er transmitted s/mbo%
F No the one-sided noise (o#er s(ectra% densit/
' Bor an &ncoded s/stem ( e
-Eb1No

' Bor a coded s/stem #ith minim&m distance d
F decoding errors occ&r #hen b/ 2t?1 W d
F nEs < kEb and th&s
(
c
e
-d12 k1n Eb1No

' CO(CL6SIO() make the fator C 7 d8* k8n 9 1
A.J. Han Vinck 85
' +etection r&%es
' (er$ormance
' -:am$les of odes
A.J. Han Vinck 86
Eichard Hamming 91N15-1NNO:
,n 1NA^, Hamming #as one o$ the ear%iest &sers o$ (rimitive com(&ters at *e%% Gaboratories. He #as $r&strated b/ their
%ack o$ $&ndamenta% re%iabi%it/. He there$ore (&SS%ed over the (rob%em o$ ho# a com(&ter might check and correct its
o#n res&%ts. .ithin severa% months Hamming discovered that e-tra bits co&%d be added to the interna% binar/ n&mbers o$
the com(&ter to red&ndant%/ encode n&merica% ;&antities. his red&ndanc/ enab%ed re%ative%/ sim(%e circ&itr/ to identi$/
and correct an/ sing%e bit that #as bad #ithin the encoded b%ock o$ bits 9t/(ica%%/ one #ord o$ data:. his encoding
scheme, no# kno#n as Hamming Code, a%so detects the condition o$ an/ t#o bits in the encoded b%ock that $ai%
sim&%taneo&s%/.
,n 1N50, Hamming (&b%ished his #ork as Error +etecting and Error Correcting Codesa in the *e%% )/stem echnica%
Jo&rna%, vo%. 2N, ((. 1A^-1K0.
-5 -K
-^
-1 -2
-!
-A
-1, -2, ..., -^ are binar/ va%&es 90 or 1:
-1, -2, -!, -A are in$ormation bits
-5 makes -1?-!?-A?-5 even
-K makes -2?-!?-A?-K even
-^ makes -1?-!?-2?-^ even
-1, -2, ..., -^ are transmitted 9stored:
.ny sin0le bit han0e an be orreted
A.J. Han Vinck 87
)im(%er code, same (rinci(%e
-5
-K
-1
-2
-!
-A
in$o check
1 ; ; 0 0 0
-1 -2 -! -A -5 -K
in$o check
1 0 0 0 1 1 transmit
-1 -2 -! -A -5 -K
+ecode received
1 0 0 0 ; 1
C8 ransmission e$$icienc/0
A.J. Han Vinck 88
he ne-t code has %ength 15 and 11 in$ormation bits
-10
-5
-2
-K
-!
-O
C8 Check that 1 error can be corrected
C8 ransmission e$$icienc/0
-^
-A
-11
-N
(A
51
52
5!
-1
A.J. Han Vinck 8*
.h/ making things com(%icated0
' he 9!,1: code
/+
/- .+
A.J. Han Vinck *,
' Can #e do a genera% (resentation 0 >E)

06+D-64
A.J. Han Vinck *+
Hamming codes
:
Mini)%) #isance 6
:
C!nsr%ci!n
= G < I
k
All )9%$les !; Ha))ing ?eig" W +
=
?"ere k < -
)
= ) 9 +
:
*+ec t+at t+e minim,m distance is )-
:
.ive t+e efficiency of t+e code
A.J. Han Vinck *-
E-am(%e k < A, n < ^, m < !
+,,, ++,
,+,, +,+
G <
,,+, ,++
,,,+ +++
,n *%&etooth #e have a shortened 910,15: Hamming Code
4 + 6 - k
6

A.J. Han Vinck *6
)/ndrome decoding
Le G < R I
k
/ S "en c!nsr%c H
T
< /
I
n9k
@!r all c!#e ?!r#s c < 3GD cH
T
< 3GH
T
< ,
HenceD ;!r a receiCe# n!is1 Cec!r
0 c n 4 H
T
< c H
T
n H
T

< n H
T
< > 7
A.J. Han Vinck *4
e-am(%e
100 110
7 < 010 101
001 011
110
101
011
100
010
001
H
<
- < 1 0 1
c < 1 0 1 1 0 1
c H

< 0 0 0
n < 0 1 0 0 0 0
c n < 1 1 1 1 0 1
[c n\ H

< ) < 1 0 1
"bvio&s $ast decoder8 (reca%c&%ate a%% s/ndromes at receiver
$or correctab%e errors
A.J. Han Vinck *5
,n s/stem $orm
c n
Calc%lae
s1n#r!)e
[c n\ H

< )
/recalc%lae#
s1n#r!)es
n_
c n
c n n_
#hen n < n_
then n n_ < 0
H!)e?!rk> c"!!se $ara)eers "a can &e i)$le)ene#
A.J. Han Vinck *6
he (arit/ check matri-
' .e sa# that C < , 7 7 is k - n
,7H
< 0H
is n - n-k
5ro(osition8 "n%/ $or code#ords CH
< 0
5roo$8 take the s/stematic encoder (resentation
- there are 2
n-k
di$$erent s/ndromes
- 2
k
di$$erent vectors ca&se the same s/ndrome
nH
< )H mH
< )H 9n m:H
< 0 i.e. the/ di$$er b/ a code#ord

- the 2
k
code#ords give the s/ndrome < 0
A.J. Han Vinck *7
' A (ro(ert/ to be &sed %ater8
F
An/ d
min
-1 ro#s o$ H
are %inear%/ inde(endent

Eeason8 ,$ not, then %ess than d
min
ro#s give the s/ndrome 0
"n%/ Code#ords give s/ndrome 0 and have minim&m #eight d
min
A.J. Han Vinck *8
Varshamov-*o&nd
Let us onstrut a matri: 3
1
'ith dimensions n : n&k for a 0i"en minimum distane d<
' )tart #ith the identit/ matri- #ith dimension 9n-k: - 9n-k:.
' hen, constr&ct a %ist o$ vectors s&ch that an/ d-2 vectors are %inear%/ inde(endent
' ,$ at %ist siSe i,
#e can add one more vector, not e;&a% to the a%% Sero vector, di$$erent $rom the %inear
combinations and th&s a%% d-1 combinations are %inear%/ inde(endent.
+e$ine n8< i?1. ake the %argest va%&e o$ n s&ch that
th&s,

+ -
- #
i
-
i
+
i
k n
<
,
_
+ +
,
_
,
_

k n
- #
, X
+ k n
-
X
+ n
-

<
,
_
,
_
- #
, i
+ n
k
i
+ n
-
-
A.J. Han Vinck **
' (ro(ert/
$or a code #ith minim&m distance d
min
< 2t?1,
a%% error events o$ #eight b t give rise to a di$$erent s/ndrome.
Eeason8
,$ not, then the s&m o$ t#o events #ith s&m-#eight %ess than d
min
give the
s/ndrome 0.
his contradicts the ass&m(tion that on%/ code#ords give s/ndrome 0 and
have minim&m #eight d
min
< 2t?1
A.J. Han Vinck +,,
Jee# 7!l!)!n C!#es 0CDD DVD4
:
7r%c%re>
)
k in;!r)ai!n s1)&!ls n9k c"eck s1)&!ls
Properties: )ini)%) #isance < n9k8+ 0s1)&!ls4
leng" -
)
9+
A.J. Han Vinck +,+
7enera% remarks on E) codes
' he genera% (rob%em is the decoding
F E) codes can be decoded &sing
' E&c%ids a%gorithm
' *er%ekam( 2asse/ a%gorithm
A.J. Han Vinck +,-
Eeed-2&%%er based codes 962):
' )tarting code m < 1H C < I 00,01,10,11 J
F has minimum distane *
m&1

F *
m=1
ode 'ords 6 of len0th *
m
' NE. code #ith 2
m?2
code #ords o$ %ength 2
m?1
F 96,6: and 96,6:
distance [ 96,6:, 96,6:\ < 2
m
#h/0 Convince /o&rse%$T
distance [ 96,6:, 9V,V:\ < 2 _ 2
m-1
< 2
m
distance [ 96,6:, 9V,V:\ < 2 _ 2
m-1
< 2
m
9&se com(%.(ro(.:
A.J. Han Vinck +,6
)&((ose #e have a generator 7 $or certain m
he constr&ction o$ 7 $or m?1 is sim(%e
E-am(%e8
m < 1 m<2 m < ! etc.
7 < 01 7 < 0 1 0 1 0 1 0 1 0 1 0 1
11 1 1 1 1 1 1 1 1 1 1 1 1
0 0 1 1 0 0 1 1 0 0 1 1
0 0 0 0 1 1 1 1
'
d
min
< 1 2 A c
A.J. Han Vinck +,4
' *asis $or constr&ction
' ake 2 %inear code generators 71 and 72
' o$ %ength n
' and minim&m distance +1 and +2, res(ective%/
' hen, 71 71
7 < has d
min
< minI2+1,+2J
0 72
' 5roo$T
A.J. Han Vinck +,5
,)*N n&mbering code
'
$or the ,)*N n&mbers 9a
1
, a
2
, c, a
10
: #e have
#here
'
.e &se a s(ecia% s/mbo% = in case a
10
< 10.
' A sing%e transmission error gives as a res&%t
since 11 is a (rime n&mber.
(ote) check that the code can a%so detect a do&b%e trans(osition 9interchange o$ 2 n&mbers:.
++ )!# , ia
+,
+ i
i

. ++ )!# , a ia a +, ia
+,
*
+ i
i +,
*
+ i
i
+

D ++ )!# , Xe Xe ia
X X
+,
+ i
i
+
A.J. Han Vinck +,6

Varshamov-enengo%S codes
'
$or the binar/ n&mbers 9-
1
, -
2
, c, -
n
: #e have
' A sing%e transmission error gives as a res&%t
' Check that a%% n&mbers are di$$erent and th&s #e can correct 1 errorT
' he cardina%it/ o$ the code
' 3ome'ork) com(are the rate o$ the code #ith the rate o$ the Hamming code.
4 + n - )!#0 a i3
n
+ i
i
+
4D + n - )!#0 X ' a Xe i3
X
n
+ i
i
+ + +
+ n -
-
2 C 2
n
+
A.J. Han Vinck +,7

' +etection r&%es
' (er$ormance
' E-am(%es o$ codes
' >hy error orretion?
A.J. Han Vinck +,8
.h/ error correction0
' )/stems #ith errors can be made
a%most error $ree
F C+, +v+ #o&%d not #ork #itho&t E) codes
A N)%liH %ser a$$licai!n
:
(NCODING> ?! %sers rans)i 8+ !r = +
9 7%$$!se !ne %ser %ses an err!r c!rrecing c!#e a rae Y.
9 T"e sec!n# %ser X%s sen#s "is in;!r)ai!n a rae +.
:
CHANN(L> a##s "e Cal%es an# "%s "e receiCer !&serCes 8-D , !r 9-
:
D(COD(J> #ec!#es "e in;!r)ai!n ;!r %ser + %sing an eras%re #ec!#ing
$r!ce#%reA A;er #ec!#ingD "e in;!r)ai!n ;r!) %ser - can &e calc%lae#
:
7UM JAT( < +.5 &i'rans)issi!n
:
NOT(> Ti)e DiCisi!n giCes a rae +Q
A.J. Han Vinck +,*
A N)%liH %ser a$$licai!n
A.J. Han Vinck
++,
?1- 1
?1- 1
?1- 2H 0
add
6ser 1H E < 112
6ser 2H E <1
EEA)6EE Channe% $rom &ser 1 to receiver
?1
-1 -2
?2
0
U
U
U
U
E1
E2
U
U
)&m e$$icienc/
time sharing
1
1
A.J. Han Vinck +++
A.J. Han Vinck ++-
' )&((ose8
transmit 9n-k bits: receiver
> < = N9oise: >H

NH

N = < >
add =H

add =
= = = =

transmit n bits
Bor n-k < nh9(: #e transmitted n ? nh9(: bits instead o$ 2n
' Ee$erence8 )%e(ian and .o%$
A.J. Han Vinck ++6
' ,n AEC s/stems s/stems co%%a(s can be (ost(onedT
tro&gh(&t
100D
k1n D
Channe% error (robabi%it/
0 1
A.J. Han Vinck ++4
Combining error detection-correction
' 719=: 729=: generates a code #ith minim&m distance +2
' 719=: generates a code #ith minim&m distance +1 P +2
' C9=: < ,9=: 719=: 729=: < ,d9=:719=:
decoding8
ste( 18 correct a ma-im&m o$ 9 +1 F 1 :12] errors #ith 719=:
ste( 28 detect the remaining errors #ith 719=:729=:
(ro(erties8
1. t 9 +1 F 1 :12] errors correctab%e
2. t +2 - 1 - 9 +1 F 1 :12] errors detectab%e
Home#ork8 constr&ct some e-am(%es
A.J. Han Vinck ++5
Combining error detection-correction
' E-am(%e8
n k d
min
generator (o%/nomia% 9octa%:
K! 5^ ! 71 < 10! 9octa%:
51 5 72 - 71 < 71 - 912^:
A5 ^ 7! - 72 - 71< 72 - 71 - 91A^:
A.J. Han Vinck ++6
E-am(%e8 b%&etooth
' hree error correction schemes are de$ined $or *%&etooth8
F 11! rate BEC, a sim(%e re(etition
F 21! rate BEC, a shortened Hamming Code 910,15:
F AEC scheme $or data 9a&tomatic retransmission re;&est:
' BEC schemes red&ce the n&mber o$ retransmissions
'
A CEC code decides #hether a (acket1header contains errors, i.e.
' transmit C9=: < ,9=: g9=:
' receive E9=: < C9=: ? E9=:
' check E9=: mod&%o g9=: < 0 0
Code shortening
' )&((ose that #e have a E) code in s/stematic $orm
n4
1 0 ... 0
0 1 ... 0 he generator takes the $irst k ? n4-k co%&mns
5arit/ (art minim&m distance is n4-k?1
...
0 0 ... 1
k n4-k
k4 n-k
take the %ast k4 ro#s o$ %ength k4 ? n - k
100...0 5arit/ (art minim&m distance is n4 F k4 ? 1
00... 1
++7
Case a
Case b
A.J. Han Vinck ++8
Channe% #ith insertions and de%etions
' *ad s/nchroniSation or c%ock recover/ at receiver8
F insertion
''' 1 0 0 0 1 1 1 0 0 1 0 ''' ''' 1 0 0 1 0 1 1 1 0 0 1 0 '''
F de%etion
''' 1 0 0 0 1 1 1 0 0 1 0 ''' ''' 1 0 0 1 1 1 0 0 1 0 '''
5rob%em8 $inding start and end o$ messages

A.J. Han Vinck ++*
' E-am(%e8
F the $o%%o#ing code corrects
' a sing%e inversion error
' or a sing%e de%etion1insertion error
' C < I 0 0 0 0 0 0, 1 1 1 1 1 1, 0 0 1 0 1 1, 1 1 0 1 0 0 J
F
d
min
< !
F An insertion1de%etion makes the #ord %onger1shorter
A.J. Han Vinck +-,
' +&e to errors in bit (attern
$%ag < 1 1 1 1 1 0, avoid 1 1 1 1 1 in $rame
''' 0 1 1 1 1 1 0 0 1 1 0 1 ''' ''' 0 1 1 1 1 0 1 0 0 1 1 0 1 '''
''' 0 1 1 0 1 0 1 0 0 1 1 0 1 ''' insertion
''' 0 1 1 1 0 0 0 0 1 1 0 1 ''' ''' 0 1 1 1 1 0 0 0 1 1 0 1 '''
''' 0 1 1 1 1 0 0 1 1 0 1 ''' de%etion
A.J. Han Vinck +-+
Channe%s #ith inter$erence
' E-am(%e 9o(tica% channe%:
Error (robabi%it/ de(ends on s/mbo%s in neighboring s%ots
A.J. Han Vinck +--
Channe%s #ith memor/ 9e-8 recording:
: (3a)$le> 5
i
< .
i
8 .
i9+
.
i
O 8+D 9+ P

X
i
X
i-1
Y
i
{ +2, 0, -2

,5.,-.+4 +-6
he (ass#ord (rob%em
server
Pass'ord Pass'ord < Honar/
Hash9Honar/:
memor/ Hash9Honar/:
Ca%c&%ate
Hash9(ass#ord:
com(are
server
, am8 *ahram
Hash9 :
memor/ Hash9 :
Ca%c&%ate Hash9 :
com(are
, am8 *ahram
biometri biometri
,5.,-.+4 +-4
5rob%em in a&thentication8
(ass#ords8 need to be e-act
biometrics8 are on%/ ver/ simi%ar
,5.,-.+4 +-5
he (rob%em A8 sec&re sketch
* * )tore *H )tore *H

9s/ndrome: 9s/ndrome:
*d< * *d< * N N
ca%c&%ate ca%c&%ate
9* 9* *d:H *d:H

< NH < NH

*dH *dH

+EC"+E N +EC"+E N
* * N N
*dH *dH

@rom NH NH

) *
k
$ossible "etors ! e:$<A orret ! 7 *
k
: B!B8*
n

$rob< orret 0uess 7 *
n&k
8 B!B *
&k

enro%%
sketch
reconstr&ction
*H *H

,5.,-.+4 +-6
A&thentication8 sec&re storage, #ith errors
-nrollment) -nrollment)

) < * E9E:
random E
Condition8
given ) and Hash9E:
it is hard
to estimate *, E
store )
Hash9E:
E
*
E9E:
authentiation) authentiation) )
Hash9E:
*4 ) *4 < * E9E: *4 < E9E: N <W E4

<W Hash9E4:
Hash9E:
,5.,-.+4 +-7
A&thentication8 sec&re storage, #ith errors
attak) attak)
)
Hash9E:
7&ess
E or *
*4 < ) E9E4: < * E9E: E9E4:
or
) H
< [* E9E:\ H
< *4 H

-nrollment) -nrollment)

) < * E9E:
random E
Condition8
given ) and Hash9E:
it is hard
to estimate *, E
store )
Hash9E:
E
*
E9E:
12!
*inar/ entro(/
inter(retation8
%et a binary se;&ence contain (n ones, then
#e can s(eci$/ each se;&ence #ith
%og
2
2
nh9(:
< n h9(: bits
0 4
-
nh p
n
pn
_

,
4 $ 0 "
$n
n
l!g
n
+
li)
-
n

,
_

Home#ork8 5rove the a((ro-imation &sing %n NT e N %nN $or N %arge.
6se a%so %og
a
- < / %og
b
- < / %og
b
a
he )tir%ing a((ro-imation
Q -
N N
N NN e

12"
0 0. 1 0. 2 0. 3 0. 4 0.5 0. 6 0.7 0. 8 0. 9 1
0
0. 1
0. 2
0. 3
0. 4
0. 5
0. 6
0. 7
0. 8
0. 9
1
p
h
T+e /inary Entropy> "0$4 < 9$l!g
-
$ = 0+9$4 l!g
-
0+9$4
N!e>
"0$4 < "0+9$4

8b Error Correction

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

8b Error Correction

Caricato da

Copyright:

Formati disponibili

DIGITAL COMMUNICATION

A.J. Han Vinck +5

in(&t se;&ence m-1 de%a/ o$ 9m-1:b e%ements

: Dec!#e "e c!#e ?!r# ?i" )ini)%) M !; #i;;erences 0NM!s Likel1H4

A.J. Han Vinck 6-

Zthe com(onent #ise mod&%o-2 s&m o$ t#o code

Conse;&ent%/, the all 2ero 'ord is a code #ord.

C9=: < ,9=: g9=: a%% o(erations mod&%o-2.

A.J. Han Vinck 8,

A.J. Han Vinck 86

< 0 i.e. the/ di$$er b/ a code#ord

are %inear%/ inde(endent

A.J. Han Vinck +,6

A.J. Han Vinck +,7

Potrebbero piacerti anche