Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
EE2
Dr. Cong Ling
Department of Electrical and Electronic Engineering
Course Information
Handouts
Slides: exam is based on slides
Notes: contain more details
Problem sheets
Course homepage
http://www.commsp.ee.ic.ac.uk/~cling
You
Y can access llecture
t
slides/problem
lid / bl
sheets/past
h t /
t papers
Also available in Blackboard
Grading
g
1.5-hour exam, no-choice, closed-book
Lectures
Introduction and background
1. Introduction
2. Probability and random
processes
3 Noise
3.
Effects of noise on analog
communications
4. Noise performance of DSB
5. Noise performance of SSB and
AM
6. Noise performance of FM
7. Pre/de-emphasis
p
for FM and
comparison of analog systems
Digital communications
8. Digital representation of signals
9. Baseband digital transmission
10. Digital modulation
11 Noncoherent
11.
N
h
td
demodulation
d l ti
Information theory
12 Entropy and source coding
12.
13. Channel capacity
14. Block codes
15. Cyclic codes
Communications
C
How do communication systems perform in the presence of noise?
About statistical aspects and noise.
noise
This is essential for a meaningful comparison of various
communications systems.
Learning Outcomes
Describe a suitable model for noise in communications
Determine the signal
signal-to-noise
to noise ratio (SNR) performance of
analog communications systems
Determine the probability of error for digital
communications systems
Understand information theoryy and its significance
g
in
determining system performance
Compare the performance of various communications
systems
References
B.P.
B
P Lathi,
Lathi Modern Digital and Analog
Communication Systems, 3rd ed., Oxford
University Press, 1998
J.G. Proakis and M. Salehi,, Communication
Systems Engineering, Prentice-Hall, 1994
L.W. Couch II, Digital and Analog
Communication Systems, 6th ed., PrenticeH ll 2001
Hall,
7
Multitude of Communications
Telephone network
Internet
Radio and TV broadcast
Mobile communications
Wi-Fi
Satellite and space communications
Smart power grid, healthcare
Analogue communications
AM, FM
Digital communications
Transfer of information in digits
Dominant technology today
Broadband, 3G, DAB/DVB
8
Whats Communications?
Communication involves the transfer of information from
one point to another.
Three basic elements
Transmitter: converts message into a form suitable for
transmission
Channel: the physical medium, introduces distortion, noise,
interference
Receiver:
R
i
reconstruct
t t a recognizable
i bl fform off the
th message
Speech
Music
Pictures
Data
Communication Channel
The channel is central to operation of a communication
y
system
Linear (e.g., mobile radio) or nonlinear (e.g., satellite)
Time invariant (e.g., fiber) or time varying (e.g., mobile radio)
10
Noise in Communications
Unavoidable presence of noise in the channel
Noise refers to unwanted waves that disturb communications
Signal is contaminated by noise along the path.
External noise: interference from nearby channels, humanmade noise, natural noise...
Internal noise: thermal noise, random emission... in
electronic devices
Noise is one of the basic factors that set limits on
communications.
i ti
A widely used metric is the signal-to-noise (power) ratio
(SNR)
signal power
SNR= noise p
power
11
13
Information Theory
Shannon
Many concepts
M
t were fundamental
f d
t l and
d paved
d th
the
way for future developments in communication
theory.
Provides a basis for tradeoff between SNR and
bandwidth, and for comparing different communication
schemes.
Kolmogorov
14
Milestones in Communications
Milestones (2)
16
17
Wi--Fi
Wi
Wi-Fi connects local computers (usually within 100m
g )
range)
19
802.11a
Standard for 5GHz band
20-70 Mbps, variable range
Similar to HiperLAN in Europe
802.11g
Standard in 2.4 GHz and 5 GHz bands
Speeds up to 54 Mbps, based on orthogonal frequency division
multiplexing (OFDM)
802.11n
Data rates up
p to 600 Mbps
p
Use multi-input multi-output (MIMO)
20
Satellite/Space Communication
Satellite communication
Cover very large areas
Optimized for one-way transmission
Radio (DAB) and movie (SatTV)
broadcasting
Two-way systems
The only choice for remote-area and
maritime communications
Propagation delay (0.25 s) is
uncomfortable in voice
communications
Space communication
21
22
Communication Networks
Todays communications networks are complicated
systems
A large n
number
mber of users
sers sharing the medi
medium
m
Hosts: devices that communicate with each other
Routers: route date through the network
23
Concept of Layering
Partitioned into layers, each doing a relatively simple task
Protocol stack
Application
Network
Transport
Network
Link
Ph i l
Physical
OSI Model
TCP/IP protocol
stack (Internet)
Physical
2-layer model
24
25
Outline
Probability
Random processes
p
Definition
Stationary random processes
Power spectral density
References
Notes of Communication Systems,
y
, Chap.
p 2.3.
Haykin & Moher, Communication Systems, 5th ed., Chap. 5
Lathi, Modern Digital and Analog Communication Systems, 3rd ed.,
Chap 11
Chap.
26
Probabilistic Concepts
What is a random variable (RV)?
It is a variable that takes its values from the outputs
p
of a random
experiment.
Example
For tossing a coin
coin, S = { H
H, T }
For rolling a die, S = { 1, 2, , 6 }
28
Probability Properties
PX(xi): the probability of the random variable X taking on
the value xi
The probability of an event to happen is a non-negative
number, with the following properties:
The probability of the event that includes all possible outcomes of
the experiment is 1.
The probability of two events that do not have any common
outcome is the sum of the probabilities of the two events
separately.
Example
Roll a die:
PX(x = k) = 1/6
for k = 1, 2, , 6
29
FX ( x) P( X x)
Properties
FX () 0, FX () 1
FX ( x1 ) FX ( x2 ) if x1 x2
dF X ( x )
dx
x
FX ( x )
f X ( y ) dy
P ( a X b ) FX (b ) FX ( a )
f X ( y ) dy
f X ( x)
dF X ( x )
dx
If x is sufficiently small,
P ( x X x x )
x x
f X ( y)
f X ( y ) dy f X ( x ) x
Area
f X ( x)
x
y
E[ X ] X
x f X ( x ) dx
E[ ]: expectation
operator
X2 E[( X X ) 2 ]
( x X ) 2 f X ( x)dx E[ X 2 ] X2
31
f X ( x)
FX ( x )
1
2
1
2
( x m )2
2 2
( y m )2
2 2
f x
for
dy
E[ X ] m
X2 2
: rm s v a lu e
32
Uniform Distribution
fX(x)
f X ( x) b a
0
0
xa
FX ( x )
b a
1
a xb
elsewhere
xa
a xb
ab
2
(b a ) 2
12
E[ X ]
X2
xb
33
Joint Distribution
Joint distribution function for two random variables X and Y
FXY ( x , y ) P ( X x , Y y )
Joint probability density function
f XY ( x, y )
Properties
2 FXY ( x , y )
xy
1) FXY (, )
f XY (u , v)dudv
d d 1
2)
f X ( x)
f XY ( x, y ) dy
3)
fY ( x )
f XY ( x, y ) dx
4)
X , Y are independent f XY ( x, y ) f X ( x) fY ( y )
5)
f ( x) 21
for 0 x 2
E[ XY ]
1
2
Locus of
X and Y
X
cos sin d 0!
35
Joint pdf
f X 1 X 2 ... X n ( x1 , x2 ,...xn )
Independent
FX 1 X 2 ... X n ( x1 , x2 ,...xn ) FX 1 ( x1 ) FX 2 ( x2 )...FX n ( xn )
f X 1 X 2 ... X n ( x1 , x2 ,...xn ) f X 1 ( x1 ) f X 2 ( x2 )...
) f X n ( xn )
x1
x1 + x2
+ x3
x1 + x2
x1 + x2+
x3 + x4
38
An Ensemble of Signals
39
X (t ) E[ X (t )] x f X ( x; t )dx
In general
general, the mean is a function of t.
t
Autocorrelation function
xy f X ( x, y; t1 , t2 )dxdy
40
X (t ) X
Its autocorrelation function only depends on time difference
RX (t, t ) RX ( )
Example
Show that sinusoidal wave with random phase
X (t ) A cos(( c t )
with phase uniformly distributed on [0,2] is stationary.
Mean is a constant:
1
2
f ( )
, [0
[0, 2 ]
1
X (t ) E[ X (t )] A cos(ct ) d 0
2
0
2
Autocorrelation function only depends on the time difference:
RX (t , t ) E[ X (t ) X (t )]
E[ A2 cos(c t ) cos(c t c )]
A2
A2
E[cos(2c t c 2)]
E[cos(c )]
2
2
A2 2
1
A2
cos(2c t c 2 )
d
cos(c )
0
2
2
2
A2
RX ( )
cos(c )
2
42
S X ( f ) R X ( )e j 2f d
2
P E[ X (t )] R X (0) S X ( f )df
The
Th frequency
f
content off a process depends
d
d on how
h
rapidly the amplitude changes as a function of time.
This can be measured by the autocorrelation function
function.
43
L
Lett Y(t) obtained
bt i d b
by passing
i random
d
process X(t) th
through
h
a linear system of transfer function H(f). Then the PSD of
Y(t)
2
(2.1)
SY ( f ) H ( f ) S X ( f )
Proof: see Notes 3
3.4.2.
42
Cf. the similar relation for deterministic signals
If X(t)
( ) is a Gaussian p
process,, then Y(t)
( ) is also a Gaussian
process.
Gaussian processes are very important in communications.
44
Lecture 3: Noise
Dr. Cong Ling
Department of Electrical and Electronic Engineering
45
Outline
What is noise?
White noise and Gaussian noise
Lowpass noise
Bandpass noise
In-phase/quadrature representation
Phasor representation
p
References
Notes of Communication Systems, Chap. 2.
Haykin & Moher, Communication Systems, 5th ed., Chap. 5
Lathi, Modern Digital and Analog Communication Systems, 3rd ed.,
Chap. 11
46
Noise
Noise is the unwanted and beyond our control waves that
g
disturb the transmission of signals.
Where does noise come from?
External sources: e.g., atmospheric, galactic noise, interference;
Internal sources: generated by communication devices themselves.
This type of noise represents a basic limitation on the performance of
electronic communication systems
systems.
Shot noise: the electrons are discrete and are not moving in a
continuous steady flow, so the current is randomly fluctuating.
Thermal
Th
l noise:
i
caused
db
by th
the rapid
id and
d random
d
motion
ti off electrons
l t
within a conductor due to thermal agitation.
47
White Noise
The additive noise channel
n(t)
( ) models all types
yp of noise
zero mean
White noise
Its power spectrum density (PSD) is constant over all frequencies,
i.e.,
N
f
SN ( f ) 0 ,
2
Factor 1/2 is included to indicate that half the power is associated
with positive frequencies and half with negative.
The term white is analogous to white light which contains equal
amounts of all frequencies (within the visible band of EM wave).
It
Itss only defined for stationary noise
noise.
48
SN(f)
PSD
Rn()
Ideal Low
Low--Pass White Noise
Suppose white noise is applied to an ideal low-pass filter
of bandwidth B such that
N0
, | f | B
SN ( f ) 2
0,
otherwise
Power PN = N0B
By
B Wiener-Khinchine
Wi
Khi hi relation,
l i
autocorrelation
l i ffunction
i
Rn() = E[n(t)n(t+)] = N0B sinc(2B)
(3.1)
where sinc(x) = sin(x)/x.
x
Samples at Nyquist frequency 2B are uncorrelated
Rn() = 0,
0
= k/(2B),
k/(2B) k = 1,
1 2,
2
50
Bandpass Noise
n(t)
Any
y noise that enters the receiver will therefore be bandpass
p
in nature:
its spectral magnitude is non-zero only for some band concentrated
around the carrier frequency fc (sometimes called narrowband noise).
51
Example
If white noise with PSD of N0/2 is passed through an ideal
bandpass
p
filter,, then the PSD of the noise that enters the
receiver is given by
N0
, f fC B
SN ( f ) 2
0,
otherwise
Power PN = 2N0B
Autocorrelation function
Rn() = 2N0Bsinc(2B)cos(2fc)
which follows from (3.1) by
applying
l i th
the ffrequency-shift
hift
property of the Fourier transform
g ( t ) G ( )
g ( t ) 2 cos 0 t [ G ( 0 ) G ( 0 )]
-ffk
fk
53
f k f c k f
(3.2)
nc (t ) ak cos(2 ( f k f c )t k )
k
(3.3)
ns (t ) ak sin( 2 ( f k f c )t k )
k
nc(t)
nc(t)
ns(t)
ns(t)
55
56
(3.4)
This is analogous to
g (t ) G ( )
g (t )2 cos 0t [G ( 0 ) G ( 0 )]
A rigorous
g
p
proof can be found in A. Papoulis,
p
, Probability,
y, Random
Variables, and Stochastic Processes, McGraw-Hill.
The PSD can also be seen from the expressions (3.2) and (3.3)
where each of nc(t) and ns(t) consists of a sum of closely spaced
base-band sinusoids.
57
Noise Power
For ideally filtered narrowband noise, the PSD of nc(t)
Sc(f)=Ss(f)
and ns((t)) is therefore g
given by
y
N 0 , | f | B
Sc ( f ) S s ( f )
0, otherwise
(3.5)
Phasor Representation
We may write bandpass noise in the alternative form:
n(t ) nc (t ) cos(2 f c t ) ns (t ) sin(2 f c t )
r (t ) cos[2 f c t (t )]
r (t ) nc (t ) 2 ns (t ) 2
ns ( t )
nc (t )
(t ) tan 1
( ) 2 fct + (t)
(t)
()
59
60
Summary
White noise: PSD is constant over an infinite bandwidth.
Gaussian noise: PDF is Gaussian
Gaussian.
Bandpass noise
In-phase
In phase and quadrature compoments nc(t) and ns(t) are low-pass
low pass
random processes.
nc(t) and ns(t) have the same PSD.
nc(t) and ns(t) have the same variance as the band-pass noise n(t).
Such properties will be pivotal to the performance analysis of
bandpass communication systems.
62
Outline
SNR of baseband analog transmission
Revision of AM
SNR of DSB-SC
References
Notes of Communication Systems
Systems, Chap
Chap. 3
3.1-3.3.2.
1-3 3 2
Haykin & Moher, Communication Systems, 5th ed., Chap. 6
Lathi, Modern Digital and Analog Communication Systems, 3rd ed.,
Chap. 12
63
SNR
We must find a way to quantify (= to measure) the
performance of a modulation scheme.
p
We use the signal-to-noise ratio (SNR) at the output of the
receiver:
average power of message signal at the receiver output PS
SNRo
dB
If x is power,
X (dB) = 10 log10(x)
If x is amplitude,
X ((dB)) = 20 log
g10((x))
Transmitted Power
PT: The transmitted power
Limited by: equipment capability
capability, battery life
life, cost
cost,
government restrictions, interference with other channels,
green communications etc
The higher it is, the more the received power (PS), the
higher the SNR
For a fair comparison between different modulation
schemes:
PT should be the same for all
66
67
Output SNR
PT
N 0W
Revision: AM
General form of an AM signal:
Modulation index:
mp
A
69
Signal Recovery
n(t)
Receiver model
1) 1 A m p : use an envelope
l
d
detector.
t t
This is the case in almost all commercial AM radio
receivers
receivers.
Simple circuit to make radio receivers cheap.
2) Otherwise: use synchronous detection = product
detection = coherent detection
Th terms
The
t
detection
d t ti and
d demodulation
d
d l ti are used
d iinterchangeably.
t h
bl
70
71
DSB--SC
DSB
Double-sideband suppressed carrier (DSB-SC)
2
72
~
y Am(t ) nc (t )
Signal power at the receiver output:
PN N 0df 2 N 0W
W
73
Comparison
A2 P
SNRo
2 N 0W
To which transmitted power does this correspond?
A2 P
2
2
2
PT E{ A m(t ) cos (2 f c t )}
2
So
SNRo
Comparison with
SNRbaseband
PT
SNRDSB SC
N0W
PT
SNRDSB SC SNRbaseband
N 0W
Conclusion: DSB
DSB-SC
SC system has the same SNR performance as a
baseband system.
74
75
Outline
Noise in SSB
Noise in standard AM
Coherent detection
(of theoretic interest only)
Envelope detection
References
Notes of Communication Systems, Chap. 3.3.3
3.3.3-3.3.4.
3.3.4.
Haykin & Moher, Communication Systems, 5th ed., Chap. 6
Lathi, Modern Digital and Analog Communication Systems, 3rd ed.,
Chap. 12
76
SSB Modulation
Consider single (lower) sideband AM:
A
A
s(t )SSB m(t ) cos 2f ct m (t ) sin
i 2f ct
2
2
where m (t ) is the Hilbert transform of m(t).
m (t ) is obtained by passing m(t) through a linear filter
with transfer function jsgn(f ).
m (t ) and m(t) have the same power P .
The average power is A2P/4.
77
Noise in SSB
Receiver signal x(t) = s(t) + n(t).
Apply a band-pass
band pass filter on the lower sideband
sideband.
Still denote by nc(t) the lower-sideband noise (different
from the double
double-sideband
sideband noise in DSB).
Using coherent detection:
y (t ) x(t ) 2 cos(2 f c t )
A
A
m (t ) ns (t ) sin(4 f c t )
2
y (t ) m(t ) nc (t )
2
78
Noise Power
Noise power for nc(t) = that for band-pass noise =
N0W ((halved compared
p
to DSB)) ((recall ((3.4))
))
SN(f)
N0/2
f
-ffc
-ffc+W
fc-W
W
fc
Lower-sideband noise
SNc(f)
N0/2
f
-W
W
Baseband noise
79
Output SNR
Signal power A2P/4
SNR at output
A2 P
SNRSSB
4 N 0W
For a baseband system with the same transmitted power
A2P/4
A2 P
SNRbaseband
4 N 0W
Conclusion: SSB achieves the same SNR performance
as DSB-SC (and the baseband model) but only requires
half the band-width.
80
~
y A m(t ) nc (t )
81
Output SNR
Signal power at the receiver output:
PS E{m2(t)} P
Noise power:
PN 2 N 0W
SNR at the receiver output:
P
SNRo
SNRAM
2 N0W
Transmitted power
A2 P A2 P
PT
2 2
2
82
Comparison
SNR of a baseband signal with the same transmitted
power:
p
A2 P
SNRbaseband
2 N 0W
Thus:
SNRAM
Note:
P
SNRbaseband
2
A P
P
1
2
A P
x(t)
Envelope
y (t ) envelope of x (t )
[ A m(t ) nc (t )]2 ns (t ) 2
Equation is too complicated
Must use limiting cases to put it in a form where noise and
message are added
85
n ( t ) [ A m ( t )]
Then
Then
Thus
ns (t ) [ A m(t ) nc (t )]
y (t ) [ A m(t ) nc (t )]
SNRo
P
SNRenv
2 N 0W
P
2
SNRbaseband
A P
n(t ) [ A m(t )]
Isolate the small quantity:
y2 (t ) [ A m(t) nc (t )]2 ns2 (t)
m
t
(
(
))
2( A m(t))nc (t )
2
2
[nc (t ) ns (t )]1 2
2
2
2
2
nc (t) ns (t )
nc (t) ns (t )
2[ A m(t )]nc (t )
y (t ) [n (t ) n (t )]1 2
2
nc (t ) ns (t )
2
c
2
s
2[ A m(t )]nc (t )
E (t )1
2
En (t
(t )
2
n
En (t ) nc2 (t ) ns2 (t )
87
y(t) En (t) 1
U
Use
1 x 1
x
for x 1
2
2[ A m(t)]cosn (t)
En (t)
[ A m(t )] cos n (t )
y (t ) En (t )1
En ( t )
En (t ) [ A m(t )] cos n (t )
88
Summary
A: carrier amplitude
amplitude, P: power of message signal
signal, N0: single-sided PSD of noise
noise,
W: message bandwidth.
89
90
Outline
Recap of FM
FM system model in noise
PSD of noise
References
Notes of Communication Systems, Chap. 3.4.1-3.4.2.
Haykin & Moher, Communication Systems, 5th ed., Chap. 6
Lathi,
Lathi Modern Digital and Analog Communication Systems
Systems,
3rd ed., Chap. 12
91
Frequency Modulation
Fundamental difference between AM and FM:
AM: message information contained in the signal
amplitude Additive noise: corrupts directly the
modulated signal.
FM: message information contained in the signal
frequency the effect of noise on an FM signal is
determined by the extent to which it changes the
frequency of the modulated signal.
Consequently, FM signals is less affected by noise than
AM signals
92
Revision: FM
A carrier waveform
s(t)
( ) = A cos[[i((t)]
)]
When
d
1 d
2f f
dt
2 dt
Generalisation: instantaneous frequency:
1 d i ( t )
f i (t )
2 dt
93
FM
In FM: the instantaneous frequency of the carrier varies
g
linearlyy with the message:
fi(t) = fc + kf m(t)
where kf is the frequency sensitivity of the modulator.
i (t ) 2 fi ( )d 2 f c t 2 k f m( )d
0
Modulated signal:
Note:
s(t ) A cos 2f c t 2k f m( )d
94
Bandwidth of FM
mp = max|m(t)|: peak message amplitude.
fc kf mp < instantaneous frequency
q
y < fc + kf mp
Define: frequency deviation = the deviation of the
instantaneous frequency from the carrier frequency:
f = kf mp
Define: deviation ratio:
f / W
W: the message bandwidth.
Small
: FM bandwidth 2x message
g bandwidth ((narrow-band FM))
Large : FM bandwidth >> 2x message bandwidth (wide-band FM)
95
FM Receiver
n(t)
96
Nonetheless, it can be shown that for high SNR, noise output and
message signal are approximately independent of each other:
Output Message + Noise (i.e., no other nonlinear terms).
97
Output of discriminator:
k f m((t )
98
x(t ) A cos 2 f c t 2 k f m( )d
0
nc (t ) cos(2 f c t ) ns (t ) sin(2 f c t )
~
x (t ) A cos(2f c t ) nc (t ) cos(2f c t ) ns (t ) sin(2f c t )
99
Phase Noise
Phasor diagram of
the FM carrier and
noise signals
ns(t)
A
Instantaneous
I t t
phase
h
noise:
i
i (t) tan1
nc(t)
ns (t )
A nc (t )
ns (t ) ns (t )
A
A
1 di (t )
1 dns (t )
2 dt
2 A dt
100
Discriminator Output
The discriminator output in the presence of both
g
and noise:
signal
1 dn
d s (t )
k f m( t )
2A dt
What is the PSD of
Fourier theory:
dns (t )
nd (t )
d
dt
x(t ) X ( f )
dx(t )
then
j 2 fX ( f )
dt
if
Noise PSD
It follows from (2.1) that
So ( f ) | H ( f ) |2 Si ( f )
PSD of nd (t ) | 2 f |2 N 0
BT
2
f BT / 2
1 dns (t ) 1
f2
2
PSD of fi (t )
| 2 f | N 0 2 N 0
2 A dt 2 A
A
Aft
After the
th LPF,
LPF the
th PSD off noise
i output
t t no(t)
( ) is
i restricted
t i t d iin
the band W
f2
S No ( f ) 2 N 0
A
f W
(6 1)
(6.1)
102
(a) Power spectral density of quadrature component ns(t) of narrowband noise n(t).
(b) Power spectral density of noise nd(t) at the discriminator output.
(c) Power spectral density of noise no(t) at the receiver output.
103
Lecture 7: Pre/de
Pre/de--emphasis for FM and
Comparison of Analog Systems
Dr. Cong Ling
Department of Electrical and Electronic Engineering
104
Outline
Derivation of FM output SNR
Pre/de-emphasis to improve SNR
Comparison with AM
References
Notes of Communication Systems, Chap. 3.4.2-3.5.
Haykin & Moher, Communication Systems, 5th ed., Chap. 6
Lathi,
Lathi Modern Digital and Analog Communication Systems
Systems,
3rd ed., Chap. 12
105
Noise Power
Average noise power at the receiver output:
W
PN S No ( f )df
W
Thus,
Thus from (6
(6.1)
1)
PN
2 N 0W 3
f2
N 0 df
2
A
3 A2
(7.1)
carrier power A2
A Noise,
Noise called the quieting effect
106
Output SNR
Since PS k 2f P , the output SNR
2 2
PS 3 A k f P
SNRO
SNRFM
3
PN 2 N 0W
From
A2
PT
2
k f mp
PT
and
:
SNRbaseband
N 0W
W
3k 2f P
2 P
SNR FM
SNR
3
SNRbaseband
baseband
2
2
W
mp
Threshold effect
The FM detector exhibits a more pronounced threshold
p detector.
effect than the AM envelope
The threshold point occurs around when signal power is
10 times noise power:
A2
10,
2 N 0 BT
BT 2W ( 1)
ns(t)
A
nc(t)
108
Qualitative Discussion
As the noise changes randomly, the point P1 wanders
around P2
High SNR: change of angle is small
Low SNR: P1 occasionally sweeps around origin, resulting in
changes
h
off 2
2 in
i a short
h t ti
time
109
Improvement Factor
Assume an ideal pair of pre/de-emphasis filters
Hde ( f ) 1/ Hpe ( f ),
)
f W
f BT / 2,
f2
recall S No ( f ) 2 N 0
A
f2
2
H
(
f
)
N 0 df
de
2
A
3
PN without pre / de - emphasis
2
W
2
I
W 2 3A 2
W
2
f
2
pre / de - emphasis
p
PN with p
(
f
)
df
W A2 H de ( f ) N0dff 3W f H de
d
112
Example Circuits
R r , 2 frC 1
(b) D
De-emphasis
h i filt
filter
1
H de ( f )
1 jf / f 0
Improvement
I
2W 3
3
f 2 / (1 f 2 / f 0 2 )df
(W / f 0 )3
3[(W / f 0 ) tan 1 (W / f 0 )]
In commercial FM,
FM W = 15 kHz
kHz, f0 = 2.1
2 1 kHz
I = 22 13 dB (a significant gain)
113
Assumptions:
single-tone
g
modulation,, i.e.: m(t)
( ) = Am cos(2
( fmt);
);
the message bandwidth W = fm;
for the AM system, = 1;
for the FM system, = 5 (which is what is used in commercial FM
transmission, with f = 75 kHz, and W = 15 kHz).
without pre/deemphasis
114
115
Conclusions
(Full) AM: The SNR performance is 4.8 dB worse than a
baseband system, and the transmission bandwidth is BT =
2W .
DSB
DSB: Th
The SNR performance
f
iis id
identical
ti l tto a b
baseband
b d
system, and the transmission bandwidth is BT = 2W.
SSB: The SNR performance is again identical, but the
transmission bandwidth is only BT = W.
FM: The SNR performance is 15.7 dB better than a
b
baseband
b d system,
t
and
d th
the ttransmission
i i b
bandwidth
d idth iis BT =
2( + 1)W = 12W (with pre- and de-emphasis the SNR
performance is increased by about 13 dB with the same
transmission bandwidth).
116
117
Outline
Introduction to digital communication
Quantization (A/D) and noise
PCM
Companding
References
Notes of Communication Systems, Chap. 4.1-4.3
Haykin & Moher
Moher, Communication Systems,
Systems 5th ed
ed., Chap
Chap. 7
Lathi, Modern Digital and Analog Communication Systems,
3rd ed., Chap. 6
118
119
Why Digital?
Advantages:
Digital
g
signals
g
are more immune to channel noise by
y using
g channel
coding (perfect decoding is possible!)
Repeaters along the transmission path can detect a digital signal
and retransmit a new noise
noise-free
free signal
Digital signals derived from all types of analog sources can be
represented using a uniform format
Digital signals are easier to process by using microprocessors and
VLSI (e.g., digital signal processors, FPGA)
Digital systems are flexible and allow for implementation of
sophisticated functions and control
More and more things are digital
Sampling
How densely should we sample an analog signal so that
p
its form accurately?
y
we can reproduce
A signal the spectrum of which is band-limited to W Hz,
can be reconstructed exactly from its samples, if they are
taken uniformly at a rate of R 2W Hz.
Nyquist frequency: fs = 2W Hz
121
Quantization
Quantization is the process of transforming the sample
p
into a discrete amplitude
p
taken from a finite set
amplitude
of possible amplitudes.
The more levels, the better approximation.
Dont need too many levels (human sense can only detect
finite differences).
Quantizers can be of a uniform or nonuniform type.
122
Quantization Noise
Quantization noise: the error between the input signal and
p signal
g
the output
123
q
2
2
If is sufficiently small,
small it is reasonable to assume that q is
uniformly distributed over this range:
Noise variance
1
, q
fQ (q)
2
2
0, otherwise
1 /2 2
PN E{e } q fQ (q)dq q dq
/2
3 /2
1q
/2
1 3 ()3 2
24 12
24
124
SNR
Assume: the encoded symbol has n bits
the maximum number of q
quantizing
g levels is L = 2n
maximum peak-to-peak dynamic range of the quantizer = 2n
125
SNR
From (8.1)
I dB,
In
dB
mp
2
n 1
SNRo
2m p
2
12 P
4 m2p
22 n
4m 2p
22 n
3P 2 n
22
mp
(8.2)
3P
SNRo (dB) 10 log10 (2 ) 10 log10 2
mp
2n
3P
20n log10 2 10 log10 2
m
p
3P
6n 10 log10 2 (dB)
m
p
Hence, each extra bit in the encoder adds 6 dB to the output SNR
quantizer.
of the q
Recognize the tradeoff between SNR and n (i.e., rate, or bandwidth).
126
Example
Sinusoidal message signal: m(t) = Am cos(2 fmt).
Average signal power:
Am2
P
2
Maximum signal value: mp = Am.
Substitute into (8
(8.2):
2):
3 Am2 2 n 3 2 n
SNRo
2 2
2
2 Am
2
In dB
SNRo (dB) 6n 1.8
1 8 dB
129
130
nonuniform
if
quantized
131
132
-Law vs. A
A--Law
DPCM (differential
(diff
ti l PCM):
PCM) quantize
ti the
th difference
diff
between
b t
consecutive samples; can save 8 to 16 kbps. ADPCM (Adaptive
DPCM) can go further down to 32 kbps.
Delta modulation: 1-bit DPCM with oversampling; has even lower
symbol rate (e.g., 24 kbps).
Audio CD: 16
16-bit
bit PCM at 44
44.1
1 kHz sampling rate
rate.
MPEG audio coding: 16-bit PCM at 48 kHz sampling rate
compressed to a rate as low as 16 kbps
kbps.
134
Demo
Music playing in Windows Sound Recorder
135
Summary
Digitization of signals requires
Sampling:
p g a signal
g
of bandwidth W is sampled
p
at the Nyquist
yq
frequency 2W.
Quantization: the link between analog waveforms and digital
representation.
representation
SNR (under high-resolution assumption)
3P
SNRo (dB) 6n 10 log
l 10 2 (dB)
m
p
Companding can improve SNR.
137
Outline
Line coding
Performance of baseband
digital transmission
Model
Bit error rate
References
Notes of Communication
Systems, Chap. 4.4
Haykin & Moher, Communication
y
, 5th ed.,, Chap.
p 8
Systems,
Lathi, Modern Digital and Analog
Communication Systems, 3rd ed.,
Chap. 7
138
Line Coding
The bits of PCM, DPCM etc need to be converted into
g
some electrical signals.
Line coding encodes the bit stream for transmission
through a line, or a cable.
Line coding was used former to the wide spread
application of channel coding and modulation techniques.
Nowadays, it is used for communications between the
CPU and peripherals, and for short-distance baseband
communications, such as the Ethernet.
139
Line Codes
Unipolar nonreturn-to-zero
(NRZ) signaling (on-off
signaling)
Polar NRZ signaling
Unipolar Return-to-zero (RZ)
signaling
Bipolar RZ signaling
Manchester code
140
141
Lowpass
Lowpass
filter
filter
n(t)
n(t)
Distribution of Noise
Effect of additive noise on digital transmission: at the
y
1 mayy be mistaken for 0,, and vice versa.
receiver,, symbol
bit errors
What is the probability of such an error?
After the LPF, the predetection signal is
y(t) = s(t) + n(t)
s(t): the binary-valued function (either 0 or A volts)
n(t): additive white Gaussian noise with zero mean and variance
W
N 0 / 2df N 0W
2
143
Decision
Y: a sample value of y(t)
If a symbol 0 were transmitted: y(t) = n(t)
Y will have a PDF of N (0, 2)
choose symbol 0
choose symbol 1
144
Errors
Two cases of decision error:
((i)) a symbol
y
0 was transmitted,, but a symbol
y
1 was chosen
(ii) a symbol 1 was transmitted, but a symbol 0 was chosen
Case (i)
Probability of (i) occurring = (Probability of an error,
given symbol
g
y
0 was transmitted)) ((Probabilityy of a 0 to
be transmitted in the first place):
p(i) = Pe0 p0
where:
p0: the a priori probability of transmitting a symbol 0
Pe0: the conditional probability of error
error, given that symbol 0 was
transmitted:
Pe 0
n2
1
dn
exp
2
2
2
146
Case (ii)
Probability of (ii) occurring = (Probability of an error,
given symbol
g
y
1 was transmitted)) ((Probabilityy of a 1 to
be transmitted in the first place):
p(ii) = Pe1 p1
where:
p1: the a priori probability of transmitting a symbol 1
Pe1: the conditional probability of error
error, given that symbol 0 was
transmitted:
( n A)2
1
dn
Pe1
exp
2
2
2
147
(n A) 2
n2
1
1
exp
exp 2 dn
p1
dn (1 p1 ) T
2
2
2
2
2
T
148
Derivation
Leibnitz rule of differentiating an integral with respect
parameter: if
to a p
I ( )
b( )
a ( )
f ( x; )dx
then
b ( ) f ( x; )
dI ( ) db( )
da ( )
f (b( ); )
f ( a ( ); )
dx
a
(
)
d
d
d
Therefore
(T A)2
T2
dPe (T )
1
1
(1 p1 )
p1
exp
exp 2 0
2
2
dT
2
2
2
T 2 (T A) 2
p1
exp
1 p1
2
T 2 T 2 A2 2TA
A(2T A)
exp
exp
2
2
2
2
149
Optimum Threshold
p1
A(2T A)
ln
1 p1
2 2
p1
2 ln
A(2T A)
1 p1
2
p1
p1
A
2 2
2
ln
ln
2T A T
A
1 p1
A 1 p1 2
150
Calculation of Pe
Define a new variable of integration
n
1
z dz
d dn
d dn
d dz
d
Then
1
z 2 /2
Pe 0
e
dz
A /(2 )
2
1
z 2 /2
e
dz
A /(2 )
2
Then:
t2
x exp 2 dt
A
Pe Q
151
Example
Example 1: A/ = 7.4 Pe = 104
For a transmission rate is 105 bits/sec,
bits/sec there will be an
error every 0.1 seconds
Example 2: A/ = 11.2 Pe = 108
For a transmission rate is 105 bits/sec, there will be an
error everyy 17 mins
A/ = 7.4: Corresponds
p
to 20 log
g10((7.4)) = 17.4 dB
A/ = 11.2: Corresponds to 20 log10(11.2) = 21 dB
Enormous increase in reliabilityy by
y a relativelyy small
increase in SNR (if that is affordable).
152
Probability
y of Bit Error
153
Q-function
Upper bounds and good
pp
approximations:
Q( x)
1 x2 / 2
e , x0
2 x
1 x2 / 2
Q( x) e
,x0
2
which is a better upper
bound for small x.
154
155
156
Outline
ASK and bit error rate
FSK and bit error rate
PSK and bit error rate
References
Notes of Communication Systems, Chap. 4.5
Haykin & Moher, Communication Systems, 5th ed., Chap. 9
Lathi, Modern Digital and Analog Communication Systems,
3rd ed., Chap. 13
157
Digital Modulation
Three Basic Forms of Signaling Binary Information
(a) Amplitude-shift
keying (ASK).
(c) Frequency
Frequency-shift
shift
keying (FSK).
158
Demodulation
ASK
Amplitude shift keying (ASK) = on-off keying (OOK)
s0((t)) = 0
s1(t) = A cos(2 fct)
or
s(t) = A(t) cos(2fct),
A(t) { 0, A}
Coherent detection
Coherent Demodulation
Pre-detection signal:
x(t ) s (t ) n(t )
A cos(2 f c t ) nc (t ) cos(2 f c t ) ns (t ) sin(2 f c t )
[ A(t ) nc (t )]cos(2 f c t ) ns (t ) sin(2 f c t )
After multiplication with 2cos(2 fct):
After low
low-pass
pass filtering:
y (t ) A(t ) nc (t )
161
162
PSK
Phase shift keying (PSK)
s(t) = A(t) cos(2 fct),
A(t) { A, A}
Use
U coherent
h
td
detection
t ti again,
i tto eventually
t ll gett th
the
detection signal:
y (t ) A(t ) nc (t )
Probability density functions for PSK for equiprobable 0s
and 1s in noise (use threshold 0 for detection):
(a): symbol 0 transmitted
((b):
) symbol
y
1 transmitted
163
Analysis
Conditional error probabilities:
Pe0
(n A)2
1
exp
dn
2
2
2
(n A)2
1
Pe1
exp
dn
2
2
2
0
0 n A
In the first set n n A dn dn and when n 0,
1
Pe 0
2
n~ 2 ~
A exp 2 2 dn
n 2
1
exp 2 2 (1)dn 2
A
n 2
A exp 2 2 dn
164
n2
1
Pe0 Pe1 Pe,PSK
exp 2 dn
A 2
2
2
Remember that
1
Q x
2
exp( t 2 / 2)dt
165
FSK
Frequency Shift Keying (FSK)
s0((t)) = A cos(2
( f0t),
),
if symbol
y
0 is transmitted
s1(t) = A cos(2 f1t),
if symbol 1 is transmitted
Symbol recovery:
Use two sets of coherent detectors, one operating at a frequency f0
and the other at f1.
166
Output
Each branch = an ASK detector
i
A + noise
LPF output on each branch =
noise
if symbol
b l presentt
if symbol not present
A
Q
2
168
170
Comment
To achieve the same error probability (fixed Pe):
PSK can be reduced by 6 dB compared with a baseband
or ASK system (a factor of 2 reduction in amplitude)
FSK can be reduced by 3 dB compared with a baseband
or ASK (a factor of 2 reduction in amplitude)
Caution: The comparison is based on peak SNR. In terms
of average SNR, PSK only has a 3 dB improvement over
ASK, and FSK has the same performance as ASK
171
Q-function
172
Examples
Consider binary PSK modulation. Assume the carrier
p
A = 1 v,, and noise standard deviation = 1/3.
amplitude
Determine the bit error probability.
Answer: Pe = 1.03 10-3.
173
Application: GMSK
Gaussian minimum shift keying (GMSK), a special form of
preceded by
y Gaussian filtering,
g, is used in GSM
FSK p
(Global Systems for Mobile Communications), a leading
cellular phone standard in the world.
Also known as digital FM, built on some
of FM-related advantages of AMPS,
g
analog
g system
y
the first-generation
(30 KHz bandwidth).
Binary data are passed through a
Gaussian filter to satisfy stringent
requirements of out-of-band radiation.
Minimum Shift Keying: its spacing between the two frequencies of
FSK is minimum in a certain sense (see problem sheet).
GMSK is allocated bandwidth of 200 kHz, shared among 32 users.
This provides a (30/200)x32
(30/200)x32=4
4.8
8 times improvement over AMPS
AMPS.
174
175
Outline
Noncoherent demodulation of ASK
Noncoherent demodulation of FSK
Differential demodulation of DPSK
References
Haykin & Moher, Communication Systems, 5th ed., Chap. 9
Lathi, Modern Digital and Analog Communication Systems,
3 d ed.,
3rd
d Ch
Chap. 13
176
Noncoherent Demodulation
Coherent demodulation assumes perfect synchronization.
Needs a p
phase lock loop.
p
When
e the
e ca
carrier
e p
phase
ase is
su
unknown,
o ,o
one
e must
us rely
eyo
on nono
coherent detection.
No provision is made for carrier phase recovery.
Bandpass
filter
Envelope
detector
Threshold
device
Decision
t = nTb
when 0 is sent
R ( A nc (t )) 2 ns2 (t )
when
h 1 iis sentt
178
f (r)
r2 /(2 2 )
, r 0
( r 2 A2 ) /( 2 2 )
Ar
I 0 2 , r 0
179
Rayleigh Distribution
Define a random variable R X Y where X and Y
are independent
p
Gaussian with zero mean and variance 2
R has Rayleigh distribution:
2
f R (r )
r 2 / ( 2 2 )
r0
v = r/
fV(v) = fR(r)
tan
t 1
Y
X
180
Derivation
Consider a small area dxdy = rdrd. One has
f R , (r , )drd f X ,Y ( x, y )dxdy f X ,Y ( x, y )rdrd
1
2
Hence
f R , ( r , )
e
2
r
2
e
2
x2 y 2
2 2
rdrd
x2 y 2
2 2
e
2
r2
2 2
pdf of R:
f R (r )
pdf of
f R , (r , )d
f ( ) f R , (r , )dr
0
e
2
r2
2 2
r0
1
, [0, 2 ]
2
181
Rician Distribution
If X has nonzero mean A, R has Rician distribution:
(r2 A2 )/(2 2 )
fR (r) 2 e
r
where
I 0 ( x)
1
2
A
I0 (Ar
2 ),
r 0
e x cos d
v = r/
a = A/
fV(v) = fR(r)
182
Derivation
Similarly,
f R , (r , )drd f X ,Y ( x, y )rdrd
1
2
e
2
1
2
( x A )2 y 2
2 2
r 2 A2 2 Ar cos
Hence
f R , ( r , )
r
2
e
2
rdrd
2 2
rdrd
r 2 A2 2 Ar cos
2 2
Bessel function
pdf of R:
f R (r)
f R , ( r, )d
e
2
r 2 A2
2 2
1
2
Ar cos
d ,
r0
183
Error Probability
Let the threshold be A/2 for simplicity.
The error p
probability
y is dominated by
y symbol
y
0,, and
is given by
1 r r 2 /(2 2 )
Pe
e
dr
2
A
/2
2
1 A2 /(8 2 )
e
2
Cf.
Cf coherent demodulation
Pe, ASK ,Coherent
A
Q
2
1 A2 /(8 2 )
e
2
Bandpass
filter
centered at f1
R1
If R1 > R2,
choose 0
If R1 < R2,
choose 1
Bandpass
filter
centered at f2
R2
185
Distribution of Envelope
When a symbol 1 is sent, outputs of the BPFs
y1((t)) = n1((t))
y2(t) = n2(t) + A cos(2 f2t)
Again, the first branch has Rayleigh distribution
f R1 (r1 )
r1 /(2 )
, r1 0
2 e
r1
r2
( r22 A2 )/(2 2 )
I 0 ( Ar22 ),
)
r2 0
186
Error Probability
Error occurs if Rice < Rayleigh
Pe P( R2 R1 )
e
1
2
r2
2
r2
( r2 2 A2 )/(2 2 )
r2
(2 r2 2 A2 )/(2 2 )
r1
I0 ( 2 ) 2 e
I0 ( 2 )
e
2
( r2 2 A2 )/(
)/(2 2 )
r2
Ar2
Ar2
r2
A2 /(4 2 )
r12 /(2 2 )
dr1dr2
r1 /(2
/( )
dr
d 1dr
d2
2 e
r1
I 0 ( Ar22 )dr2
( x 2 2 )/(2 2 )
I 0 ( x2 )dx
x 2r2 , A / 2
A 1 A 2 / ( 4 2 )
Q
2e
2
187
input
an
DPSK
signal
bn-1
Delay Tb
cos(ct)
188
Differential Demodulation
signal
plus
noise
Bandpass
filter
Lowpass
filter
Threshold
device
Decision
D
i i
on an
Delay Tb
A 1 A2 /( 2 2 )
Q e
2
189
Illustration of DPSK
Information symbols {an}
-1
-1
-1
-1
{bn-1}
-1
-1
Differentially encoded
sequence {bn}
-1
-1
Transmitted phase
(radians)
ec s o
Decision
-1
-1
-1
-1
Coherent ASK
Q(A/2)
Coherent FSK
Q(A/2)
Coherent PSK
Q(A/)
Noncoherent
ASK
exp(
exp(-A
A2/82)
Noncoherent
FSK
exp(-A
p( 2/42)
DPSK
exp(-A2/22)
Noncoherent
ASK, FSK
Coherent
ASK, FSK
Coherent PSK
Conclusions
Non-coherent demodulation retains the hierarchy of
p
performance.
Non-coherent
Non coherent demodulation has error performance slightly
worse than coherent demodulation, but approaches
coherent performance at high SNR.
Non-coherent demodulators are considerably easier to
build.
192
Application: DPSK
WLAN standard IEEE 802.11b
Bluetooth2
Digital audio broadcast (DAB): DPSK + OFDM (orthogonal
frequency division multiplexing)
Inmarsat (International Maritime Satellite Organization):
now a London-based mobile satellite company
193
194
Outline
What is information theory?
Entropy
Definition of information
Entropy of a source
Source coding
Source coding theorem
Huffman coding
References
N
Notes
t off Communication
C
i ti S
Systems,
t
Ch
Chap. 5
5.1-5.4
154
Haykin & Moher, Communication Systems, 5th ed., Chap. 10
Lathi, Modern Digital and Analog Communication Systems, 3rd ed.,
Chap. 15
195
196
What is Information?
Information: any new knowledge about something
How can we measure it?
Messages containing knowledge of a high probability of
occurrence Not very informative
Messages containing knowledge of low probability of
occurrence More informative
A small change in the probability of a certain output should
not change the information delivered by that output by a
large amount.
198
Definition
The amount of information of a symbol s with
probability
p
y p:
1
I (s) log
p
Properties:
p = 1 I(s)
( ) = 0: a symbol
y
that is certain to occur contains no
information.
0 p 1 0 I(s) : the information measure is monotonic and
non-negative.
p = p1 p2 I(s) = I(p1) + I(p2): information is additive for
statistically independent events.
199
Example
In communications, log base 2 is commonly used,
g in unit bit
resulting
Example: Two symbols with equal probability p1 = p2
= 0.5
Each symbol represents I(s) = log2(1/0.5) = 1 bit of
information.
In summary:
I ( s ) log 2
1
bits
p
log10 x
3.32 log10 x
log10 2
200
201
Source Entropy
If symbol sk has occurred, then, by definition, we have received
I (sk ) log2
bits of information.
1
log2 pk
pk
k 1
k 1
E{I ( sk )} pk I ( sk ) pk log
l 2 pk
Source entropy: the average amount of information per source
symbol:
K
H ( S ) pk log2 pk
k 1
202
Meaning of Entropy
What information about a source does its entropy give us?
It is the amount of uncertainty before we receive itit.
It tells us how many bits of information per symbol we
expect to get on the average.
Relation with thermodynamic entropy
In thermodynamics: entropy measures disorder and randomness;
In information theory: entropy measures uncertainty.
Some argue that to obtain 1 bit information, the minimum energy
needed is 10-23 Joules/Kelvin degree (extremely cheap!).
This may have implications on green computation.
computation
203
H(p)
Maximum uncertainty
when p = 1/2.
p
204
Source entropy:
H ((SS ) 0.7 log2 (0.7) 0.2 log2 (0.2) 0.1 log2 (0.1)
0.7 0.515 0.2 2.322 0.1 3.322
1.157 bits/symbol
A = 00, B = 01, C = 10
Entropy prediction: the average amount of information is
only 1.157 bits per symbol.
We
W are wasting
ti
bits!
bit !
205
L p k lk
k 1
208
1
log 2 ( p1Np1 p2 Np2 ... pK NpK )
p(S N )
Np
N 1 log
l 2 p1 Np
N 2 log
l 2 p2 ... Np
N K log
l 2 pK
K
N pk log 2 pk NH ( S )
k 1
L H (S )
210
Huffman Coding
Example
The average
g codeword length:
g
L (2 0.4) (2 0.2) (2 0.2) (3 0.1) (3 0.1) 2.2
213
Summary
The entropy of a discrete memoryless information source
K
H ( S ) pk log2 pk
k 1
214
215
Outline
Channel capacity
Channel coding theorem
Shannon formula
Comparison with
practical systems
p
y
References
Notes of Communication Systems, Chap. 5.5-5.6
Haykin & Moher, Communication Systems, 5th ed., Chap. 10
Lathi, Modern Digital and Analog Communication Systems,
3rd ed., Chap. 15
216
P
C B log 2 1 SNR B log 2 1
N
B
0
bps
Shannon Limit
Tradeoff between power and bandwidth: to get the same
g capacity
p
y C, one mayy
target
Increase power, decrease bandwidth;
Decrease power, increase bandwidth.
Unachievable
region
Achievable region
1/B
B
P
21/ B 1
1/ B
B(2 1)
N0
1/ B
Capacity boundary
P
21/ B ln 2( B 2 )
lim
lim
ln 2
2
B N
B
B
0
P/N0 (dB)
0.693
0 693 1.6
1 6 dB
Example
A voice-grade channel of the telephone network has a
bandwidth of 3.4 kHz.
(a) Calculate the capacity for a SNR of 30 dB.
(b) Calculate the SNR required to support a rate of 4800 bps.
Answer:
(a) 30 dB SNR = 1000
C B log 2 (1 SNR)
3.4 log
g 2 ((1 1000))
33.9 kbps
(b)
SNR 2C / B 1
24.8/3.4 1
1.66
1 66 2.2
2 2 dB
220
221
Input/Output SNR
The maximum rate at which information may arrive at the
receiver is
Cin B log
l 2 (1 SNR
S in )
Co W log2 (1 SNRo )
P
W P
N 0 B B N 0W
B /W
SNRbaseband
SNRo 1
1
B /W
e SNRbaseband as B / W
1
lim 1 e
x
x
223
SNRo (d
dB)
B/W=12
B/W 2
B/W=2
B/W=1
SNRbaseband (dB)
224
225
226
Discussion
Known analogue communication systems do not achieve
performance in g
general.
the ideal p
Something must be missing with analogue
communications.
Similarly, it can be shown that simple digital modulation
schemes cannot achieve capacity too.
One must resort to digital communications and coding to
approach the capacity.
The idea of tradeoff between bandwidth and SNR is useful
in spread-spectrum communications (CDMA, 3G mobile
communications) and ultra wideband
wideband.
227
228
Outline
Introduction
Block codes
Error detection and
correction
Generator matrix and
parity-check
parity
check matrix
References
Haykin & Moher, Communication Systems, 5th ed., Chap. 10
Lathi, Modern Digital and Analog Communication Systems,
3rd ed
ed., Chap
Chap. 16
229
Taxonomy of Coding
Error
Correction
Coding
= FEC
- no feedback
channel
Cryptography
(Ciphering)
Error
Detection
Coding
- used
in ARQ
as in TCP/IP
- feedback channel
- retransmissions
Source
Coding
Error Control
Coding
- Makes bits
equally
probable
- Strives to
utilize
channel
capacity by
adding
extra bits
Compression
Coding
- Redundancyy removal:
- Destructive (jpeg, mpeg)
- Non-destructive (zip)
Line Coding
- for baseband
communications
230
(n,k)
n bits
k input bits
encoder
Convolutional codes
Each source bit influences n(L+1) output bits
L is the memory length
Like
Lik convolution
l ti iin a lilinear filt
filter
i
input
bit
bi
k = 11, n = 22, L = 2
231
232
Channel Model
We can measure the effect of noise in different ways.
The most common is to specify an error probability,
probability p.
p
Consider the case of a Binary Symmetric Channel
with noise.
1
1-p
p
1
p
1-p
If the error probability is small and the information is fairly fault tolerant,
it is possible to use simple methods to detect errors.
Repetition Repeating each bit in the message
If the two symbols in an adjacent pair are different, it is likely that an error
has occurred.
However, this is not very efficient (efficiency is halved).
Repetition provides a means for error checking, but not for error correction.
P it bit Use
Parity
U off a parity
it bit att the
th end
d off the
th message
A parity bit is a single bit that corresponds to the sum of the other
message bits (modulo 2).
This allows any odd number of errors to be detected, but not even
numbers.
As with repetition,
p
, this technique
q only
y allows error checking,
g, not error
correction.
It is more efficient than simple repetition.
234
Block Codes
An important class of codes that can detect and correct
some errors are block codes
The first error-correcting block code was devised by
Hamming around the same time as Shannon was working
on the foundation of information theory
Hamming codes are a particular class of linear block code
Block codes
Encode a series of symbols from the source, a block, into a
longer string: codeword or code block
Errors can be detected as the received coded block will not be
one of the recognized, valid coded blocks
Error correction: To decode and associate a corrupted block to a
valid coded block by its proximity (as measured by the Hamming
distance)
d
s a ce )
235
101 5
101={1}{0}{1}
236
Example
i.
ii
ii.
iii.
i
i.
ii.
iii
iii.
Answers
00111
00010
1100 0011 = 0000
237
Hamming Distance
Hamming Weight
The Hamming weight of a binary vector, a (written as wH(a)), is the
number of non
non-zero
zero elements that it contains
contains.
Hence,
g weight
g of 5.
001110011 has a Hamming
000000000 has a Hamming weight of 0.
Hamming Distance
The Hamming Distance between two binary vectors, a and b, is
written dH(a,b) , and is equal to the Hamming weight of their
((Boolean)) sum.
dH(a,b) = wH(a+b)
Hence, 01110011 and 10001011 have a Hamming distance of
dH = wH (01110011+10001011)
= wH (11111000) = 5
238
239
Generator Matrix
240
Systematic Codes
k information bits
codeword
Example
Given the generator matrix for a (7,4) systematic code
1
0
G
0
0 0 0 0 1 1
1 0 0 1 0 1
0 1 0 1 1 0
0 0 1 1 1 1
Error Detection
243
Error Correction
The number t of errors that can be corrected is simply
the number of errors that can be detected divided by two
and rounded down to the nearest integer, since any
output vector with less than this number of errors will
nearer to the input code word.
dmin 2t
2 +1
dmin< 2t
244
Syndrome
If the received coded block y contains errors, then the
product of the received block with the transpose of the
parity check matrix will not be zero,
y HT 0
Writing y = x + e which is the sum of the original coded
block x and the error e, we find that
y HT = e HT
This value is referred to as the syndrome, s = y HT = e
HT.
The syndrome is a function of the error only, and contains
the information required to isolate the position (or positions)
of the error (or errors).
246
Syndrome Decoding
s is an (n-k) row vector, taking 2(n-k) -1 possible values (the
p
g to no error).
)
zero vector corresponding
This means that it is necessary to calculate and store 2(n-k)
-1 syndromes as a look-up table to be able to pin-point the
positions of the errors exactly.
Problem: this is impractical for large values of (n k).
Received
vector
t y
Parity-check
matrix H
Syndrome
s
Syndrome
table
Error
E
vector e
Decoded
codword x
247
Summary
A linear block code can be defined by a generator matrix
G and the associated p
parity-check
y
matrix H.
Every linear block code is equivalent to a systematic code.
The key parameter of a linear block code is the minimum
Hamming distance dmin:
Up to dmin 1 errors can be detected;
Up to (dmin 1)/2 errors can be corrected.
248
249
Outline
Hamming codes
A special type of cyclic codes that
correct a single error
Cyclic codes
Can correct more errors
The most important
p
class of block codes
Implementation takes advantage
of polynomial multiplication/division
References
Haykin
ay
& Moher,
o e , Communication
Co
u cat o Systems,
Syste s, 5t
5th ed
ed.,, C
Chap.
ap 10
0
Lathi, Modern Digital and Analog Communication Systems,
3rd ed., Chap. 16
250
Hamming Codes
Hamming codes are a class of linear block codes that
g error. Theyy satisfy
y the condition
can correct a single
r = n k = log2(n+1) n = 2r 1, k = 2r 1 r.
From this expression, it is easy to see that the first few
Hamming codes correspond to
(n, k) = (7,4), (15,11), (31,26),
They are easy to construct and are simple to use.
For all Hamming codes, dmin = 3.
All (correctable) error vectors have unit Hamming weight, and the
syndrome associated with an error in the ith column of the vector
is the ith row of HT .
251
Syndrome Table
G
Generator
t matrix
ti
1
0
G
0
0 0 0 0 1 1
1 0 0 1 0 1
0 1 0 1 1 0
0 0 1 1 1 1
s eH T
000
0000000
001
0000001
010
0000010
100
0000100
111
0001000
110
0010000
101
0100000
011
1000000
When a coded
Wh
d d vector
t iis received,
i d th
the syndrome
d
iis calculated
l l t d and
d any
single error identified, and corrected by exchanging the relevant bit
with the other binary value However, problems can occur if there is
more than one error.
252
Example
Consider the (7,4) Hamming code. If the code vector
1000011 is sent while 1000001 and 1001100 are received,,
decode the information bits.
Answer:
The first vector
s=(1000001)HT=(010)
e=(0000010)
x=(1000001)+(0000010)=(1000011)
u=(1000)
correct (as there is one error)
The second vector
s=(1001100)HT=(000)
error-free
x=(1001100)
(
)
wrong
g ((as there are 4 errors))
u=(1001)
253
Cyclic Codes
254
Generator Matrices
A cyclic code is still a linear block code, so all of the
properties
p
p
p
previously
y discussed hold for cyclic
y
codes.
They are constructed by defining a generator matrix, and
an associated parity check matrix and are decoded using
syndromes in exactly the same way as the other linear
block codes that we have discussed.
A generator matrix is defined in the same way as before,
except that the rows are now cyclic shifts of one ndimensional basis vector
vector.
g0
0
G
g1 g n k
g0
g1
g n k
g0
g1
0
0
g n k
255
Encoding
xn-1 = uk-1gn-k
These elements take the general form,
k 1
xl ui gl i
i 0
256
Polynomial Representation
257
Advantage
The main advantage is that it simplifies the cyclic shift
operation and thereby simplifies the hardware
operation,
implementation of the code considerably.
Multiplication of the code word by z shifts all of the
coefficients along by one, and replacing the term (x0zn)
byy a term ((x0z0)), g
gives a cyclic
y
shift.
A simpler way to achieve the same result is to multiply
the polynomial by z, divide by zn-1, and take the
remainder term, which can be written as
(z x(z)) mod(zn 1)
258
Encoding/Decoding
Encoding: x(z)=u(z)g(z);
Parity check polynomial:
h(z) =[zn+1] /g(z)
Decoding: (y(z) is the received vector)
Calculate the syndrome polynomial:
s(z)=(y(z)h(z))mod(z
s(z)
(y(z)h(z))mod(zn+1)
1)
Look up the syndrome table to get e(z) from s(z);
x(z)=y(z)+e(z);
u(z)=x(z)/g(z);
Thi
This iis th
the en/decoding
/d
di procedure
d
ffor non-systematic
t
ti
codes. A modified version works for systematic codes
(non-examinable)
(non-examinable).
260
Hardware Implementation
g2
g1
g3
b(z)
g(z)=
) g0+g1z+g2z2+g3z3, g0=g3=1;
1
Binary polynomial division: divide-by-g(z)
g0
g1
g2
b(z)
a(z)
g(z)=g0+g1z+g2z2+g3z3, g0=g3=1;
261
262
Applications of Coding
The first success was the application of convolutional
p space
p
p
probes 1960s-70s.
codes in deep
Mariner Mars, Viking, Pioneer missions by NASA
264
265
266
T
Trigonom
metic Iden
ntities
Revision Lecture
Dr. Cong Ling
Department of Electrical and Electronic Engineering
267
Lectures
Introduction and background
1. Introduction
2. Probability and random
processes
3 Noise
3.
Effects of noise on analog
communications
4. Noise performance of DSB
5. Noise performance of SSB and
AM
6. Noise performance of FM
p
for FM and
7. Pre/de-emphasis
comparison of analog systems
Digital communications
8. Digital representation of signals
9. Baseband digital transmission
10. Digital modulation
11 Noncoherent
11.
N
h
td
demodulation
d l ti
Information theory
12 Entropy and source coding
12.
13. Channel capacity
14. Block codes
15. Cyclic codes
268
The Exam
The exam paper contains 3 questions. All questions are
p
y
compulsory.
For example, the questions may look like
Question 1 (40 marks): Basic knowledge of communication
systems, elements of information theory and/or coding, mostly
bookwork
Question 2 (30 marks): Analog or digital communications
communications,
bookwork, new example/application/theory
Question 3 (30 marks): Digital or analog communications or
i f
information
i theory/coding,
h
/ di
b
bookwork,
k
k new
example/application/theory
Sample questions:
Past papers
Problems in classes
269
S X ( f ) RX ( )e j 2f d
270
Noise
Why is noise important in communications? How does
performance? What types
yp of noise exist?
noise affect the p
White noise: PSD is constant over an infinite bandwidth.
Gaussian noise: PDF is Gaussian.
Additive white Gaussian noise
Bandlimited noise, bandpass representation, baseband
noise nc(t) and ns(t), power spectral density
n(t ) nc (t ) cos((2f ct ) ns (t ) sin(
i ( 2f ct )
S ( f f c ) S N ( f f c ), | f | B
Sc ( f ) S s ( f ) N
otherwise
0,
271
Noise performance of AM
Signal-to-noise ratio
SNR
PS
PN
SNRbaseband
PT
N 0W
272
Noise performance of FM
FM modulator and demodulator
Method to deal with noise in FM: linear argument at high
SNR
Derivation of the output SNR, threshold effect
Digital communications
PCM: sample, quantize, and encode
3P
Quantization noise and SNR SNRo (dB) 6n 10 log10 2 (dB)
mp
274
1
x2 / 2
Q( x)
e
,x0
2 x
275
Coherent ASK
Q(A/2)
Coherent FSK
Q(A/2)
Coherent PSK
Q(A/)
Noncoherent
ASK
exp(
exp(-A
A2/82)
Noncoherent
FSK
p( 2/42)
exp(-A
DPSK
exp(-A2/22)
Noncoherent
ASK, FSK
Coherent
ASK, FSK
Coherent PSK
Information theory
The entropy of a discrete memoryless information source
K
H ( S ) pk log2 pk
k 1
277
P
C B log 2 1 SNR B log 2 1
N
B
0
278
Channel coding
Block vs. convolutional codes
Binaryy fields and vector space,
p
, Hamming
g distance/weight
g
A linear block code can be defined by a generator matrix
G and the associated parity-check matrix H.
Every linear block code is equivalent to a systematic code.
The key parameter of a linear block code is the minimum
Hamming distance dmin:
Up to dmin 1 errors can be detected;
Up
p to (d
( min 1)/2
) errors can be corrected.
Appendix:
More Background on Probability
Probability
Sample Space
S : Set of all random experiment outcomes
S = { s : s iis an outcome
t
}
Examples
For tossing a coin, S = { H, T }
Roll a die, S = { 1, 2, , 6 }
Event E S
Roll a die twice: S = { (H,H), (H,T), (T,H), (T,T) }
Event E = { (H,H), (T,T) }
A collection of events, F. Obviously, F S
281
Random Variables
-$5
$5
$5
X (s) 0
5
s W
sD
S W D L
sL
P ( X ( s ) 5) P ( s : s W ) P (W )
3
8
P ( X ( s ) 0) P ( s : s D ) P ( D )
1
4
X(s) P ( X ( s ) 5) P ( s : s L ) P ( L )
3
8
CDF : F X ( x ) P ( X x )
pdf : f X ( x )
dF X ( x )
dx
FX ( x )
f X ( y ) dyy
FX ( )
f X ( y ) dyy 1
P ( a X b ) FX ( b ) FX ( a )
f X ( y ) dy
dF X ( x )
dx
0
283
Interpretation of pdf
If x is sufficiently small,
P ( x X x x )
f X ( y)
x x
f X ( y ) dyy f X ( x ) x
Area
f X ( x)
E[ X ] X
Variance:
y f X ( y ) dy
X2 E[( X X )2 ] ( y X )2 f X ( y)dy E[ X 2 ] X2
284
Moments: r E [ X r ]
Average (mean) of X:
for r 1 ,2 ,3 ,...
1 E [ X ] or m X
r ' E [| X m X |r ]
for r 1 ,2 ,3 ,...
C
Comments
1 ' E[ X m X ] E[ X ] m X 0
Variance : 2 ' E[| X m X |2 ]
Standard deviation : X 2 '
which gives a measure of dispersion of X about its mean
285
Let p + q = 1
Bernoulli distribution
P ( X 1) p
P ( X 0) q
Bi
Binomial
i l distribution
di t ib ti
n
P (Y k ) p k q n k
k
k 0 ,1, 2 ,..., n
Exercise: Verify
E [ X ] p , X2 pq
E [Y ] np , Y2 npq
286
f X ( x)
f X ( x ) e x
E
Exercise:
i
V if
Verify
for x 0
E(X )
X2
287
f X ( x)
f X ( x)
FX ( x )
1
2
1
2
( x m )2
2 2
for x
( y m )2
2 2
dy
Exercise: Verify E ( X ) m
2
288
f R (r )
r 2 /( 2 2 )
r0
f R (r )
where
I0 ( x)
1
2
e
2
r
( r 2 A2 ) /( 2 2 )
e x cos d
I 0 ( Ar2 ) r 0
289
Conditional Probability
By definition
N ( A)
as
N
N ( A,B )
B ) N (B)
P ( A)
P(A |
P(A B)
Similarly,
N ( A,B )
N
N ( A,B ) N ( A)
N ( A)
N
P ( B | A)P ( A)
P( A B ) P( A | B )P(B )
P( A | B) P( A) P( A B) P( A)P(B)
290
Random variables :
y
FXY ( x , y )
X ,Y
XY
( u , v ) dudv
f X ( x)
fY ( x )
f XY ( x, y )dy
f XY ( x, y )dx
2 FXY ( x , y )
xy
3)
f XY ( x, y )
4)
X , Y are independen t f XY ( x, y ) f X ( x ) fY ( y )
291
FY ( y | x X x x ) P ( YP (xy, xXXx xx ) x ) xx x
f X ( u ) du
x
f XY ( x ,v ) xdv
f X ( x ) x
As x 0,
FY ( y | x )
f XY ( x ,v ) dv
fX (x)
fY ( y | x )
dFY ( y | x )
dy
fY ( y | x )
f XY ( x , y )
fX (x)
292
X 1 ,..., X n
FX 1 X 2 ... X n ( x1 , x2 ,... xn ) P ( X 1 x1 , X 2 x2 ,... X n xn )
f X1 X 2 ... X n ( x1 , x2 ,...xn )
E[ X i X j ] E [ X i ]E [ X j ] i, j , i j
293
Covariance of X and Y:
cov( X , Y ) E [( X m X )( Y mY )]
Correlation Coefficient:
X ,Y )
XY cov(
Property 1: 1
E [( X m X )(Y mY )]
XY
XY 1
2 2
1
2
1 XY
exp
1
2
2 (1 XY
)
2
( x 2 2 XY xy y 2 )
fY ( y )
XY
( x , y ) dx
1
2
exp{
f XY ( x , y )
fY ( y )
1
2
2 1 XY
y2
2 2
exp
1
2
2 (1 XY
)
2
( x XY y ) 2
Note that this conditional pdf is also a normal density function with
2
2
mean XY y and variance (1 XY ) . The effect of the condition
(having Y=y on X) is to change the mean of X to XY y and to
2
reduce
d
th
the variance
i
b
by 2 XY
.
295
296
f ( x) 21 for 0 x 2
Define random variables X and Y as
X cos Y sin
XY E[( X m X )(Y mY )]
X
E [ XY ]
2
21 0 cos sin d
0!
297
Define S n X i
i 1
E[ X ] ,
X2 2
S n n
n
lim P ( R n x )
n
1
2
y2 / 2
dy
298