DETECTION THEORY

Gram-Schmidt Procedure

Permits the representation of any set of

M energy signals, {si(t)}, as linear
combinations of N orthonormal basis
functions { j (t ) }, where N M

Gram-Schmidt Procedure

Represent the given set of real-valued energy

signal s1(t), s2(t), , sM(t), each of duration T
seconds, in the form
N

si (t ) sij j (t )
j 1

sij si (t ) j (t )dt
0

0t T
i 1,2,...., M

i 1,2,...., M
j 1,2,...., N

Gram-Schmidt Procedure

The real-value basis functions

are orthonormal

1
0 i (t ) j (t )dt 0

1 (t ) , 2 (t ) .
i j
i j

Each basis function is normalized to have unit energy

The basis functions are orthogonal with respect to
each other over the interval 0 t T

Example

Gram-Schmidt Procedure

Stage 1:

We have to establish whether or not the given

set of signal s1(t), s2(t), , sM(t) is linearly
independent
Let s1(t), s2(t), , sN(t) denote subset of linearly
independent signals, where N M each
member of the original set of signals s1(t), s2(t),
, sM(t) may be expressed as a linear
combination of this subset of N signals

Gram-Schmidt Procedure

Stage 2:

Construct a set of N orthonormal basis

functions 1(t), 2(t), , N(t) from the linearly
independent signals s1(t), s2(t), , sN(t)

Geometric Interpretations of
Signals

With orthonormal basis functions {i(t)}, j = 1,

2, , N, each real-valued energy signal s1(t),
s2(t), , sM(t), can be expanded as
N

j 1

0t T
i 1,2,...., M

sij si (t ) j (t )dt

i 1,2,...., M
j 1,2,...., N

si (t ) sij j (t )
T

Geometric Interpretations of
Signals

Geometric Interpretations of
Signals

Example

Solution

Geometric Interpretations of
Signals

Each signal in the set {si(t)} is completely determined

by the vector of its coefficients

si1
s
i2

si


siN
Si is called the signal vector

i 1,2,...., M

Geometric Interpretations of
Signals

Set of signal vector {si}, i = 1, 2, , M as

defining a corresponding set of M points in
a N-dimensional Euclidean space
N-dimensional Euclidean space is called
signal space

Geometric Interpretations of
Signals

In an N-dimensional Euclidean space,

define lengths of vectors and angles
between vectors

Geometric Interpretations of
Signals

Geometric Interpretations of
Signals

Square-length of any signal vector si is define to

be the inner product or dot product of si

s i (s i , s i ) sij2
j 1
The cosine of the angle between the vectors si
and sj equals the ratio
2

(s i , s j )
si s j

Geometric Interpretations of
Signals

Energy of a signal si(t) of duration T seconds is

equal to
T

Ei s (t )dt sij2
2
i

j 1

Euclidean distance between the points

represented by the signal vector si and sk

si s k

s
N

j 1

ij

[s (t ) s (t )] dt
T

skj

Model of a Digital
Communication System

Model of a Digital
Communication System

At the transmitter input, we have a

message source that emits one symbol
every T seconds, denote by m1, m2,, mM
Assume that all symbols are equally likely
1
Pi P(mi )
M

Model of a Digital
Communication System

The output of message source is presented

to a vector transmitter, producing a vector
of real numbers
si1
s
i2
.
si
.
.

si N

Model of a Digital
Communication System

The channel is assumed to have two

characteristics:

The channel is linear, with a bandwidth that is

large enough to accommodate the transmission
of the si(t) without distortion
The transmitted signal si(t) is perturbed by an
additive, zero-mean, stationary, white Gaussian
noise process, denote by W(t)

Model of a Digital
Communication System

We expressed the received random

process, X(t)

X (t ) Si (t ) W (t )

0t T
i 1, 2,..., M

x(t) is referred as the received signal

Model of a Digital
Communication System

The requirements is to design the vector

receiver so as to minimize the average
probability of symbol error defined as

mi )
Pe P(m

is estimate
mi is the transmitted symbol and m
produced by the vector receiver

Model of a Digital
Communication System

If the receiver is phase locked to the

transmitter, it is referred to coherent
receiver
On the other hand, the receiver is referred
to non-coherent receiver

Response of Bank of Correlators

to Noisy Input

Suppose that the input of the bank of N

product-integrator or correlators is
random process X(t)
X (t ) Si (t ) W (t )

0t T
i 1, 2,..., M

W(t) is a white Gaussian noise process of zero

mean and power spectral density N0/2

Response of Bank of Correlators

to Noisy Input

Response of Bank of Correlators

to Noisy Input

The output of each correlator is

T

X i X (t ) j (t )dt
0

sij W j

j 1, 2,..., N

Response of Bank of Correlators

to Noisy Input

The first component, sij, is a deterministic

quantity contributed by the transmitted signal
si(t)
T

sij si (t ) j (t )dt
0

The second component, Wj, is a random variable

that arises because of the presence of noise at
the input
T

W j W (t ) j (t )dt
0

Response of Bank of Correlators

to Noisy Input

Consider next a new random process, X(t)

N

X '(t ) X (t ) X j j (t )
J 1

X '(t ) si (t ) W (t ) ( sij w j ) j (t )
j 1

W (t ) W j j (t )
j 1

W '(t )

Response of Bank of Correlators

to Noisy Input

We may express the received random

process as
N

X (t ) X j j (t ) X '(t )
j 1

X j j (t ) W '(t )
j 1

Response of Bank of Correlators

to Noisy Input

The noise process W(t) has zero mean,

hence the random varialbe Wj extracted
from W(t) has zero mean
The mean value of jth correlator output Xj
depends only on sij
mx j E X j
E sij w j sij E W j sij

Response of Bank of Correlators

to Noisy Input

The variance of Xj
2 x j Var X j
2

E X j sij

E W 2 j

Response of Bank of Correlators

to Noisy Input

2
Xj

T
T

E W (t ) j (t )dt W (u ) j (u )du
0

T T

E j (t ) j (u )W (t )W (u )dtdu
0 0

j (t ) j (u ) E W (t )W (u )dtdu

j (t ) j (u) RW (t , u)dtdu

Response of Bank of Correlators

to Noisy Input

Since
Then

N0
RW (t , u )
(t u )
2

2
Xj

N0 T T

j (t ) j (u ) (t u )dtdu

2 0 0
N0 T 2

j (t )dt

2 0

N0

Response of Bank of Correlators

to Noisy Input

Similarly, we find that Xj are mutually uncorrelated

Cov X j X k E X j mX j

mX k

E X j sij X k sik
E W jWk

T
T
E W (t ) j (t )dt W (u )k (u )du
0

N
0
2
N0

j (t )k (u ) RW (t , u )dtdu
T

j (t )k (u ) (t u )dtdu

j (t )k (t )dt 0

Response of Bank of Correlators

to Noisy Input

Define the vector of N random variables at

the correlator outputs as
X1
X
2
.
X
.
.

X N

Response of Bank of Correlators

to Noisy Input

Since the elements of the vector X are

statistically independent, we may express the
conditional probability density function of the
vector X, given that the signal si(t) or
corresponding the symbol mi was transmitted
f X x | mi f X j x j | mi
N

i 1, 2,..., M

j 1

This is called likelihood functions

Response of Bank of Correlators

to Noisy Input

Since each Xj is a Gaussian random variable

with mean sij and variance N0/2, we have
fX j

2
1
1
x j | mi N exp N x j sij
0

Response of Bank of Correlators

to Noisy Input

Likelihood functions of AWGN channel are

defined by
f x x | mi N 0

N 2

1
exp
N0

x
N

j 1

sij

i 1, 2,..., M

Maximum Likelihood Detector

Suppose that, when the observation vector

has the value x, we make decision

m
m

The average probability of symbol error in

this decision, denoted by Pe(mi,x), is
Pe (mi , x) P mi not sent|x
1 P mi sent|x

Maximum Likelihood Detector

The optimum decision rule is

mi if
set m
p mi sent|x p mk sent|x
for all k i

Where k = 1, 2, , M
This decision rule is referred to as maximum a
posteriori probability (MAP)

Maximum Likelihood Detector

Applying Bayes rule, we may restate this

decision rule as
set m
mi if

pk f x x | mk
fx

is maximum for k =i

Maximum Likelihood Detector

Since fX(x) is independent of the transmitted

signal, and the a priori probability pk= pi when all
the signals are transmitted with equal probability,
we may simplify the decision rule as
mi if
set m
f x x | mk is maximum for k =i

The decision rule is referred to as maximum likelihood, and the

device for its implementation is referred as maximum-likelihood
detector

Maximum Likelihood Detector

Since likelihood function is always

nonnegative, we may restate the decision
rule
mi if
m
ln f x x | mk is maximum for k = i

set

Maximum Likelihood Detector

Let Z denote the N-dimensional space of all

possibly observed vector x. we refer to this space
as observation space.
Because we have assumed that the decision rule
must say m mi , where i = 1, 2, , M, the total
observation space Z is correspondingly
partitioned into M decision regions, denoted by
Z1, Z2, , ZM

Maximum Likelihood Detector

We may restate the decision rule as follow:

vector x lies inside region Zi if
ln f x x | mk is maximum for k = i

Maximum Likelihood Detector

Maximum Likelihood Detector

We may formulate the maximum likelihood

decision rule for AWGN channel as
vector x lies inside region Zi if
1

N0

x
N

j 1

skj

is maximum for k = i

Maximum Likelihood Detector

Equivalently, we may restate

vector x lies inside region Zi if
N

(x
j 1

skj ) is minimum for k = i

2

Maximum Likelihood Detector

Since
N

(x
j 1

skj ) x s k
2

We may rewrite the decision rule

vector x lies inside region Zi if

x s k is minimum for k = i
2

Maximum Likelihood Detector

This show that maximum-likelihood

decision rule is simply to choose the
message point closest to the received
signal point

Probability of Error

The average probability of symbol error

equals
M

Pe P(mi sent)P(x does not lie inside Zi | mi sent)

i 1

1
1
M

P(x lies inside Z

i 1

| mi sent)

Probability of Error
P(x lies inside Zi | mi sent)

Zi

1
Pe 1
M

f
i 1 Zi

(x | mi )dx

( x | mi )dx

Correlation Receiver

The optimum receiver consists of two subsystems

The detector part of the receiver consists of bank of M

product-integrators or correlators supplied with a
corresponding set of coherent reference signals or
orthonormal basis functions
The second part of the receiver, namely the vector
receiver, is implemented in the form of a maximumlikelihood detector that operates on the observation
vector x to produce an estimate m of the transmitted
symbol mi, i = 1, 2, , M, in a way that would minimize
the average probability of symbol error

Correlator Detector

Correlator Receiver

Matched Filter Receiver

Since each of the orthonormal basis functions

1(t), 2(t), , N(t) is assumed to be zero outside
the interval 0 t T may be avoided because
analog multipliers are usually hard to build
Consider a linear filter with impulse response hj(t).
With the receiver signal x(t) used as the filter
input, the resulting output yj(t)

y j (t ) x( )h j (t )d

Matched Filter Receiver

Suppose
h j (t ) j (T t )

The resulting filter output is

y j (t ) x( ) j (T t )d

Matched Filter Receiver

Sampling the output at time t = T, we get

y j (T ) x( ) j ( )d

Since j(t) is zero outside the interval 0 t T,

finally we get
T

y j (T ) x( ) j ( )d
0

Note that yj(t) = xj, where xj is the jth correlator output

produced by the received signal x(t)

Matched Filter Receiver

The detector part of the optimum

receiver may also be implemented as
bank of matched filter
The optimum receiver based on this
detector is referred as the matched filter
receiver

Matched Filter Receiver

Matched Filter

Consider a linear filter of impulse response h(t),

with an input that consist of a known signal, (t),
and an additive noise component, w(t)

x(t ) (t ) w(t )

0t T

T is the observation instant

w(t) is the sample function of a white Gaussian noise
process of zero mean and power spectral density N0/2

Matched Filter

The resulting output, y(t), may be expressed as

y(t ) 0 (t ) n(t )

0(t) and n(t) are prodeuced by the signal and noise

components of the input x(t), respectively

Matched Filter

The output signal component 0(t) be considerably

greater than the output noise component n(t) is to
have the filter make the instantaneous power in the
output signal 0(t), measured at time t = T, as large
as possible compared with the average poser of the
output noise n(t)
This is equivalent to maximizing the output signal-tonoise ratio

( SNR)O

0 (T )

E n 2 (t )

Matched Filter

(f) denote the Fourier transform fo the known

signal (t), and H(f) denote the transfer function
of the filter

0 (t ) H ( f )( f ) exp j 2ft df

When the filter output is sampled at time t = T

0 (T )
2

H ( f )( f ) exp j 2fT df

Matched Filter

The power spectral density SN(f) of the output

noise n(t)
SN ( f )

N0
2

H( f )

The average power of the output noise n(t)

E n (t ) S N ( f )df
2

N0

H ( f ) df

Matched Filter

The output signal-to-noise ratio

SNR O

H ( f ) ( f ) exp j 2fT df
N0
2

H ( f ) df

Matched Filter

According to Schwarzs inequality

H ( f )( f ) exp j 2fT df

SNR O

( f ) df
2

H ( f ) df ( f ) df

The output signal-to-noise ratio

2

N0

Matched Filter

The signal energy given

(t ) dt ( f ) dt

The noise power spectral density N0/2

Matched Filter

Consequently, the output signal-to-noise ratio will

be a maximum when H(f) is chosen so that the
equality holds

SNR O,max

N0

( f ) df
2

The optimum value of this transfer function is

defined by

H opt ( f ) * ( f ) exp j 2fT

Matched Filter

The impulse response of matched filter

hopt (t ) * ( f ) exp j 2f (T t )df

Since for real-valued signal (t), we have

*(f) = (-f)
hopt (t ) * ( f ) exp j 2f (T t )df

T t

Matched Filter

Properties of Matched filter

The spectrum of the output signal of a matched

filter with the matched signal as input is, except
for a time delay factor, proportional to the energy
spectral density of the input signal

0 ( f ) H opt ( f )( f )
* ( f ) ( f ) exp j 2fT
f exp j 2fT
2

Matched Filter

Properties of Matched filter

The output signal of a matched filter is

proportional to a shifted version of the
autocorrelation function of the input signal to
which the filter is matched

0 (t ) R t T

0 (T ) R (0) E

Matched Filter

Properties of Matched filter

The output signal-to-noise ratio of a matched filter

depends only on the ratio of the signal energy to
the power spectral density of the white noise at
filter input

N0
E n (t )
2
2

SNR O,max

N0
( f ) df 2 E

E2
2E

N0 E 2 N0

Matched Filter

Properties of Matched filter

The matched filter operation may be separated

into two matching conditions; namely, spectral
phase matching that produces the desired output
peak at time T, and spectral amplitude matching
that gives this peak value its optimum signal-tonoise density ratio
H ( f ) H ( f ) exp j ( f ) j 2fT
H ( f ) ( f )

Detection of Signals With

Unknown Phase In Noise

Up to this point in our discussion, we have assumed that

the information bearing signal is completely known at the
receiver.
In practice, however, it is often found that in addition to
the uncertainty due to the additive noise of a receiver,
there is an additional uncertainty due to the randomness
of certain signal parameters
The usual cause of this uncertainty is distortion in the
transmission medium. The most common random signal
parameter is the phase, which is especially true for
narrow-band signals.

Detection of Signals With

Unknown Phase In Noise

Synchronization with the phase of the

transmitted carrier may be too costly, and
the designer may simply choose to
disregard the phase information in the
received signal at the expense of some
degradation in the noise performance of
the system

Detection of Signals With

Unknown Phase In Noise

Consider a digital communication system in

which the transmitted signal equals
si (t )

2E
cos 2 fi t
T

0t T

E is the signal energy

T is the duration of the signaling interval
fi is an integral multiple of 1/2T

Detection of Signals With

Unknown Phase In Noise

When no provision is made to phase synchronize the

receiver with the transmitter, the received signal will, for
an AWGN channel, be of the form

2E
x(t )
cos 2 fi t w t
T

0t T

w(t) is the sample function of a white Gaussian noise process of

zero mean and power spectral density N0/2
The phase is unknown, and is usually considered to be the
sample value of a random variable uniformly distributed between
0 and 2 radians

Detection of Signals With

Unknown Phase In Noise

Using a well-known trigonometric identity, we may write

x(t )

2E
2E
cos cos 2 fi t
sin sin 2 fi t w t
T
T

0t T

Suppose that the received signal x(t) is applied to a pair of

correlators; we assume that one correlator is supplied
with the reference signal 2 T cos 2 fit and the other is
supplied with the reference signal 2 T sin 2 fit
For Both correlators, the observation interval is 0 t T

Detection of Signals With

Unknown Phase In Noise

In the absence of noise, we find that the first

correlator output equals E cos and the second
correlator output equals E sin
The dependence on unknown phase may be
removed by summing the squares of the two
correlator outputs, and then taking the square
root of the sum
When the noise w(t) is zero, the result of these
operations is simply E , which is independent of
the unknown phase

Detection of Signals With

Unknown Phase In Noise

The suggests that for the detection of a

sinusodal signal of arbitrary phase, and
which is corrupted by an additive white
Gaussian noise, we may use the so-called
quadrature receiver
This receiver is optimum in the sense that
it realizes this detection with the minimum
probability of error

Detection of Signals With

Unknown Phase In Noise

Detection of Signals With

Unknown Phase In Noise

To obtain the second equivalent form of the

quadrature receiver, suppose we have a filter that
is matched to
2
s t
cos 2 fi t
T

0t T

The envelope of the matched filter output is

obviously unaffected by the value of the phase

Detection of Signals With

Unknown Phase In Noise

The output of such a filter in response to

the received signal x(t) is given
T

2
y t
x cos 2 fi T t d

T0
T

cos 2 fi T t x cos 2 fi d
T
0
T

sin 2 fi T x sin 2 f i d
T
0

Detection of Signals With

Unknown Phase In Noise

The envelop of matched filter output, evaluated

at time t = T, will be

2
2

li x
cos 2 fi d x
sin 2 fi d
T
T
0

0
T

1/ 2

This is just the output of the quadrature receiver.

Therefore, the output at time T) of a filter matched to the
signal s t 2 T cos 2 fit , of arbitrary phase ,
followed by an envelope detector is the same as the
corresponding ouput of the quadrature receiver

Detection of Signals With

Unknown Phase In Noise

Detection of Signals With

Unknown Phase In Noise

The combination of matched filter and

envelope detector is called a noncoherent
matched filter

Detection of Signals With

Unknown Phase In Noise