D7.
8
MULTILAYER PERCEF'TRON STRUCTURES APPLIED TO ADAPTIVE EQUALISERS FOR
DATA COMMUNICATIONS
Gavin J . Gibson, Sammy Siu and Colin F. N. Cowan
Department of Electrical Engineering, University of Edinburgh, Mayfield Rd.,
Edinburgh EH9 31L.
ABSTRACT
The purpose of this paper is to demonstrate the applica-
tion of multilayer perceptrons to the problem of equalisation in
digital communications systems. Following a brief description
of the architecture of the multilayer perceptron we investigate
its capabilities as a channel equaliser and show that they are
superior to those of the linear equaliser. The comparison is
effected by simple examples which show graphically why the
nonlinear architecture provides better results, and also by an
investigation of bit-error rates in a higher dimensional situa-
tion.
1. Introduction
In this paper we investigate the application of a mul-
tilayer perceptron [l] as an adaptive channel equaliser and we
compare its performance with that of the linear transversal
equaliser [Z].
The system which we consider throughout this paper is
depicted in Fig.1.
ni
I EQUALISER I
Fig.1. Schematic of data transmission System
A random sequence xj is passed through a linear dispersive
channel of finite impulse response, here modelled by a FIR
.
filter with response function a, C aiz-' + ' . + a k z + ,
where the coefficients ai, 05j 5 k , are all real valued, to pro-
duce a sequence of outputs, y.,where y, = ~ a j x l - j A. term,
/-1
ni , which represents additive noise in the system, is then added
to each y, to produce an observable sequence 9,. The function
of the equaliser is tn utilise the information represented by the
observed channel outputs 9:. $,-,, ' . . , to produce an
estimate of the input symbol, x,. The integer m is known as
the order of the equaliser. Throughout, the input samples are
chosen from {-1, 1) with equal probability and are assumed to
be independent of one another. The additive noise samples,
a,, are chosen independently from a Gaussian distribution with
mean 0 and variance n2. The above system has been used to
model a variety of communications systems.
One structure employed as an equaliser is the linear
rransversol equaliser which estimates the input symbol x, by
-
i j = sgn(h.$)
1183
where pi denotes the vector of observed channel outputs
e,, . . . ,3,,+i), 6 denotes the vector of equaliser coef-
ficients (bo. bi, ..., b,,,-i), . represents the usual inner product
defined on R" and s g n ( x ) = 1 if x 2 0 and -1 otherwise. As
will be described later a linear transversal equaliser is
inherently incapable of perfectly reconstructing the input
sequence, x,, regardless of its order, when certain conditions
on the channel coefficients, a j are violated, and it is this ina-
bility which provides the motivation to investigate alternative
equaliser architectures.
2. Mullilayer Perceplrons
In this section we give a brief description of the architec-
ture and capabilities of the multilayer perceptron. The basic
building block of the multilayer perceptron is the single neuron
or node which we depict in Fig2 below. A node receives a
number of inputs xi, ..., x . , say, which are then multiplied by
a set of weights wi, ..., wm and the resultant values are
summed. To this weighted sum of inputs is added a constant
9, known as the node threshold and the output of the node is
obtained by evaluating a nonlinear function, f, of the total. In
this paper we restrict our attention to perceptrons where the
node activation function, f, is defined by
f ( x ) = (1 - e-')/(l + e")
the graph of which is shown in Fig. 2.
Fig.2. Node slruclure and activation function
A nrulrilayer perceptron comprises a number of nodes of
the type described above which are arranged in layers, as dep-
icted in Fig.3. A multi-dimensional input is passed to each
node in the first layer. The outputs of the first layer nodes
then become inputs to the nodes in the second layer, and so
on. The output of the network is therefore the outputs of the
nodes lying in the final layer. Thus, weighted connections
exist from a node to every node in the succeeding layer, but no
-m
connections exist between nodes in the same layer.
Z
H s e
-I
C
Fig.3. Multilayer perceplron archileelure
CHt673-2/89/0000-1183S1.00 0 1989 IEEE
We shall describe the architecture of a perceptron by a is nonstationary, such as the H F channel [5], it may become
-
sequence of integers i i 8 1 1 , - . . . - 18, where n o is the necessary to alter this delay if a linear equaliser is used. An
dimension of the input to the network, and the number of equaliser which can deal with both minimum and non-
nodes in each layer, ordered from input to output, is minimum phase channels, without the introduction of any
. delay, would not suffer from this problem and is therefore
duces a nonlinear mapping, g 3 ' -
n , , . . , n,. In this notation, the perceptron therefore pro-
' R"5
In this paper we shall concern ourselves with perceptrons
worthy of investigation. Figures 4 and 5 illustrate the sets
P.(l) and Pm(-l) where m = 1.2 for two different channels,
one minimum phase ( 1.0 + 0.8z-' + 0.5r-' ) and the other
fore produce a mapping g *"*-
which have a single node in the output layer and which there-
R , For such a perceptron, we
non-minimum phase ( 0.5 + l.Oz-' ). Throughout the paper
we represent elements of P, (1) and P, (-1) by the symbols 0
define its decision regiori to be the set of p i n t s and X respectively.

It is clear that for the simplest perceptron, that is a percepon

composed of a single node, the decision region is the half-
space bounded by the hyperplane w.x + 0 = 0, where tt! and m=l
e represent the vector of input weights and the node threshold
rcspectively. It follows that a single layer perceptron is incapa-
ble of correctly classifying inputs as coming from either of two
sets in Euclidean space unless the two sets in question can be
separated by a hyperplane. Using the terminology of [3]. we
say such sets are linearly separable. However if we increase
the number of layers in the perceptron to three we are able to
form decision regions which are considerably more complex
and have highly nonlinear boundaries. It is this latter capabil- rn.2
ity of multilayer perceptrons which will prove invaluable when
we apply them to the problem of channel equalisation.
Multilayer perceptrons can be trained to perform a partic-
ular task by the back-propagation algorithm [4]. This a simple
stochastic gradient descent algorithm whose mode of operation
is summarised below. At the i" iteration a training sample x,
is presented to the perceptron and the resultant output, g(&),
compared with a desired output, d ( g , ) , to produce an error,
e , . The weights and thresholds of the nodes in the network
are then updated in such a manner as to decrease the square of
the error ej , according to the following equations.

Fig.4. Example of minimum phase channel outputs with

separating hyperplane for m =2.

where B is the adaptive gain and a is a momentum parameter

which "smooths out" high frequency variations in the
weight/threshold vector during training.
rn-1
-3
I r - . 0, . " --
3
3. The Equalisation Problem
In this section we consider in more detail the equalisation
problem described in the introduction and we shall employ the
definitions and notation introduced therein. We begin by
describing how the performance of the linear transversal
equaliser is affected by firstly the coefficients of the channel to
be equalised, and secondly the power of the additive noise. In
order to relate the channel coefficients to the complexity of the
problem we assume that additive noise is absent, so that m= 2
n2 = 0 and we define

with P,(l) equivalently defined. Thus P , ( l ) and P M ( - l )

represent the sets of possible channel output vectors
(y,, ..., ytn+') which can be produced from sequences of
channel inputs beginning with x, = 1 and x, = -1
respectively. It is clear that a linear transversal equaliser can
successfully reconstruct the input sequence .ti if and only if the
sets P. (-1) and P,(1) are linearly separable for some integer
rn . By an argument involving convex sets it can be demon-
strated that this latter condition is satisfied precisely when the
roots of the polynomial
aGk + alzk-' + . . . + a,
all lie strictly within the unit circle in the complex plane.
Channels whose impulse response functions satisfy this condi-
tion are said to be nrinimumphase. At this point we remark
that nonminimum phase channels can be equalised by linear
equalisers if some delay, d , is introduced in the calculation of
xi so that, on the irh iteration, the equaliser estimates the
input symbol x i d . However, in situations where the channel
-3 1
li
Fig.5. Example of nonminimum phase channel outputs
It is now clear that in order to design structures to equalise
non-minimum phase channels we must look to architectures
which are more sophisticated than the linear transversal
equaliser. Now the linear transversal equaliser and the basic
perceptron node are essentially similar in their decision making
capabilities, a fact which suggests that the multilayer percep
tron is a natural candidate to consider in the search for this
required sophistication.

1184
 Further support for employing structures capable of form- (NLMS) algorithm and the back-propagation algorithm respec-
ing decision regions with nonlinear boundaries as channel tively. In all the simulations considered the following system
equalisers is provided by considering the effect of the additive was employed.
noise on the problem. When significant amounts of noise are
present the observed channel output vectors will be distributed
as a sum of m -dimensional Gaussian distributions, with vari-
ance I? in each component, centred on the noise free channel
output vectors represented by P m ( l ) and P - ( - l ) . The
equaliser must therefore make a decision as to whether an
observed channel output vector represents the "noise corrup-
tion" of an element of P..(1) or P . (-1). Given knowledge of
the channel to be equalised and the power of the additive
noise it is possible to design a theoretically "optimal" equaliser,
of a given order m , for the channel as follows. Let /,(y) be
the p.d.f. of observed channel outputs corresponding to Zctual
channel output vectors in P.(1) with f - l ( y ) similarly defined.
Now consider the equaliser which calculates its estimate of the EOURLISER
input sample xj , given a vector of observed channel outputs 2,.
according to
:x. = .WUlLv,) -f-,CV,)).
It is a simple matter to show that the above equaliser is optimal
for its order in the sense that it minimises the probability of a
wrong decision in the estimate of x , . In certain low dimeo-
sional situations it is possible to represent the decision boun-
dary associated with this optimal equaliser graphically. Fig.6 Fig.7. System employed for adaptive cquallsrtion
depicts the optimal decision boundary for an equaliser which
utilises two succwive observed channel outputs si,$,-1 in cal- The first simulation results presented demonstrate the per-
culating the estimate of x , , together with the sets P,(1) and formance of a multilayer perceptron employed to equalise the
+
P z ( - l ) for a channel with impulse response 1.0 O.Sz-' and channel with response function 0.5 + 1.01-I. As this channel
noise power given by n2 = 0.2. is nonminimum phase, a linear equaliser is incapable of recon-
structing the input sequence ( without the introduction of some
delay ). In Fig.8 we show the decision region formed by a
2-6-4-1 perceptron after a training run of loo0 samples. The
3T I t power of the additive noisc was 0.01, representing a signal to
noise ratio of approximately 21dB. The shaded area represents
x-1 the set of points in R 2 which when presented as inputs result in
a positive output from the perceptron, corresponding to
$, = 1. It is clear that for this situation P,(1) lies within the
1.5
t shaded region and P z ( - l ) lies in its complement indicating
that the perceptron is correctly reconstructing the input
sequence. Of course errors will occur if the additive noise
components are such as to cause the observed channel output
vector (g,, il-l)to lie in the 'krong" region, but such
occurrences will be infrequent in good signal to noise condi-
tions.

1.5 -
++++++++ ++++
%-I::++ ++++++ +++++
+++++++++ ++++++
++++++++ ++++++
1.5.. +++++++ + x ++)++++
+++++++ ++++++++
++++++ ++++++++
-3 -1.5 0 1.5 Y; ++++++ +++++++++
+++++ ++++++++++
0.5- x ++e+ +.+++++++++
Fig.6. Example of optimal decision boundary for equaliser ++++ +++++++++++
+ +++++++++++
of order 2 ++++++++++
++++++++
It is clear that this boundary is highly nonlinear and deviates
markedly from any decision boundary which can be formed by
a linear transversal equaliser of order m =2. This provides
further justification for the use of multilayer perceptrons, with
their nonlinear decision-making capabilities, as channel
equalisers.

4. Simulation results
In this section we present the results of computer simula-
tions which demonstrate how a multilayer perceptron can
operate as an adaptive channel equaliser and compare its per-
formance with that of a linear transversal equaliser of similar
order. The algorithms employed to train the linear equaliser Fig.8. Decision region formed by perceptron equaliser (non-
and the perceptron were the normalised least mean square minimum phase channel)

1185
 The following figure shows the decision region formed by
a 2-9-5-1 perceptron after being trained to equalise the channel
with response 1.0 + 0.5s-'. The power of the additive noise,
nz, was 0.2, representing a signal to noise ratio of approxi-
mately 8dB. The solid curve in Fig.9 shows the position of the
decision boundary formed by the optimal equaliser discussed in
the previous section. It can be seen that the decision region
formed by the perceptron coincides in the region of interest
with that of the optimal equaliser displayed in Fig.6, which
suggests that the perceptron is utilising the available informa-
tion with something approaching maximum efficiency.
1.5D
' X '
0.00
-1.50
-3.D O
-3.00 -1.50 0.00 1.50 j?i 3.00
Fig.9. Decision region formed by perceptron equaliser
(minimum phase channel)
The last simulations to be presented demonstrate quanti-
tatively the advantages offered by the perceptron equaliser over
the linear structure. This is obtained by comparing the bit
error rates achieved by the respective equalisers, over a wide
range of signal to noise ratios, when they are set to the task of
equalising the channel with response function
0.3482 + 0.8704r-' + 0.3482s-', which is more representative
of the kind of channels encountered in practical situations than
the simple illustrative examples considered above. Io order to
allow the linear equaliser to successfully reconstruct the input
signal a delay of one sample interval is introduced in the esti-
mate of xi ( so that at the i" iteration the equaliser calculates
an estimate of x , - ~ ) . The equalisers used in these simulations
are a 5-9-3-1 perception and a linear transversal equaliser of
similar order. The results of these simulations are displayed in
Fig.10 and they show clearly that the perceptron equaliser
enjoys a lower bit error ratio than the linear equaliser across
the range of signal to noise ratios considered.
1186
O 7----t
-51 I I I I I I
0 5 1 0 1 5 2 0 2 5 3 0
SIGNAL TO NOISE RATIO (dB )
Fig.10. Comparison of bit error ralios achieved by linear
and perceptron equalisers
5 . Conclusions
The multilayer perceptron offers advantages over the
linear transversal equaliser in a number of areas. Its ability to
form decision regions which have nonlinear boundaries enables
it to equalise both minimum phase and nonminimum phase
channels without the introduction of any timing delay, a capa-
bility which may be of value in nonstationary environments.
Furthermore, and for the same reasons, the perceptron is less
susceptible to the effects of high levels of additive noise. How-
ever the perceptron architecture and the training algorithm
used are considerably more complex than the linear equaliser
and its associated training algorithms.
ACKNOWLEDGEMENTS
This research was supported by the British Science and
Engineering Research Council, the National Science Council
of the Republic of China and Telecommunications Labs.,
Taiwan.
REFERENCES
[!] Minsky, M., and Papert, S., "Perceptrons: An Introduc-
tion to Computational Geometry", MIT Press. 1969.
[2] Qureshi, S. U. H., "Adaptive Equalisation", Proc. IEEE,
vo1.73, no.9, pp.1349-1387, Sept. 1985.
[3] Widrow, B., Winter, R. G., and Baxter, R. A., 'Layered
Neural Nets for Pattern Recognition", IEEE Trans. Acoust.,
Speech, Signal Processing, vo1.36, 110.7, pp.1109-1118, July
1988.
[4] Rumelhart, D. E., Hinton, G. E., and Williams, R. J.,
"Learning Internal Representations by Error Propagation",
Learning Internal Representations by Error Propagation", in
Rumelhart, D. E. and McClelland, J.L. (eds), "Parallel Distri-
buted Processing: Explorations in the Microstructure of Cogni-
tion", MIT Press, 1986.
151 Proakis, J. G . , "Digital Communications", McGraw Hill,
1983.