USING FISHERS LINEAR DISCRIMINANT

TO CLASSIFY FEATURE VECTORS OF ECG

SIGNALS
Mirjam Jonkman, Aya Matsuyama, Mohamed Elgendi, Friso de Boer
School of Engineering.
Charles Darwin University
Darwin, Austrailia.
Phone: +61-(0)-8-89-46-6671 Fax: +61-(0)-8-89-46-6680.
E-mail: mirjam.jonkman@cdu.edu.au

Abstract: In this paper a method is presented to classify

normal and abnormal ECG signals. It employs a
combination of wavelet decomposition, feature extraction
and classification methodology using Fishers linear
discriminant.
It is shown that the method is effective for quantifying the
classification of ECG abnormalities. The results indicate
that successful classification relies on the first two wavelet
approximations. Further decomposition leads to less
accurate classification results. It has also been shown that
selection of a suitable wavelet is critical. It could be
concluded that the bior3.1 wavelet is not suitable for this
method of classification of ECG signals.
Keywords: ECG, feature vectors, wavelets, classification, Fishers
linear discriminant

1. INTRODUCTION
The Electrocardiogram (ECG) is one of the most effective diagnostic
tools to detect cardiac diseases. Traditionally skilled physicians analyse
ECG recordings in the time-domain. However, ECG recording in the
frequency domain has also been studied for subtle pathological
conditions which may not always be obvious in the original time
domain [1-3]. Signal processing techniques for the information in the
frequency domain include Fourier transforms and wavelet transforms.
The latter overcomes the important limitation of Fourier transforms,
which is uncertainty of the information in time after the transform [4].
The wavelet transform has been applied to the ECG for a wide range of
purposes: feature extraction [5-8], feature detection [9-13], noise
reduction [14], and data compression [15].
The combined technique of wavelet decomposition and feature
extraction was previously applied to an ECG signal to separate normal
beats and abnormal beats [7, 8]. However, the separation was not
quantified.
In this paper, we combine the previously described technique with a
classification using Fishers linear discriminant, leading to a
quantitative classification system.

2. ECG SIGNAL PROCESSING

Figure 1 shows the ECG signal processing flow. The ECG signals
were first decomposed with the wavelet transform, after which feature
vectors were extracted. These feature vectors were used to classify the
signal. The details of each stage are described next.

Original
ECG
signals

Wavelet
Transfer

Decompose
d
ECG
signals

Beat
Classification

Feature
Extraction

Feature
Vectors
of
the ECG
signals

Figure 1. ECG Signal Processing Flow

2.1 Wavelet Decomposition

By applying the wavelet transform, ECG signals were decomposed
to the approximate (low frequency components) and detail information
(high frequency components) [16], refer to Figure 2.
Original
ECG
signals

Wavelet
Transfer

Approximate
Information

Detail
Information

(Low
Frequency)

(High
Frequency)

Figure 2. ECG Signal Decomposition

All ECG signals were obtained from the Physionet Database [17].
Using the wavelet packet decomposition command wpdec in Matlab
[18], each ECG signal was decomposed to Level 4. To investigate the
suitability of the type of wavelet for ECG signal analysis, several types

of wavelets were applied (db2, db4, bior1.3, bior3.1, bior5.5, bior6.8).

The start of the QRS complex was defined as the beginning of each
beat and normal beats which occurred immediately before or after
abnormal beats were removed.

2.2 Feature Extraction

Two features were extracted from the decomposed ECG signals,
normalised energy and entropy. The purpose of feature extraction is to
select and retain relevant information from the original signals.
2.2.1 Normalised Energy
The normalised energy at decomposition level n for each beat was
calculated as variance

E ( j )n

1 N
(xi m)2
N 1 i 1

(1)

(j: beat number, N: number of samples in one beat, i: sample number, n: decomposition
level, m: sample mean)

This energy of each beat, was then normalised across the

decomposition levels, which allows comparison between the
decomposed signals in different levels. The normalised energy of the
beat j at decomposition level n is defined as:

E ( j )norm _ n

E ( j )n
E(j

2
)1

E ( j )2 ... E ( j )n

(2)

(j: beat number, n: decomposition level)

2.2.2 Entropy
In signal processing, the entropy can be viewed as a measure of
uncertainty [19]. The classical log energy entropy was used in this
study. The entropy of the beat j at decomposition level n was obtained
as follows.
N

Ent( j )log_ n log(x i )

i 1

(3)
(j: beat number, n: decomposition level, N: sample size, i: sample number)

2.2.3 Feature Vectors

Each beat of the decomposed signals at each decomposition level
now has two features: normalised energy: E ratio( j )n and entropy:
Entratio( j ) n . These feature vectors of normal and abnormal beats
compose different groups of vector points, known as clusters. We have
obtained feature vectors for six different wavelets (db2, db4, bior1.3,
bior3.1, bior5.5, bior6.8) and four decomposition levels.

3. CLASSIFICATION
The purpose of classification is to assign an object to a certain class.
Many classification methods have been described [20]. Here we use
Fisher's linear discriminant [21]. Fishers linear discriminant is
particularly useful for discriminating between two classes in a
multidimensional space. Since it is based only on the first and second
moments of each distribution, it is not a computationally intensive
method. A limitation is that it assumes that the two classes are Gaussian
with equal covariance [22]. If this is not the case the discriminant may
not give the minimum classification error.
3.1 Fishers Linear Discriminant
Fishers linear discriminant is a classification method that projects
high-dimensional data onto a line and performs classification in a onedimensional space. The objective of the method is to reduce the
dimensionality while preserving as much of the class discriminatory
information as possible. Let the feature vectors be denoted by x .
Fishers linear discriminant is defined as the linear function y w T x
that maximizes the criterion function [23]
~

J ( w)

| 1 2 |2
~2

(4)

~2

S1 S 2
~2

~2

where (S 1 S 2 ) is the total within-class scatter of the projected

~

samples and 1 2 is the difference of the projected means. The

projection results in the optimum separation of the two classes in one
dimension.
3.2 Training

The training phase consisted of applying Fishers linear discriminant

to a half of the feature vectors for a particular wavelet and
decomposition level. It was necessary to take the logarithm of both
coordinates of the feature vectors before applying Fishers linear
discriminant in order to produce an approximately normal distribution.
The threshold, which would be used for classification, was defined as
the average of the projected means of the classes.
3.3 Testing
The testing phase consisted of applying the previously found linear
discriminant to a new set of feature vectors. Different wavelets and
decomposition levels were applied to a number of ECG recordings. A
record was kept of all classified heartbeats and true positive, true
negative, false positive and false negative rates were calculated for each
patient and for each beat.

4. RESULTS
The results indicate that the method described above does indeed
result in a good separation of the classes in the testing phase. In fact, all
abnormal beats were correctly classified (there were no false
negatives). However a number of normal beats were classified
incorrectly. This may be caused by a variance difference between the
projections of the normal and abnormal beats.

Figure 3. Wavelets used for analysis

To evaluate the suitability of a variety of wavelets for classifications

purposes, results were calculated for each beat (number of incorrectly

classified beats) and each patient (number of incorrectly classified

patients, one incorrectly classified beat of a patient will result in an
incorrectly classified patient). The db2, db4, bior1.3, bior3.1, bior5.5,
bior6.8 wavelets were used for the analysis since they are commonly
used. Figure 3 shows the shape of the various wavelets involved.
Previous research [7, 8] indicates that successful separation of
normal and abnormal beats can be developed using the various levels of
approximation of the wavelet decomposition and approximation levels
were therefore varied from level 1 to level 4.
Table 1. False positive classification using the bior1.3 wavelet
Wavelet approximation level bior1.3
wavelet
A1
A2
A3
A4
False positive
mis-classifications of patients
[No]
False positive
mis-classifications of beats [%]

0.09%

0.00%

26.13%

22.22%

Table 1 shows the result for 9 patients using the bio1.3 wavelet. At
the A1 decomposition level two patients had normal beats incorrectly
classified as abnormal. However, only 0.09% of the normal beats were
incorrectly classified as abnormal beats. Furthermore, it can be seen
that approximation levels A3 and A4 do not lead to good results any
more as 3 patients for the A3 level and 4 patients in the A4 level were
incorrectly classified and about a quarter of all beats were incorrectly
classified (or about half of the beats of the incorrectly classified
patients). This indicates that A3 and A4 do not have the enough relevant
information of the abnormality to accurately classify.
Table 2 presents the aggregated results for all wavelets. It shows
that, on average, 9.3% of the patients are incorrectly classified using the
A1 approximation and increasing to 42.6% using the A4 approximation.
A similar trend exists for incorrectly classified beats, increasing from
2.0% to 38.9%. Clearly the A3 and A4 approximations of the signal do
not contain enough information to successfully classify ECG signals.

Table 2. False positive classification of aggregated results

Wavelet approximation level
A1
A2
A3

A4

False positive
mis-classifications of patients
[%]

9.26%

7.41%

33.33%

42.59%

False positive
mis-classifications of beats [%]

2.02%

2.04%

23.18%

38.89%

After further analysis of individual wavelet results, it became

apparent that there was a significant difference in classification
accuracy between the bior3.1 and the remaining wavelets. The results
for the bior3.1 wavelet are shown in Table 3.
Table 3. False positive classification using the bio3.1 wavelet
Wavelet approximation level - bior3.1
wavelet
A1
A2
A3
A4
False positive
mis-classifications of patients
[No]
False positive
mis-classifications of beats [%]

12.03%

12.26%

33.59%

33.33%

It can be concluded that the bior3.1 wavelet is unsuitable for ECG

classification techniques.

5. CONCLUSIONS
The combination of wavelet decomposition and feature extraction,
using normalised energy and entropy with classification using Fisher's
linear discriminant is an effective method for quantifying the
classification of wavelet analysis of ECG abnormalities. In particular it
has been shown that the classification relies on information present in
level A1 and A2 of the wavelet decomposition and that A3 and A4 do
not include this information any more. Furthermore, it has been shown
that selection of a suitable wavelet is critical to the success of
classification. It was shown that bior3.1 is not suitable for this method
of classification of ECG signals.

REFERENCES
[1] J. Millet-Roig, J. J. Lopez-Soriano, A. Mocholf, R. Ruiz-Granell, and F. J. Chorro,
"Study of frequency and time domain parameters extracted by means of wavelet
transform applied to ECG to distinguish between VF and other arrhythmias,"
presented at Computers in Cardiology 1998, 1998.
[2] A. Spaargaren and M. J. English, "Analysis of the signal averaged ECG in the
time-frequency domain," presented at Computers in Cardiology 1999, 1999.
[3] G. Sierra, P. Morel, P. Le Guyader, F. Trellez, R. Nadeau, and P. Savard,
"Frequency analysis of the signal-averaged ECG of postinfarction patients for
prediction of cardiac death," presented at Engineering in Medicine and Biology
society, 1997. Proceedings of the 19th Annual International Conference of the
IEEE, 1997.
[4] C. Valens, "A Really Friendly Guide to Wavelets," vol. 2003, 1999.
[5] G. K. Prasad and J. S. Sahambi, "Classification of ECG arrhythmias using multiresolution analysis and neural networks," presented at TENCON 2003. Conference
on Convergent Technologies for Asia-Pacific Region, 2003.
[6] B. Castro, D. Kogan, and A. B. Geva, "ECG feature extraction using optimal
mother wavelet," presented at Electrical and Electronics Engineers in Israel, 2000.
The 21st IEEE Conference, Tel-Aviv, Israel, 2000.
[7] A. Matsuyama and M. Jonkman, "The Application of Wavelet and Feature Vectors
to ECG Signals," presented at IEEE, Tencon 2005, Melboune, Australia, 2005.
[8] A. Matsuyama and M. Jonkman, "The Application of wavelet and feature vectors
to ECG signals," Australian Physical & Engineering Sciences in Medicine, vol. 29,
pp. 14-17, 2006.
[9] S. M. Szilagyi, L. Szilagyi, and L. David, "Comparison between neural-networkbased adaptive filtering and wavelet transform for ECG characteristic points
detection," presented at Engineering in Medicine and Biology society, 1997.
Proceedings of the 19th Annual International Conference of the IEEE, 1997.
[10] T. Stamkopoulos, K. Diamantaras, N. Maglaveras, and M. Strintzis, "ECG analysis
using nonlinear PCA neural networks for ischemia detection," Signal Processing,
IEEE Transactions on [see also Acoustics, Speech, and Signal Processing, IEEE
Transactions on], vol. 46, pp. 3058-3067, 1998.
[11] D. Bojanic and D. B. Popovic, "QRS detection from an ongoing ECG recordings
by using dyadic wavelets," presented at EMBEC 02, Vienna, 2002.
[12] F. Schlegelmilch, M. Helbig, V. Natchkova, G. Ivanova, K. Schellhorn, V.
Detschew, D. Mandler, and G. GrieBbach, "A Multi-Channel Approach for OnlineIdentification of QRS-Regions Using Adaptive Recursive Estimators," presented at
EMBEC 02, Vienna, 2002.
[13] A. Mousa and A. Yimaz, "Neural network detection of ventricular late potentials in
ECG signals using wavelet transform extracted parameters," presented at
Engineering in Medicine and Biology Society, 2001. Proceedings of the 23rd
Annual International Conference of the IEEE, 2001.
[14] L. Senhadji, J. J. Bellanger, G. Carrault, and J. L. Coatrieux, "Wavelet Analysis of
E.C.G. Signals," presented at the Twelfth Annual International Conference of
IEEE, Philadelphia, 1990.
[15] I. Provaznik and J. Kozumplik, "Wavelet Transform in Electrocardiography - Data
Compression," International Journal of Medical Informatics, vol. 45, pp. 111-128,
1997.

[16] P. O. Ogunbona, M. Milliss, F. De Boer, and M. Fernandes, "Classification of Gas

Metal Arc Welds Using Wavelets," presented at Engineering Mathematics and
Applications Conference, Adelaide, Australia, 1998.
[17] "PhysioBank," vol. 2004: Physionet.
[18] M. Misiti, Y. Misiti, G. Oppenheim, and J.-M. Poggi, Wavelet Toolbox User's
Guide: Mathwork, 2000.
[19] S. Tong, A. Beserianos, J. Paul, Y. Zhu, and N. Thakor, "Nonextensive entropy
measure of EEG following brain injury from cardiac arrest," Physica A, pp. 619628, 2002.
[20] Anil K. Jain, Robert P.W Duin, and Jianchang Mao. Statistical Pattern
Recognition: A Review. IEEE Transactions on Pattern Analysis and Machine
Intelligence, Vol. 22 No. 1, 2000.
[21] Fisher, R.A. The Use of Multiple Measurements in Taxonomic Problems. Annals
of Eugenics, 7: 179-188 (1936)
[22] T. Cooke and M. Peake, The Optimal Classification Using a Linear Discriminant
for Two Point Classes having Known Mean and Covariance, IEEE Transactions
On Pattern Analysis And Machine Intelligence, Vol. 24, No. 2 2002
[23] R.O. Duda, P.E. Hart and D.G. Stork, Pattern Classification, New York: John
Wiley & Sons, 2001.

10