Robust Face Recognition Approaches Using PCA,
ICA, LDA Based on DWT, and SVM algorithms
Zied Bannour Lahaw Dhekra Essaidani
Products Research Centre (CEREP) Products Research Centre (CEREP)
ENSIT, 5 Av. Taha hussein, 1008, ENSIT, 5 Av. Taha Hussein, 1008,
Tunis, Tunisia Tunis, Tunisia
ziedd.bannour@gmail.com dhekraessaidani89@gmail.com
Abstract—Face Recognition represents one of the attracting
research areas. It has drawn the attention of many researchers
due to its varying applications such as database identification,
identity authentication, surveillance and security access control
human computer interaction…. In this order, different face
recognition algorithms are proposed. These methods are based
on the face representations. In order to use the most important
information in the face representation, we propose to exploit the
concept of 2D-DWT for the image compression as a pre-
processing for feature extraction. In fact, the DWT is performed
at different scales and orientations so it is sensitive to strong
lighting conditions and facial details. Based on the DWT
advantages, the LL sub-band of the processed image is used as
input image for feature extraction process based on ICA, PCA,
LDA and SVM algorithms. The importance of the proposed
approaches is evaluated by experimental and comparative studies
tested on AT&T database.
Keywords—Face Recognition; Discrete Wavelet Transform ;
Principal Component Analysis; Linear Discriminant Analysis;
Independent Component Analysis; Support Vector Machine
I. INTRODUCTION
In the last few years, face recognition represents an
important research issue in person identification and pattern
recognition projects. It has been used in many applications
such as: terrorists, criminals, and missing children. In this
order, several approaches have been developed such as
Principal Component Analysis (PCA) [1], Linear Discriminant
Analysis (LDA) [2], Independent Component Analysis (ICA)
[3], Local Binary Pattern [4], Extreme Machine Learning [5],
Discrete Wavelet Transform [6] for features face extraction
and classification. In 2016, M. Chihaoui et al. presented an
overview of some well-known methods in 2D face recognition
techniques. They classified these techniques into three
categories according to the used method for image treating.
These categories are global, local and Hybrid approaches.
Also, they realized a comparative study between different face
recognition approaches [7]. Similarly, A. Bouzalmat et al.
represented in [8] a comparative study between three face
recognition schemes on AT&T database. The first one used
the PCA for feature extraction and the Support Vector
Machine (SVM) for classification. The second one used the
ICA for feature extraction and the SVM for classification. The
978-1-5386-4695-3/18/$31.00 ©2018 IEEE 413
Hassene Seddik
Products Research Centre (CEREP)
ENSIT, 5 Av. Taha Hussein, 1008,
Tunis, Tunisia
seddik-hassene@ieee.org
third one used the LDA for feature extraction and the SVM for
classification. The several experimental studies are performed
on the AT&T database. Also, G.Linge and M.Pawar proposed
a face recognition approach based on the PCA algorithms for
feature extraction and the neural network algorithm for
eigenvalues vectors classification [9]. Moreover, H. Makwana
and T. Singh presented a comparative study between the PCA
and LDA face recognition methods. This study is applied to
the ORL database. The author compared the accuracy, the
training time and the total execution time between these
approaches [10]. Also, S. K. Jameel Proposed a new face
recognition system based on the PCA, the DCT and the
Hidden Markov Model (HMM) techniques [11]. Y. Liu et al.
proposed a Face recognition approach based on independent
component. The features are extracted from the low-
frequency band of different levels of 2D-DWT by using the
2D-ICA. The authors used the FSVM algorithm for data
classification. For the experimental study, they used the ORL
and YALE databases [12]. Moreover, R. Sharma and M.S
Patterh combined the PCA, LDA and ICA algorithms with the
Adaptive Neuro Fuzzy Interface System (ANFIS) to develop
three pose invariant face recognition approaches. The PCA,
LDA and ICA are used in features extraction process and the
ANFIS is used for classification process [13]. S. A. Z.
Bagherzadeh, A. N. Sarcheshmeh et al. proposed a new
hybrid face recognition algorithm based on Discrete Wavelet
Transform (DWT) and the LDA algorithm for dimensionality
reduction, and the support vector machine for classification
[14]. S. N. Borade et al. implemented a new face recognition
algorithm by using the ICA and InfoMax algorithms [15].
Also, J. Kim et al. proposed a new recognition by parts
approach based on ICA method [16].
In order to develop an effective approach for face
recognition, we propose and compare, in this work, three
approaches based on the combined DWT, ICA, PCA, LDA
and SVM algorithms. The performances of these approaches
are carried out by simulation and comparative study on AT&T
database[17]. The proposed paper is organized as follows: the
following section describes the three proposed approaches and
the details of the used algorithm. In the third section, we
present the obtained results and a comparative study is carried
out in the section 4.
TSP 2018
 Fig. 2. Principle of the image’s decomposition by the 2D-DWT at level 2 :
(a) Original image; (b) One level; (c) Two level.

B. Principal Component Analysis (PCA)

The Principal Component Analysis is used for data
representation with a high dimensional space. The used steps
for feature extraction are as follows [1]:

Fig.1: Diagram representation of the proposed face recognition system Each image I i with size ( m × n ) is decomposed to three
vectors Γ1 , Γ 2 , Γ3 , .., Γ M , where M represents a set of face
II. PROPSED FACE RECOGNITION APPROACH
images. The mean of these images is given by:
All face recognition algorithms are composed by three steps:
pre-processing image’s face, feature extraction and face 1 M
classification. The feature extraction based on PCA, ICA and Ψ= ¦ Γi (1)
M i =1
LDA algorithms are more efficient where the input image has
a most details representation. For this reason, we propose to The covariance matrix is given by:
use the discrete wavelet transform as image’s pre-processing
to generate good accuracies. The localized nature of the DWT 1 M
T T
C= ¦ Φ i Φ i = AA (2)
allows the easy extraction of features such as spikes, edges M i =1
and objects. Also, it measures the variations around a small
region of the data array. For these reasons, we propose to Where Φ i = Γ i − Ψ .
extract the features from the sub-band of the important
information (LL) by using the PCA, ICA or LDA techniques. The first k eigenvectors is corresponding to the largest
Finally, we have used the SVM algorithm for feature eigenvalues.
classification. Based on these methods, we present three face C. Linear Discriminant Analysis (LDA)
recognition approach: the first one is based on 2D-DWT +
Linear Discriminant Analysis (LDA) is developed in 1936
PCA+ SVM algorithms. The second one is based on 2D-DWT
by Robert Fisher for taxonomic classification. It is very used
+ ICA+ SVM algorithms. The first one is based on 2D-DWT
in pattern recognition [2]. Let a set of C-class where each
+LDA+ SVM algorithms. The proposed face recognition
schemes are resumed in the following figure: class C j as N j face images, the scatter matrices SW (within-
The different used methods for these approaches are
class) and S B (between-class) are calculated as:
recalled in the following subsections.
C Nj

A. Two Dimensional Discrete Wavelet Transform (2D-DWT) Sw = ¦¦ (ī

j=1 k=1
k - Ȍ C j )(ī k - Ȍ C j )
T
(3)
The wavelet transform is used almost as widely as analyze
signals and images. Its ability to capture localized time- C
frequency information of image motivates its use for feature
extraction. The 2D-DWT divides the information of an image
SB = ¦ (Ȍ
j=1
cj
T
- Ȍ) (Ȍ c j - Ȍ)
T
(4)
into decomposing images to approximate sub-band Low-Low
(LL) and three detail sub-bands Low-High (LH), High-Low Where Ȍ is the total mean and M is the set of all face images:
(HL), High-High (HH). They represent, respectevely, the 1 M
horizontal, vertical and diagonal details of the original image.
The approximate sub-band corresponds to the low frequency
Ȍ=
M i=1
īi ¦ (5)

information which contains the most important information

[18]. The following figure illustrates two-level resolutions in a And Ȍ c j is the mean of class C j :
2D-DWT.

414
 Nj n
1 1
Ȍ =
cj Nj ¦ī
k=1
k (6) maximize - ¦ Įi Į j yi y j K( xi , y j ) + ¦ Įi
2 i, j i =1
(9)

The objective of LDA is to find an optimal projection Wopt by Such that:

n
maximizing the between-class scatter S B and minimizing the ¦ α i yi = 0 (10)
i=1
within-class scatter SW :
Where x i are the input vectors and y are the class labels,
i
T
W SB W 0 ≤ Į i ≤ c , Į i are the Lagrange multipliers parameters to be
Wopt = arg max W (7)
WT S W W adjusted, i=1,2...n , c is the penalty parameter of the
classification error term and K( xi , y ) is the kernel matrix.
j
The solution for this optimization is to solve the
We chose RBF kernel function for classifier in our face
-1
eigenvectors and eigenvalues of SW S B . Similar to PCA, we recognition.
2
used the first K eigenvectors in the set for face projection. K(x i ,y j )=exp(- Ȗ x i ,x j (11)
D. Fast Independent Component Analysis
Where Ȗ 0 is kernel parameter and parameterized by using
Independent Component Analysis (ICA) is a statistical and
computational technique for separate independent sources 1
Ȗ= .
from mixed sources. It is successfully used for feature 2ı 2
extraction in image processing [19]. Let the observed The output of the classifier for an unknown x is given by:
T N

¦ Į y K(x , x) + b)
variable x i a vector X = [x , x ,..., x ] and the variables s i a
1 2 m y = sign ( i i i (12)
vector S=[s1 ,s 2 ,...,s n ]T . The relation between X and S is: i=1

X=AS (8) Where N is the number of non-zero’s Į in the results of the

previous optimization problem.
Where A has the size ( m× n ) . It represents the mixing
matrix. In feature extraction, each column in A represents a
III. EXPERIMENT RESULTS
th
features and s i is the coefficient of the i feature in the We have used the AT&T face database and the platform of
observed data vector X . The goal of this technique is therefore Phyton 2.7 for simulation. This database contains 400 face
T images of 40 subjects, each people have 10 images which are
to find the separation matrix w x . For this, the algorithm uses captured at different times and several positions and
the fixed-point algorithm [19]. If we consider that each output conditions with subjects wearing sunglasses. These image are
T
of the network is given as a function of type G ( w x ) , where gray scale with size (112×92). Four different samples of the
G is non quadratic function. The simplified form of the AT&T database are shown in Fig.2.
algorithm for the estimation of an independent component can
be reformulated as follows:
1. I initialize the weight vector w.
T ' T
2. Let w = E ( x.G ( w x )) - w .E (G ( w x ))
i +1 i i i
w
3. Let w = i +1
wi +1
4. If not converged, go back to 2
E. Supported Vector Machine (SVM)
Support Vector Machines represents a supervised learning
algorithm based on the concept of decision planes that define
decision boundaries. SVM is introduced as a hyper plane
maximizing the margin of separation and minimum
classification error to separate data of different classes. For
nonlinear classifiers, the general idea is to map the input data
to a high dimensional space and separate it there by a linear Fig.3. Example of Face images of Four different person from AT&T
classifier. This method is given by the following relationship: database

415
 Firstly, the images are subdivided into two equal parts
where each person is presented by five images. The first one is
used for the training process and the second one is used for the
test process. Firstly, the important informations are extracted
from the original images by using the 2D-DWT.

(a) (b)
Fig.4. An example of applying 2D-DWT: (a) Original image; (b) 2 Level
of 2D-DWT Decomposition Outputs.

Then, the important features are extracted from the LL

Fig. 7. Example of Independent Components using the ICA method
sub-bands of each image by using the PCA, LDA and ICA
algorithms. The given five first important eigenvalues in the
training process are represented by the figures 5, 6 and 7.
Finally, the obtained eigenvectors are classified by the
SVM algorithm. The obtained rate of recognition accuracy
with variable component number and by using the
DWT+PCA+SVM, DWT+LDA+SVM, DWT+ICA+SVM
approaches on both the AT&T database are shown in the
following figure.

Fig. 8. Recognition rate for AT&T database with variable component number
for the proposed recognition face approaches

The performance evaluation of proposed recognition face

approaches on AT&T database is given in figure 8. The
DWT+LDA+SVM approach show better performance in small
number of components where the Recognition rate achieves
up 96% for 14, 15 and 16 components. We found higher
Fig. 5. Example of Eigenfaces of the training process
recognition rate for several components number of the
proposed approach based on DWT+PCA+SVM algorithms. It
achieves up 96% where the number of components are
between 27 and 34, 36 and 39 and between 93 and 99.
However, the prosed approach based on DWT+ICA+SVM has
the less recognition accuracy which achieves up 94.5% for 29
components. Hence the given results are more important when
compared with PCA+SVM, ICA+SVM and LDA+SVM
recognition face schemes. This efficiency is confirmed by the
given comparative study in the next section.

IV. COMPARATIVE STUDY

In order to prove the importance of the obtained results, we
have compared the proposed approach to the proposed ones in
[8], [13], [14], [15] and [16]. This comparative study is given
by the following table.

Fig. 6. Example of Fisherfaces of the training process

416
 TABLE I. COMPARATIVE STUDY [3] M.S. Bartlett, J.R. Movellan and T.J. Sejnowski, “Face Recognition by
Independent Component Analysis,” IEEE Transactions on Neural
Approaches Number of Recognition Networks, vol. 13, no. 6, pp. 1450-1464, November 2002.
Used algorithms
subjects Rate (%)
PCA+SVM 90.24 [4] D. Huang, C. Shan, M. Ardebilian and L. Chen, “Local Binary Patterns
[8] LDA+SVM 200-200 93.9 and Its Application to Facial Image Analysis: A Survey,” IEEE
ICA+SVM 91 Transactions on Systems, Man, and Cybernetics, vol.41, no. 6 ,pp. 765-
PCA+ ANFIS 96.66 781, November 2011.
[13] LDA+ ANFIS Not defined 68 [5] W. Zong and G. Huang, “Face recognition based on extreme learning
ICA+ ANFIS 71.33 machine,” School of Electrical and Electronic Engineering, vol.74, no.
DWT-PCA-SVM 86.6 16 ,pp. 2541-2551, September 2011.
[14] Not defined
DWT-DLDA-SVM 95 [6] D. P. Wadkar and M.. Wankhade, “Face Recognition Using Discrete
[15] ICA 200-200 87.5 Wavelet Transforms,” International Journal of Advanced Engineering
[16] LS-ICA 200-200 90 Technology, vol.3, no. 1 ,pp. 239-242, 2012.
DWT+PCA+SVM 96 [7] M. Chihaoui , A. Elkefi, W. and C. B. Amar “A Survey of 2D Face
propsed
DWT+LDA+SVM 200-200 96 Recognition Techniques,” Computers, vol.5, no. 4, 2016.
approach
DWT+ICA+SVM 94.5 [8] A. Bouzalmat, J. Kharroubi and A. Zarghili, “Comparative of PCA,
ICA, LDA using SVM Classifier,” Journal of Emerging Technologies in
This comparative study proves the effectiveness of the Web Intelligence, vol. 6, no. 1, pp., 64-68, 2014
proposed approaches for face recognition applications. The [9] G. Linge and M. Pawar, “Neural Network Based Face Recognition
Using PCA,” International Journal of Computer Science and Information
2D-DWT pre-processing increases the recognition rate Technologies, vol. 5, no. 3, pp. 4011-4014, 2014
compared to other works. This effectiveness is due to the [10] H. Makwana and T. Singh, “Comparison of Different Algorithm for
advantage of the DWT which is performed based on the Face Recognition,” Global Journal of Computer Science and
different scales and orientations of the processed image. Technology Graphics & Vision, vol. 13,no. 9, 2013
Consequently, it is sensitive to strong lighting conditions such [11] S. K. Jameel, “Face Recognition System Using PCA and DCT in
as facial expressions and details. This propriety improves the HMM,” International Journal of Advanced Research in Computer and
Communication Engineering, vol. 4, no. 1, January 2015
performance of the feature extraction classification process.
[12] Y. Liu, G. Chen, J. Lu, and W. Chen, “Face recognition based on
Therefore, the proposed approaches based on DWT, PCA, independent component analysis and fuzzy support vector machine, ” In
ICA, LDA and SVM algorithms give an important recognition 2006 6th World Congress on Intelligent Control and Automation, vol. 2,
rates. pp. 9889–9892, 2006.
[13] R. Sharma and M.S Patterh, “Pose Invariant Face Recognition using
V. CONCLUSSION Neuro-Fuzzy Approach, ” IOSR Journal of Computer Engineering, vol.
17, no.3, pp. 20–27, Juin 2016.
In this paper, we have proposed DWT+PCA+SVM, [14] S. A. Z. Bagherzadeh, A. N. Sarcheshmeh, S. H. Z. Bagherzadeh and
DWT+ICA+SVM and DWT+LDA+SVM face recognition M. M. Khalilzadeh, “New Hybrid Face Recognition Algorithm Based on
approaches. The performance of these approaches is evaluated Discrete Wavelet Transform and Direct LDA, ” Biomedical Engineering
by simulation on the AT&T database and comparative study. and 2016 1st International Iranian Conference on Biomedical
Engineering, November 2016
This effectiveness is due to the DWT pre-processing which
[15] S. N. Borade, R. Deshmukh and P.Shrishrimal, “Independent
allows keeping the most important information in the input Component Analysis and Number of Independent Basis Vectors”,
images. Also, this pre-processing increases the recognition Procedia Computer Science, Vol. 58, pp. 380-386, 2015
rate by approaches compared to other approaches based on [16] J. Kim, J. Choi and J. Yi , “Face Recognition Based on Locally Salient
PCA+SVM, LDA+SVM or ICA+SVM. ICA Information”, Lecture Notes in Computer Science, Vol. 3087,pp. 1-
9, 2004
REFERENCES [17] AT&T. Laboratories. Cambridge, http://www.cl.cam.ac.uk/Research/
DTG /attarch ive:pub/data/att_faces.tar.Z.
[1] M. Turk and A. P. Pentland, “Eigenfaces for recognition,” Journal of
Cognitive Neuroscience, vol. 3, no. 1,pp. 71-86, 1991. [18] Del Sallam M. Haider, Munya Abdulmajid Arasi, “Face Recognition
Based on PCA, DCT, DWT and Distance Measure”, Egyptian Computer
[2] P. N. Belhumeur, J. a. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. Science Journal, Vol. 37, No. 5, pp. 16-30, 2013
fisherfaces: Recognition using class specific linear projection,” IEEE
Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp.711-720, July 1997. [19] A. Hyvarinen, “Fast and robust fixed-point algorithms for independent
component analysis,” IEEE Transactions on Neural Networks, vol. 10,
no. 3, pp. 626-634, May 1999.

417