Face recognition techniques with permanent changes: A review
Alka Jindal * Dr Savita Gupta
PEC university of Technology, India UIET,Panjab University,Chandigarh
alka er@rediffmail.com
Abstract-Biometrics has become an integral part for
person identification. Out of all biometrics, face is used more
often as it can be captured easily. Face recognition plays an
important role in digital world and hence has gained a lot of
attention. In this paper, focus is laid on the face recognition
with permanent changes. Permanent changes in face can be
naturally or artificially induced. In natural change, changes in
face happen due to change in age while artificially face can be
changed by plastic surgery, both these changes are permanent.
Innovation of this paper is that survey for age and plastic
surgery exist separately in literature but here, survey exists in
the direction of those techniques which leave a permanent
mark/change in face. In this paper, the comparison is made
among various techniques for face recognition dealing with
age and plastic surgery. This paper also provides future
directions for further research.
Keywords-face recognition, age, plastic surgery
L INTRODUCTION
Biometrics have changed our life to a great extend. In
modem times in the digital world, everyone wishes to get
access without producing any documents like I card, library
card etc. Automatic access is possible with personal
identification that is carried in one's own biometrics.
Biometrics like fingerprint, iris, face, gait, etc have recently
gained a lot of weight because of their natural availability over
traditional password Icard system. Further, face as biometrics
have several advantages from Among various biometrics
available. It is easy to use, natural and can be captured without
anyone's special notice i.e. passive acquiring [l]. Surveillance
cameras are deployed at many public places for security
reasons. A lot of research is going on in the field of face
recognition as it has many applications in the field of
biometrics, access and security[2]. It is used in Airport
Security, Homeland, Defense, Surveillance, Law enforcement,
Access Control and Security to name a few [1].
There are a lot of challenges in the field of face recognition
which can withstand all environments. The face recognition
should be able to deal with change in pose, expression, human
race, occlusion, illumination, age, surgery etc. Unlike
illumination, pose, expression , human race, occlusion, etc.
age and plastic surgery leave a permanent change in one's face
leading to low performance of face recognition algorithms
over time. Aging is a natural process and plastic surgery is
artificial ,but variations by both aging and plastic surgery on
face are permanent.
In order to identify the correct person, the security concerns
have posed a great challenge for face recognition. Since,
expression, pose, lightening etc are environmental controlled
978-1-4673-7910-6115/$31.00 © 2 015 IEEE
Dr. Lakhwinder Kaur
UCOE,Punjabi University, Patiala
challenges but changes due to age and changes due to plastic
surgery leave a permanent mark on one's face. Hence, face
recognition with changes due to age and plastic surgery are
challenge to be focused upon in the forensics world.
Through plastic surgery, one can change face globally or
some regions locally. This can alter shape, appearance, texture
etc. It is very difficult to determine the features or parts of the
face which remain same or change hence making face
recognition difficult after plastic surgery. Individuals can
make misuse of the technology by concealing their original
identity for illegal purposes like fraud commitment, breaking
law or theft. Use of plastic surgery can be used by terrorist or
criminals to change their facial identity and move freely.
Hence, they can make mockery of the system being identified
by face[3].
Through Aging, one experiences changes in his facial
features differently at each age. Process of aging is different in
different people. It is a complex phenomenon which depends
on their genes and also influenced by lifestyle conditions such
as environment, stress, disease, changes in weight, and
exposure to extreme climates[4], habits like smoking
Idrinking. The environmental factors like exposure to
ultraviolet radiation may cause aging of face. If aging is not
taken into consideration then there will be need for frequently
updating of the databases maintained by various organizations
for systems working with automatic face recognition system.
If aging in face recognition is taken into consideration then
our life would be much easier due to its various applications.
Automatic Age invariant face recognition model can also be
useful in Internet of Things (lOT) [5]. Using satellites or
advanced machines, a person (terrorist or criminal) can be
traced. Machine to machine connections in Internet of things
(lOT), can be used to track someone throughout age
progression which otherwise will be a difficult task when done
manually. Electronic Customer Relationship Management will
be possible i.e. we will be able to customize items on the basis
of customer's age as people in different age groups have
different needs. [6]. Further, criminals and missing persons at
different ages can be found if we can predict the face
according to the age or we can match correctly the current face
image with the older available face image. It can act as a
check on children and keep them under parental control for
internet usage etc.. Computer programs or TV shows can be
customized acc to different age. Also, it can be used to stop
children indulge into drinking and can prohibit them from
entering bars or wine shops. It can also be used to check de­
duplication such as in passports, Adhaar-cards, Ids, etc. Aging
being one complex phenomenon can further pose a challenge
689
 in age estimation via usage of anti aging medicines [4] or
cosmetics or plastic surgery [3], making face recognition
harder.
This paper is further organized in four sections. In the next
section, various algorithms of face recognition across age are
given along with the respective comparison. In the third
section, various algorithms of face recognition across plastic
surgery are given and comparison of those algorithms is
drawn and In the last and fourth section, conclusion is drawn.
11. ALGORITHMS FOR FACE RECOGNITION
ACROSS AGE
In face recognition, a query face is found to be matched in
the database of images. Aging plays a great role in ones facial
imaging as features of the face change along age Progression.
Face Recognition with change in age are broadly classified
into two main categories - generative and discriminative
approach. Face is generated at a desired age based on available
face images and hence matched with face in query.
In discriminative approach, the features of both the query
face and the database image are matched. A detailed survey on
face recognition with age till 20 10 has been nicely described
in [12]. In this paper rest of the techniques for face
recognition across age are reviewed and compared.
In the various face recognition algorithms, after feature
extraction and reduction different methods are used to match
the query face with the available database. Various feature
extraction techniques like AAM, LFA, LBP, SIFT, MLBP,
CI, HOG, HIGH-DIMENSIONAL LBP, MEFD etc have been
used to extract features from the face across age. Feature
reduction techniques like LDA, PCA ,MFDA etc are used. For
face matching, block matching, LDA, Semi-NaIve Bayesian
method, Mahalanobis distance etc have been used. Databases
used are FGnet[7], Morph[8], FERET [9], Browns[lO],
IIITD[II] aging face database have been used by researchers
in their respective work.
The recognition accuracy of various techniques related
with age are compared and are given in table 1 below.

Table 1 is the comparison of various techniques available for face recognition across age

s.no Reference Method Database Performance Accuracy %

1 Park etai, 2010[13] 3D shape and texture using FG-NET 37.4
Generative Approach [uses AAM MORPH Album 1 66.40
for feature extraction] BROWNS 28.1
2 Mahalingam Graph Matching method [uses LFA FG-NET [18,69] 25.4
&Kambhameetu, 2010 and LBP for feature extraction] (EER) [0, 69] 29.2
[14] GMM+ Graph
3 Li etal, 2011[15] Discriminative Approach [uses FG-NET 47.50
SIFT and MLBP for feature MORPH Album 2 83.9
extraction]
4 Xia etai, 2011[16] Age Simulation Generative FG Recognition Rate (Gauss Function Simulation»
Approach[uses LDA for feature] - (Without Simulation)
NE Age Specific Simulation:
T With Filling> Without Filling
5 Klare etai, 2011 [17] Discriminative Model -Improved MORPH 2 79.08%
Training Method
[uses LDA and RS-LDA for
feature extraction]
6 Wu etai, 2012rl81 Efficient Landmarks Formation FG-NET 23.6(EER)
7 Du etai, 2012[19] Generative Approach [uses non MORPH Efficiency(After NMF»
negative matrix for feature FG-NET Efficiency(Before NMF)
extraction]
8 Mao etai, 2012[20] Local Classifier Ensemble Model MORPH CAR and CRR is calculated and
Performs Better than GOP+SYM
,When Age Gap Is 1 To 6 Years
9 Wang etai, 2013 [21] Modified input images [uses LBP FG-NET 33.92
for feature extraction]
10 Yadav etai, 2013 [22] Bacteria Foraging Fusion FG-NET 64.5
lIlT Delhi 54.3
II Gong etal , 2013[23] Hidden Factor Analysis [uses HOG MORPH 91.14
for feature extraction] FG-NET 69.0
12 Bareta et ai, 2013[24] LBP, gabor variants etc FG-net 45
13 Chun etai, 2014[25] Cross Age Reference model [uses MORPH 92.8
HD-LBP for feature extraction]
14 Gong etai, 2015[26] Maximum Entropy Descriptor- MORPH 92.26
Discriminative Approach[uses
FG-NET 76.2
MEFD and LBP for feature
extraction]

690 2015 International Conference on Green Computing and Internet of Things (ICGCloT)
 III. ALGORITHMS FOR FACE RECOGNITION
ACROSS PLASTIC SURGERY
Face recognition post plastic surgery is challenging
task. Research in face recognition with plastic surgery is
relatively new dimension and has not been explored much.
Features of the face undergo many changes during plastic
surgery. It can be local or global depending on the extent of
the surgery done. So, the changes introduced after plastic
surgery need to be tackled by face recognition methods. After
an individual undergoes the process of plastic surgery, the
features of the face are affected to different extend either
locally or globally. The plastic surgery makes changes in the
face which makes it difficult for existing face recognition
algorithms to recognize. IT degrades the performances of face
recognition algorithm
Method for Feature Extraction! Reduction are PCA, FDA,
LFA, Circular LBP, Speeded Up Robust Features (SURF), 2-
D Log Polar Gabor Transform (GNN) Neural , SLBT, Gabor
local binary pattern (GLBP), Gabor filters, EUCLBP, SIFT,
UCLBP etc. Database Used are plastic surgery face database
of l800 face images (before and after surgery) of 900 subjects
[3], some have used Self created database, Publicly accessible
plastic surgery database. Table 2 is the comparison of various
techniques available for face recognition involving plastic
surgery.

Table 2 is the comparIson of varIOUS techniques available for face reco2nition mvolvin2 plastic sur2ery
S.No. Reference Method Database Performance

I. R singh et al 2010[3] Using CLBP, FDA, A plastic surgery database 54%.recognition rate for GNN for
GNN, SURF, LFA, (1800 images for 900 persons) plastic surgery db.
PCA [authors prepared this database] Evaluated for local and global
surgery also.
2. R.S. Khedgaonkar et Near set theory. Self created database of certain Identified 55.55% accuracy for
al [27] celebrities local plastic surgery compared
with PCA,GNN and LBP
3. Lakshmiprabha and SLBT plastic surgery face database, 72
Majumder [28] AAM with LBP MUCT 83.2 respectively
followed by PCA Cascading improves
4. R. Jillela and A. Fusion method for plastic surgery database 87.4%
Ross,[29] face and ocular region
5. Y.Sun, SSIM plastic surgery database 77.5%. in components
M. Tistarelli[30]
6. Bhatt et al [31] Multiobjective Plastic surgery DB, Combined 87 to 89 accuracy
Evolutionary method Heterogeneous face DB
7. Feng and Prabhakaran Sum of Facial Parts Plastic surgery face database., 85.35%
[32] Recognition MUCT regular 96.8%

IV CONCLUSION AND FUTURE DIRECTIONS accuracy dealing with both age and plastic surgery, which are
Face recognition has been of interest to image processing permanent in nature.
researchers. Performance of the face recognition has not been Also, it is further necessary to state that robust face
upto the mark due to several covariates (like illumination, recognition method is required which can consider all the
pose, makeup, expression, age , plastic surgery). The effect of covariates at the same time. But this can begin only if problem
some variants is temporary or environmental based, but the of two or more variants can be dealt at one time. In this paper,
changes due to age and plastic surgery (in face) are permanent the comparison of techniques is done based on age and plastic
in nature. Plastic surgery can rejuvenate face image and can surgery.
reduce the aging effect. It is a complex phenomenon to deal
REFERENCES
with. Wrinkles can be made hidden and many more changes
either locally or globally can be induced! reduced in a face, [1] S.Z. Li, A.K. Jain, Handbook of Face Recognition,
and plastic surgery can even defy the aging effects. Also, Springer, Spring Street, NY, USA, 2005.
Aging is complex as it is individual specific and depends on [2] Zhao, W., Chellappa, R., Phillips, P. J., & Rosenfeld, A.
various factors including environment, race, ailment, habits (2003). Face recognition: A literature survey. Acm
etc. Computing Surveys (CSUR), 35(4), 399-458
There is a need to take into account both these changes! [3] Singh, R., Vatsa, M., Bhatt, H. S., Bharadwaj, S., Noore,
variants at the same place.1) Area of plastic surgery need to be A., & Nooreyezdan, S. S. (2010). Plastic surgery: A new
further explored individually as this has been explored to
dimension to face recognition. Information Forensics and
small extend. 2) There exist some discriminative features on
Security, IEEE Transactions on, 5(3), 441-448.
the face which never change across age and also in some local
[4] A.Stone, "The Aging Process of the Face and Techniques
plastic surgery. Those features need to be identified and hence
of Rejuvenation, "Available:.aaronstonemd.coml
targeted for face recognition across permanent changes. 3) No
Facial_Aging_Rejuvenation.shtm, 2010
common technique exists in literature which can lead to good

2015 International Conference on Green Computing and Internet of Things (ICGCloT) 691
[5] D. Miorandi, S. Sicari, F. De Pellegrini, and I. Chlamtac, Classifier Ensemble, International Conference on Pattern
"Internet of things: vision, applications and research Recognition, pp. 2408-2411, dec 2012.
challenges," Ad Hoc Networks, vol. 10, no. 7, pp. [21] Wang Shaoyu, Xia Xiaoling, Huang Yongfeng & Le
14971516,2012. Jiajin, 2013, Biologically-Inspired Aging Face
[6] Electronic Customer Relationship Management, Recognition Using Cl and Shape Features, IEEE Fifth
http://en.wikipedia.org/wiki/ECRM,2010. International Conference on Intelligent Human-Machine
[7] The FG-NET Aging Database,
Systems and Cybernetics,voI.2, pp 574-577, Jan,2013.
http://www.fgnet.rsunit.com/ [22] Yadav Daksha, Vatsa Mayank, Singh Richa & Tistarelli

K. Ricanek, Jr and T. Tesafaye, "Morph: A longitudinal
[8]
image database of normal adult age-progression," Int.
Conf. Automatic Face gesture Recognition., 2006, pp.
341345.
Massimo, 2013, Bacteria Foraging Fusion For Face
Recognition Across Age Progression, IEEE Conference
on Computer Vision and Pattern Recognition Workshops,
pp 173-179, Sep, 2013.

P. J. Phillips, H. Moon, P. J. Rauss, and S. Rizvi, "The [23] Gong Dihong, Li Zhifeng, Lin Dahua, 2013, Hidden
[9]
FERET evaluation methodology for face recognition Factor Analysis for Age Invariant Face Recognition,

algorithms", IEEE Transactions on Pattern Analysis and IEEE International Conference on Computer Vision, pp.

Machine Intelligence,Vol. 22,No. 10, October 2000 2872-2879,dec 2013.

N. Nixon and P. Galassi, "The Brown Sisters, Thirty­ [24] Bereta, M., Karczmarek, P. , Pedrycz, W., Reformat, M.:
[10]
Three Years. The Museum of Modern Art," 2007. ,,Local descriptors in application to the aging problem in

Daksha Yadav, Mayank Vatsa, Richa Singh, Massimo face recognition'. Proceedings Pattern Recognition,2013,
[11]
Tistarelli, "Bacteria Foraging Fusion For Face 46(10),pp. 2634-2646.

Recognition Across Age Progression," IEEE Conference [25] Chen Bor-Chun, Chen Chu-Song, & Hsu Winston H.,

on Computer Vision and Pattern Recognition Workshops, 2014, Cross-Age Reference Coding for Age-Invariant

pp 173-179,2013. Face Recognition and Retrieval, Computer Vision -

Yun Fu, Guodong Guo and Thomas S. Huang, "Age ECCV, pp. 768-783, DOl 10.1007/978-3-319-10599-
[12]
Synthesis and Estimation via Faces: A Survey," IEEE 4 49.

Transactions on Pattern Analysis and Machine [26] Gong Dihong, Li Zhifeng, Dacheng Tao, Jianzhuang Liu

Intelligence, VOL. 32, NO. 11,2010.
[13] Jain Anil K., Tong Yiying & Park Unsang, Age-Invariant
Face Recognition, IEEE Transactions On Pattern
Analysis And Machine Intelligence, vol- 32(5),Jan 2010.
[27]
[14] Mahalingam Gayathri & Kambhamettu Chandra, 20 I0,
Age Invariant Face Recognition Using Graph Matching,
& Xuelong Li, 2015, A Maximum Entropy Feature
Descriptor for Age Invariant Face Recognition, The
IEEE
Conference on Computer Vision and Pattern Recognition
(CVPR), pp. 5289-5297,Feb 2015.
R. S. Khedgaonkar et aI., "Identifying Resemblance in
Local Plastic Surgical faces using near sets for Face

IEEE International Conference on Biometrics Recognition," in Communication Systems and Network

Compendium in Washington, DC, pp.I-7, Sep 2010. Technologies (CSNT), 2011, pp. 589- 592

Li Zhifeng, Unsang Park, & Jain Anil K., A [28] N. S. Lakshmiprabha and S. Majumder, "Face
[IS]
Discriminative Model for Age Invariant Face Recognition System Invariant to Plastic Surgery," in

Recognition, IEEE Transactions On Information Intelligent Systems Design and Applications (ISDA), 12th

Forensics and Security, vol-6 (3), 2011. International Conference, 2012, pp. 258 - 263.

Xia Tian, Lu Jiwen, Tan Yap-Peng, 2011, Face [29] R. Jillela and A. Ross, "Mitigating Effects of Plastic
[16]
recognition using an enhanced age simulation method, Surgery: Fusing Face and Ocular Biometrics," in

IEEE Visual Communications and Image Processing, pp. Biometrics: Theory, Applications and Systems (BTAS),

1-4,May 2011. 2012 IEEE Fifth International Conference, 2012, pp. 402-

Klare Brendan & Jain Anil K., 2011, Face Recognition 411.
[17]
[30] Y. Sun, M.Tistarelli, "Structural Similarity Based Image
Across Time Lapse: On Learning Feature Subspaces,
IJCB International Joint Conference on Biometrics, pp. 1- Quality Map for Face Recognition across Plastic

8, oct 2011. Surgery," in Biometrics: Theory, Applications and

Wu Tao, Turaga Pavan & Chellappa Rama, 2012, Age Systems (BT AS), 2013 IEEE Sixth International
[18]
Estimation and Face Verification Across Aging Using Conference, 2013, pp. 1- 8.

Landmarks, IEEE Transactions on Information Forensics [31] H. S. Bhatt, S. Bharadwaj, R. Singh, M.Vatsa,

and Security, vol-.7, pp. 1780-1788, August 2012. "Recognizing Surgically Altered Face Images Using

Du Ji-Xiang, Zhai C-M, & Ye Y-Q, 2012, Face aging Multiobjective Evolutionary Algorithm," IEEE
[19]
simulation based on NMF algorithm with sparseness Transactions On Information Forensics and Security, Vol.
8, NO. 1,pp. 89-100, January 2013.
constraints, Advanced Theory and Methodology in
Intelligent Computing, Neurocomputing, dec 2012. [32] R. Feng and B. Prabhakaran, "A Novel Method for Post­

Mao Xiao-Jiao, Yang Yu-Bin, Li Ning & Zhang Yao, Surgery Face Recognition Using Sum of Facial Parts
[20]
2012, Age-Invariant Face Verification Based on Local Recognition," in Applications of Computer Vision

692 2015 International Conference on Green Computing and Internet of Things (ICGCloT)
(WACY), 2014 IEEE Winter Conference, 2014, pp. 1082- 1089.

2015 International Conference on Green Computing and Internet of Things (ICGCloT) 693