Feature extraction using Deep Learning for
Intrusion Detection System
Mohammed Ishaque1, Ladislav hudec2
Slovak University of Technology, Bratislava, Slovak Republic
1
xishaque@is.stuba.sk
2
lhudec@fiit.stuba.sk
Abstract— Deep Learning is an area of Machine
Learning research, which can be used to manipulate
large amount of information in an intelligent way by
using the functionality of computational intelligence.
A deep learning system is a fully trainable system
beginning from raw input to the final output of
recognized objects. Feature selection is an important
aspect of deep learning which can be applied for
dimensionality reduction or attribute reduction and
making the information more explicit and usable. Deep
learning can build various learning models which can
abstract unknown information by selecting a subset of
relevant features. This property of deep learning makes
it useful in analysis of highly complex information one
which is present in intrusive data or information
flowing with in a web system or a network which
needs to be analyzed to detect anomalies. Our
approach combines the intelligent ability of Deep
Learning to build a smart Intrusion detection system.
Keywords — Deep learning, Intrusion Detection
System, Computational Intelligence.
I. INTRODUCTION
The web system is an environment containing huge number
of mechanisms and technologies which includes HTTP
protocol, client side and server-side applications, web
browser, and scripting mechanisms like java script, CGI.
Applications which are made on these infrastructures are
widely accepted but at a cost of their security because of
inconsistency found in these technologies, so
implementation of security on these infrastructures is a big
challenge and as a result many web applications deployed
on these systems are exposed to security vulnerabilities.
There are three basic types of security vulnerabilities within
a web system, Input validation, session management, and
application logic. Some common attacks which exploits
these vulnerabilities are discussed here. Script injection and
dataflow attacks belongs to the class of input validation
vulnerability which usually inject malicious scripts in web
contents which results in malfunctioning of the output of
web system e.g. Cross site scripting XSS and SQL injection.
Copyright Notice: 978-1-7281-0108-8/19/$31.00 ©2019 IEEE
To deal with the problem of pattern change or attack
modification Computational Intelligence technique a
suitable technique as it has a high detection accuracy used
for making intelligent detection model and automatically
detect anomalous activities. The basic requirement of these
methodologies is to train the labelled data and understand
the behavior of data resulting in high resource cost. These
kind of classifiers follows an iterative process for
understand the behavior of patterns, making some
mathematical adjustment and then predicting the outcome.
Deep learning can learn or extract features directly from raw
data also it has the capability to find out the features
automatically which are important and useful for
classification. Different Deep learning techniques are used
for extraction process like Deep belief network, restricted
Boltzmann machine and Auto encoders. One more
advantage of using deep learning is its ability to handle huge
and raw data. In this research, we propose to handle
intrusion detection using Deep Learning approach for a web
system. The Deep learning approach extracts key features
from the data obtained from the web system which is
exposed to lot of vulnerabilities as it is on internet. Then a
stacked denoised autoencoder is used as an enhancement
stage. This classifier is basically used to classify the attacks
and distinguish between normal and anomalous traffic.
II. RELATED WORK
Sharma et al [1] they proposed a system comprising of
Deep Belief Network for anomaly detection and at the same
time, they tried to compare its performance with the classic
neural network. The comparision was done by considering
the features learned and how well it can detect the
anomalies. They also explored the amount of set of features
learn by each layer of the deep architecture. A simple, more
reliable, accurate and fast mechanism is been provided by
them for visualizing the features at upper level. Here the
authors tried to to develop a system for abnormality
detection which reduces the amount of data to be processed
manually by focusing only on a specific part of data, for this
a good set of features selection system is required and this
selection of good feature results in good abnormality
detection. Deep Learning is a modern tool which can be
used to learn features directly from raw data by deriving
high level features from low level features and form a
hierarchical representation. The performance of the system
is tested on two types of data with Precision, Recall and F-
score, one set is discrete in nature and the other is normal
data. The author used MNIST [2] dataset which includes
digit and non-digit dataset like handwritten data as digit data
and face data as non-digit.
Experimentally the authors explored and compared two
types of structures, one is the classic neural network, and
another is Deep belief network. The deep belief model is
made by stacking Restricted Boltzmann machines. With the
comparison of deep architecture with classic neural network
it is observed that deep architecture performs better
compared conventional neural network also deep
architecture can be further explored and applied to detect
anomalies. On an average 42.4% to 66.5% improvement
achieved on the two datasets used in experimentation.
Our approach implies concept of Deep learning to reduce
the dimensionality or extract the key feature of the data
which the web system is dealing with. Restricted Boltzmann
machine and auto encoders are the two methods in Deep
learning which has the capability to extract key features
from unlabeled data [3, 4]. Alrawashdeh, Khaled, and Carla
Purdy [5] In this work the main idea is detection of
intrusions over a network by using the concepts of Deep
Belief Network which is formed by layering the Restricted
Boltzmann Machines together. The feature extraction is
done here by using a single hidden layer of RBM. The
processed output from this layer is passed to another RBM
which results in deep belief network. The trained data is
then passed through a layer of logistic regression to classify
the data and put it according to its class. Here the obtained
output from logistic regression can be categorized into
anomalous and normal data
Until now in the name of anomaly detection, neural network
is the only technique used in the name of classification but
in our approach, we use a stacked denoised auto encoder
classify normal and anomalous traffic on the web system.
[6] come up with an approach for detectiong anomalies in
codes using the concept of deep learning by combining
Autoencoders and Deep belisf networks together.
Autoencoders are used to extract the key features and deep
belief networks are applied to detect anomalies in code. A
new dataset will be generated over a web system which will
carry both normal and anamolus data.
III WEB SYSTEM
Web systems can easily be exploited by the use of
HTTP/HTTPS protocols. Http is a protocol designed to
make communication between client and server and https is
mainly used for a secured connection by encrypting the
connection. One disadvantage of https protocol with respect
to intrusion detection system is that the encryption wraps the
network-based detection system which results in failure of
detection of packets as the data packets are in encrypted
format. On the other hand, Host based intrusion detection
system does not face this problem as the end point is
protected and get decrypted to original form [7]. In our
experiment we will be using a small web application which
can upload, download and manipulate images online. It will
have a database system, where normal SQL queries can be
used to define and manipulate the data.
In [8] the authors come up with an approach that detects
intrusions using state transition analysis technique (STAT).
A comparison of machine learning techniques like Random
Forest, Logistic Regression, Decision Tree, AdaBoost
classifier and SGD classifier is done here on web intrusion
detection like SQL injection, buffer overflow and XSS. Out
of the compared machine learning techniques, Logistic
regression comes out as the best learning method for this
problem. The performance is being further improved with
different feature extractions and parameter tuning. It uses
CSIS HTTP 2010 Data set.
A large number of web vulnerabilities can be targeted here
like Injections, Broken authentic and session management,
Cross site scripting, Insecure direct object reference, Web
application and server misconfiguration, Sensitive data
exposure, Broken access control, Cross site request forgery,
Using components with known vulnerabilities and
invalidated redirects and forwards. These vulnerabilities are
included in the test data which is used to test the web
applications.
The web intrusion detection architecture is divided into two
parts, the first part trains and validates the machine learning
techniques and the other part captures packets on the
network card. The training and validation process use a
dataset to train and test the machine learning models which
are trained until these models fulfil the required condition in
the phase where validation is done, later the live captured
packets are classified with these models to detect attacks.
Here the authors have used SciKit Learn [9] to apply
learning methods to the selected dataset. A comparative
analysis is done where this data set is applied on all the
machine learning techniques mentioned above, each
machine learning algorithm’s performance is measured by
the precision, recall and the f1-score [8] on the testing
dataset where precision indicates how useful the search
results are, recall indicates how complete the search results
are and f1-score indicates measure of test accuracy.
Experimental results show that Logistic regression comes up
with good results with respect to high recall rate and high
precision rate as compared to other machine learning
algorithms.
IV INTRUSION DETECTION SYSTEM
In a computer system an Intrusion detection system scans
occurrence of events in web or network system and
investigates for threats, inaccessible intrusions and
unauthorized access or entry [10]. An unstable system with
many loopholes is the main victims of these intrusions
which attempts to access or manipulate information and
make the system non usable or unreliable. Denial of Service
(DoS) makes the machine unavailable for the user, worms
and viruses over the network exploits information of users
and take advantage of privileged access of host systems
vulnerabilities [11]. An Intrusion Detection System
basically protects and make them ready to handle attacks. A
detailed description of different types of Intrusion detection
systems is shown in fig 1.

Fig-1 Intrusion Detection System
V CONVENTIONAL FEATURE LEARNING
ALGORITHMS
Conventional feature extracting algorithms are shallow in
nature and its main purpose is to learn changes happening in
data that makes it simpler to extract important information
when building classifiers [12]. Mostly, feature extracting
algorithms are either linear or nonlinear, generative or
discriminative, supervised or unsupervised, local or global.
Principal Component Analysis (PCA) is an unsupervised,
linear, global and generative feature extracting algorithm
whereas Linear Discriminative Analysis (LDA) is a
supervised, nonlinear, local and discriminative algorithm.
Principal Component Analysis algorithm, because of its
simplicity has the capability to transform raw data into
orthogonal dimensional space. High variance Principal
components are suitable for solving the machine learning
problems as it is a result of conversion from correlated
variables to linear uncorrelated variables. Predicting if a
datapoint in a dataset belongs to one or two classes can
easily be done by the principle components with high
variance points and hence the best features will be selected
in order to separate the data. In case of unsupervised
learning the model does not consider label of each data point
since the PCA observes the dataset as a whole and finds the
direction of high variance later the next direction finds the
next high variance which is orthogonal to the last one.
Though the datasets where the separation of data is not
based on high variance then even the most important
principle component of PCA will not work and this is one of
the important drawbacks of PCA [13] which can be
eliminated with the use of Deep Learning.
VI DEEP LEARNING FOR FEATURE EXTRACTION
It is a technology for learning and representation where
many levels exist comprising of modules, they are nonlinear
and transforms the information present at the first level to
higher upper levels which are more abstract in nature
basically used for feature extraction and transformation.
Each layer present here utilizes the output obtained from the
previous layer input. With an arrangement of such
transformation, very complex and integrated functions can
be simplified and learned. The classification process takes
place when layers presents at higher level amplifies the
inputs from lower level and can easily be distinguishable.
With high end hardware, high dimensional dataset can
easily be manipulated and can be used in various
applications. Our main purpose of using deep learning in our
research is to reduce the dimensionality of the dataset
obtained from the web system.
Fig-2 Deep learning system
Feature selection or extraction is a process of optimization
which decreases the dimensionality of a dataset by selecting
only those features which are interesting without duplication
and irrelevancy. The selected best features are highly
optimal for increasing the performance with respect to
accuracy rate of the classification model by reducing the
computation time and the space required for storing the
information. It also takes care of the noise reduction of data
and avoid over-fitting problem [14], [15]. The use of deep
learning in numerous fields is because of the huge
development of three aspects, the first is its ability to learn
features, availability of huge labelled data and finally
because of available modern-day computing power of
graphical processing units (GPUs).
Auto Encoders is a deep learning method proposed by G.E.
Hinton in 2006 and is basically used for feature Extraction.
The complete structure of auto encoder is divided into two
parts one is the encoding part and another is the decoding
part having three layers, input layer, hidden layer and output
layer. At the cross section of encoder and decoder there is a
layer called as code layer and this layer is the core of Auto
Encoder and this core represents the important features of
high dimensional data sets with nested structure and at the
same time set the dimensions of this high-dimension data
set. The hidden layer neurons when they are less in number
as compared to both input layer neurons and output layer
neurons then the data dimensionality reduction is obtained.
Auto encoders are basically comprising of three steps
pretraining, Unrolling and fine-tuning [16].
y = s (Wx + b) transformed to
z = s (W`y + b`)
where y is the latent representation or code, s is a non-
linearity W is the weight matrix.
And the average re construction error can be represented as
VII PROPOSED SYSTEM ARCHITECTURE
Fig-3 Proposed architecture for Intrusion detection system
IV FEATURE EXTRACTION AND CLASSIFICATION
USING STACKED DENOISED AUTO ENCODER
Stacked autoencoders contains more than one hidden layer
[17]. By stacking the expressing capacity of the model
increases allowing the autoencoder to distinguish between
attacks and normal traffic.
In stacked autoencoders the output of each layer is fed to the
next corresponding layer
h1 = f (x), hi = f (hi−1) …encoding
g1 = g(hi ), gi = g(gi−1) …decoding
This stacked autoencoder is now pretrained using greedy
layer training where the raw input is fed to the first layer of
encoder and parameter sets are obtained from it, then this
layer will transform the raw input to the hidden units in the
first layer. The second layer is trained their after and obtain
the parameters. For all the layers, this process is repeated
where parameters are trained at each layer individually
while on the other layers, the parameters are unchanged.
The denoising of the autoencoders is necessary as it
prevents autoencoders from over-fitting. Our aim is not just
to target those attacks which are known but also to identify
the new or unknown attacks. Our system should be able to
consider cases that were not focused in the training phase.
Denoising corrupts the original input by inserting some
noise in the input data hence allowing the encoder to repair
the input by reconstructing the corrupted version making the
hidden layer to get the statistical dependencies between the
inputs. If the onset value obtained from the stacked denoised
auto encoder is greater then the onset value then the request
is taken as a normal request but if the onset value is less
then the request is taken as attack abnormal request.
VIII PERFORMANCE ANALYSIS
For our approach we propose to study the following
performance metrics.
Accuracy: This measure is the result whose degree of
classification is conducted based on the proposed
framework and the input data.
False Positive rate: It is the ratio between the occurrences of
events which are negative and wrongly categorized as
positive and the total number of actual negative events
Capability to detect attack or vulnerability: It is the
capability of the system how it can detect attacks or
vulnerabilities as compared to other systems which are not
hybridized, where the deep learning approach is not used for
feature extraction. A comparison will be done after the
experimentation part showing the clear difference between
our approach and other approaches mentioned in the
literature.
[1] Sharma, Manoj Kumar, Debdoot Sheet, and Prabir Kumar Biswas.
"Abnormality Detecting Deep Belief Network." Proceedings of the
International Conference on Advances in Information Communication
Technology & Computing. ACM, 2016.
[2] PaxsonV. Bro: a system for detecting network intruders in real-time.
In: Proceedings of the 7th USENIX security symposium, (1998), San
Antonio, TX.
[3] E. Albin, “A Comparative Analysis of Snort and Suricata Intrusion
Detection Systems”, Naval Postgraduate School, Dudley Know
Library, September 2011.
[4] OpenWIPS-ng. http://www.openwips-ng.org/. Accessed: August 8,
2013.
[5] Alrawashdeh, Khaled, and Carla Purdy. "Toward an Online Anomaly
Intrusion Detection System Based on Deep Learning." Machine
Learning and Applications (ICMLA), 2016 15th IEEE International
Conference on. IEEE, 2016.
[6] Li, Yuancheng, Rong Ma, and Runhai Jiao. "A hybrid malicious code
detection method based on deep learning." methods 9.5 (2015).
[7] Agarwal, Nancy, and Syed Zeeshan Hussain. "A closer look on
Intrusion Detection System for web applications." arXiv preprint
arXiv:1803.06153 (2018).
[8] Pham, Truong Son, and Tuan Hao Hoang. "Machine learning
techniques for web intrusion detection—A comparison." Knowledge
and Systems Engineering (KSE), 2016 Eighth International
Conference on. IEEE, 2016.
[9] Ilgun, Koral, Richard A. Kemmerer, and Phillip A. Porras. "State
transition analysis: A rule-based intrusion detection approach." IEEE
transactions on software engineering 21.3 (1995): 181-199.
[10] J. P. Anderson, “Computer Security Threat Monitoring and
Surveillance,” James P Anderson Co, Fort Washington, Pennsylvania,
Tech. Rep., April 1980.
[11] Cannady,J. Artificial neural networks for misuse detection. In :
Proceedings of the National Information Systems Security
Conference, 1998, pp.443–456.
[12] Bengio Y, Courville A, Vincent P. Representation learning: a review
and new perspectives. IEEE Trans Pattern Anal Mach Intell. 2013;
35:1798e1828.
[13] Limitations of Applying Dimensionality Reduction using PCA by
Roberto Reif, https://www.robertoreif.com/blog/2018/1/9/pca.
[14] Liu, H., Yu. L,“Toward integrating feature selection algorithms for
classification and clustering,” IEEE Transactions on knowledge and
data engineering, Vol. 17, No. (4), PP. 491-502, 2005.
[15] Bostani, H., Sheikhan.M, “Hybrid of binary gravitational search
algorithm and mutual information for feature selection in intrusion
detection systems,” Soft computing, Vol. 21, No. (9), PP. 2307-2324,
2017.
[16] Li, Yuancheng, Rong Ma, and Runhai Jiao. "A hybrid malicious code
detection method based on deep learning." methods 9.5 (2015).
[17] Vincent, Pascal, et al. "Stacked denoising autoencoders: Learning
useful representations in a deep network with a local denoising
criterion." Journal of machine learning research11.Dec (2010): 3371-
3408.