SECURITY ISSUES AND CHALLENGES OF BIG DATA ANALYTICS

AND VISUALIZATION
1
Bipin Bihari Jayasingh, 2M. R. Patra, 3D Bhanu Mahesh
1
IT Dept., CVR College of Engineering, Ibrahimpatan (M), RR Dist-501510, Telangana State.
Email: bbjayasingh9@rediffmail.com
2
Dept. of Computer Science, Berhampur University, Berhampur-7, Orissa, India.
Email: mrpatra12@gmail.com
3
IT Dept., CVR College of Engineering, Ibrahimpatan (M), RR Dist-501510, Telangana State.
Email: dbhanumahesh@gmail.com

Abstract- The big data environment supports to
resolve the issues of cyber security in terms of
finding the attacker. There are security
challenges of big data as well as security issues the
analyst must understand. In this paper, the
challenges faced by an analyst include the fraud
detection, network forensics, data privacy issues
and data provenance problems are well studied. It
is more focused on the tools and techniques of
data mining for the use of big data analytics in
terms of security and the use of techniques for
security to protect big data in terms of applying
encryption capabilities. This rigorous study leads
to make use of available tools and techniques of
data analytics in the systematic implementation of
a security system as well as forensic system. The
paper also proposes the Bayesian classification
algorithm of classification mining is suitable for
predicting the attack type in the internetworked
environment. The forensic aspects of big data
issues and challenges are more emphasized in
order to help the forensic investigator in case of a
network attack.
Keywords: Network Forensics, Big Data Analytics,
Fraud Detection, Privacy Issues, Data Provenance
Problems, Visual Analytics.
I. INTRODUCTION
There are various techniques of big data for data
analysis, future data prediction and the incidence
monitoring [10]. The more attractive system for
many organizations is security incident and event
management (SIEM) which is used for fraud
detection and play a major role in the security
environment. The organizations deploy various
intrusion detection sensors, sensor for network
monitoring, tools for login and other tools to protect
the networked system which in turn generates diverse
data types is very difficult to manage. The security
vendors have to develop such a system which can
aggregate the diverse data sets and correlate to
present the data sets using statistics in a dashboard to
data scientist. It is the major requirement for a data
scientist how to develop a tool that can correlate and
consolidate the diverse datasets and to make as a
source for longer period [2].
The big data visualization and analytic techniques are
useful to analyze the huge network traffic data
available in the storage hub to address the cyber
security [2, 8]. The other way of analyzing the
network traffic data after the attack is also important
to extract the evidence against the crime is through
big data analytics and visualization [3, 4]. The
analysis after the attack using big data analytics in the
storage hub of an enterprise is called a post hoc
forensic analysis.
There is a work related to HACE theorem that shows
the large volume of data, diverse data sources,
centralized and distributed control of data which
require to explore the relationships among the data. It
also includes the processing model and the resolution
of big data in the perspective of data mining [15].
The data mining emerged with privacy preserving
data mining (PPDM) where data can be prepared
according to the suitability of the data mining
algorithms without compromising the security [16].
The remainder of the paper is followed as:
section 2 focuses more on the security challenges of
the big data; section 3 discusses specifically the
security issues of big data analytics and the use of
security mechanism to protect the big data; section 4
proposes the Bayesian classification algorithm to
predict the class level; section 5 provide the
concluding remark.

978-1-5090-5256-1/16/$31.00 2016
c IEEE 204
 II. SECURITY CHALLENGES
The big data analytics has several security challenges
that must be addressed to understand the real
potential. The challenges we consider in this section
are the fraud detection, network forensics, data
privacy issues and data provenance problems.
A. Fraud Detection
There are quintillion bytes of data [1] are created and
10 to 100 billion events are generated by a large
enterprise [2]. As the events are generated from
multiple and heterogeneous sources, the enterprise
must deploy more devices, hire more employees and
run more software for post hoc forensic analysis. As
a result, the efficacy of the big data is achieved
through efficient analysis of data and prediction of
data class. Existing analytical techniques are not
sufficient in the large scale analysis and processing of
big data events, so the big data analytics has attracted
the interest of the security community [6].
The current visible challenge in the society is the
fraud detection that must use big data analytics. The
most affected companies who really need fraud
detection are commercial Banks, credit cards ad
phone companies etc. however, it is not enough
economically to mine big data with a custom built
infrastructure [7]. Hence, big data technologies are
useful though it facilitate a wide variety of
institutions with an affordable infrastructure for
security monitoring. In particular, there are a variety
of big data technologies in the market like the
Hadoop map reduce that includes Pig, Hive, Mahout,
and RHadoop. There are also analysis tools which
work faster with an unprecedented scale when it
apply to large scale heterogeneous datasets are like
mining of stream, processing of complex event and
NoSQL databases. These are the only technologies
that facilitate the storage, maintenance, and analysis
of security information. However, the examples of
efficient data process for security are dependent on
MapReduce that is used by the WINE and Bot-
Cloud2.
B. Network forensics
Network forensics is a system that continuously
monitoring the network traffic in order to uncover the
unusual behavior in the normal traffic or to discover
the abnormal pattern in the network traffic that may
give indication towards some kind of network attack
[7]. Network forensics is an investigation process that
presents handling of large amount of network traffic
and analyzes the huge volume of data in the
internetwork environment. There are also severe
2016 2nd International Conference on Contemporary Computing and Informatics (ic3i)
challenge which includes the problems associated in
accessing the network devices. The challenges face
by the forensic investigator can be resolved through
big data analytics only. The paper discusses a data
mining classification technique called Bayesian
classification may be very useful to predict the class
level of an attack.
C. Data Privacy Issues
The privacy of data is entirely dependent on the
organization how the organization uses the data. The
data has to be used for the purpose it is collected and
no more apart from that particular. The data should
not be shared among others to get business profits or
to get names in the market. However, there are many
tools and techniques in big data environment that
extract the private data and make violations easier.
The security policies have to be enforced along with
privacy policy for all the entities involved in the
process. Now the requirement is to develop an
application by understanding the privacy policies and
practices to avoid the privacy violations [2].
The role of the architects and designers play a major
role in safeguarding the data stores as well as the
privacy of data. The privacy cannot be preserved due
to availability of data to many parties to attract
towards marketing and advertising. Sometimes the
private data can be used by the law enforcement
agencies for national security. There is also
possibility of stealing the private data by the intruders
to pretend to be authorized personnel [2]. The privacy
issues related to data mining are viewed from a wider
perspective and investigate various approaches that
can help to protect sensitive information [16].
D. Data Provenance Problem
The term data provenance refers to the
origin of data i.e. when and where the data was first
generated and the location of that data [16]. Data
provenance is one of the problems in big data
because data is coming from so many sources which
some may be trust worth and some may not. Data
provenance causes problems related to
trustworthiness of data. One example might be old
articles or news about a company may be posted by
the big data analytical tools which cause to fall down
of shares of that company. So we must consider the
authenticated and trustworthy data when analyzing
the data using big data tools. Integrity and
authenticity are the two parameters that we have to
consider when we want to analyze the data. We must
implement statistical and machine learning
205
 algorithms to examine the malicious data [2].
Researchers have to develop some innovative
techniques to overcome data provenance problem [5].
Though there are problems with data provenance it is
having its own advantages like we can keep track of
data from origin of the data to analysis of data. More
exploration is need in data provenance and
visualization techniques will help to easily identify
the data provenance problems [17].
III. SECURITY ISSUES OF BIG DATA
Every day the world is generating zetta bytes of data.
The data that is generating by world in this 2 years is
equal to 90% of data generated earlier. Big data can
be viewed in terms of four dimensions. They are
Volume, Velocity, Variety, and Veracity. Analytics is
the science of processing large volumes of data with
the help of mining algorithms and high end
performance computing devices and generates results
within no time or taking considerable amount of time.
Security issues is one of the major concerns when we
are dealing with Big Data. Some of the security
issues in big data includes analyzing the data in real
time, storage of transaction logs and data in a secured
manner, access controls at granular level and data
provenance.
There is a need to process the vast and variety of
data with a lighting speed as the data is coming with
huge velocity. The analysts must use various methods
to find the patterns and store the data securely. The
security mechanisms includes encryption and
decryption of data and must be part of analysis as the
unethical bigdata experts may analyze or gather the
data without users notice. Big Data is changing the
landscape of the security technologies for network
monitoring and forensics. New engineers must be
educated in knowing the value of privacy issues and
to develop the tools according to the commonly
agreed privacy guidelines[2].
A. Use of Big Data Analytics for Security
Encrption and decryption must be part of analysis as
the unethical big data experts my analyze or gather
the data without user or organization notice. We can
use some of the encryption algorithms like blow
fish,Rijndeal 3DES algorithms for encryption. In big
data analytics we can explore the data and gain new
insight knowledge using advanced data mining
techniques. Hadoop is one of the tool used to process
the data and we need tools to improve the security of
systems. Traditional security mechanisms are not
sufficient to examine the huge amount of data [2].
206 2016 2nd International Conference on Contemporary Computing and Informatics (ic3i)
Big data refers to huge and heterogeneous data where
traditional database management system cannot
process. In Big data the data is gathered in real-time
from different sources like IoT devices, facebook,
twitter etc for analysis. Big data tools are used to
analyze huge amount of data in short time to extract
interesting patterns. Big data is implementing by
organizations to gain insights into the patterns present
in the data in order to increase revenue, to overcome
competitiveness and providing better customer
satisfaction [3]. Big data mining helps decision
makers to gain insights of data which will help in
viewing future opportunities in business [4].
B. Use of Security to Protect Big Data
Protecting the data is one of the important task
to be performed by any organization. When talking
about the big data there must be sophisticated
mechanisms to protect the data along with analysis of
data. The security breaches may be big for big data as
the data is coming from different sources. Different
solutions for securing the big data is security at
application level using API, at the level of columns in
databases and at the level of file system, account
monitoring and maintaining log analysis. Log
analysis can be performed from multiple sources with
the help of intrusion detection and intrusion
preventive systems. Some of the encryption methods
includes transparent encryption and application
encryption. Transparent encryption controls access at
the file level. Application encryption encrypts
specific columns in an application before it writes the
fields to a database.
Some vendors offer big data encryption capabilities
at the level of nodes but not at internal levels of data.
Big data poses serious threats without right
encryption and right security. The encryption
techniques even do not protect the log files due to the
intelligence of the attacker beyond the limitation of
security. Further, the IT experts have to contend the
key fragment, policy and administration to apply
standards consistently to improve the performance in
big data environment [5].
C. Visual Analytics
Visualization is one of the most powerful
representations of data [2]. It helps humans in
viewing the data in the form of graphs, images,
piecharts etc. Even though considerable work was
done visualizing the big data is one of the challenges
researchers have to focus. Visual analytics is not
about representing simply the data but providing an
environment to visualize and tools to analyze the data
which will help to gain insights into the data.
Dynamic data visualization is one the area which is
becoming popular in today’s world. There are many
software which provides visualization for processing
of big data like tableau, d3.js, timeline etc. in order to
make overviews, summaries and drill down to a level
where we can extract patterns and correlation from
the data sets [6]. Visual Analytics is the combination
of big data analytics and interactive visualization
techniques. The main challenge of visual analytics is
to embed or to support the big data to represent the
data. The Visual Analytics must help in visualizing
the data and support the different views of big data
analysis. Application of visual analytics includes
early fraud detection in usage of credit card, weather
monitoring, network analysis and forensic analysis.
The data analyst may put complex analytical queries
which will initiate the underlying analytical
algorithms that must be supported by the machine
learning and statistics to get accurate results from
larger datasets. The results can be visualized through
graphical presentations and interactive user interfaces
for further manipulations [9].
IV. BAYESIAN CLASSIFICATION MINING
Forensic profiling is the technique used to develop
information and helps in deriving the evidence with
the help of data mining. The forensic profiling
patterns will be derived or generated from different
types of data sources. Applying data mining
techniques on storage devices of portable type is not
focused heavily by the researchers [12]. The
extraction of historical data can be drawn from
SCADA (Supervisory Control And Data Acquisition)
systems is one of the area the where we combine the
data mining and network forensics. We propose the
Naïve Bayesian Classifier to predict the class level of
network flows under the category of network
attacks[13,16]. Decision trees, regression methods
are the algorithms used for SCADA systems. But the
similarity based machine learning techniques are
more robust and can be implemented using sparse
matrix [18]. Recent architectures also incorporate
mechanisms for monitoring process behavior,
analyzing trends, and optimizing plant performance
[14].
Naïve Bayesian classification is a supervised
learning algorithm which will help in predicting the
2016 2nd International Conference on Contemporary Computing and Informatics (ic3i)
class labels. The basic assumption made in Naïve
Bayesian classification is that each attribute is
independent of other attribute. This kind of
assumption we call as class conditional
independence. This assumptions help in retrieving the
results in a effective and fast manner. This Naïve
Bayesian classifier is used when we want to predict
the class labels probabilistically. The naïve Bayesian
classifier is discussed below.
Consider D be the data set which consists of
tuples. Let T be a tuple in the Dataset D, and in naïve
Bayesian classification T is treated as “evidence”.
We consider T belongs to a class C using some
hypothesis S. According to Bayes Theorem
Probability(S|T)= Probility(T|S) * probability(S) /
probability(T).
Where Probability(T) is the prior probability of T.
Probability(S) is the prior probability of S.
Probability(T|S) is the post probability of T
conditioned on S. Here we have to find the post
probility of S conditioned on T i.e. Probability(S|T).
In Naïve Bayesian Classifier, the classifier estimates
that T belongs to a class having highest posterior
probability. It means that a tuple T belongs to class C
if and only if Probability(Ci|T) > probability (Cj|T).
V. CONCLUSION
We propose to develop a system using big
data analytics technique that can provide an analyzed
information for forensic experts and reduce the time
and cost of forensic analysis because not all the
information captured or recorded can be useful for
analysis or evidence. The study undertakes more
emphasis on advanced data mining tools and
techniques relating with big data analytics. In future
the analysis of particular network traffic will be
carried out in order to maintain sufficient information
as evidence against an unusual event using big data
analytics.
REFERENCES
[1] “IBM What Is Big Data: Bring Big Data to the
Enterprise,” http://
www-01.ibm.com/software/data/bigdata/, IBM, 2012.
[2] Alvaro A. Cárdenas, Pratyusa K. Manadhata,
Sreeranga P. Rajan, Big Data Analytics for Security,
IEEE Security & Privacy, 1540-7993/13/$31.00 ©
2013 IEEE, pp.74-76.
207
[3] A. Katal, M. Wazid, R. Goudar. “Big data Issues,
challenges, tools and good practices”, in the sixth
international conference on contemporary computing,
Aug. 2013, pp. 404-409.
[4] D. F. Nettleton, commercial data mining:
processing analysis and modeling for predictive
analytics projects. 1st ed. Boston, United states,
Morgan Kauffman Publishers-Elsevier, 2014.
[5] Domenico Talia, Clouds For Scalable Big Data
Analytics, IEEE Computer Society, 0018-9162/13,
2013, pp.98-101.
[6] R. Nambiar, R. Bharadwaj, A. Sethi and R.
Vargheese.“ a look at challenge and opportunities in
big data analytics in health care”, in IEEE
international conference in Big Data, Oct. 2013, pp.
17-22.
[7] Khoa Nguyen, Dat Tran , Wanli Ma, Dharmendra
Sharma, An approach to detect network attacks
applied for network forensics, IEEE 11th
International Conference on Fuzzy Systems and
Knowledge Discovery (FSKD),
DOI:10.1109/FSKD.2014.6980912 19-21 Aug. 2014,
pp. 655 – 660.
[8] E. J. Palomo, J. North ; D. Elizondo ; R. M.
Luque, Visualisation of network forensics traffic data
with a self-organising map for qualitative features,
IEEE International Joint Conference on Neural
Networks (IJCNN), The 2011, DOI:
10.1109/IJCNN.2011.6033434, July 31 -Aug. 5 2011,
pp. 1740 – 1747.
[9] D. Bruschi, M. Monga ; E. Rosti, Trusted Internet
forensics: design of a network forensics appliance,
IEEE Workshop of the 1st International Conference
on Security and Privacy for Emerging Areas in
Communication Networks, 2005, DOI:
10.1109/SECCMW.2005.1588292, 5th -9th Sept.
2005, pp. 33 – 35.
[10] Yangseo Choi, Joo-Young Lee ; Sunoh Choi ;
Jong-Hyun Kim, Introduction to a network forensics
system for cyber incidents analysis, 18th IEEE
208 2016 2nd International Conference on Contemporary Computing and Informatics (ic3i)
International Conference on Advanced
Communication Technology (ICACT), DOI:
10.1109/ICACT.2016.7423270, Jan. 31 2016-Feb. 3
2016, pp. 50 – 55.
[11] Sherif Saad, Issa Traore, Method ontology for
intelligent network forensics analysis, Eighth IEEE
Annual International Conference on Privacy Security
and Trust (PST), DOI: 10.1109/PST.2010.5593235,
17-19 Aug. 2010, pp. 7 – 14.
[12] V.H. Bhat, “A Novel Data Generation Approach
for Digital Forensic Application in Data Mining,”
Proc. 2nd Int’l Conf. on Machine Learning and
Computing (ICMLC 10), IEEE, 2010, pp. 86-90.
[13] F. Camastra, A. Ciaramella, and A. Staiano,
“Machine Learning and Soft Computing for ICT
Security: An Overview of Current Trends,” J.
Ambient Intelligence and Humanized Computing,
Oct. 2011; doi:10.1007/s12652-011-0073-z.
[14] T. Kilpatrick et al., “An Architecture for
SCADA Network Forensics,” Proc. IFIP Int’l Conf.
Digital Forensics (IFIP 06), Nat’l Center for Forensic
Science, 2006, pp. 273-285.
[15] Xindong Wu, Xingquan Zhu, Gong-Qing Wu,
Wei Ding, Data Mining with Big Data, IEEE
TRANSACTIONS ON KNOWLEDGE AND DATA
ENGINEERING, VOL. 26, NO. 1, JANUARY 2014,
pp. 97-107.
[16] Lei Xu, Chunxiao Jiang, Jian Wang, Jian Yuan,
Yong Ren, Information Security In Big Data: Privacy
And Data Mining, DOI
10.1109/Access.2014.2362522, IEEE ACCESS,
October 20, 2014, Pp. 1149-1176.
[17] Y. L. Simmhan, B. Plale, and D. Gannon, A
survey of data provenance in e-science,'' ACM
Sigmod Rec., vol. 34, no. 3, 2005, pp. 31-36.
[18] Dorit S. Hochbaum, Philipp Baumann, Sparse
Computation for Large-Scale Data Mining, IEEE
TRANSACTIONS ON BIG DATA, VOL. 2, NO. 2,
APRIL-JUNE 2016, pp. 151-174.