Query Pattern Access and Fuzzy Clustering Based Database
Intrusion Detection System.
Indu Singh
Department of Computer Science and Engineering
Delhi Technological University
Delhi, India
indu.singh.dtu14@gmail.com
Abstract- Hackers and malicious insiders
perpetually try to steal, manipulate and corrupt
sensitive data elements and an organization’s
database servers are often the primary targets of
these attacks. In the broadest sense, misuse
(witting or unwitting) by authorized database
users, database administrators, or
network/systems managers are potential insider
threats that our project intends to address. Insider
threats are more menacing because in contrast to
outsiders (hackers or unauthorised users), insiders
have authorised access to the database and have
knowledge about the critical nuances of the
database. Database security involves using
multitude of information security controls to
protect databases against breach of
confidentiality, integrity and availability (CIA). It
involves plethora of controls such as technical,
procedural/administrative and physical. We hence
intend to propose an Intrusion Detection System
(IDS). QPAFCS(Query Pattern Access and Fuzzy
Clustering System) that monitors a database
management system and prevents inference
attacks on sensitive attributes, by means of
auditing user access patterns .
Keywords: Database Intrusion Detection, Fuzzy
Clustering, User Access Pattern, Insider Attacks,
Dubiety Score
I. INTRODUCTION
Data protection from insider threats is essential to
most organizations. Attacks from insiders could be
more damaging than those from outsiders, since in
most cases insiders have full or partial access to the
data; therefore, traditional mechanisms for data
protection, such as authentication and access
control, cannot be solely used to protect against
insiders. Since recent work has shown that insider
attacks are accompanied by changes in the access
Shivam Gupta, Shivam Maini, Shubham, Simran
Seth
Computer Science and Engineering
Delhi Technological University
Delhi, India
shivam_bt2k16@dtu.ac.in,
shivammaini_bt2k16@dtu.ac.in,
shubham_bt2k16@dtu.ac.in,
simran_bt2k16@dtu.ac.in
patterns of users, user access pattern mining [1] is
a suitable approach for the detection of these
attacks. It creates profiles of the normal access
patterns of users using past logs of users accesses.
New accesses are later checked against these
profiles and mismatches indicate potential attacks.
A security technique called Access control[2] can
regulate who can view or use resources in a
computing environment. There are diverse access
control systems that perform authorization,
identification, authentication and access approval.
Intrusion Detection Systems[3]. scrutinise and
unearth surreptitious activities perpetrated by
malevolent users. IDS work by either looking for
signatures of known attacks or deviations of normal
activity. Normally, IDS undergo a training phase
with intrusion free data wherein they maintain a log
of benign transactions. Pattern matching [4] is then
used to detect whether or not an action is
malign. This is called anomaly based detection[5].
When errors are detected using their known
“signatures” from previous knowledge of the
attack, it is called signature based detection[6].
These malicious actions once detected are then
either blocked or probed depending upon the
organisation’s policy. However, IDS need to be
dynamic, robust and quick. Different architectures
for IDS function differently and have different
measures of performance. Every organisation
needs to make sure that the IDS it uses satisfies its
requisites.
Several AD techniques have been proposed to
detect anomalous data accesses. Some rely on the
analysis of input queries based on the syntax.
Although these approaches are computationally
efficient, they are unable to detect anomalies in
scenarios like the following one. Consider a clerk in
an organization who issues queries to a relational
database that typically selects a few rows from
specific tables. An access from this clerk that selects
all or most of the rows of these tables should be
considered anomalous with respect to the daily
access pattern of the clerk. However, approaches
based on syntax only are not able to classify such
access as anomalous. Thus, syntactic approaches
have to be extended to take into account the
semantic features of queries such as the number of
result rows. An important requirement is that
queries should be inspected before their execution
in order to prevent malicious queries from making
changes to the database.
From the technical perspective, the main purpose is
to ensure the effective enforcement of security
regulations. Audit is an important technique of
examining whether user behaviours in a system are
in conformance with security policies. Many
methods audit a database processing by comparing
a user SQL query expression against some
predefined patterns so as to find out an anomaly.
But a malicious query may be made up as good
looking so as to evade such syntactic detection. To
overcome this shortcoming, the data-centric
method further audits whether the data a user
query actually accessed has involved any banned
information. However, such audit concerns a
concrete policy rather than the overall view of
multiple security policies. It requires clear audit
commands that are articulated by experienced
professionals and much interactive analysis. Since
in practice an anomaly pattern cannot be
articulated in advance, it is difficult to detect such
fraud by the current audit method.
The anomaly detection technology is used to
identify abnormal behaviours that are statistical
outliers. Some probabilistic methods learned
normal patterns, against which they detected an
anomaly. But these methods assume very few users
are deviated from normal patterns. In case there
are a number of anomalous users, the normal
pattern would be diverged. These works do not
examine user behaviour from either a historical or
an incremental view, which may overlook some
malicious behaviour. Furthermore, if a group of
people collude together, it is difficult to find them
by the current methods.
In QPAFCS we tackle the insider threat problem
using different approaches. We take into
consideration the fact that certain data elements
are more critical to the database as compared with
other data elements. Thus we pay special attention
to the security of such critical data elements. We
also recognise the presence of data attributes in a
system which can be manipulated to indirectly
influence the crucial data attributes. We address
the threat to our critical data elements using such
attributes also.
We investigate a suspected user also from the
diachronic view by analysing his/her historical
behaviour. We store a measure denoting how
suspicious a user has been. The greater this
measure, the greater the chances of the query
being malicious. This measure also solves the
problem of gradually malicious threat since the
historical statics measures the accumulative
results.
The main purpose of our project is to recognise user
access pattern. Our Intrusion Detection
System(IDS) pays special attention to certain
semantically critical data elements, along with
those elements which can be used to infer the
aforementioned elements. We present an
innovative approach to combine a user’s historic
and present access pattern and hence classify the
incoming transaction as malicious/non-malicious.
Using FCM, we partition the users into fuzzy
clusters. Each of these clusters contains a set of
rules in their cluster profiles. In the detection phase,
new transactions are checked against rules in these
clusters, and then a suitable action is taken
depending upon the nature of transaction. The
main advantage of our IDS lies in its ability to
prevent inference attacks on Critical Data Elements.
The remainder of this work is organized as follows.
In Sect. 2, we present prior research related to this
work. Section 3 introduces the fuzzy clustering and
belief update framework. Section 4 discusses the
approach using examples. In Sect. 5, we discuss how
to apply our method into practical system.
Experimental evaluation is discussed in Section. 6.
I. RELATED WORK
Numerous researchers are currently working in the
field of Network Intrusion Detection Systems, but
only a few have proposed research work in
Database IDSs. Several systems for Intrusion
Detection in operating systems and networks have
been developed, however they are not adequate in
protecting the database from intruders.[11] ID
system in databases work at query level,
transaction level and user (role) level. Bertino et.
Al described the challenges to ensure data
confidentiality, integrity and availability and the
need of database security wherein the need of
database IDSs to tackle insider threats was
discussed.
Panda et. Al[19] propose to employ data mining
approach for determining data dependencies in the
database system. The classification rules reflecting
data dependencies are deduced directly from the
database log. These rules represent what data
items probably need to be read before an update
operation and what data items are most likely to be
written following this update operation.
Transactions that are not compliant to the data
dependencies generated are flagged as anomalous
transactions.
Database IDSs include Temporal Analysis of queries
and Data dependencies among attributes, queries
and transaction. Lee et al.[28] proposed a Temporal
Analysis based intrusion detection method which
incorporated time signatures and recorded update
gap of temporal attributes. Any anomaly in update
pattern of the attribute was reported as an
intrusion in the proposed approach. The
breakthrough introduction to association rule
mining by Aggarwal et al.[22] helped in finding data
dependencies among data attributes, which was
incorporated in the field of intrusion detection in
Databases.
During the initial development of data dependency
association rule mining, DEMIDS ,a misuse
detection system for relational database systems
was proposed by Chung et al.[7] Profiles which
specified user access pattern were derived from the
audit log and Distance Metrics were further applied
for recognizing data items These were used
together in order to represent the expanse of users.
But once the number of users for a single system
becomes substantial, maintaining profiles becomes
a redundant procedure. Another flaw was the
system assuming domain information about a given
schema.
Hu et al.[16] presented a data mining based
intrusion detection system, which used the static
analysis of database audit log to mine
dependencies among attributes at transaction level
and represented those dependencies as sets of
reading and writing operations on each data item.
In another approach proposed by Hu et al.
,techniques of sequential pattern mining have
been applied on the training log, in order to
identify frequent sequences at the transaction
level. This approach helped in identifying a group of
malicious transactions, which individually complied
with the user behavior. The approach was
improved in by Hu et. al by clustering legitimate
user transaction into user tasks for discovery of
inter-transaction data dependencies.
The method proposed extends the approach by
assigning weights to all the operations on data
attributes. The transactions which didn’t follow the
data dependencies were marked as malicious. The
major disadvantage of user assigned weights is the
fact that they are static and unrelated to other data
attributes. Kamra et. [27] al employed a
clustering technique on an RBAC model to form
profiles based on attribute access which
represented normal user behavior. An alarm is
raised when anomalous behavior of that role profile
is observed.
(Bezdek, Ehrlich & Full, 1984) proposed the Fuzzy C-
Means Algorithm. The basic idea behind this
approach was to illustrate the similarity a data point
may share with each of the clusters with help of a
function often referred to as membership function.
This measure of similarity lies between zero and
one signifies the extent of similarity between the
data point and the cluster and is termed as the
membership value. The main aim of this technique
is to construct fuzzy partitions of a particular data
set.
Y. Yu et. al.[29] illustrated a fuzzy logic based
Anomaly Intrusion Detection System. A Na¨ıve
Bayes Classifier is used to classify an input event as
normal or anomalous. The basis of classifier is
formed by the independent frequency of each
system call from a process in normal conditions.
The ratio of the probability of a sequence from a
process and the probability not from the process
serves as the input of a fuzzy system for the
classification.
A hybrid approach was described by Doroudian et.
al [26] to identify intrusion at both transaction and
inter-transaction level. At the transaction level, a
set of predefined expected transactions were
specified to the system and a sequential rule mining
algorithm was applied at the inter transaction level
to find dependencies between the identified
transactions. The drawback of such a system is that
sequences with frequencies lower than the
threshold value are neglected. Therefore, the
infrequent sequences were completely overlooked
by the system, irrespective of their importance. As
a result, the True Positive Rate falls down for the
system.
The above drawback was overcome by Sohrabi et.
Al[20] who proposed a novel approach ODARDM, in
which rules were formulated for lower frequency
item sets, as well. These rules were extracted using
leverage as the rule value measure, which
minimized the interesting data dependencies. As a
result, True Positive Rate increased while the False
Positive Rate decreased. In recent developments,
Rao et. Al [21] presented a Query Access
detection approach through Principal Component
Analysis and Random Forest to reduce data
dimensionality and produce only relevant and
uncorrelated data. As the dimensionality is
reduced, both, the system performance and True
Positive rate increases.
In 2009, majumdar et. Al[15] propose a
comprehensive database intrusion detection
system that integrates different types of evidences
using an extended Dempster-Shafer theory. Besides
combining evidences, they also incorporate
learning in our system through application of prior
knowledge and observed data on suspicious
users.In 2016, bertino et. Al[14] e tackle the insider
threat problem from a data-driven systemic view.
User actions are recorded as historical log data in a
system, and the evaluation investigates the date
that users actually process. From the horizontal
view, users are grouped together according to their
responsibilities and a normal pattern is learned
from the group behaviours. They investigate a
suspected user also from the diachronic view by
comparing his/her historical behaviours with the
historical average of the same group.
Anomaly detection has been an important research
problem in security analysis, therefore
development of methods that can detect malicious
insider behavior with high accuracy and low false
alarm is vital [10]. In this problem layout, McGough
et al [8] designed a system to identify anomalous
behavior of user by comparing of individual user’s
activities against their own routine profile, as well
as against the organization’s rule. They applied two
independent approaches of machine learning and
Statistical Analyzer on data. Then results from these
two parts combined together to form consensus
which then mapped to a risk score. Their system
showed high accuracy, low false positive and
minimum effect on the existing computing and
network resources in terms of memory and CPU
usage.
Bhattacharjee et al proposed a graph-based
method that can investigate user behavior from
two perspectives: (a) anomaly with reference to the
normal activities of individual user which has been
observed in a prolonged period of time, and (b)
finding the relationship between user and his
colleagues with similar roles/profiles. They utilized
CMU-CERT dataset in unsupervised manner. In their
model, Boykov Kolmogorov algorithm was used and
the result compared with different algorithms
including Single Model One-Class SVM, Individual
Profile Analysis, k-User Clustering and Maximum
Clique (MC). Their proposed model evaluated by
evaluation metrics Area-Under-Curve (AUC) that
showed impressive improvement compare to other
algorithms [9].
T. Rashid et al. proposed that parameter learning
task in HMMs is to find, given an output sequence
or a set of such sequences, the best set of state
transition and emission probabilities. The task is
usually to derive the maximum likelihood estimate
of the parameters of the HMM given the set of
output sequences. No tractable algorithm is known
for solving this problem exactly, but a local
maximum likelihood can be derived efficiently using
the Baum–Welch algorithm or the Baldi–Chauvin
algorithm. The Baum–Welch algorithm is a special
case of the expectation-maximization algorithm. If
the HMMs are used for time series prediction, more
sophisticated Bayesian inference methods, like
Markov chain Monte Carlo (MCMC) sampling are
proven to be favorable over finding a single
maximum likelihood model both in terms of
accuracy and stability.[12] Log data are considered
as high-dimensional data which contain irrelevant
and redundant features. Feature selection methods
can be applied to reduce dimensionality, decrease
training time and enhance learning performance .

3. OUR APPROACH
3.1 Basic Notations
Large organisations deal with tremendous amount
of data whose security is of prime interest. The data
in databases comprises of attributes describing real
life objects called as entities. The attributes have
varying levels of sensitivity, i.e. not all attributes are
equally important to the integrity of database. As
an example, the signatures and other biometric
data are highly sensitive data attributes for a
financial organisation like Bank in comparison to
others like name, gender etc. So, unauthorised
access to the crucial attributes is of a greater
concern. Only certain employees may have access
to such data elements and access by all others must
be blocked instantaneously to ensure
Confidentiality and consistency of data.
Our proposed model QPAFCS(Query Pattern Access
and Fuzzy Clustering System) pays special attention
to sensitive data attributes and they have been
referred to as CDE (Critical Data Elements) in the
text. The attributes that can be used to indirectly
infer CDEs are also critical to the functioning of the
organisation. For instance, account number of a
user may be used to access the signatures and other
crucial details about him. Such attributes have been
referred to as DAE (Directly Associated Elements) in
the text.
We propose a two-phase detection and prevention
model that clusters users based on similarity of
their attribute access patterns and the types of
queries performed by them, i.e. our model tries to
track the user access pattern of each user and
further classify it as normal or malicious. The
superiority of our model lies in its ability to prevent
unauthorised retrieving and modification of most
sensitive data elements (CDEs). Our model also
makes sure that the query pattern for access of
CDEs is specific and fixed for a particular user to
avoid data breaches, i.e. the user associates himself
with his regular access behaviour. Any deviation
from the regular arrangement may lead to
depreciation of user’s confidence and may act as
representative of user’s malicious intent. The
following terminologies are used:
Definition 1 (Transaction) Set of queries executed
by a user. Each transaction is represented by a
unique transaction ID and also carries the user’s ID.
Hence <Uid,Tid> act as unique identification key for
each set of query patterns. Each Transaction T is
denoted as
<Uid, Tid, <q1, q2, … qn>>
where
qi denotes the ith query, i ∈ [1 … n]
For example, suppose a user has id 1001. He/she
then executes the following set of SQL queries:
q1: SELECT a,b,c
FROM R1,R2
WHERE R1.A>R2.B
q2: SELECT P
FROM R5
WHERE R5.P==10
Then this is said to be a transaction of the form:
t=<1001,67,<q1,q2>>
Definition 2 (Query) A query is a standard database
management system token/request for inserting
and retrieving data or information from a database
table or combination of tables. We define query as
a read or write request on an attribute of the
relation. A query is represented as
<O(D1), O(D2), … O(Dn)>
where,
D1 , D 2 , … D n ∈ R s
where Rs is the relation schema and Di are the
attributes. O represents the operations i.e. Read or
write Operations. O ∈ {R, W}
For example, examine the following transaction:-
start transaction
select balance from Account where
Account_Number='9001';
select balance from Account where
Account_Number='9002';
update Account set balance=balance-900 where
Account_Number='9001' ;
update Account set balance=balance+900 where
Account_Number='9002' ;
commit; //if all SQL queries succeed
rollback; //if any of SQL queries failed or error
The query corresponding to this transaction is:
<<R(Account_Number),R(balance)>,
<R(Account_Number),R(balance)>,
<R(Account_Number),R(balance),W(balance)>,
<R(Account_Number),R(balance),W(balance)>>
Definition 3 (Read Sequence) A read sequence is
defined as
{R(x1), R(x2), … O(xn)}
where O represents the operations i.e. Read or
write Operations. O ∈ {R, W}. The Read sequence
represents that the transaction may need to read
all data items x1, x2, …, xn-1 before the transaction
performs operation (O∈ {R, W}) on data item xn.
For example, consider the following update
statement in a transaction.
Update Table1 set x = a + b + c where d = 90;
In this statement, before updating x, values of a, b,
c and d must
be read and then the new value of x is calculated.
So <R(a), R(b),
R(c), R(d), W(x)> ∈ RS(x).
Definition 4 (Write Sequence) A write sequence is
defined as
{O(x1), W(x2), … W(xn)}
where O represents the operations i.e. Read or
write Operations i.e. O ∈ {R, W} which represents
that the transaction may need to write all data
items x1, x2, …, xn-1 in this order after the
transaction operates on data item xn.
For example, consider the following update
statements in one transaction.
Update Table1 set x = a + b + c where a=50;
Update Table1 set y = x + u where x=60;
Update Table1 set z = x + w + v where w=80;
Using the above example, it can be noted that
<W(x), W(y),W(z)>
is one write sequence of data item x, that is <W(x),
W(y),W(z)> ∈
WS(x), where WS(x) denotes the write sequence set
of x.
Definition 5 (Read Rules (RR)) Read rules are the
association rules generated from Read sequences
whose confidence is greater than the user defined
threshold (Ψconf). A read rule is represented as
{R(x1), R(x2) ...} ⇒ O(x).
For all sequential patterns <R(x1), R(x2), …, R(Xn-1),
O(xn) > in read sequence set, generate the read
rules with the format {R(x1), R(x2) ...} ⇒ O(xn). If the
confidence of the rule is larger than the minimum
confidence (Ψconf), then it’s added to the answer
set of read rules, which implies that before xn, we
need to read x1,x2…….. xn-1
For example:
The Read Rule corresponding to the read sequence
<R(a), R(b),
R(c), R(d), W(x)> is:
{R(a), R(b), R(c), R(d)} ⇒ W(x)
Definition 6 (Write Rules (WR)) Write rules are the
association rules generated from write sequences
whose confidence is greater than the user defined
threshold (Ψconf). A write rule is represented as
O(x) ⇒ {W(x1), W(x2) …}
For all sequential patterns O(x), W(x1), W(x2), …,(xk)
in the write sequence set, generate the write rules
with the format O(x)→W(x1), w(x2), …, w(xk). If the
confidence of the rule is larger than the minimum
confidence (Ψconf), then it’s added in the set of write
rules which depicts after updating x, data
items x1, x2, …, xk must be updated by the same
transaction.
For Example: The write rule corresponding to the
write sequence
<W(x), W(y),W(z)> is W(x) ⇒ {W(y),W(z)}
Definition 7 (Critical Data Elements (CDE)) They are
semantically defined data elements crucial to the
functioning of the system. They are the data
attributes of prime significance having direct
correlation to the integrity of the system. In a
vertically hierarchical organisation, these are the
attributes accessed only by the top level
management, and the access by lower levels of
hierarchy is strictly protected.
 Type of Attribute
Critical data Elements
Directly Associated
Elements
Sensitivity Level historic transactional data. This score summarizes
the user’s historic malicious access attempts.
Highest Dubiety Score attempts to quantify the personnel
Medium vulnerability that the organisation faces because of
a particular user.

Normal Attributes Low Dubiety Score is indicative of the amount of

Table 3.1 Types of attributes and their sensitivity
levels
CDEs are tokens of behaviour that our model uses
for the malicious activity recognition of users of
system.
deviation between the user’s access pattern and his
designated role. Dubiety Score combined with the
deviation of user’s present query from his normal
behaviour pattern, yields the output of the
proposed IDS.
For our paper:

Definition 8 (Critical Rules (CR)) A set of rules that 0<=φ<=1.

contain a Critical Data Element in its antecedent or
Higher the Dubiety Score, more is the evidence
consequent.
against user following the assigned role, that is
CR = {ζ | (ζ ∈ RR ∨ ζ ∈ WR) ∩ (x ∈ CDE ∩ ({R(x1), more is the malicious intent i.e. rogue behaviour.
R(x2) …} ⇒ O(x) ∪ O(x) ⇒ {W(x1), W(x2) …}))}
Definition 11 (Dubiety Table) A table maintaining
We propose a method of user Access Pattern the record of dubiety scores of each user. It
Recognition using the Critical Rules. CR recognize contains two attributes: UserID and Dubiety Score.
the actions and goals of Users from a series of
The initial Dubiety scores are set to 1.
observations on the users' actions and the
environmental conditions, i.e. the user query Uid φ
pattern associated to the Critical data elements.
1001 1
Definition 9 (Directly Associated elements (DAE))
The attributes except those present in CDE, which 1002 1
are either part of antecedents or consequents of
1003 1
Critical Rules.
DAE = {μi| μi ∈ CR ∩ μi ∉ CDE}. 1004 1
The query patterns as perceived by our model 1005 1
QPAFCS are explored using DAEs that represent the
first level of access of the CDEs. A user's behaviour Table 3.2 Initial Dubiety Table
is represented by a set of first-order statements
The dubiety table is updated each time a user
(derived from queries) called attribute hierarchy
performs query.
encoded in first-order logic, which defines
abstraction, decomposition and functional For example:
relationships between types of access
arrangements. The unit-transactions accessing Let user 1001’s deviation from normal query is
CDEs are decomposed into attribute hierarchy quantified as 0.81, Then the updated Dubiety table
comprising of DAEs, which further represents the is as shown.
user’s most sensitive retrieval pattern. Where:
Example: ds = deviation from normal query
 R(b) → R(a) φi = Initial dubiety score.
 R(b), R(c) → R(a)
If a is a CDE, then the set {b,c} represents DAEs. Uid 𝑑𝑠 ∗ фi

Definition 10 (Dubiety Score(φ)) A measure of 1001 0.9

anomaly exhibited by a user in the past based on his
 1002 1 transaction log and one data file that is exclusive to
1003 1 input to the learning phase algorithm is the
1004 1 transaction log, with only authorised and consistent
1005 1 activity and is used to form user profiles, role
Table 3.3 Updated Dubiety Table
The Updated Dubeity table is hence stored in
memory for further processing.
3.2 Learning Phase
We start our learning phase by reading the
training dataset into the memory and extracting
useful patterns out of it. Our system requires non-
malicious training dataset composed of
transactions executed by trusted users. The model
aims at generating user-profiles from the
transaction-logs and quantifies deviation from
normal behaviour i.e. this phase aims to recognise
and characterise the user activity pattern on the
basis of their queries arrangement. The following
are various components of architecture of the
proposed model:
Fig 3(a) Learning Phase Architecture
COMPONENTS OF ARCHITECTURE:
Training data: A transaction log is a sequential
record of all changes made to the database while
the actual data is contained in a separate file. The
transaction log contains enough information to
undo all changes made to the data file as part of any
individual transaction. The log records the start of a
transaction, all the changes considered to be a part
of it, and then the final commit or rollback of the
transaction. Each database has at least one physical
the database for which it was created. Our initial
transactions. This data is free of any unauthorised
profiles etc based on normal user transactions. The
logs are scanned, and the following elements are
extracted:
a. SQL Queries
b. The user executing a given query
SQL query parser: This is a tool that takes SQL
queries as input, parses them and produces
sequences (read and write) corresponding to the
SQL query as output. The query parser also assigns
a unique Transaction ID. The final output consists of
two 3 columns: (TID), UID (User ID) and the read and
write sequence generated by the parsing algorithm.
As an Example, if the following transaction
performed by user U1001 is examined:
start transaction
select balance from Account where
Account_Number='9001';
commit; //if all SQL queries succeed
rollback; //if any of SQL queries failed or error
The parser generates a unique Transaction ID say
T1234 followed by parsing the transaction. The
parser finally yields:
< T1234,U1001,<R(Account_number),R(balance)>>
Frequent sequences generator: After the SQL
query parser generates the sequences, the
generated sequences are pre-processed. Then
weights are assigned to data items, for instance the
CDEs are given greater weight as compared to DAEs
and other normal attributes. Then finally these pre-
processed sequences are given as inputs to
frequent sequences generator. It uses the prefix
span algorithm to generate frequent sequences out
of input sequences corresponding to each UID.
Rule generator: The frequent sequences are
given as inputs to the rule generator module which
uses association rule mining to generate read rules
and write rules out of the frequent sequences.
As an example, if the input frequent sequences are:
 1. <R(m),R(n),R(o),W(a)>
Algorithm 1: DAE Generator
2. <R(m),R(n),W(o),W(a)>
Data: CDE, Set DAE = {}, RR = Set of Read
3. <R(m),W(n),W(o),W(a)>
4. <W(a),R(b),W(o)> Rules, WR = Set of Write Rules
5. <R(a),R(b),R(m),W(a)> Result: The set of Directly Associated
6. <R(a),R(b),W(m),W(b)> elements DAE
Function: DAE Generator (CDE, RR, WR)
S.No Frequent Sequences Associated
for Ω є RR ∪ WR do
. Rules
for α є Ω do
1 <R(m),R(n),R(o),W(a)> R(m),R(n),R(o) if α є CDE
→W(a) while β є Ω do
2 <R(m),R(n),W(o),W(a) R(m),R(n),W(o)
DAE {} ⃪ β
> →W(a) end
end
3 <R(m),W(n),W(o),W(a) R(m),W(n),W(o end
> ) →W(a) end
4 <W(a),R(b),W(o)> W(a),R(b)
User vector generator: Using the frequent
→W(o)
sequences for the given audit period, it generates
5 <R(a),R(b),R(m),W(a)> R(a),R(b),R(m) the user vectors. A user vector is of the form
→W(a) BID = < UID, w1, w2, w3, ... wn >
6 <R(a),R(b),W(m),W(b) R(a),R(b),W(m) where wi = |O(ai)|.
> →W(b)
|O(ai)| represents the total number of times user
Table 3.4 Rule Generator for given Example with the given Uid performs operation (O ∈ {R, W})
DAE generator: In our approach, we semantically on the aforesaid attribute ai in the pre-decided
define a class of data items known as Critical data audit period. An audit period τ refers to a period of
elements or CDEs. These CDEs and rules are given time such as one year, a time window τ = [t1, t2] or
as input to our DAE (Directly associated element) the recent 10 months. User vector is representative
generator which specifies all those elements as DAE of user’s activity.
which are present in either the antecedent or the Each of these wi would represent how frequently a
consequent of those rules that involve at least one user performs the operation on the particular data
of the CDEs. item. It also can be used in a normalized form, as is
used in our proposed model QPAFCS.

UVID = <UID, < p(a1), p(a2), p(a3), … p(an)>>

where,
𝑤
p(𝑎 ) =
∑ 𝑤

p(ak) is defined as the probability of accessing the

attribute ak.

Value of p(𝑎 ) close to 1 would mean that the user

accesses the given attribute frequently.

Cluster generator: It takes user vectors and rules

as input and generates fuzzy clusters. Users are
clustered into different fuzzy clusters based on the
similarity of their user vectors. A cluster profile
would include
Ci = <CID, {R}> UVy = <Uy, < py(a1), py(a2), py(a3), … py(an)>>

where, CID represents the cluster centroid, and of equal length n, the modified Jensen Shannon[27]
distance is computed as
{R} is a set of rules which is formed by taking the
union of all the rules that the members of the given 𝐷(𝑈𝑉 ||𝑈𝑉 )
fuzzy cluster abide by. 1 + 𝑝 (𝑎 ) ∗ 𝑤(𝑎 )
1 + 𝑝 (𝑎 ) ∗ 𝑤(𝑎 ) log
We have used Fuzzy c-means[26] clustering to ⎛ 1 + 𝑝 (𝑎 ) ∗ 𝑤(𝑎 ) ⎞
⎜ + ⎟
create cluster. Each user belongs to a cluster to a ⎜ ⎟
certain degree wij. 1 + 𝑝 (𝑎 ) ∗ 𝑤(𝑎 )
1 + 𝑝 (𝑎 ) ∗ 𝑤(𝑎 ) log
⎝ (1 + 𝑝 (𝑎 ) ∗ 𝑤(𝑎 ))⎠
Where: =
2
wij represents the membership coefficient of the ith
where, w(ai) is the semantic weight associated with
user (ui) with the jth cluster
the aith attribute
The centre of a cluster (α) is the mean of all points,
User profile generator: This module takes user
weighted by their membership coefficients[28].
vectors and the cluster profiles as input and
Mathematically,
generates user profiles. A user profile is of the form
𝑤 = Ui=<UID, < p(a1), p(a2), p(a3) … p(ak) >, < c1, c2, … cC >
|| ||
∑
| | >

where

∑ 𝑤(𝑢) 𝑢 UID is a unique ID given to each user,

𝛼 =
∑ 𝑤(𝑢)
<p(a1), p(a2), p(a3), … p(an)> is a 1-D matrix
The objective function that is minimized to create containing the probability of the user accessing a
clusters is defined as: particular attribute, and

< c1, c2, … cC > is a vector representing the

Inputs Outputs

C1 C2 C3 C4 User Vector User profile

0.2 0.2 0.2 0.4 <U1001,0.2,0.109,0.9, 0.6> <U1001,<0.2,0.1,0.9,0.6>,<0.2,0.2,0.2,0.4>>

membership coefficients of the given user for C

𝑎𝑟𝑔 𝑚𝑖𝑛 𝑤 ||𝑢 − 𝛼 || different clusters.

As an Example:
where
Table 3.5 User profile for the given Example
n is the total number of users,
Consider a system with 4 fuzzy clusters and 4
C is the number of clusters, and attributes, the given table illustrates the profile of
m is the fuzzifier. user U1001.

The dissimilarity/distance function used in the 3.3 Testing Phase

formation of fuzzy clusters is the modified Jenson
Shannon distance which is illustrated as:
Given two user vectors[13]
UVx = <Ux, < px(a1), px(a2), px(a3), … px(an)>> and
In section 3.2, the learning phase is described, in
which the system is trained using non-malicious or
benign transactions. Now the trained model can be
used to detect malicious transactions. In this phase,
a test query is obtained as input and it is compared
with the model’s perception of user’s access
pattern, and the model perpetually evaluates if the
test transaction is malicious. It is first checked
whether the user is trying to access a CDE. If yes,
the transaction is allowed only if the given user has
accessed that CDE before. Next, it is checked if any
DAE is being accessed. A user can perform write
operation on a DAE iff it is previously written by the
same user, otherwise the transaction is termed as
malicious. Next, we check if the transaction abides
by the rules that are generally followed by similar
users.

PHASES OF TESTING PHASE:

Rule generator: This module takes the sequence

as generated by the SQL query parser and gives the
rule that the input transaction follows. This can be
a read rule or a write rule and indicates the
operations done by the user, data attributes
accessed by the user and the order in which they
Algorithm 2: CDE Detector
are accessed. Now this rule can be checked for Data: Set of rules (ϒ) from test transaction, Set
maliciousness. χCDE, UID, User Profile(ϴ)
CDE Detector: The semantically critical elements
Result: Checks whether the test transaction is
referred to in our approach as CDEs are detected in malicious or normal with respect to CDE
this module. The read/ write rule corresponding to for Ѓє ϒ do
the incoming transaction is checked for the for ϱ є Ѓ do
presence of CDEs. If the rule being checked for
if ϱ є χCDE then
maliciousness contains a CDE, then it is dealt with
using the following policy:-
if w(ϱ) є Ѓ & ϴ[UID][w(ϱ)] == 0 then
Raise Alarm;
a. If read operation has been performed on any
end
CDE, i.e. r(CDE) is present in the rule and
UV[i][r(CDE)] = 0 and UV[i][w(CDE)] = 0 for
if r(ϱ) є Ѓ & ϴ[UID][r(ϱ)] == 0 &
the given user, then the transaction is termed ϴ[UID][w(ϱ)] == 0 then
as malicious. Raise Alarm;
b. If write operation has been performed on any end
CDE i.e. w(CDE) is encountered and
end
UV[i][w(CDE)] = 0 for the given user, then the
transaction is termed as malicious.
end
end

DAE Detector: This module addresses the issue of

inference attacks on CDEs. As discussed earlier,
certain data elements can be used to access the
CDEs, i.e. first order inference. This module uses the
rules mined in the learning phase to determine
which elements can be used to directly infer the
DAEs.
 Our system seeks to prevent inference attacks by matched with the corresponding rules in
especially monitoring the DAEs. We lay emphasis on the cluster of which a user is as part.
write operations on DAEs. If write operation has
been performed on any DAEs i.e. w(DAE) is Algorithm 4: Modified Jaccard Distance
present in the rule to be checked and Data: Rules R1, R2; 𝛿1, 𝛿2; Set χR1, χR2
UV[i][w(DAE)] = 0 for the given user, then the Result: Distance between the two rules (Ԏ)
transaction is termed as malicious.
Function jcDistance (R1, R2)
for Ω є R1 do
Dubiety Score Calculator and Analyser: If the χR1 ← Ω;
transaction has not been found malicious in the end
previous two modules, we check if the for Ω’ є R2 do
transaction is malicious based on the previous χR2 ← Ω’;
history of the user and the behaviour pattern of
end
all similar users (modified Jenson Shannon
∗ – ∗ –
distance). To do so, we maintain a record of Ԏ= ;
action of all users by keeping the measure of
Dubiety Score(φi).
return Ԏ;

The deviation of a user’s new transaction with his

normal access pattern is referred to as Dubiety, and  In order to quantitatively measure the
the relative measure of Dubiety is the Dubiety similarity between two rules, we use
Score. Our IDS keeps a log of the DS (Dubiety Score) modified Jaccard distance[30]:
in a separate table. A user who is a potential threat JD = 1-𝛿1(R1 R2) - 𝛿2(R1 R2- R1 R2)
tends to have a high dubiety score. Another R1 R2
intuition that our system follows is that any R2| μi > 𝛿 and i [1, k]
transaction that a user makes matches significantly
either with the transactions the same user or
similar users have made in the past.

We use a measure ds to keep a track of the

maximum similarity of the given rule. We combine
Algorithm 3: DAE Detector
ds with φ i to get the final measure of dubiety score
φ f for the given user. We define 2 thresholds ФLT Data: Set of rules (ϒ) from test transaction, Set
and ФUT. ФUT represents the upper limit for the χDAE, UID, User Profile(ϴ)
dubiety score of a non-malicious user whereas ФLT Result: Checks whether the test transaction is
denotes the lower limit. This means that if φf for a malicious or normal with respect to DAE
user comes out to be greater than ФUT, the user is
for Ѓє ϒ do
malicious. On the other hand, φf value less than ФLT
denotes a benign user. for ϱ є Ѓ do
if ϱ є χDAE then
 If the incoming rule (R1) is a write rule, then
if w(ϱ) є Ѓ & ϴ[UID][w(ϱ)] == 0 then
the consequent of the incoming rule is
matched with the corresponding rules in Raise Alarm;
the cluster of which a user is as part. A user end
is said to be the part of the ith cluster iff: end
μi > 𝛿. end
Where,
end
μi is the fuzzy membership coefficient of the
given user for the ith cluster.  The minimum value of JD is regarded as ds.
𝛿 is a user defined threshold. φi is fetched directly from dubiety table.
 If the incoming rule (R1) is a read rule, then Final dubiety score for the given user is
the antecedent of the incoming rule is calculated as:
 φf = 𝑑𝑠 ∗ фi 1003 0.2
 If φf < ФLT, the transaction is termed as non-
1004 0.6
malicious. In this case, the current dubiety
score in the dubiety table for the given user 1005 0.46
is reduced by a factor known as
“amelioration factor(Å)”. Table 3.8 calculated dubiety scores table
Thus, φi is updated as
φi = Å φi
 If ФUT > φf ≥ ФLT, the transaction is termed
as non-malicious and the dubiety table
entry for the given user is updated with φf.
 If φf ≥ ФUT the transaction is termed as
malicious.
 As an Example, Let the initial dubiety table
be:

Uid φ

1001 0.9 Taking ФLT=0.3 and ФUT=0.6

1002 0.8

1003 0.2 Uid φf Nature of Updated

Transaction
1004 0.6 φf

1005 0.7 1001 0.42 Non- 0.42

malicious
Table 3.6 Initial dubiety table
1002 0.49 Non- 0.49
Let the minimum value of ds corresponding to
malicious
each user be:
1003 0.2 Non- 0.198
malicious
Uid ds
1004 0.6 Malicious 0.6
1001 0.2
1005 0.46 Non- 0.46
1002 0.3 malicious

1003 0.2 Table 3.9 Summary of transactions of various users

1004 0.6 The Malicious Transactions are blocked in a

straightforward fashion and the Non Malicious
1005 0.3 transactions are processed. Updated Dubiety Table
is stored in database.
Table 3.7 Minimum ds values for various Users

The calculated dubiety score table:

4. Discussion

With regard to a typical credit card company

Uid φf= 𝑑𝑠 ∗ фi
dataset, some examples of critical data elements
1001 0.42 (CDEs) are: -

1002 0.49 1. CVV (denoted by a)

Card verification value (CVV) is a combination of  R(b) → R(a)
features used in credit, debit and automated teller  R(b), R(c) → Ra)
machine (ATM) cards for the purpose of
establishing the owner's identity and minimizing 5. Example to our Approach
the risk of fraud. The CVV is also known as the card 1. JC Distance
verification code (CVC) or card security code (CSC).
R1: R(c), R(b) → R(a)
When properly used, the CVV is highly effective
against some forms of fraud. For example, if the R2: R(d), R(b) → R(a)
data in the magnetic stripe is changed, a stripe
The modified JC Distance between R1 & R2 where
reader will indicate a "damaged card" error. The
the hyperparameters are 𝛿1 = 0.70 and 𝛿2 = 0.20,
flat-printed CVV is (or should be) routinely
is calculated as
required for telephone or Internet-based
purchases because it implies that the person JC Distance = 1-𝛿1(R1 R2) - 𝛿2(R1 R2- R1 R2)
placing the order has physical possession of the
card. Some merchants check the flat-printed CVV R1 R2
even when transactions are conducted in person. R1 R2 = 2
CVV technology cannot protect against all forms of R1 R2 = 4
fraud. If a card is stolen or the legitimate user is
tricked into divulging vital account information to JC Distance = 0.75
a fraudulent merchant, unauthorized charges
2. User Profile Vector
against the account can result. A common method
B1 = <U1, <0.7, 0.1, 0.6, 0.2, 0.4, 0.0, 0.2, 0.0>,
of stealing credit card data is phishing, in which a
<0.2, 0.3, 0.1, 0.2, 0.167, 0.033> >
criminal sends out legitimate-looking email in an
Here the values in the second tuple <0.7, …0.0>
attempt to gather personal and financial
represent the probability of User U1 accessing
information from recipients. Once the criminal has
particular attributes, for instance 0.7 denotes
possession of the CVV in addition to personal data
that there is a 70% probability that U1 accesses
from a victim, widespread fraud against that
the first attribute.
victim, including identity theft, can occur.
The values in the third tuple represent the
The following are directly associated membership of user U1 in the various(k) fuzzy
elements (DAEs) to CVV:- clusters, which is 6 in our case.

a. Credit card number (denoted by b) 3. Dubiety Score

b. Name of card holder (denoted by c)
c. Card expiry date (denoted by d)
Credit Card Number, Name of card holder, Card
expiry date are elements that are read before CVV
and hence used to validate the CVV entered by the
user. Hence the above-mentioned attributes have
been classified as DAEs, by our system.
Some normal data attributes are: -
1. Gender of Customer (denoted by e)
2. Credit Limit (denoted by f)
3. Customer’s phone number (denoted by g)
Suppose the Dubiety Score φi for User U1 is 0.8.
The JC Distance of the test transaction with its
cluster is 0.6. Then,
φf = 𝑑𝑠 ∗ φi
φf =√0.6 ∗ 0.8 = 0.69
Setting our hyperparameter ФUT as 0.65. We
observe that φf > ФUT. Hence the test transaction is
malicious, and an alarm is raised.

These are the attributes that have been collected

for the fraud detection and are not directly used to
access the CDE but are crucial for the process.

Some examples of transactions for our proposed

approach:
6. Experimentation
In this section, we describe the method of
evaluation of the proposed algorithm. Firstly, we
describe our dataset. We then calculate various
accuracy measures considering different
parameters as reference.
6.1 Description of dataset
This paper is about anomaly detection of user
behaviours. An ideal dataset should be obtained
from a practical system with concrete job functions.
But in fact, it is very sensitive for almost every
organization or company.
The performance of the algorithm was
analyzed by carrying out several experiments on a
credit card company dataset adhering to the TPC-C
benchmark[18]. The TPC-C schema is composed of
a mixture of read only and read/write transactions
that replicate the activities found in complex OLTP
application environment. The database schema,
data population, transactions, and implementation
rules were designed to broadly represent modern
OLTP systems. We used two audit logs: one for
training the model and the second for testing it. The
training log comprised of normal user transactions
and testing log consisted of a mixture of normal as
well as malicious user transactions. Although there
are unusual records in real dataset, we also inject
some anomalies for detection. The injected
anomalies are set differently with the normal
behaviour pattern from several aspects. In totality,
about 20,000 transactions were used. In total,
about 99% of data was non-malicious while less
than 1% of data was malicious. Fig. 6(a) shows the
distribution of malicious and benign data in the
dataset used:
Fig 6(a) Frequency of data items and their access
frequency
The details of CDEs, DAEs and Normal data items
has already been given in Section 3 and examples
have been discussed in Section 5.
The access pattern data hereby shows that CDEs are
rarely accessed, that too only by a few user roles
and hence, protection of CDEs from malicious
access is of a greater significance as compared to
DAEs and Normal data elements.
6.2 Cluster Analysis
When the number of users/user roles exceeds a
given limit, it becomes exceedingly difficult for the
IDS to keep track of individual user access patterns
and hence detect anomaly. This is the reason that
clustering is a better and computationally efficient
solution for better performance of IDS. We prefer
Fuzzy clustering over hard clustering. Fuzzy
clustering (also referred to as soft clustering) is a
form of clustering in which each data point can
belong to more than one cluster. In non-fuzzy
clustering (also known as hard clustering), data is
divided into distinct clusters, where each data point
can only belong to exactly one cluster. In fuzzy
clustering, data points can potentially belong to
multiple clusters. Membership grades are assigned
to each of the data points(tags). These membership
grades indicate the degree to which data points
belong to each cluster. Thus, points on the edge of
a cluster, with lower membership grades, may be in
the cluster to a lesser degree than points in the
center of cluster. When we evaluate various
performance measures keeping the number of
clusters as a reference parameter, it is observed
that a particular count for clusters is the most
efficient in predicting results.
Fig 6(b) Variation of performance with number of
clusters
Fig 6(b) depicts variation in precision, recall, TNR,
accuracy with change in number of clusters. From
the graph, we can see that :-
 TNR does not vary with the number of clusters,
i.e. TNR is invariant.
 The precision is always greater than 0.94 and is
more or less constant.
 Recall reaches optimum value when number of
Fuzzy Clusters is greater than 3.
 Accuracy also reaches the optimum value
when number of clusters is greater than 3.
6.3 Distances and thresholds
In section 3.2, we have described Modified Jensen-
Shannon distance as a measure to calculate
distance between two user vectors of same length.
In probability theory and statistics, the Jensen–
Shannon divergence is a method of measuring the
similarity between two probability distributions. It
is also known as information radius (IRad) or total
divergence to the average. It is based on the
Kullback–Leibler divergence, with some notable
(and useful) differences, including that it is
symmetric and it is always a finite value. The square
root of the Jensen–Shannon divergence is a metric
often referred to as Jensen-Shannon distance. We
preferred to use modified Jenson-Shannon distance
to give weights to data attributes and avoid curse of
dimensionality. The variation of modified Jenson-
Shannon distance with Euclidean distance is shown
in the fig 6(g).
In section 3.3, we have defined modified Jaccard
distance to quantitatively measure the similarity
between two rules. The Jaccard index, also known
as Intersection over Union of the Jaccard similarity
coefficient, is a statistical measure used for
comparing the similarity and diversity of sample
sets. The Jaccard coefficient measures similarity
between finite sample sets, and is defined as the
size of the intersection divided by the size of the
union of the sample sets. The variation of modified
Jaccard index with Jaccard index is shown in fig 6(h).
The variation of precision, recall, TNR, accuracy
with the various thresholds, namely 𝛿1, 𝛿2, фUT , фLT
that were defined in section 3 is shown in the
following figures:
Fig 6(c) shows the variation of Precision, recall, TNR,
accuracy with 𝛿1. It can be observed from the graph
that Precision, TNR and Accuracy increase with the
increase in value of 𝛿1, while the value of Recall
decreases with increase in value of 𝛿1.
Fig 6(e) shows the variation of Precision, recall, TNR,
accuracy with 𝛿2. It can be observed from the graph
that the value of Precision, TNR and Accuracy starts
decreasing when the value of 𝛿2 increases beyond
a certain value. Recall, on the other hand, increases
for higher values of 𝛿2.
Fig 6(d) shows the variation of Precision, recall,
TNR, accuracy with фUT. It can be observed from the
graph that the value of Precision first decreases and
then exponentially increases with the increase in
value of фUT. An identical trend is followed by
Accuracy. Somewhat similar trend is followed by
TNR except that it does not decrease initially. On
the contrary, the value of Recall decreases with the
increase in value of фUT.
Fig 6(f) shows the variation of Precision, recall, TNR,
accuracy with фLT. It can be observed from the
graph that the values of all the parameters
fluctuate a little but remain more or less constant
with the increase in value of фLT.
With regards to the dataset we have used, following
inferences can be done from the graphs:
1. Value of 𝛿1 should be close to 0.65 for
optimum performance.
2. Value of 𝛿2 should be close to 0.55 for
optimum performance.
3. Value of фUT should be close to 0.59 for
optimum performance.
4. Value of фLT should be close to 0.2 for optimum
performance.
6.4 Comparison with related methods
Table 1 shows the performance measures used for
comparison of approaches. Using these
performance measures, we will compare our
approaches with other related works. Our various
approaches are:-
Approach 1. Our approach using modified Jenson-
Shanon distance and modified Jaccard index.
Approach 2. Using unmodified Jaccard index with
Jenson-Shanon distance.
Approach 3. Using Euclidean distance with
unmodified Jaccard index.
S.No. PERFORMANCE FORMULA
MEASURE
1 TNR TN
TN + FP
2 Precision TP
TP + FP
3 Accuracy TP + TN
TN + FP + TP + FN
4 F1 Score 2 ∗ Precision ∗ Recall
Precision + Recall
5 PPV TP
TP + FP
6 ACC TP + TN
TP + TN + FP + FN
7 NPV TN
TN + FN
8 FDR FP
FP + TP
9 FOR FN
TN + FN
10 BM TPR + TNR – 1
11 FPR FP
FP + TN
12 FNR FN
FN + TP
13 MK PPV + NPV – 1
14 MCC TP × TN − FP × FN
(TP + FP)(TP + FN)(TN + FP)(TN + FN)
Table 1 ( Performance Measures)
In table 2 we have compared the three approaches If we compare Approach 1 with Approach 3, we
with each other. observe that:-
Sensitivity Approach 1 Approach 2 Approach 3  TNR and precision of Approach 1 is a lot better
Measures
than the TNR and precision for Approach 3
PPV 0.96 0.73 0.74
 .It has also got better accuracy as compared to
TPR 0.81 0.95 1.00
Approach 3.
ACC 0.89 0.80 0.83
 Approach 1 also has a much lower FPR and FDR
F1 Score 0.88 0.83 0.85 score as compared to Approach 3.
NPV 0.83 0.93 1.00  Amongst other performance measures, MK
FDR 0.04 0.27 0.26 and MCC values of Approach 1 are also slightly
FOR 0.17 0.07 0.00 better than that of Approach 3.
BM 0.77 0.60 0.65  Approach 3, on the other hand has got better
FPR 0.03 0.34 0.34 TPR, NPV and FOR measures as compared to
TNR 0.96 0.65 0.65 Approach 1. In fact, it has the best values for
FNR 0.19 0.05 0.00 these parameters in the entire table.
MK 0.79 0.66 0.74  Also, both Approach 1 and Approach 3 have got
MCC 0.78 0.63 0.70 somewhat similar F1 score.

Table 2 (Comparison of our approaches)
From the table, following observations can be
made:-
If we compare Approach 1 with Approach 2, we can
observe that:-
 TNR and FPR of Approach 1 is a lot better than
the TNR and precision for Approach 2.
 Approach 1 has also got better accuracy as
compared to Approach 2.
 Approach 1 has a much lower FPR and FDR
score as compared to Approach 2.
 Amongst other performance measures, MK
and MCC values of approach 1 are also better
than that of Approach 2.
 Approach 2, on the other hand has got better
TPR, NPV and FOR measures as compared to
Approach 1.
 Both Approach 1 and Approach 2 have got
somewhat similar F1 score.
In the measures like FPR and TNR where Approach
1 has good performnce, Approach 2 performs
rather poorly. However, in measures like TPR and
NPV, where Approach 2 performs better, Approach
1 also has good performance. For example, both
Approach 1 and Approach 2 have similar NPV scores
with Approach 2 performing slightly better As
Approach 1 performs far better than Approach 2 in
most of the measures, we can conclude that the
overall performance of Approach 1 is better than
Approach 2.
In the measures like TNR and precision,where
Approach 1 has one of the best score in the entire
table, Approach 3 performs rather poorly. Also,
Approach 3 lags far behind in measures like FPR and
FDR score. On the other hand, in the measures in
which Approach 3 performs better than Approach
1, Approach 1 is also performing quiet nicely. For
example, in case of NPV, both the approaches have
good scores, with Approach 3 performing better.
similar trends are observed in case of all other
measures except FNR, where Approach 3 has is far
superior. Considering all the above scenario, we can
say that the overall even though Approach 3 has the
best values for some performance measures, its
poor performance in other measures are clearly a
disadvantage due to which Approach 1 is better
than Approach 3.
Table 3 shows a comparison of our approaches with
various other related works. If we compare our
approach with other related approaches, we
observe that:-
 In comparison to HU Panda, our approach
works better with respect to all the
performance measures considered for the
purpose of comparison.
 In comparison to the work of Mostafa et al. our
approach performs better with respect to all
the performance measures that are considered
for comparison.
Sensitivity Approach Approach Approach HU Panda Hashemi Mostafa Mina Majumdar EliSa UP Rao et
Measures 1 2 3 et al. et al. Sohrabi et al. (2006) Bertino al.(2016)
et al. et al.
PPV 0.96 0.73 0.74 0.88 0.97 0.94 0.93 0.88 0.94 0.61

TPR 0.81 0.95 1.00 0.73 0.71 0.75 0.66 0.70 0.91 0.70

ACC 0.89 0.80 0.83 0.81 0.84 0.85 0.80 0.80 0.93 0.64

F1 Score 0.88 0.83 0.85 0.79 0.82 0.83 0.77 0.78 0.92 0.65

NPV 0.83 0.93 1.00 0.77 0.77 0.79 0.73 0.75 0.91 0.68

FDR 0.04 0.27 0.26 0.12 0.03 0.06 0.07 0.13 0.06 0.39

FOR 0.17 0.07 0.00 0.23 0.23 0.21 0.27 0.25 0.09 0.32

BM 0.77 0.60 0.65 0.63 0.69 0.70 0.60 0.60 0.85 0.35

FPR 0.03 0.34 0.34 0.10 0.02 0.05 0.05 0.10 0.06 0.45

TNR 0.96 0.65 0.65 0.90 0.98 0.95 0.94 0.90 0.94 0.65

FNR 0.19 0.05 0.00 0.28 0.29 0.25 0.35 0.30 0.09 0.30

MK 0.79 0.66 0.74 0.65 0.74 0.73 0.66 0.63 0.85 0.29

MCC 0.78 0.63 0.70 0.63 0.72 0.71 0.63 0.61 0.85 0.29

Table 3 (Comparison of our approaches with related works)

 In comparison to the work of Hashemi et al.
even though our approach scores just a little
less in measures like TNR and precision, it
scores a lot better with respect to rest of the
performance measures.
 If we consider the work of Mina Sohrabi et al.
our approach performs better with respect to
all the performance measures that are present
in the table.
 In comparison to the work of Majumdar et al.
our approach performs better with respect to
all the performance measures that we have
considered for the purpose of comparison.
 With comparison to the work of UP Rao et al.
our approach performs better in context to all
the measures that are considered in the table
for comparison.
 In comparison to the work of Elisa Bertino, our
approach gives better TNR and precision
scores. It also gives comparatively better FDR
and FPR scores. In other measures, except TPR
and recall, both approaches have somewhat
similar score. Since our work is mostly related
to finding Critical Data Items in a dataset,
higher TNR and precision scores are more
desirable as compared to other performance
measures. Since our approach performs quiet
well with respect to other performance
measures as well, better TNR and precision
scores can easily cover up lower recall values.
7. Analysis and Conclusion
In this paper we have tried to detect malicious
transactions keeping in mind that certain data
elements hold more critical information. We also
take into consideration user behaviour pattern in
this approach. A user regularly behaving as a
normal user will be gradually improving his
suspicion score. We then analyse the approach
w.r.t different parameters by conducting
experiments. Finally, we conclude that the
approach works efficiently in determining the
nature of a transaction.
 References
1. I-Yuan Lin ; Xin-Mao Huang ; Ming-Syan Chen “Capturing user access patterns in
the Web for data mining” Published in: IEEE ;Proceedings 11th International
Conference on Tools with Artificial Intelligence pp 9-11 Nov. 1999
2. R.S. Sandhu ; P. Samarati “ Access control: principle and practice” Published in:
IEEE Communications Magazine ( Volume: 32 , Issue: 9 , Sept. 1994 )
3. Denning, D.E. (1987) An Intrusion Detection Model. IEEE Transactions on Software
Engineering, Vol. SE-13, 222-232.
4. Knuth, Donald E., James H. Morris, Jr, and Vaughan R. Pratt. "Fast pattern
matching in strings." SIAM journal on computing 6.2 (1977): 323-350.
5. Wang, Ke. "Anomalous Payload-Based Network Intrusion Detection" . Recent
Advances in Intrusion Detection. Springer Berlin. doi:10.1007/978-3-540-30143-1_11
6. Douligeris, Christos; Serpanos, Dimitrios N. (2007-02-09). Network Security:
Current Status and Future Directions. John Wiley & Sons. ISBN 9780470099735.
7. Christina Yip Chung, Michael Gertz and Karl Levitt (2000), “DEMIDS: a misuse
detection system for database systems”, Integrity and internal control information
systems: strategic views on the need for control, Kluwer Academic Publishers,
Norwell, MA.
8. A. S. McGough, D. Wall, J. Brennan, G. Theodoropoulos, E. Ruck-Keene, B. Arief, et
al., "Insider Threats: Identifying Anomalous Human Behaviour in Heterogeneous
Systems Using Beneficial Intelligent Software (Ben-ware)," presented at the
Proceedings of the 7th ACM CCS International Workshop on Managing Insider
Security Threats, Denver, Colorado, USA, 2015.
9. S. D. Bhattacharjee, J. Yuan, Z. Jiaqi, and Y.-P. Tan, "Context-aware graph-based
analysis for detecting anomalous activities," presented at the Multimedia and Expo
(ICME), 2017 IEEE International Conference on, 2017.
10. P. A. Legg, O. Buckley, M. Goldsmith, and S. Creese, "Automated insider threat
detection system using user and role-based profile assessment," IEEE Systems
Journal, vol. 11, pp. 503-512, 2015.
11. I. Agrafiotis, A. Erola, J. Happa, M. Goldsmith, and S. Creese, "Validating an
Insider Threat Detection System: A Real Scenario Perspective," presented at the
2016 IEEE Security and Privacy Workshops (SPW), 2016.
12. T. Rashid, I. Agrafiotis, and J. R. C. Nurse, "A New Take on Detecting Insider
Threats: Exploring the Use of Hidden Markov Models," presented at the Proceedings
of the 8th ACM CCS International Workshop on Managing Insider Security Threats,
Vienna, Austria, 2016.
13. Zamanian Z., Feizollah A., Anuar N.B., Kiah L.B.M., Srikanth K., Kumar S. (2019)
User Profiling in Anomaly Detection of Authorization Logs. In: Alfred R., Lim Y.,
Ibrahim A., Anthony P. (eds) Computational Science and Technology. Lecture Notes
in Electrical Engineering, vol 481. Springer, Singapore
14. Yuqing Sun, Haoran Xu, Elisa Bertino, and Chao Sun. 2016. A Data-Driven
Evaluation for Insider Threats. Data Science and Engineering Vol. 1, 2 (2016), 73--85.
[doi>10.1007/s41019-016-0009-x]
15. S. Panigrahi, S. Sural and A. K. Majumdar, "Detection of intrusive activity in
databases by combining multiple evidences and belief update," 2009 IEEE
Symposium on Computational Intelligence in Cyber Security, Nashville, TN, 2009, pp.
83-90. doi: 10.1109/CICYBS.2009.4925094
[16] Yi Hu, Bajendra Panda, A data mining approach for database intrusion detection,
SAC '04 Proceedings of the 2004 ACM symposium on Applied computing Pages 711-
716, doi>10.1145/967900.968048
[17] Abhinav Srivastava , Shamik Sural , A. K. Majumdar, Weighted intra-
transactional rule mining for database intrusion detection, Proceedings of the 10th
Pacific-Asia conference on Advances in Knowledge Discovery and Data Mining, April
09-12, 2006, Singapore [doi>10.1007/11731139_71]
18 TPC-C benchmark: http://www.tpc.org/tpcc/default.asp
19 Mina Sohrabi, M. M. Javidi, S. Hashemi, ”Detecting intrusion transactions in
database systems: a novel approach”, Journal of Intelligent Info Systems 42:619-644
DOI 10.1007 Springer 2014.
20 UP Rao et. al ,“Weighted Role Based Data Dependency Approach for Intrusion
Detection in Database”, International Journal of Network Security, Vol.19, No.3,
PP.358-370, May 2017 (DOI: 10.6633/IJNS.201703.19(3).05).
[21] R. Agrawal, T. lmieliiski, and A. Swami, ”Mining Association Rules between Sets
of Items in Large Databases”, in Proceedings of the 1993 ACM SIGMOD International
Conference on Management of data, 1993.
[22] Sattar Hashemi, Ying Yang,Davoud Zabihzadeh and Mohammadreza Kangavari,
“Detecting intrusion transactions in databases using data item dependencies and
anomaly analysis”, Article in Expert Systems 25(5):460-473 · November 2008 DOI:
10.1111/j.1468-0394.2008.00467.
[23] Mostafa Doroudian, Hamid Reza Shahriari, “A Hybrid Approach for Database
Intrusion Detection at Transaction and Inter-transaction Levels”, 6th Conference on
Information and Knowledge Technology (IKT 2014), May 28-30, 2014, Shahrood
University of Technology, Tehran, Iran.
24 E. Bertino, A. Kamra, E. Terzi and A. Vakali (2005), "Intrusion detection in RBAC
administered databases, " in Proceedings of the Applied Computer Security
Applications Conference (ACSAC).
25 Lee, V. C.S., Stankovic, J. A., Son, S. H. Intrusion Detection in Real-time Database
Systems Via Time Signatures. In Proceedings of the Sixth IEEE Real Time Technology
and Applications Symposium, 2000.
26. Weina Wang, Yunjie Zhang, Yi Li and Xiaona Zhang (2006), "The Global Fuzzy C-
Means Clustering Algorithm," 2006 6th World Congress on Intelligent Control and
Automation, Dalian, 2006, pp. 3604- 3607.
27. Fuglede, Bent; Topsøe, Flemming (2004). "Jensen-Shannon divergence and
Hilbert space embedding - IEEE Conference Publication". ieeexplore.ieee.org.
28. Dunn, J. C. (1973-01-01). "A Fuzzy Relative of the ISODATA Process and Its Use in
Detecting Compact Well-Separated Clusters". Journal of Cybernetics. 3 (3): 32–57.
doi:10.1080/01969727308546046. ISSN 0022-0280.
29. A. Mangalampalli and V. Pudi (2009), "Fuzzy association rule mining algorithm for
fast and efficient performance on very large datasets," 2009 IEEE International
Conference on Fuzzy Systems, Jeju Island, 2009, pp. 1163-1168
30. Vorontsov, I.E., Kulakovskiy, I.V. & Makeev, V.J. Algorithms Mol Biol (2013) 8: 23.
https://doi.org/10.1186/1748-7188-8-23 “ Jaccard index based similarity measure to
compare transcription factor binding site models”