Self Organised-Distributed Database Systems
[Using Swarm Intelligence]
Vamsi Krishnam Raju Chiluvuri
Department of computer science
California state university
Fullerton,CA
vamsyraju@gmail.com
ABSTRACT
This paper discusses in detail about the distribute database
system, swarm intelligence and how swarm intelligence tech-
niques can be used in distributed database systems.
General Terms
Distributed database, swarm intellligence
Keywords
Distributed database, swarm intellligence, replication, con-
currecy control
1. INTRODUCTION
Distributed databases are currently the main focus of re-
search in the area of databases. Early databases in the
seventies and early eighties moved towards centralization,
and resulted in large monolithic databases. Since then, the
trend reversed more towards decentralization and auton-
omy of processing. With the advances in distributed pro-
cessing and distributed computing, the database research
community performed considerable work to address issues
of data distribution, distributed query, and transaction pro-
cessing. A few prototypes have been implemented, how-
ever, a full-scale comprehensive distributed database man-
agement system (D-DBMS) that implements the functional-
ity and techniques proposed in distributed database (D-DB)
research never emerged as a commercially viable product.
Most major vendors redirected their efforts to developing
systems based on client-server architecture or towards de-
veloping technology for accessing distributed heterogeneous
data sources.
Swarm intelligence is a relatively new field in computer sci-
ence. It deals with collective behavior in decentralized, self-
organized systems. The theory behind swarm intelligence
comes from biological swarm behavior. In nature simple
creatures like ants and bees follow some basic rules when
living in a colony. Each creature only acts upon a tiny bit
of information available to it. No single individual sees the
overall picture. The creatures act as peers with little hier-
archical chain of command and usually only communicate
basic information about the environment. No individual in-
structs others, yet together they are able to function well and
achieve some impressive results. Swarm intelligence can be
used to optimally solve many classic computer science prob-
lems such as the Traveling Salesman.
In this paper, we will discuss in detail about distributed
database systems and swarm intelligence. The total paper
is divided into five sections. Section 2 talks about the dis-
tributed database systems. Section 3 gives a overview on
swarm intelligence. In section 4 we will discuss how swarm
intelligence techniques can be used in distributed database
systems. Section 5 concludes the paper with a summary and
future work.
2. DISTRIBUTED DATABASE SYSTEMS
A distributed database is a collection of multiple, logically
interrelated databases distributed over a computer network.
A distributed database management system is then defined
as the software system that permits the management of the
distributed database and makes the distribution transparent
to the users. [11] The terms DDBMS and DDBS are often
used interchangeably. Assumptions regarding the system
that underlie these definitions are: [8]
• Data is stored at a number of sites. Each site is as-
sumed to logically consist of a single processor. Even if
some sites are multiprocessor machines, the distributed
DBMS is not concerned with the storage and manage-
ment of data on this parallel machine.
• The processors at these sites are interconnected by a
computer network rather than a multiprocessor con-
figuration. The important point here is the empha-
sis on loose-interconnection between processors which
have their own operation systems and operate inde-
pendently. Even though these multiprocessor archi-
tectures are quite similar to the loosely interconnected
distributed systems, they have different issues to deal
with. (e.g., task allocation, migration, load balancing,
etc)
• The DDB is a database, not some collection of files that
can be individually stored at each node of a computer
network. This is a distinction between a DDB and a
collection of files managed by a distributed file system.
 Figure 1: Simple Distributed database system
• The system had full functionality of a DBMS. It should
provide functions like query processing, structures or-
ganization of data and so on.
Figure 1 shows an example for a simple distributed database
system. We can see six sites with their own databases which
are connected through a common communication network.
2.1 Advantages of Distributed Databases:
• Many real world organizations can be represented more
naturally.
• It greatly increases the scalability. For example the
size of DB can be incremented.
• It ensures security by replicating vital data.
• It increases the availability and reliability because there
is no single point of failure.
• It increases the performance of the system because it
speeds up the query processing and by local processing.
• It provides local anatomy because the local organiza-
tion is fully responsible for accuracy and safety of its
own portion of DB.
2.2 Distributed Database Issues:
There are three important issues with distributed database
systems.
Figure 2: Replication approaches
2.2.1 Replication
Replication plays a major role in increasing the performance
of a distributed database. This is because in a distributed
database, data is distributed across many sites. Every site
has its own local database. But the Database management
system operates on all the databases simultaneously. So to
improve system reliability and performance the data items
are to be stored redundantly at multiple sites. The other
advantage is fault- tolerance, when a node crashes then the
replica of that in another site can be used. Replication in-
creases scalability too, for example clusters can be used in-
stead of mainframes. With the ever increasing use of mobile
users, data warehouses replication is almost a must.
There are many approaches for replication, most of them
solving certain classes of problems. The figure 2 shows the
different scale of approaches. Right hand side classes of
technologies are appropriate for supporting operational sys-
tems which need real time transactions that are appropriate
real time transaction processing. Left hand side classes of
technologies are well suited for supporting decision making,
browsing and research on LAN based PCs or other plat-
forms. [9]
Replication Problems
1. How to replicate data?
It can be done synchronously or asynchronously de-
pending on where the updates take place.
2. When to replicate data?
 Update Everywhere:
W(X) W(X) W(X)
Primary Copy:
W(X)
Read Read
Only Only
Figure 3: Approaches for submitting Updates
It depends on the data transfer patterns in the net-
work. For example a particular site may use a partic-
ular piece of data more often, so a replica near that
site will be very useful.
3. Where to update?
There are two ways it can be done.
• First approach is Update Everywhere. I.e. exe
cute update transactions at every site. The ad
vantage of this approach is simplicity , but it’s a
costly approach.
• Second approach is Update only Primary Copies.
Primary copies are maters copies, whereas sec
ondary copies (slave) are read-only. Figure 3 shows
how this is done. W(X) is the update process. We
can see in the Update every where all the replicas
are updates. In the primary copy approach all
the Read -only replicas are left alone.
4. When to update?
There are two approached for this.
• First approach is Eager. It means update within
the boundaries of the transaction, i.e., transac
tions terminate usually with Two Phase Proto-
col
• Second approach is Lazy. It means update only
after the commit of transaction. The disadvan
tage of this approach is it leads to inconsistency.
Figure 4 shows the two approaches in detail. We
Eager:
1)T:W(X)
3)Commit
2) Propagate W(X)
Lazy:
1)T:W(X)
2)Commit
3) Propagate W(X)
Figure 4: Approaches for propagating updates
can see in the Eager approach, propagation of up-
dates occurs even before the commit occurs, but
whereas in the Lazy only after the commit occurs
does the propagation of updates happens.
2.3 Concurrency Control
Concurrency control is one of the correctness criteria for
replicated databases. A replicated database system that
achieves replication and concurrency control has the same
input/output as a centralized, one-copy database systems
that executes user requests one at a time. [13] Concurrency
control problem is exacerbated in a distributed database be-
cause:
• Users may access data stored on different computers
in a distributed system.
• A concurrency control mechanism at one computer
cannot instantaneously know about interactions at other
computers.
Concurrency control has been actively investigated for the
past several years, and the problem for non-distributed DBMSs
is well understood. The two-phase locking protocol is ac-
cepted as a standard solution. [2]
2.3.1 Two-phase locking protocol:
Two phase locking is a process used to gain ownership of
shared resources without creating the possibility for dead-
lock. The technique is extremely simple, and breaks up the
modification of shared data into ”two phases”, this is what
gives the process its name. [1]
There are actually three activities that take place in the ”two
phase” update algorithm:
1. Lock Acquisition
2. Modification of Data
3. Release Locks
The modification of data, and the subsequent release of the
locks that protected the data are generally grouped together
and called the second phase.
Two phase locking prevents deadlock from occurring in dis-
tributed systems by releasing all the resources it has ac-
quired, if it is not possible to obtain all the resources required
without waiting for another process to finish using a lock.
This means that no process is ever in a state where it is hold-
ing some shared resources, and waiting for another process
to release a shared resource which it requires. This means
that deadlock cannot occur due to resource contention. The
resource (or lock) acquisition phase of a ”two phase” shared
data access protocol is usually implemented as a loop within
which all the locks required to access the shared data are ac-
quired one by one. If any lock is not acquired on the first
attempt the algorithm gives up all the locks it had previ-
ously been able to get, and starts to try to get all the locks
again.
2.4 Query Optimization
The queries in distributed data base systems often cannot
be answered by a single local unit. An aggregate of data,
spanning over different data bases in a network is needed.
To do this often there will be many ways. Our goal is finding
the best way. [4]
2.4.1 Centralized Vs Distributed Query Processing:
In centralized query processing the number of I/O opera-
tions and the usage of CPU to process the query are the main
concerns. Whereas in the distributed query processing along
with this, the amount of data transmission between the sites
is also an important concern. Two new operators send and
receive are included in distributed query processing. These
operators are used for transferring the data between sites.
The other important difference is the heterogeneity in data
formats and data models in distributed databases. In dis-
tributed databases the data is replicated in various locations
to increase the performance. This leads to more complex
problems while trying process the queries. The usage of
resource vectors, interconnect matrix, and caching in dis-
tributed environment will make a huge difference.
2.4.2 Query Processing Objectives:
The cost function of a distributed query is
I/O cost + CPU cost + Communication Cost.
The minimization of this function is the main objective of
distributed query processing. These costs may have different
weights in different distributed environments. For example
the communication cost will dominate in a wide area net-
work, whereas in a local area network it is negligible.
3. SWARM INTELLIGENCE
We face dynamic optimization problems in almost all fields.
Even with today’s ever increasing computing power, some
of these problems are still hard to solve. Finding solutions
to these problems in most of the times is not finding the
extema, but to find something that is as close as possible.
Most recently scientists are turning to insects like ants and
bees for ideas to solve such problems. This form of artificial
intelligence based on the collective behavior of decentralized,
self organized systems is called swarm intelligence.
A single ant or bee isn’t smart, but their colonies are. The
study of swarm intelligence is providing insights that can
help humans manage complex systems, from truck routing
to military robots [9]
Following a trail of insects as they work together to accom-
plish a task hovers unique possibilities for problem solving -
[15]
Swarm intelligence algorithms can be divided into two classes.
They are
• Pheromone based navigational algorithms inspired by
biological ant-colony behavior.
• Non-pheromone based navigational algorithms inspired
by biological bee-colony behavior.
3.1 Pheromone based Algorithms
Let us look at the collective behavior of ants. The objec-
tives of ants are very simple, finding food and building a
nest. To achieve these every single ant follows some sim-
ple set of rules. No one is in-charge; no one knows the
complete picture. But despite this they achieve some ex-
traordinary solutions to problems like finding the shortest
path to food, allocating workers to different tasks, defend-
ing their nests from predators, etc,. Now let us look at one
of the simple day to day tasks of a Ant, finding the shortest
path to food from nest, which is analogous to finding short-
est path problems like travelling sales person [5] Whenever
a ant bring food to nest it leaves a trail with a chemical
called Pheromone. So as many and many ants go through
this trail the track gets reinforced, if no ant uses the trail
for sometime the chemical slowly evaporates (deleting the
trail in our computer science words). Suppose half of the
ants choose the long path and other half choose the shorter
path. The shorter path will have more intense pheromone
trail then the longer one, because on the longer path the
pheromone evaporates faster than on the shorter path. On
the next rounds the ants choose the path with more intense
pheromone. So after a while all the ants choose the shorter
path. This total process is shown in figures 5,6 and 7.
3.2 Non-Pheromone based Algorithms:
They are also called bee colony. Bee colony is the area of
swarm intelligence which studies (1) the behavior of bees and
similar behavior in other insects, and (2) the applicability of
 Figure 5: At the start

Figure 8: Distance and direction by waggling dance

the underlying principles. Non-pheromone based algorithms

are [7]

• Significantly more efficient when finding and collecting

food, i.e., it uses fewer iterations to complete the task.
• More scalable, i.e., it requires less computation time
to complete the task, even through in small worlds,
Figure 6: After some time pheromone-based algorithms are faster on a time-per-
iteration measure
• Less adaptive than pheromone-based algorithms.
Honeybee foraging behavior consists of two behaviors;

– Recruit behavior
– Navigation behavior

In order to recruit members of the colony for food sources,

honeybees inform their nest mates of the distance and di-
rection of these food sources by means of a waggling dance
performed on the vertical combs in the hive [6] This dance is
actually the bee language. It is made by a series of alterna-
tive left and right loops. All these moves are intercepted by
a segment in which the bee waggles her abdomen from side
to side. The duration of the waggle phase is a measure of
the distance to the food, and the angle between the sun and
the axis of the waggle segment on the vertical comb repre-
sents the azimuthal angle between the sun and the direction
in which the recruit should fly to find the target.
Figure 7: By the end
[7] Figure 8 shows this process. The advertisement for a food
source can be adopted by other members of the colony. The
decision mechanism for adopting an advertised food source
location by a potential recruit, is not completely known. It
is considered that the recruitment amongst bees is always a
function of the quality of the food sources.

4. METHODOLOGY
4.1 Agent Based Systems:
Swarm intelligence is been widely used in various distributed
environments. For example in telecommunications to solve
routing problems [10] and load balancing [14] .They were
also used in distributed pattern detection and classification
[3] and in robotics [12] . In this paper we will try dealing
the problem of replication management using bio-inspired
algorithms. The interesting feature of these bio-inspired al-
gorithms is the solutions to complex problems by following
some simple set of rules in the individual levels. 1.Query Executed 2.Data Item
Local Agent is Accessed
These Swarm biological systems can be quite naturally em- Generated. Condition
ulated with any distributed system through the multi-agent checked
paradigm . The main advantages of using such systems are: 3.Data item is
replicated

1. Self organization:
All decisions are made based on local information.

2. Adaptability:
Can adapt themselves to any dynamic environment.

3. Stigmergy:
It is the mechanism of spontaneous, indirect coordi-
nation between agents or actions, where the trace left
in the environment by an action stimulates the per-
formance of a subsequent action, by the same or a
different agent.
4.2 Replication Issues:
Issue 1:
In a distributed database system, if data items are far away
in the network, then the systems performance will be af-
fected. This is because it leads to network load which causes
a decline in the overall performance. So a system should
have its most accessed data itemś replica nearby.
Issue 2: Over doing replication is also a major problem,
because when we over replicate, then the memory resources
will be affected. So when a system no longer uses a replica
it should be deleted.
Issue 3: In distributed database systems often a query will
be operated on more than one data item. If such operations
are very common on a set of data items that are not on a
single site, then replicating a merged copy of those data item
would drastically increase the performance of the system.
Issue 4: Maintaining the consistency of a system is the
main issue when it comes to replication . Whenever a copy is
updated, there should be consistency among all the replicas.
4.3 Agent
An Agent is a piece of code. It is the mobile agent we use in
our swarm approach in distributed database system. There
will be two types of agents.
Figure 9: Working of a local agent
1. Local Agent
2. Global Agent
The local agent will make replicas. The Global Agent is
responsible for maintaining consistency with the replicas.
So local agent deals with the issues 1,2 and 3, whereas the
global agent deals with the issue 4 described above.
4.3.1 Local Agent
A local agent is generated whenever a query is executed.
Every agent has a ID which is unique for each source, i.e.
all the agents from the same source will have a same ID.
Every time a data item is accessed the local agent checks if
it has to be replicated. The condition on which this checking
takes place will be discussed in the next section. For now
just think that the agent knows it.
There are two possible cases here
• Yes, it has to be replicated
• No, leave it like that.
If it has to be replicated then a replica of the data item is
generated and transferred to the source site. Otherwise no
action takes place.
Figure 9 shows this process in a simple distributed database.
How does the agent know whether to replicate or
not?
Number Of Replicas
Every time a data item is accessed then the agent leaves
a time stamp there, with the agent ID. So when the data
item is accessed the agent generates a time stamp and then
searches for a time stamp with its agent ID. It then computes
the difference between timestamps. It then checks if the
difference with the X which is a constant, If the difference
is less than X, it means the data item is accessed recently.
Then it increments a count in there with its agent ID. After
this it checks the number of notes, if the count is greater than
Y (which is another constant) then it replicates. Otherwise X
it just increases the count. Graph showing value of X Versus Number of Replicas
Number Of Replicas
This count is reset after some interval of time.

How to calculate X and Y?

The value of X is the constant with which we compare the

timestamp difference. This value depends on the activity in
the distributed database. If the value of X is too less it will
decrease the number of replicas, else if the value is too high
Y
then it increases the replicas as shown in the graph from
Graph showing value of Y Versus Number of Replicas
figure 10

The value of Y is the constant with which we compare the

count. This value depends on the structure of distributed
database. If the value of Y is too less then it will increase
the number of replicas, else if its too high then it decreases Figure 10: Graphs showing the effect of constants X
the number of replicas as shown in the graph from the figure and Y on the number of replicas
10

So using the two constants X and Y we can fine tune the

replication management to exactly suit the requirement of
the system.

Deleting old replicas:

2. Replica Updated
Replicas which are no longer used should be deleted, so that 3.Global Agent
memory is not wasted. Every site takes care of the replicas Generated
which are stored in it. It check the usage of replicas in
certain intervals of time, if it sees that the replica is no longer
used then it deletes it. This check also uses the timestamps
of the recently visited agents.

4.3.2 Global Agent

Global Agent is generated whenever a update transaction
occurs on a replica or the master copy. It maintains the con- 1.Query Executed
sistency by propagating updates to all replicas. This process
works in conjunction with other locking protocols like Two-
Phase-Locking protocol. Figure 11 shows this process in a
simple distributed database.
4.Global Agent Maintains consistency among
5. CONCLUSIONS AND FUTURE WORK Replicas
In this paper we started with discussion about replication
management, its problems and why it is important. We
followed it with a discussion about swarm intelligence basics.
Then we started talking about the implementation of some
swarm methods for replication management. We discussed
about the local agents and global agents, their functions.
We discusses about the effect of values of X and y on the Figure 11: Global Agent
number of replicas.
There is a lot of scope for future work in this area. Imple- A.4.2 Replication Issues
mentation of this is to be tried. The constants X and Y need A.4.3 Agent
some formulation on based on the properties of the system.
Actually a program which can generate these constants au- A.5 Conclusion and Future Work
tomatically will be very useful A.6 References

6. REFERENCES
[1] Arnold.P. Two phase locking protocol, march 2009.
Good read with overview of two phase locking
protocol.
[2] Bernstein, P. A., and Goodman, N. Concurrency
control in distributed database systems. acm 13
(1981), 37.
[3] Brueckner, S., and Parunak, H. V. D. Swarming
agents for distributed pattern detection and
classification. pages (forthcoming),.
[4] Cem Evrendilek Asuman Dogac, F. O.
Multidatabase query optimization. Kluwer 39 (1997),
27.
[5] Dorigo, M., and Gambardella. And colonies for
the traveling salesman problem. acm 43 (1997), 13.
[6] Frisch, K. The dance language amd orientation of
bees. acm 5 (1967), 14.
[7] Karl Tuyls, A. N. A bee algorithm for multi-agent
systems. acm 7 (2000), 5.
[8] M. Tamer Özsu, P. V. Distributed database
systems: Where are we now? IEEE 18 (2001), 19.
[9] Miller. National geographic swarm theory. web site,
july 2007.
[10] Ns, T. I. Swarm intelligence and problem solving in.
[11] Ozsu. M. T, V. Principles of distributed database
systems. paperback, 1991.
[12] Pettinaro, G. C., Kwee, I. W., Gambardella,
L. M., Mondada, F., and louis Deneubourg, J.
Swarm robotics: A different approach to service
robotics. 71–76.
[13] Philip A. Berstein, N. G. An algorithm for
concurrency control and recovery in replicated
distributed databases. ACM 9 (1984), 20.
[14] Schoonderwoerd, R., Holland, O., Bruten, J.,
and Rothkrantz, L. Ant-based load balancing in
telecommunications networks.
[15] tarasewitch Patrick R. McMullen, P. Swarm
intelligence, power in numbers. ACM 45,No 8 (Aug
2002), 6.

APPENDIX
A. HEADINGS IN APPENDICES
A.1 Introduction
A.2 Distributed Database Systems
A.2.1 Advantages of Distributed Databases
A.2.2 Distributed Database Issues
Replication Concurrency Control Query Optimization

A.3 Swarm Intelligence

A.3.1 Pheromone based Algorithms
A.3.2 Non-Pheromone based Algorithms
A.4 Methodology
A.4.1 Agent Based Systems