February 2008

Discussion

Introduction
Tomcat is a popular application server used to host web applications. Apache is a popular web server which
provides services like https encryption and decryption, URL rewriting etc. Apache can also be used a load balancer
to balance load between several Tomcat application servers.

This article briefly discusses some alternatives for load balancing an application server. It discusses
implementation details for setting up load balancing with Apache using ‘mod_proxy’ module. It also looks at some
of the features provided by apache such as ‘server affinity’ and safe removal of node.

Downloadable web application is provided which can be used to test load balancing with Apache. Jmeter script is
also provided for load testing Apache.

Setting up Apache and load balancing is the role of the System Administrator. Unless a java developer works in a
small team or is setting up a test environment, he wont get involved in setting up load balancing. However, it is
good to understand the principles behind load balancing. The knowledge might help a developer to fix issues in
live environment.

This article discusses load balancing in the context of a web application. The application is accessed using https,
requires a user to login and some user specific information is stored in session.

Background knowledge

It is assumed that the reader is familiar with the following concepts and technologies:

Apache (Version 2.2)

Tomcat (version 5.5.23)
Jmeter for load testing web applications

Some important terms

Load balancing – user requests are processed by more than one server with all servers sharing the load ‘equally’
Server affinity (sticky session) – With server affinity, multiple requests from a user are processed by the same
server. This is required for non clustered servers as the user session data is held on one server only and all requests
from that user have to go to the server which has the session data for the user.
Transparent failover – User is not aware of a server crash. Transparent failover can be request level or session
level. Transparent failover can be achieved by clustering application servers. With clustering, all the servers are the
same and so the loss of a server does not interrupt the service. With load balancing alone, the user has to login
again when server crashes.
Server Cluster – A group of servers which appear to be a single server to the user. Loss of a server is transparent
to the user. User data is held on all servers in the cluster group. Any server can process any user request and loss of
any server does not lead to interruption of service. The user does not have to login after a server failure. Since the
user session data is replicated over the network to more than one server, there is a performance overhead. So
clustering should be avoided unless transparent failover is required.
Scalability – measure of the ability of a system to handle increasing load without reducing response time
Response time – time taken to process a user request
Real workers – Term used by apache to refer to servers which are components of a load balanced system. The real
workers do the actual work and the real worker is usually a remote host.
Virtual worker – In Apache, the load balancer is referred to as the virtual worker which delegates processing to
real workers.

Load balancing algorithms

Round robin – requests are Reduced likelihood of version conflicts

Weighted round robin – servers of different capacity are assigned requests in proportion to their capacity (as
defined by a load factor). Apache as 2 versions of this:
Request counting algorithm – requests are delegated in round robin manner irrespective of the nature
of the request
Weighted traffic counting algorithm – Apache delegates traffic to real worker based on the number of
bytes in the request

Compare Load balancing with Clustering

Both Load balancing and clustering aim to improve scalability by spreading load over more than one server. They
both aim to provide horizontal scalability.

Load balancing Clustering

User has to login after server crash User does not have to login after server crash. So
failover is transparent to the user

Load balancing is done by the web server or Clustering capability is provided by the
using DNS or using hardware load balancer or application server. Clustering also requires load
using Tomcat balancer web application balancing

The application servers (e.g. Tomcat) do not The application servers (e.g. Tomcat)
communicate with each other communicate with each other.

There is minimal effect on response time when Response times could deteriorate when moving
moving up from a single server to load balanced to a clustered system from a single server as the
servers under the same load session data is now replicated over the network to
other servers.

More the session data, more is the deterioration

in performance compared to a single server.

Response time also depends on number of nodes

in the cluster. More the number of nodes, more is
the deterioration in performance as data is
replicated using TCP to every single node in
cluster. This can be reduced by using UDP to
replicate session data. With UDP, session data
could be lost during session replication. So a user
might have to login again if a server crashes

Load balancing can be used independently of Clustering also requires load balancing but makes
clustering ‘server affinity’ redundant. It provides
 ‘transparent failover’ capability over load
balancing at cost of decreased response time and
more complex configuration

Usually no changes are required to move an
application from a single server to a load
balanced set of servers
Application must meet certain criteria for it to
work in a clustered environment. The user
variables stored in the session must be
‘serializable’. To get good response time, only
small objects must be stored in the session.

Choices for implementing Load balancing

Hardware based load balancing

Pros
Fast
Cons
Expensive
Proprietary
Less flexible

Software based load balancing (e.g. Apache or Tomcat balancer)

Pros
Open source and free to implement with Apache and Tomcat balancer application
Easy to configure
More flexib
Cons
Lower performance compared to hardware based solution

Alternatives for software based load balancing

Apache ‘mod_proxy’ module or ‘mod_jk’ module. However ‘mod_proxy’ is easier to configure and newer
than ‘mod-jk’ module.
Using Tomcat balancer application
Linux virtual server
Using DNS for load balancing

The rest of this article discusses load balancing using Apache ‘mod_proxy’ module.

Load balancing with server affinity

A simple load balanced setup which does not provide ‘server affinity’ is not suitable for stateful web applications.
In stateful web applications, user state (session data) is held on one server. All further requests from that user must
be processed by the same server. Hence server affinity (sticky sessions) is necessary for stateful web applications
which don’t use clustering. The minimum load balanced setup is with 2 application servers and one web server
(load balancer). If https decryption is required, then the same Apache server can also be used for https decryption.In
a load balanced system with server affinity, all requests from user 1 go to Tomcat instance 1. This is shown below:
Apache implements server affinity by rewriting the ‘jsessionid’ sent by Tomcat to the browser. The Tomcat worker
name is added to the end of ‘jsessionid’ before the ‘jsessionid’ is sent to the browser. In the next request from the
same user, the Tomcat worker name is read from the ‘jsessionid’ and the request delegated to this Tomcat real
worker.The ‘jsessionid’ stored as a cookie in the browser has the tomcat worker name as shown in screenshot
below:

The Apache configuration (in httpd.conf) to setup Apache as a load balancer for 2 application servers is shown
below:
ProxyPass /apache-load-balancing-1.0 balancer://mycluster stickysession=JSESSIONID

<Proxy balancer://mycluster>
BalancerMember ajp://tomcat1:8009/apache-load-balancing-1.0 route=tomcat1 loadfactor=50
BalancerMember ajp://tomcat2:8009/apache-load-balancing-1.0 route=tomcat2 loadfactor=50
</Proxy>

The above setup requires ‘mod_proxy’ module to be loaded.The first line sets up a reverse proxy for request
‘/apache-load-balancing-1.0’ and delegates all requests to load balancer (virtual worker) with name ‘mycluster’.
Sticky sessions are enabled and implemented using cookie ‘JSESSIONID’.The ‘Proxy’ section lists all the servers
(real workers) which the load balancer can use. For each participant in load balancing, we define the url (using http,
ftp or ajp protocol) and give it a name which matches the name of the Tomcat engine defined in the Tomcat
‘server.xml’. The ‘loadfactor’ can be a number between 1 and 100. Set it to 50 so it can be increased or decreased
dynamically later on.The tomcat ‘server.xml’ in ‘TOMCAT_FOLDER/conf’ folder should have this configuration:
....
<Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat2">
...

With 2 browser sessions, the Tomcat server console and the server response show that all requests from the same
user are processed by the same server.

Load balancing manager and safe removal of server node

Apache load balancing manager Apache includes a ‘balancer manager’ which can be used to check the status of
the servers used for load balancing and to prepare for safe removal of a node. To enable balancer manager, add this
section to Apache ‘httpd.conf’ file:
<Location /balancer-manager>
SetHandler balancer-manager
</Location>

Balancer manager requires ‘mod_proxy’ and ‘mod_proxy_balancer’ modules to be loaded.

Balancer manager will now be accessible at the url ‘/balancer-manager’.

Screenshot of this is shown below:

Safe removal of a server node When an application server (real worker) has to be taken offline for maintenance,
set a very low load factor for the server due to be taken offline. Few new users will be served by that server. All
existing users ‘sticking’ to this real worker will continue to be processed by this real worker. This will help reduce
the number of users who will have to login again when the real worker is taken offline.After some time, the real
worker is disabled and taken offline. Once it is ready to come online, the worker is enabled again in balancer
manager.

Tests
About the attached web application and JMeter scriptThe attached web application contains a single servlet
and 3 JSP pages. The servlet has hard coded usernames and passwords for 10 users and is used to authenticate the
user and store the username in the session. The servlet also prints the username and server name of server. The
login JSP (index.jsp) is used to login and the second JSP (greeting.jsp) is used to print a greeting after the user logs
in. This JSP also prints the username and server name in the response. The third JSP (user_details.jsp) is used to
print user details and the server name and IP address of the server name used to process the request. This JSP also
prints the username and server name of the server to the console. The name of the server is setup as a context
parameter in the web.xml. Change it before deploying it so that the 2 web applications have different names to
make it easier to identify them
<context-param>
<param-name>serverName
<param-value>Tomcat instance 1
</context-param>

The attached JMeter script sets up 10 threads with each thread being used to login 1 user and request user details.
Login is done once per thread/user and all subsequent requests from the user are requests to get user
details.Manual test with 2 browser windows to show server affinityOpen 2 browser windows and login using
different usernames. The server consoles in Tomcat will show that requests from one user will always go to a
particular Tomcat instance demonstrating server affinity. See screenshot below:
Load testing with JMeterThe attached JMeter script is used to simulate a load of 10 users. The attached JMeter
script has been setup to record the response. Comparing the response from the server and the server console, we
can see that the load of 10 users is shared equally between 2 servers and all requests from one user are processed by
the same server.

Sudden loss of one server and then restoring the serverThis can be simulated with either shutting down tomcat
to simulate a server crash. This is best simulated using JMeter as a client to simulate a load of 10 users
continuously requesting pages from the servers.Run the attached JMeter script and once the load test is running,
take one server down. Any subsequent requests to the offline server will be redirected to the second server. When
one server goes down, the user session data is lost and so all the users who have ‘affinity’ to that server will have to
login again.

Reducing single point of failure

Load balancer can become the single point of failure. This can be reduced by using round robin DNS to delegate
user requests to more than one load balancer. The load balancer delegates requests to more than one application
server. In this scenario, if the load balancer and/or the application server goes down, the other load balancer and
application servers can still provide some level of service. This is illustrated in the diagram below
 :

Conclusion
Apache can be used to load balance Tomcat servers. This setup provides other useful features such as ‘Server
affinity’ and safe removal of nodes for scheduled maintenance. Load balancing is recommended if transparent
failover is not required. It is easy to setup load balancing and ‘server affinity’ with Apache.JMeter can be used to
load test the configuration and to test the behaviour in case of a server crash.The load balancer can become the
single point of failure. This can be reduced by using 2 load balancer and using round robin DNS to delegate request
to more than one server.

Source Files
web application.zip
apache load balance load test script.jmx

Biography
Avneet Mangat 6 years experience in Java/J2EE. Currently working as Lead developer at Active Health Partners (
www.ahp.co.uk ). Bachelors degree in Software Engineering, Sun Certified Web developer and Java programmer,
Adobe certified Flash Designer and Prince2 certified (foundation). Lead developer of open source tool DBBrowser,
please see http://databasebrowser.sourceforge.net/ Outside interests include photography and travelling. Please
contact me at avneet.mangat@ahp.co.uk or avneet.mangat@gmail.com for more information.

PRINTER FRIENDLY VERSION