Load Balancing Using

Open Source Softwares

MSN: finalbsd@hotmail.com
MAIL: finalbsd@gmail.com
CUID: FinalBSD
www.sanotes.net

Layer 4-7

F5

Layer4-7
Switch

4-7

NetScale 4-7
r
LVS

HAProxy 4-7

2/29

Schedule
Basically

Hardware/GUI/CLI (Configure method)/HA (Config Sync)

Load balance related

virtual server/node/pool/pool member

Monitors
Sorry server
Maintenance Mode
Load balance method

Persistence
SNAT/RNAT
Server Protection
ACL/Content Switch
GSLB
Performance
3/29

We are here

LB related
Persistence

Basically
SNAT/RNAT
Server Protection
GSLB
ACL/CS

4/29

Hardware/GUI/CLI/HA

Commercial
F5
NetScaler
Hardware
GUI
CLI
HA

5/29

Open Source
LVS
HAProxy

HAProxy Hot Reconfiguration

mv /etc/haproxy/config /etc/haproxy/config.old
mv /var/run/haproxy.pid /var/run/haproxy.pid.old
mv /etc/haproxy/config.new /etc/haproxy/config
kill -TTOU $(cat /var/run/haproxy.pid.old)
if haproxy -p /var/run/haproxy.pid -f /etc/haproxy/config; then
echo "New instance successfully loaded, stopping previous
one."
kill -USR1 $(cat /var/run/haproxy.pid.old)
rm -f /var/run/haproxy.pid.old
exit 1
else
echo "New instance failed to start, resuming previous one."
kill -TTIN $(cat /var/run/haproxy.pid.old)
rm -f /var/run/haproxy.pid
mv /var/run/haproxy.pid.old /var/run/haproxy.pid
mv /etc/haproxy/config /etc/haproxy/config.new
mv /etc/haproxy/config.old /etc/haproxy/config
exit 0
fi

6/29

pid

We are here

LB related
Persistence
Basically

SNAT/RNAT
Server Protection
GSLB
ACL/CS

7/29

Concepts
Monitor

Incoming
request

Availability requirement
SNAT/NAT

VIP

VIP

192.168.101.1

192.168.101.2

Priority-based member
activation virtual server

192.168.101.1:443

ACTION of service
down

Intelligent
Traffic
Control

pool

pool

(name= Time (name=

Slow Ramp
cgi_boxes)
asp_boxes)

Pool/pool
member member member
member
(server=
(server=
(server=
(server=
member
statistics
10.1.1.1:80) 10.1.1.2:80) 10.1.1.3:80)
10.1.1.4:80)

(name=
ssl_boxes)

member

member

(server=
(server=
10.1.1.5:80) 10.1.1.6:80)

8/29

member

member

(look at URL,
client IP addr.,
etc.)

member

(server=
(server=
(server=
10.1.1.1:443) 10.1.1.2:443) 10.1.1.6:443)

Load
Balancing

pool

Port-based
Traffic
Direction

virtual server

192.168.101.1:80

IP Addr.based
Traffic
Direction

Monitors
Monitor

Simple

ECV

EAV

ICMP/GW
ICMP/TCP
ECHO

TCP/HTTP/HTT /FTP
PS

LTM

/IMAP/LDAP/M
SSQL/NNTP/O
racle/POP3/R
ADIUS/Real
Server/SIP/SM
TP/SOAP/WMI

monitor

9/29

HAProxy Monitor
http://www.chinaunix.net

listen webfarm 192.168.1.1:80

mode http
balance roundrobin
cookie SERVERID insert indirect
option httpchk HEAD /index.html HTTP/1.0
server webA 192.168.1.11:80 cookie A check
server webB 192.168.1.12:80 cookie B check port 81 inter 2000
server webC 192.168.1.13:80 cookie C check
server webD 192.168.1.14:80 cookie D check

10/29

HAProxy Sorry Server

http://www.chinaunix.net

listen webfarm 192.168.1.1:80

mode http
balance roundrobin
cookie SERVERID insert indirect
option httpchk HEAD /index.html HTTP/1.0
server webA 192.168.1.11:80 cookie A check
server webB 192.168.1.12:80 cookie B check port 81 inter 2000
server webC 192.168.1.13:80 cookie C check
server webD 192.168.1.14:80 cookie D check
server bkpA 192.168.1.15:80 cookie A check backup
server bkpB 192.168.1.16:80 cookie B check backup
11/29

HAProxy Maintenance Mode

http://www.chinaunix.net
503 Service Unavailable
No server is available to
handle this request.

Updating...

12/29

Load balancing algorithm

Round Robin
Wrr(Ratio(member), Ratio(Node))
Dynamic Ratio weight

Fastest(node) & Fastest(application): /

LC(Member) & LC(node)
Observed(member) & Observed(node)
Predictive(member) & Predictive(node)
Source
URL HASH
URL Param
13/29

We are here

LB related

Persistence
Basically

SNAT/RNAT
Server Protection
GSLB
ACL/CS

14/29

Persistence

TCP handshake

First Hit

GET /URI1 HTTP/1.1

HTTP request (no cookie)

pick
server

TCP handshake

Second Hit

GET /URI1 HTTP/1.1

listen webfarm
192.168.1.1:80
Cookie persistence
HTTP request (no cookie)
mode http1.1 HTTP Cookie Insert
HTTP/1.1 200 OK
HTTP/1.1
insert
balance
roundrobin
1.2 HTTP
200 OK Cookie Rewrite
HTTP reply (no cookie)
reply
rewrite
cookie
HTTP
SERVERID
(withHTTP
insertedinsert
cookie)
indirect
1.3
Cookie
Passive
Set-Cookie:
httpchk
prefix
option
HEAD Hash
/index.html HTTP/1.0
1.4SERVERID=A
Cookie
Client server webA 192.168.1.11:80 cookie A check

Destination Address affinity persistence

TCP
handshake
server webB
192.168.1.12:80

Hash persistencecookie B check

cookie
GET
/URI2
HTTP/1.1 persistence
server webC
192.168.1.13:80
cookie C check

specifies
MSRDP
TCP handshake
HTTP request (with same cookie)
server
server webD
192.168.1.14:80
cookie
D
check

SIP persistence(session Initiation

protocol)
GET /URI2
HTTP/1.1
Cookie: SERVERID=A

Souce address affnity persistence

HTTP request (with same cookie)

SSL persistence
HTTP/1.1 200 OK

Universal persistence
HTTP reply (no cookie)
HTTP/1.1 200 OK
HTTP reply (updated cookie)

15/29

Server A

SNAT & RNAT

backend private
# Connect to the servers using our 192.168.1.200 source address
source 192.168.1.200
VIP:221.238.249.177

RNAT

External vlan

backend transparent_ssl1
# Connect to the
SSL
farm from the client's source address
MAPPED
IP: 10.10.1.1
source 192.168.1.200 usesrc clientip
SNAT
server railsA 192.168.1.11:80
sourceInternal
192.168.1.201
check
vlan
eth0: 10.10.1.2
server railsB 192.168.1.12:80 minconn 4 maxconn 12 check
server railsC 192.168.1.13:80 minconn 4 maxconn 12 check
eth1: 192.168.1.2

16/29

We are here

LB related
Persistence
Basically

SNAT/RNAT

Server Protection
GSLB
ACL/CS

17/29

Server Protection
Attack (SYN Flood)listen appfarm 192.168.1.1:80
Syn Proxy
mode http
F5
Connection Limit maxconn 10000ACL/iControl/iRules
option httpclose
Timeout
Surge Queue
Slow Start

option abortonclose
NetScal Syn Cookie/TCP
option forwardfor
er
offload/Content
balance roundrobin
Filter/ACL
server railsA 192.168.1.11:80
minconn 4 maxconn 12 check
server
LVSrailsB 192.168.1.12:80
Iptables?minconn 4 maxconn 12 check
server railsC 192.168.1.13:80 minconn 4 maxconn 12 check
contimeout
HAProx60000
ACL

weight
maxconn

18/29

Timeout

Timeout client

Client

timeout clitimeout

timeout connect

( )

timeout contimeout

timeout http-request

timeout server

HTTP (
header DDoS
)

proxy queue 503

timeout srvtimeout

timeout tarpit

19/29

reqtarpit

proxy

timeout queue

server

We are here

LB related
Persistence
Basically

SNAT/RNAT
Server Protection

ACL/CS

20/29

GSLB

HAProxy ACL
Layer 4 and below

Layer 7 Content

eq 0
src/dst acl missing_cl hdr_cnt(Content-length)
method

block if HTTP_URL_STAR !METH_OPTIONS || METH_POST missing_cl

src_port/dst_port
block if METH_GET HTTP_CONTENT
req_ver

block unless METH_GET or METH_POST or METH_OPTIONS

dst_conn
path_*
To select a different backend for requests to static contents on the "www" site

nbsrv(backend)
and to every request on the "img", "video",url_*
"download" and "ftp" hosts :
url_static path_beg
Layer 4aclContent
acl url_static path_end

req_lenacl host_www

hdr_*

/static /images /img /css

.gif .png .jpg .css .js

hdr_beg(host) -i www

Pre-defined ACL

acl host_static hdr_beg(host) -i img. video. download. ftp.

HTTP_1.1

wait_end

# now use backend "static" for all static-only hosts, and for static urls

# of host "www". Use backend "www" for METH_GET

the rest.
req_ssl_ver
use_backend static if host_static or host_www url_static
use_backend www

if host_www

21/29

Content Switch (UIE/iRule/ACL)

iffrontend
(http_uripublic
ends_with .gif) {
usereqisetbe
pool image_servers
^Host:\ img
static
} url_static path_beg
acl
/static /images /img /css
else if#
(http_uri
starts_with
/foo)keyword
{
The URI
will use a specific
soon
acl url_static
path_end
.gif .png
.jpg .css
.js
use reqisetbe
pool foo_servers
^[^\ ]*\ /(img|css)/ static
acl
} host_www
-i www
reqisetbe hdr_beg(host)
^[^\ ]*\ /admin/stats
stats
else if (http_cookie(XYZ-Type) == direct) {
acl host_static hdr_beg(host) -i img. video. download. ftp.
use default_backend
pool cookie_servers
dynamic
}
else if (findstr(http_uri, ?type=, 6, &) == cgi) {
cgi_servers
#use
Thepool
static
backend backend for 'Host: img', /img and /css.
use_backend
static if host_static or host_www url_static
}backend static
else {
use_backend
www if host_www
use
pool
web_servers
backend dynamic
}

backend stats

22/29

We are here

LB related
Persistence
Basically

SNAT/RNAT
Server Protection
ACL/CS

23/29

GSLB

Illu
s

GSLB

CDN

24/29

tra

te

Performance
Keep-Alive
Compression
In-memory Cache
Server Offload
TCP Buffering

25/29

Logging
listen proxy-out
mode http
option httplog
option logasap
log global
server cache1 192.168.1.1:3128
# log the name of the virtual server
capture request header Host len 20
# log the amount of data uploaded during a POST
capture request header Content-Length len 10
# log the beginning of the referrer
capture request header Referer len 20
# server name (useful for outgoing proxies only)
capture response header Server len 20
# logging the content-length is useful with "option logasap"
capture response header Content-Length len 10
# log the expected cache behaviour on the response
capture response header Cache-Control 26/29
len 8

HTTP Header Manipulation

reqdel
reqdeny
reqpass
reqtarpit
reqsetbe
reqisetbe
reqirep
reqidel
reqideny
reqipass
reqiallow
reqitarpit
reqadd
rsp*

# remove X-Forwarded-For header and SERVER cookie

reqidel ^X-Forwarded-For:.*
reqidel ^Cookie:.*SERVER=
# refuse *.local, then allow www.*
reqideny ^Host:\ .*\.local
reqiallow ^Host:\ www\.
# refuse *.local, then allow www.*, but ignore #www.private.local"

reqipass ^Host:\ www.private\.local

reqideny ^Host:\ .*\.local
reqiallow ^Host:\ www\.
# replace "/static/" with "/" at the beginning of any request path.

reqrep ^([^\ ]*)\ /static/(.*) \1\ /\2

# replace "www.mydomain.com" with "www" in the host name.

reqirep ^Host:\ www.mydomain.com Host:\ www

27/29

HAProxy The Reliable, High Performance

TCP/HTTP Load Balancer

28/29

Bibliography

[1] HAProxy Official Site: http://haproxy.1wt.eu

[2] Willy TARREAU:

HAProxy Configuration Manual (English)

[3] Willy TARREAU:

HAProxy Architecture Guide (English)

[4] Willy TARREAU:

HAProxy Reference Manual

[5] F5 Networks:

Configuration Guide for Local Traffic Management(v9.2.2)

[6] Citrix:

THE END

NetScaler Installation and Configuration Guide - Vol. 1(2)

29/29