Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Introduction
Often an organization will require more bandwidth than a single DSL or cable Internet
connection can provide, but higher bandwidth connections (T1 or T3) are beyond the
reach of their budget. An alternative (and much less costly) solution is to combine
multiple slower Internet connections using some form of bandwidth aggregation. This is
often called "load balancing" or "site multihoming", and the specific type of aggregation I
describe in this tutorial is often called "teaming" or "connection teaming".
Ideally bandwidth aggregation would be supported by your ISP, and performed at the link
layer (using multilink PPP) or at least with BGP routing support. However, most ISPs are
selling their lines with only the most basic of services, so customers are left with basically
only one choice: connection teaming using network address translation (NAT).
Allow NAT clients to share bandwidth from multiple Internet connections with
dynamic IP addresses (cheap DSL, cable, or other Internet service) and no
upstream routing support from the ISPs
FreeBSD 4.x
ipf for packet filtering
ipnat for network address and port translation
ipfw for traffic shaping
No natd (ipnat provides vastly better performance)
Minimize disruption to NAT clients if one or more Internet connections are
unavailable
Use only a single Ethernet port on the router for all the Internet connections
It might be hard to believe, but all of these things are achievable using FreeBSD 4.x and
no other additional software. You just have to learn how to make all its various pieces
and parts work smoothly together.
The first thing to recognize is that policy routing can be used together with network
address translation to handle bandwidth aggregation. Policy routing allows you to route
packets based on policies rather than route just based on the destination address and the
routing tables. This isn't a tutorial on policy routing, so if you are unfamiliar about policy
routing, read the FreeBSD man pages for ipf and ipfw. ipf uses the "fastroute" or "to"
command. ipfw uses the "fwd" or "forward" command.
The second thing to realize is that it is impossible to use ipnat's network address
translation with multiple external IP addresses on the same Ethernet interface. The ipnat
configuration doesn't even have a way to describe such a situation. However, you can
create multiple virtual Ethernet interfaces on the same Ethernet port, and have ipnat use
each interface independently.
Also, keep in mind that dissimilar Internet services use dissimilar authentication and
connection protocols. Many DSL connections use PPPoE, some cable ISPs authenticate
using a MAC address, and a few of the cheapest ISPs require you to use their
router/modems. Supporting all these different environments on the FreeBSD router can
be done, however it would also make this tutorial unnecessarily complicated. Instead, I'm
going to use the ISP supplied NAT router/modem and perform a second network address
translation.
Topology
In this tutorial, I've configured the ISP supplied routers to translate class C (/24) subnets
within the unroutable 192.168.0.0/16 subnet. For the clients, I've used a class B subnet
(10.0.0.0/16) in the unroutable 10.0.0.0/8 subnet. This means the FreeBSD router will
translate the NAT clients in 10.0.0.0/16 to an IP address on one of its virtual Ethernet
interfaces (each of which have an IP address within 192.168.0.0/16). The corresponding
ISP supplied NAT router will then translate the 192.168.0.0/16 address into its public
dynamic IP address.
Many organizations need at least one static IP address for providing services, so I've also
included a static public IP address on sis0 (12.34.56.7). This is also configured as a
backup for the NAT clients in case the connections on sis1 are down.
Requirements
Your kernel must have been built with support for ipf, ipfw, ipnat, dummynet, and
netgraph. This includes (but is not limited to the following kernel options):
options IPFIREWALL
options IPFIREWALL_FORWARD
options IPFILTER
options DUMMYNET
options NETGRAPH
In addition, you will have to build and install the following kernel module: ng_eiface.ko
(not a kernel option as far as I know).
Finally, you will have to fix the order in which ipf and ipfw handle packets by following
this tutorial: How to Reorder the FreeBSD 4.x ipf and ipfw Packet Filters
We need to create and configure virtual ethernet interfaces on boot. To have the
interfaces automatically created, create a file named /etc/start_if.sis1 and add this:
To enable bridging on these two new interfaces, install the following script in
/usr/local/etc/rc.d: ether.bridge.0.sh
Also, configure the interfaces (and a few other things) in /etc/rc.conf with these lines:
ipfilter_enable="YES"
firewall_enable="YES"
ipnat_enable="YES"
ifconfig_sis0="inet 12.34.56.7/24"
ifconfig_ngeth0="inet 192.168.0.100/24"
ifconfig_ngeth1="inet 192.168.1.100/24"
ifconfig_sis2="inet 10.0.0.1/24"
gateway_enabled="YES"
defaultroute="12.34.56.1"
firewall_type="/etc/ipfw.rules"
To allow network address translation on all the Internet connections, add these lines to
/etc/ipnat.rules:
map sis0 10.0.0.0/24 -> 12.34.56.7/32 proxy port ftp ftp/tcp mssclamp 1360
map sis0 10.0.0.0/24 -> 12.34.56.7/32 portmap tcp/udp auto mssclamp 1360
map sis0 10.0.0.0/24 -> 12.34.56.7/32 mssclamp 1360
map ngeth0 10.0.0.0/24 -> 192.168.0.100/32 proxy port ftp ftp/tcp mssclamp
1360
map ngeth0 10.0.0.0/24 -> 192.168.0.100/32 portmap tcp/udp auto mssclamp
1360
map ngeth0 10.0.0.0/24 -> 192.168.0.100/32 mssclamp 1360
map ngeth1 10.0.0.0/24 -> 192.168.1.100/32 proxy port ftp ftp/tcp mssclamp
1360
map ngeth1 10.0.0.0/24 -> 192.168.1.100/32 portmap tcp/udp auto mssclamp
1360
map ngeth1 10.0.0.0/24 -> 192.168.1.100/32 mssclamp 1360
Policy routing is used to balance the NAT client requests between the Internet
connections. The contents of /etc/ipfw.rules:
flush
# ipfw is used only for traffic shaping (ipf is used for filtering)
# Policy routing
add 500 forward 192.168.0.1 all from 192.168.0.100 to any out xmit sis0
add 510 forward 192.168.1.1 all from 192.168.1.100 to any out xmit sis0
# Allows use of the regular routing when load balancing cannot be done
# rule 1799 reserved for check-loadbalance
add 1800 skipto 2000 all from any to any in recv sis2
add 1900 skipto 10000 all from any to any in recv sis2 keep-state
The method of balancing based on services could be improved, but this gives you a good
general idea of how to balance (both based on probability and on odd/even IP addresses).
Stateless protocols like HTTP and HTTPS tend to be very forgiving about receiving the
same client requests from different IP addresses. Other protocols, like FTP, are not.
Additional Internet connections can be added, but if you want to balance them evenly, the
number of rules and probability values get complicated (especially with check-
loadbalance script enabling and disabling the rules ). Remember that the probabilities
must be in the series [1/n, 1/(n-1), 1/(n-2), ... , 1]. For example, with four connections you
would want to use these lines:
Also, with four Internet connections, balancing with odd and even source IP addresses
won't work, but you can use mod 4 (least signifigant bits: 1 + 2) with these lines:
Now that you've got your clients connection teaming these multiple Internet connections,
you'll want to examine reliability. Imagine one of the cheap Internet connections become
unavailable. You'll get some unusual problems on your NAT clients: half of all
HTTP/HTTPS connections will fail, and half of all NAT clients won't be able to use other
protocols at all.
This would obviously not be acceptable. To handle this problem, I've scheduled a script
in the crontab to watch these connections every few minutes and modify the policy routes
when a connection becomes unavailable: check-loadbalance.sh
Change the variable definitions DST1_IP and DST2_IP to suit your situation. Ideally they
should be ping-able IP addresses within the respective ISP's (like routers one hop away,
or the ISP's DNS servers). They could be the same destination, if necessary.
This script is designed only for checking two Internet connections, but a similar idea
could be applied to handle more connections. Try unplugging one of the Internet
connections and running the script from the console to test it before scheduling it in the
crontab.
Conclusion
As you can see, even without installing any additional software, FreeBSD can be a very
powerful and flexible operating system for routing. It is always a surprise what you can
do with it if you think outside the box.