Cisco ACE config for FTP load balancing

This configuration handles both regular and passive ftp. Consult Cisco manuals for specific commands explanation. !-- using dummy tcp probe for ftp server for simplicity. probe tcp probe-tcp-21 port 21 interval 30 passdetect interval 60 connection term forced !-- optional open 1 rserver host ip address inservice rserver host ip address inservice ftp-server1 10.0.1.11 ftp-server2 10.0.1.12

serverfarm host ftp-farm probe probe-tcp-21 rserver ftp-server1 21 inservice rserver ftp-server2 21 inservice

class-map match-all ftp.mycompany.com 2 match virtual-address 55.55.55.55 tcp eq ftp ftp site IP

!-- This is

policy-map type loadbalance generic first-match lbmapftp.mycompany.com class class-default serverfarm ftp-farm policy-map multi-match pmap-vlan100 class ftp.mycompany.com loadbalance vip inservice loadbalance policy lbmap-ftp.mycompany.com loadbalance vip icmp-reply inspect ftp access-list ALL line 10 extended permit ip any any interface vlan 200 description Web Servers Lan ip address 10.0.1.2 255.255.255.0 peer ip address 10.0.1.3 255.255.255.0 alias 10.0.1.1 255.255.255.0 access-group input ALL no shutdown interface vlan 100 description OUTSIDE ip address 55.55.55.11 255.255.255.0 alias 55.55.55.10 255.255.255.0 peer ip address 55.55.55.12 255.255.255.0 access-group input ALL service-policy input pmap-vl100 no shutdown

Cisco ACE config for HTTP VIP

Consult Cisco manuals for specific commands explanation. !-- testing specific web page for string "SUCCESS" probe http probe-http request method get url /test.html expect regex SUCCESS interval 30 passdetect interval 60 open 1

rserver host http-server1 ip address 10.0.1.11 inservice rserver host http-server2 ip address 10.0.1.12 inservice

serverfarm host http-farm probe probe-http rserver http-server1 80 inservice rserver http-server2 80 inservice

class-map match-all www.mycompany.com 2 match virtual-address 55.55.55.55 tcp eq 80 ! This is ftp site IP

policy-map type loadbalance generic first-match lbmap-www.mycompany.com class class-default serverfarm http-farm

policy-map multi-match pmap-vlan100 class www.mycompany.com loadbalance vip inservice loadbalance policy lbmap-www.mycompany.com loadbalance vip icmp-reply

access-list ALL line 10 extended permit ip any any interface vlan 200 description Web Servers Lan ip address 10.0.1.2 255.255.255.0 peer ip address 10.0.1.3 255.255.255.0 alias 10.0.1.1 255.255.255.0 access-group input ALL no shutdown

interface vlan 100 description OUTSIDE ip address 55.55.55.11 255.255.255.0 alias 55.55.55.10 255.255.255.0 peer ip address 55.55.55.12 255.255.255.0 access-group input ALL service-policy input pmap-vl100 no shutdown

Cisco ACE config for FTP load balancing with source NAT

This configuration handles both regular and passive ftp. Consult Cisco manuals for specific commands explanation. !-- using dummy tcp probe for ftp server for simplicity. probe tcp probe-tcp-21

port 21 interval 30 passdetect interval 60 connection term forced !-- optional. Linux ftp server probe has issues if connection is not forced to close open 1

rserver host ftp-server1 ip address 10.0.1.11 inservice rserver host ftp-server2 ip address 10.0.1.12 inservice

serverfarm host ftp-farm probe probe-tcp-21 rserver ftp-server1 21 inservice rserver ftp-server2 21 inservice

class-map match-all ftp.mycompany.com 2 match virtual-address 55.55.55.55 tcp eq ftp ! This is ftp site IP

policy-map type loadbalance generic first-match lbmap-ftp.mycompany.com class class-default serverfarm ftp-farm nat dynamic 1 vlan 200 serverfarm primary

policy-map multi-match pmap-vlan100 class ftp.mycompany.com loadbalance vip inservice loadbalance policy lbmap-ftp.mycompany.com loadbalance vip icmp-reply inspect ftp access-list ALL line 10 extended permit ip any any interface vlan 200 description Web Servers Lan

ip address 10.0.1.2 255.255.255.0 peer ip address 10.0.1.3 255.255.255.0 alias 10.0.1.1 255.255.255.0 nat-pool 1 10.0.1.254 10.0.1.254 netmask 255.255.255.255 pat access-group input ALL no shutdown

interface vlan 100 description OUTSIDE ip address 55.55.55.11 255.255.255.0 alias 55.55.55.10 255.255.255.0 peer ip address 55.55.55.12 255.255.255.0 access-group input ALL service-policy input pmap-vl100 no shutdown

Cisco ACE One-Arm Mode Setup

In View of Public interest & noticing a huge hit on Cisco load balancer related blogs. I am coming up with Cisco ACE One-Arm Mode Setup.

Firstly, lets talk about the benefit of a One-Arm mode setup. 1) This setup doesnt require you to redesign your IP schema as you just place in your load balancer & configure a virtual IP for the traffic to hit & change the DNS :) Ya, thats all the benefit !! :) A major drawback of this setup is you half the throughput of the interface as the traffic needs to traverse the same physical interface twice thereby as an example making your 100 Mbps fastethernet link merely doing a 50 Mbps on a full duplex.

Well, lets begin anyways forgetting the drawbacks. Now, based on the diagram you need to loadbalance 3 app servers 192.168.1.11, 12, 13 on Cisco ACE. So, what are the things to do. 1) On the Layer 3 switch, you define a VLAN ID 20 with gateway as 172.16.1.1 & add the interface of Cisco ACE connected on L3 switch to VLAN 20. 2) On Cisco ACE, create the serverfarm, create the rservers (real IP address of the APP server has shown in diagram, & define the virtual IP as 172.16.1.100 as in this example & create class-maps, policy-maps & apply the service policy to the outbound VLAN (i.e 20) 3) Remember, for the servers the default gateway remains the layer 3 only (i.e 192.168.1.1). Thats all? Ya for now :D Now, lets begin the real world communication 1) Client sitting on the internet attempts a connection to (http://iamnotccie.com/) which resolves to the Virtual IP we configured (i.e 172.16.1.100). 2) The router sitting on the edge forwards the traffic to the Layer 3 switch which inturns forwards to the Cisco ACE loadbalancer. 3)Cisco ACE checks the associated realservers to the Virtual IP & forwards to the one available (depending on the predictor configured - Mostly RoundRobin, easy right?) Woow, client still waiting for the damn webpage to load !!!?? What the heck has been happening? Alrighty, let me grab a packet capture on the client PC & see the connections. Well, i notice the below 1) Client sends the initial request to 172.16.1.100 2) A subsequent packet comes back with the IP of 192.168.1.11, & client PC resets the connection (RST) So, the issue is the realserver responds directly back to the client PC. Because the source address seen by the server is from client PC only & not that of the loadbalancer. Hence, the server sends the reply via it's default GW. Default GW has neccessary routing to reach back those client PC IP subnet. How to resolve this? Any Guess? Ya you got it right... Source NAT on the ACE. You create a dynamic nat-pool with a pool or a single IP & have all the traffic routed via the cisco ACE. Below is the complete config :
interface gigabitEthernet 1/1 switchport access vlan 100 no shutdown

interface gigabitEthernet 1/2 switchport access vlan 20 no shutdown access-list everyone line 8 extended permit ip any any access-list everyone line 16 extended permit icmp any any probe http HTTP_82 port 82 interval 20 passdetect interval 40 receive 3 expect status 0 499 connection term forced open 1 rserver host EPSWEB01 ip address 192.168.1.11 inservice rserver host EPSWEB02 ip address 192.168.1.12 inservice

rserver host EPSWEB03 ip address 192.168.1.13 inservice

serverfarm host PEOPLESOFT_Web_80_to_82_Farm rserver EPSWEB01 82 probe HTTP_82 inservice

rserver EPSWEB02 82 probe HTTP_82 inservice

rserver EPSWEB03 82 probe HTTP_82 inservice

sticky ip-netmask 255.255.255.255 address source PEOPLESOFT_Web_80_to_82_Sticky

timeout 720 serverfarm PEOPLESOFT_Web_80_to_82_Farm

class-map type management match-any MANAGEMENT description Management Traffic 2 match protocol telnet any 3 match protocol ssh any 4 match protocol snmp any 5 match protocol http any 6 match protocol https any 7 match protocol icmp any

class-map match-all PEOPLESOFT_CLASS_80_to_82 2 match virtual-address 172.16.1.100 tcp eq http policy-map type management first-match MGMT class MANAGEMENT permit

policy-map type loadbalance first-match PEOPLESOFT_Web_80_to_82-l7slb class class-default sticky-serverfarm PEOPLESOFT_Web_80_to_82_Sticky

policy-map multi-match L4_LB_VIP_POLICY_CLIENT_TO_SERVER class PEOPLESOFT_CLASS_80_to_82 loadbalance vip inservice loadbalance policy PEOPLESOFT_Web_80_to_82-l7slb loadbalance vip icmp-reply nat dynamic 1 vlan 20 interface vlan 100 description Management VLAN - For Device Management ip address 10.1.100.12 255.255.255.0 service-policy input MGMT no shutdown

interface vlan 20 description "Client-Server VLAN" ip address 172.16.1.10 255.255.255.0 nat-pool 1 172.16.1.50 172.16.1.50 netmask 255.255.255.0 pat service-policy input L4_LB_VIP_POLICY_CLIENT_TO_SERVER no shutdown

ip route 0.0.0.0 0.0.0.0 172.16.1.1

Notice the configuration carefully, you will notice that realservers (i.e our PEOPLESOFT server) runs the app on Port 82. Think in this way, does a normal user always type in http://172.16.1.100:82/ to access the server? So, we need so kind of mechanism where we do port redirection from a normal Port 80 connection to Port 82. This we achieve in class-map match-all statement & define a virtual address with a layer 4 Policy & expects traffic coming in with Dst Port as Port 80. If a connection comes on Port 82, then it will drop. So, if a user attempts (http://172.16.1.100:82/) it wont work for him. This basically adds a level of security as the real listening port is hidden from a user. So, now once a connection request on port 80 arrives at Load balancer it checks the serverfarm & redirects traffic to Port 82 (notice the port number besides the real server in the serverfarm). Now, if a user types the URL as http://172.16.1.100/ all works fine ! Thanks, to lot of people who visited my blog & recommended them on Cisco supportforums. Any mistakes in the write-up can be brought to my notice & i will surely thank them for that ! Posted by Vivek Ganapathi at 6:37 AM 2 comments: Email ThisBlogThis!Share to TwitterShare to Facebook Labels: Cisco ACE

Monday, July 12, 2010

Cisco ACE 4710 Load balancer configuration

Coming back with a BANG ! Lets start up with the Cisco ACE load balancer & its configuration. NOTE : Minimum understanding of how load balancing works is required to understand this topic. Cisco ACE does an application load balancer (i.e Layer 7) Cisco ACE load balancing works on two different modes

Routed Mode Bridged Mode

Pretty simple to remember them is Routed mode is a Layer 3 function & Bridged mode is a Layer 2 function. Below diagram would provide a clean overview of them.

Above diagram shows the Routed mode strategy. Load balancer has two legs, one connected to a different network & another one towards the Serverfarm. Now, for the servers the default gateway will always be the interface VLAN 40 as the gateway & routing needs to be available on LB to pass the packet to next hop (i.e Router). Now, what if the customer wants the Layer 3 boundary on router & make a load balancer as just another Layer 2 switch. For this, there is something called "Bridged mode".

If you have a clear look @ the diagram you notice that the servers behind the LB are in the same segment as the router. VLAN 30 & VLAN 31 would be part of the bridge & a virtual Bridged interface would created to bridge these two VLANs. Configuration will provide a better overview of the two modes. Lets start up with the configuration part now. ROUTED MODE -------------------

First create Layer 3 interfaces as per the (1) Diagram interface vlan 20 description "Outside towards client interface" ip address 172.16.1.5 255.255.255.0 no shutdown interface vlan 40 description "Outside towards client interface" ip address 192.168.1.1 255.255.255.0 no shutdown Now add the physical ports into the respective VLAN. Assume, Gi1/1 is the interface connecting to the router & Gi1/2 is connected to the serverfarm segment with a Layer 2 switch in between. interface gigabitEthernet 1/1 description *** Interface connecting to the router *** switchport access vlan 20 no shutdown

interface gigabitEthernet 1/2 description *** Interface connecting to the serverfarm *** switchport access vlan 40 no shutdown Now we enter into the load balancing stuff ! We need to create real physical servers first & then associate them into a serverfarm. For an example, Within a serverfarm there could be more than 100 physical servers catering the application / web request. One such good example would be internet banking websites. Do you think there is only one server catering the request of millions of customers worldwide? No chance. Users from around the world would be hitting an IP address (Virtual IP) which is configured on the LB. LB does the load balancing between certain servers configured in the real server list. rserver host PROD1 ip address 192.168.1.10 inservice

rserver host PROD2 ip address 192.168.1.11 inservice For this example, we will take up only two real servers. The command "inservice" states that the servers are not administratively shutdown & are in operational state. This command is a must to bring the servers alive. Now, we need to create a serverfarm. Serverfarm is nothing other than a farm holding multiple servers within it. serverfarm host PROD-web rserver PROD1 inservice rserver PROD2 inservice Serverfarm has been created & the real servers are associated with it. By Default, ACE doesnt allow any management traffic on the interface. So, we need to explicitly allow them by doing the below configuration class-map type management match-any MGMT 2 match protocol ssh any 3 match protocol telnet any 5 match protocol https any 6 match protocol http any 7 match protocol icmp any Now, a policy-map needs to be created to permit the MGMT traffic. policy-map type management first-match MGMT class MGMT permit We need to use a class-map to define the VIP when a client sends their requests. Users from inside or outside will be targetting this Virtual IP address

class-map match-all slb-vip-PROD 2 match virtual-address 172.16.1.3 any Next step is to define the action when a new client requests arrives. This is called a Layer-4 policy. policy-map type loadbalance first-match LB_POLICY_PROD class class-default serverfarm PROD-web Now the Virtual IP & load balancing parameters are defined, they needs to be associated to make sure how the traffic is being handled when it hits the VIP. To define this, there is multimatch policy. This policy needs to be applied on interfaces where the traffic is hitting. icmp-reply feature needs to be enabled in order to have the VIP pingable. This policy is called a Layer 7 Policy. policy-map multi-match CLIENT-VIPS_PROD class slb-vip-PROD loadbalance vip inservice loadbalance policy LB_POLICY_PROD loadbalance vip icmp-reply active Now an ACL needs to be created to make sure all the users are allowed to reach the servers from outside the world or from the corporate network. access-list INBOUND line 8 extended permit ip any any access-list INBOUND line 16 extended permit icmp any any Once all the above is done, its time to apply the service policy onto the respective interface & a default route pointing towards the router's interface. Where will the Layer 7 service policy be applied? Any idea? Think Think ... Well, it should be applied on the interface which is facing towards serverfarm as the packet is destined to the servers. interface vlan 20 description "Outside towards client interface"service-policy input CLIENT-VIPS_PROD ip address 172.16.1.5 255.255.255.0 service-policy input MGMT no shutdown

ip route 0.0.0.0 0.0.0.0 172.16.1.1 Thanks to Sandev Singh for the correction on the service-policy direction. I was probably sleeping while writing :) Alrighty mate ! We have done with the routed mode configuration. Now we can publish the VIP into DNS with some DNS name. Example http://www.miss.you.ch/ to 172.16.1.3. You are done ! y0 y0...