VSX

workshop
Lab architecture

Web_Australia.cp
172.31.1.201 MDG 192.268.X.4
MDS 10.1.1.1 External Server
Eth1
172.31.1.2 Eth0
172.16.1.2
Main DMS
Management network
vmnet
Target DMS
External network eth2 172.16.1.0/16

Target DMS
Sync eth1 Sync eth1

6.6.6.101 6.6.6.103
VSX_GW1 6.6.6.102
eth0 10.1.1.101
VSX_GW2
eth0 10.1.1.102 VSX_GW3
Japan networks eth4 eth0 10.1.1.103
USA networks eth3 Not configured
yet

10.1.1.242

Client SDM Client

VMnet1 - Mgmt_network=10.1.1.0/24
VMnet2 - synchronization network=6.6.6.0/24
VMnet3 - Internet=172.16.1.2/16
VMnet4 - USA_corp_network=10.2.1.0/24
VMnet5 - Japan_corp_network=10.3.1.0/24
VMnet6 - Internet=172.31.1.10/16

** In this lab environment all the network interfaces on the VSX host are connected to the same
Broadcast domain / VLAN.

2
Lab 1

Objective: Create and configure VSX cluster

Time: 25 min

At this lab we'll create a new VSX cluster, using the two preinstalled modules. We will finish this lab with fully
functional VSX cluster, ready for the creation of Virtual Devices.
We will also see the policy which is used by the VSX itself, and how to change it.
In the verification will review the process from the GUI and will take a look behind the sheens of the VSX creation.
Connecting to the Management Client
On the top panel locate the Client_Mngm tab and click on it

Once you click on tab, you will see the Client_Mngm desktop inside the browser after automatic login.
(if you experiencing long time connection issue, press F5 to reload the page)

3
Creating VSX Cluster
1. Connect to VSX-GW-1 (10.1.1.101) via ssh (admin/vpn123) and enter expert mode. Run “vsx stat –v” as you can
see before we are installing VSX on these machine it’s not a VSX platform.
2. Connect to the MDS with SmartConsole to IP address 10.1.1.1 user admin password vpn123.
3. Choose the Main_VSX_Server.

4. On the object category go Right click on Network Objects -> Gateway and Servers -> VSX -> Cluster…

5. Fill in the General properties: Cluster name (" LAB_Cluster "), IP (10.1.1.104), version (R80.10) and Cluster
platform (Check Point SecurePlatform ClusterXL).
6. Choose the custom configuration in the templates screen.
7. Add the VSX-GW1-R80.10 (10.1.1.101) and VSX-GW2-R80.10 (10.1.1.102)
The Initialize (password is "vpn123")
8. No need to check for vlan trunk.
9. For Synchronization network select eth1 and configure both members (VSX-GW1-R80.10 with 6.6.6.101/24 and
VSX-GW2-R80.10 with 6.6.6.102/24), because it’s an isolated network you can use anything as long as it is in the
same subnet.
10. On the wizard page about creating an additional virtual network device, click next.

4
11. Change the VSX default policy by adding the GUI_PC/Client_Mgmt object with IP 10.1.1.242 and allow all rules
from Gui_PC object to getaway for management access by checking all the checkboxes see screen shot below:

12. Complete the creation.

13. Add licenses to the VSX from via SmartUpdate of the Main_VSX_Server DMS

License files located on C:\Users\cp\Desktop\newlic\2018 (10.1.1.5-GW1.lic 10.1.1.5-VSX-GW2-R80.10)

After importing the file, attach a license to each once of the vsx nodes.

5
Objective: Verifying VSX Creation.
Time: 45 min

1. View the installation report in the SmartConsole.

2. In the expert shell of VSX-GW1-R80.10 or VSX-GW2-R80.10 run "vsx stat –v " this command lists all virtual
devices currently configured on the Gateway and current policy information.
3. During the creation of the VSX, a process has been performed that called “Implicit Conversion” that switched the
machine from “Kernel mode” FW to “User mode” FW, vi the registry file and search for User mode and VSX.
4. Please look at the conversion log on the VSX Gateway /var/log/gw_to_vsx.log
5. Ping the VSX cluster from the GUI_PC server (ping 10.1.1.104),
6. Management Implementation review: Open GuiDBedit and connect to the Main_VSX_Server DMS. Examine
Lab_Cluster object under Network Objects/network_objects table. Identify the Cluster's mode, VS High
availability, list of members and the private network (funny IPs network).
7. In GUIDBedit (:\Program Files (x86)\CheckPoint\SmartConsole\R80.10\PROGRAM ) in to Network
Objects/vs_slots_objects Examine Lab_Cluster object
See the differences between the interfaces list of VSX-GW-1 GW-VSX-2 & Lab_Cluster
Look at the object's slots as well. See the interfaces / routes tables and their parallel "installed" tables
8. Open GuiDBedit connects each DMS separately. The slots are located on the Main_VSX_Server DMS only!
9. Using command line under Main_VSX_Server DMS $FWDIR/conf/vs_repository/VSX_NAME directory look at
files .vsrt and .vsnew
10. Using command line under Main_VSX_Server DMS $FWDIR/state/VSX_NAME/VSX look at files
(local.vs,local.vsall,local.vskeep)
11. Create a policy with Any/Any/Accept/log and install it
12. Run cphaprob stat
13. Enforcement Module Implementation review: On one of the member cd into $FWDIR/state/local/VSX. Examine
local.vskeep & local.vsall (cat or vi the files).
14. Examine $CPDIR/conf/ctxdb.C – Some of the VSX commands iterate through this file in order to display
information regarding the configured Virtual Systems. (Therefore, in case the output of vsx stat –v is not sorted,
it can be fixed by sorting this file).
15. On the VSX Gateway Cat /config/active file look for vs0 definitions.
Netstat – nap | grep 257 or Netstat – npa | grep 10.1.1.5 you should see the ports that are open to the DMS.
Do the same on the P1 and verify that those are the same ports.

6
Lab 2
In this lab we will perform Route Propagation between Virtual Systems connected to a common Virtual Router.
We will first see how to create a Virtual router, and how to connect to it Virtual Devices. We will see how to manage
the security policy of different VSs.
We will view how routes are propagated from VSs to other adjacent VSs, and how this helps the day-to-day
Main_VSX_Server tenance of a large VSX system.
In the verification you will review the process from the GUI and will take a look behind the sheens of the VSX
creation, Management database and Gateway side.

Eth1
172.31.1.2

Eth0
172.16.1.2
External network
172.16.1.0/16
eth2

EX_VR
External IP 172.16.1.1

172.18.108.1
172.17.107.1

VS_USA
VS_Japan

10.2.1.104/24 10.3.1.104/24
eth3 eth4

Eth3 Eth4

7
Objective: Configuring VSX with Virtual router and Route propagation between VSs
Time: 25 min

Creating Virtual router

1. Connect to the MDS with the IP address 10.1.1.1 user admin password vpn123.
Login to Main_VSX_Server.
2. Create a Virtual Router. (right click on the object category go Right click on Network Objects -> Gateway and
Servers -> VSX -> Virtual Router)
3. Set the name of the Virtual System to EXT_VR, and choose to create it on LAB_Cluster.
4. Add eth2 as the Virtual Router interface set the IP address to 172.16.1.1\16.
5. Add default route that will lead you to the class router IP address 172.16.1.2.
6. Click "finish" to complete the Virtual Router wizard.
7. In order to check connectivity between the virtual router and the Router_Internet, connect to the class router
User: admin , password: vpn123
Ping to the virtual router (EXT_VR) ping 172.16.1.1 ( the EXT_VR will not respond to ping, since the default policy
that is installed on in does not permit “ping”

Use Ctrl+c to stop pinging and run the command: arp

In this example you can see that no MAC address was found for the EXT_VR ip 172.16.1.1
(if you see a mac address you can move to the next section)
** This problem happens from time to time on shared heavily loaded nested environment (not a real world
Situation)
In order to solve this problem, double click on the EXT_VR

8
 Click on the Topology tab, and edit eth2, change the interface to eth3, and click ok and ok

Once the configuration deployed, set back the interface to eth2 again.
Now go back to the Router_Internet console and run ping to 172.16.1.1 afterward run arp.
Now you will see the mac address of the EXT_VR on interface 172.16.1.1

Creating Virtual Systems

1. Connect to the MDS with the IP address 10.1.1.1 user admin password vpn123.
Login to USA_Target_Server
2. Create a Virtual System (right click on the object category go Right click on Network Objects -> Gateway and
Servers -> VSX -> Virtual System)

Set the name of the Virtual System to VS_USA.

Choose to create the VS on "LAB_cluster ".
3. Define eth3 as the internal interface and IP address of 10.2.1.104 /24 (this will simulate the customer’s network),
Define an external interface that leads to the Virtual Router interface with IP 172.17.107.1.
4. make sure to check propagate the route.
5. Add a default route as the Virtual Router.
6. Click "finish" to complete the VS creation.
7. Push policy Any/Any/Accept
8. Create one more Virtual System:
Login to Japan Domain with SmartConsole IP address 10.1.1.30 user: admin password: vpn123
Create a Virtual System (Network Objects -> Check Point -> Right click -> VSX-> Virtual System)
Set the name of the Virtual System to VS_Japan.
Choose to create the VS on "LAB_cluster".
9. Define eth4 as the internal interface and IP address of 10.3.1.104 /24 (this will simulate the customer network),
make sure to check propagate the route.
10. Define an external interface that leads to the Virtual Router interface with IP 172.18.108.1.
11. Add a default route leading as the Virtual Router.
12. Click "finish" to complete the VS creation.
13. Create and Push policy Any/Any/Accept.

9
Objective: Verifying Virtual System creation – Part 1
Time: 35 min

1. In VSX-GW1-R80.10 enter Expert mode and run vsx stat –v.

Notice the ids, types, SIC and different policies on the different objects.
2. Run cpwd_admin list and check that every process has PID or
cpwd_admin list –ctx vsid - this will display the process per VS.
Netstat – nap | grep 257 or Netstat – npa | grep 10.1.1.20 - you should see the ports that are open to the DMS.
Do the same on the P1 and verify that those are the same ports.
3. Verify cluster status (cphaprob stat), list of devices (cphaprob list) and monitored interfaces (cphaprob -a if).
4. On one of the member cd into $FWDIR/state/local/VSX.
Examine local.vskeep & local.vsall (cat or vi the files).
Examine on the Gateway $CPDIR/conf/ctxdb.C – Some of the VSX commands iterate through this file in order to
display information regarding the configured Virtual Systems.
Therefore, in case the output of vsx stat –v is not sorted; it can be fixed by sorting this file.
5. Edit via vi /config/active search for the new VSs information.
6. Management Implementation review:
Open GuiDBedit C:\Program Files (x86)\CheckPoint\SmartConsole\R80.10\PROGRAM and connect to the
Main_VSX_Server DMS and to target DMSs.
Under network objects Examine EX_VR /VS_USA /VS_ Japan network object Identify the Cluster's mode, list of
members and the private network (funny IPs network).
7. See the differences between the interfaces list of EX_VR /VS_USA /VS_Japan see the Main_VSX _customer value
for (EX_VR /VS_USA /VS_Japan).
8. Look under /network objects/ vs_slots table as well. See the interfaces / routes tables and their parallel
"installed" and the vsid, cluster members ,target_customer for (EX_VR /VS_USA /VS_Japan) value of each vs
9. Open, GuiDBedit connects each DMS separately. The slots are located on the Main_VSX_Server DMS only!)
10. Using command line under Main_VSX_Server DMS $FWDIR/conf/vs_repository/VSX_NAME directory look at
files .vsrt and .vsnew.
11. Using command line under Main_VSX_Server DMS $FWDIR/state/VSX_NAME/VSX look at files
(local.vs,local.vsall,local.vskeep).

10
Objective: Verifying Virtual System creation – Part 2
Time: 40 min

1. Vsenv to the EX_VR (VSID) run ifconfig please view the interfaces.
2. On EX_VR Run fwaccel off this will disable acceleration.
3. On EX_VR ping the default route of the router 172.16.1.2 and 172.31.1.2
4. On EX_VR run tcpdump on eth2 use proto 1, view the traffic.
5. Vsenv to VS_USA (VSID) run fwaccel off - this will disable acceleration in order to see all the connection without
it to be accelerated.
Run ifconfig please view the interfaces/ips (if you don’t see ip 10.2.1.104, push policy again (This problem
Happens from time to time on shared heavily loaded nested environment (not a real world situation)
6. From to VS_USA Ping 172.31.1.201 (Australia.cp) please collect tcpdump from wrp interface and draw the packet
flow from the VS_USA till the web server.
7. Collect tcpdump from eth2 on the Virtual Router this should show us the packet from the VS to the remote
Server.
8. From CloudShare, connect to USA server (10.2.1.201) or Japan server (10.3.1.201) hosts try to browse to
http://172.31.1.201 (Australia.cp) – it fails since we do not have NAT configured in order to translate the internal
networks.
9. Run fw monitor on VS_USA .
10. From Japan_corp.cp try to browse to http://10.2.1.201 (USA.corp.cp) - it works, wonder why?
11. From USA_corp.cp try to browse to http://10.3.1.201 (JAPAN.corp.cp) it works, wonder why? That’s route
propagation.
12. Run kernel debug and fw monitor on VS_USA Use the following commands:
fw monitor –v vsid –e ‘port(80),accept;’
vsenv VS_USA vsid
fw ctl debug –buf 32000 –v vsid
fw ctl debug –v vsid –m fw + conn
fw ctl kdebug –v vsid –T –f > filename

From USA try to browse again to http://10.3.1.201 (JAPAN.corp.cp)

Search for the packets that are coming from 10.3.1.20.
13. Run kernel debug on dispatcher Use the following commands Dispatcher:
To debug the Dispatcher run a kernel debug on VS0 with the below flags:
fw ctl debug –buf 32000
fw ctl debug –m fw + user packet
fw ctl kdebug –T –f > filename
14. Search for the packets that are coming from 10.3.1.201 (JAPAN.corp.cp) and 10.2.1.201 (USA.corp.cp).
15. Edit the object of EX_VR and look at the routes. See how the routing table of EX_VR contains routes regarding
the internal networks of VS_JAPAN and VS_USA, and note the Gateway for those routes.
16. Open GuiDbedit to the Main_VSX_Server DMS under Network objects/vs_slots table under EX_VR type routes.
17. Search for the propagated routes you could see the internal routes of VS_JAPAN and VS_USA under the field of
exported_from_vs like number 2 it means that VS 2 add this route if -1 it is locally added you could see which VS
publish those routes (On Provider-1, GuiDBedit connects each DMS separately. The slots are located on the
Main_VSX_Server DMS only!) This is important when dealing with duplicate routes or routes corruptions on VRs
or VSs.
18. Play around with the "Propagate" checkbox of the internal interfaces of the VSs, and see how they affect the
routes on the Virtual Router in the GUI.
19. See how the routes on the module for VS_USA and for other VSs are added/deleted when changing the
"Propagate" route on the EX_VR.
20. Please define Network objects for your USA_corp.cp (10.2.1.0) named NET_10.2.1.0 and for Japan_corp.cp
(10.3.1.0) named NET_10.3.1.0.

11
 21. P
lease define Hide Nat for your USA_corp.cp (10.2.1.0) NET_10.2.1.0 and for Japan_corp.cp (10.3.1.0)
NET_10.3.1.0.
22. From USA or Japan hosts try to browse to http://172.31.1.201 (Australia.cp) work now since defined NAT that
hides our internal network.
23. Run the following API script in your environment this will create network objects Under USA domain using
layers.
24. Configure the rulebase on USA Target like below, any object that is not exists please create it.

25. From web server (USA_server) ping on the gateway, try to ping 172.31.1.201 and run kernel debug on VS USA
with the following flags.
26. Try to create ftp session to web Australia:
#fw ctl zdebug drop | grep 10.2.1.201 - you should see the layer that is dropping this packet.

12
Lab 3
How to create manual and automatic NAT behind VS
In this lab we will perform a manual NAT behind Virtual Systems connected to a common Virtual Router.
We will first see how to configure the NAT, then how to manage the security policy of different VSs and we will test
connectivity between our web servers.

Eth1
172.31.1.2

Eth0
External network 172.16.1.2
172.16.1.0/16
eth2

EX_VR
External IP 172.16.1.1
172.17.107.1
NAT address for
USA_corp=172.17.107.2
172.18.108.1

VS_USA
VS_Japan

10.2.1.104/24
10.3.1.104/24
eth3
eth4

Eth3 Eth4

Japan_corp.cp 10.3.1.201
Mexico_Mexico-City_DMZ 10.3.1.205
USA_New-York_DMZ 10.3.1.204
Spain_Madrid_DMZ 10.3.1.202

13
Objective: Creating manual Nat for USA.corp on VS_USA using Virtual Router
Time: 45 min

1. On VS_USA set rulebase or Any:Any :accept

2. The Nated IP is 172.17.107.2 of USA_Web_Server.
3. Please undefine Hide Nat for your USA_corp.cp (10.2.1.0) Net and for Japan_corp.cp (10.3.1.0) Net.
4. In CloudShare open the external router_new.
5. On router_new log in to clish add route run # set static-route 172.17.107.2/32 nexthop gateway address
172.16.1.1 on
6. Save config.
7. Ping to 172.17.107.2 - does not work? Ok, then what is missing here?
8. Open SmartDashbord on Main_VSX_Server DMS open Virtual Router and add route for 172.17.107.2/32 point
to the Virtual System.

9. On EX_VR Virtual Router verify route default to point to the 172.16.1.2.

10. On the DMS USA create 2 hosts objects that will represent USA_corp for the Nat Rule configure external host
named external_USA_corp with IP 172.17.107.2 and for internal host create object Internal_USA_corp with
IP10.2.1.201 (USA.corp.cp).
11. Configure the NAT rules as below, Need to replace:

NAT Rule

12. Push policy and test connection from the external router try to ping 172.17.107.2. It might not work, why?
13. Browse from Australia 172.31.1.201 (Australia.cp) to 172.17.107.2.
14. Now do the same for Japan corp with external IP of 172.17.107.4.
15. Remove all manual NAT.

14
 Creating Automatic NAT for DMZ web sites behind VS_Japan using Virtual Router

1. In Japan DMS edit object Japan_corp.cp 10.3.1.201, Mexico_Mexico-City_DMZ 10.3.1.205, USA_New-

York_DMZ 10.3.1.204,Spain_Madrid_DMZ 10.3.1.202 create automatic NAT assigning from the IP in ranges
172.18.108.5 until 172.18.108.20
2. Try browsing from web Australia to the NATed IPs or ping, not working? Wonder why?
3. On CloudShare external router_new log in to clish, add route – run:
# set static route 172.18.108.0/24 nexthop gateway address gateway 172.16.1.1 on
4. Save config.
Now routes are configured on the router that simulates the internet router.
5. Try browsing from web Australia to the NATed IPs Not working? Wonder why?
6. NAT is configured on the nodes and in the rule base but not on the Virtual_router.
7. Edit the VS under topology add Nat ranges that starts from 172.18.108.5 till 172.18.108.20.

8. Install policy.
9. Please view the virtual router topology and notice that the NAT range turned into routes.
10. Try browsing from web Australia to the NATed IPs should work?
11. Remove all NAT definitions.
12. Remove all interfaces that are connected to the Virtual Router (please keep the internal interfaces eth3 and
eth4 with the same definition).
13. Delete the Virtual Router.

15
Lab 4
In this lab we will connect our environment with Virtual Switch, we will perform route propagation between Virtual
Systems connected to a common Virtual Switch.
This lab will demonstrate connectivity using static and hide NAT, we will see how routes are propagated from VSs to
other adjacent VSs. In the verification will review the process from the GUI and will take a look behind the scenes of
the VSX creation Management database and GW side.

Eth1
172.31.1.2

Eth0
172.16.1.2
External network
172.16.1.0/24
eth2

EXT_VSW

172.16.1.202
172.16.1.201
172.16.1.204

VS_USA
VS_IL VS_Japan

10.2.1.104/24 10.3.1.104/24
eth3 eth4

Eth3 Eth4

16
Objective: Basic Virtual Switch configuration
Time: 60 min

Creating new configuration:

1. After removing the virtual router Eth2 should be free to use
2. Create a Virtual Switch named EXT_VSW and connect it to eth2.
3. Connect the two Virtual Systems to the Virtual Switch (from VS’s topology tab) using the following
IPs: 172.16.1.201, 172.16.1.202 (netmask 255.255.0.0).
4. Add a default route leading to 172.16.1.2 (external router).
5. Install a policy with "Any Any Any Accept Log" on the 2 VSs.
6. Under Main_VSX_Server DMS create 1 new a Virtual System (Network Objects -> Check Point -> Right click ->
VSX-> Virtual System)
7. Set the names of the Virtual System to VS_IL.
8. Choose to create the VS on "LAB_cluster ".
9. Define only external interface that leads to the Virtual Switch interface with IP 172.16.1.204/16.
10. Click "finish" to complete the VS creation.
11. Push policy to VS_IL.

Verifying new configuration

1. Execute vsx stat –v, and see what is the VS ID of VS_USA.
2. Execute cphaprob state and see which member is Active.
3. Enter VS_Japan context on the active member by running: vsenv <vsid>.
Test inter-VS connectivity: ping the external and internal IPs of VS_USA (172.16.1.201 and 10.2.1.104).
4. Enter VS_USA context on the active member by running:”vsenv <vsid>".
Test inter-VS connectivity: ping the external and internal IPs of VS_Japan (172.16.1.202and 10.3.1.104).
5. In VSX-GW1-R80.10 enter Expert mode and run vsx stat –v and vsx stat –l
Notice the ids, types, SIC and different policies on the different objects.
6. Run cpwd_admin list and check that every process has PID or
cpwd_admin list –ctx vsid this will display the process per VS.
7. Verify cluster status (cphaprob stat), list of devices (cphaprob –l list) and monitored interfaces (cphaprob -a if).
8. Vsenv to the EX_VSW run ifconfig note that the wrpj does not have IPs.
9. Run fwaccel off this will disable acceleration.
10. Vsenv to VS_USA run fwaccel off this will disable acceleration in order to see all the connection without it to be
accelerated.
11. Vsenv to VS_Japan run fwaccel off this will disable acceleration in order to see all the connection without it to be
accelerated.
12. Ping from internal hosts to 172.31.1.201 (Australia.cp)
Collect tcpdump from wrp interface and from the wrpj on the Virtual Switch.
Draw the packet flow from the VS until the web server.
13. Collect tcpdump from eth2 on the Switch this should show us the packet from the VS to the remote server.
14. From USA or Japan hosts please try to browse to http://172.31.1.201 (Australia.cp) -> fail since we do not have
NAT configured in order to translate the internal networks After doing this please run fw monitor ‘accept
ip_p=1;’ on the Vs.
Looks like we are hiding behind the wrong IP address, see the difference between the behaviors of Virtual Switch
to a Router.

17
15. From Japan_corp.cp try to browse to http:// 10.2.1.201 (USA.corp.cp)
16. From USA_corp.cp try to browse to http://10.3.1.201 (JAPAN.corp.cp)
17. Edit the objects of VS_USA and VS_JAPAN and look at the routes. See how the routing table of contains routes
regarding the internal networks of VS_JAPAN and VS_USA, and note the Gateway for those routes.
18. Open GuiDbedit to the Main_VSX_Server DMS under Network objects/vs_slots table under VS_USA and
VS_JAPAN type routes.
19. Search for the propagated routes you could see the internal routes of VS_JAPAN and VS_USA under the field of
exported_from_vs you could see which VS publish those routes on VS_Japan serch for route 10.1.3.0 see that
value exported_from_vs -1 it means that the route was locally defied by the system and was not publish by
other VSs this is important when dealing with duplicate routes or routes corruptions on VRS or VSs.
20. Uncheck propagate route on (VS_USA 10.2.1.0/24 for eth3) and (VS_Japan10.3.1.0/24 for eth4) and create
static NAT for host (10.2.1.201 (USA.corp.cp) use 172.16.1.206 for NAT IP and for host (10.3.1.201
(JAPAN.corp.cp) use 172.16.1.205 for NAT IP.
21. Browse from the web server Japan.corp.cp to 172.16.1.206 and from USA_corp.cp 172.16.1.205.
From USA or Japan hosts please try to browse to http://172.31.1.201 (Australia.cp) work now since defined NAT
that hides our internal network.
22. Run fw ctl arp and check to which interface the mac is belong to.
23. On VSX-GW1-R80.10, run fw monitor –e -v "accept,http;" or on the VS fw monitor –v vsid –e "accept http;"
24. See that the http packets are being passed on VS_USA and VS_JAPAN.
25. Where X is the VSID you have seen in ifconfig. This will show in more detail on the packet on VS_USA.
26. Stop the fw monitor.
27. Test cluster fail over by killing fwd process for vsid2.
28. See that the VSX Gateway failed over and not only the VS, this is how it works in HA mode without monitor per
VS enabled.
29. In order to prevent fail over like this in production do ClusterXL_admin down on the standby member and then
restart the process.

18
Lab 5
In this lab we will demonstrate how to convert High Availability cluster to Load Sharing cluster.

Objectives: Convert Cluster to VSLS

Time: 40 min

Converting the Cluster to VSLS

1. On both members, execute cpconfig and enable the Per Virtual System State monitoring.
2. Reboot the modules for the new parameters to be applied.
3. See that after machines reboot, "cphaprob state" says that the clustering mode is "Single VS failover".
4. Test VS fail over by running “vsenv <VS_USA id>”.
5. From vs USA “clusterXL_admin down”.
6. See that only VS_USA failed over.
7. SSH to the P1 and mdsenv to the Main_VSX_Server DMS run ”vsx_util convert_cluster -s 10.1.1.5 –u admin“.
8. Convert LAB_cluster to LS. Choose option #1 to equally distribute VSs.

Verifying Cluster conversion:

9. Run "cphaprob stat". Verify that the cluster mode is now "Virtual System Load Sharing".
10. Examine the distribution of the active Virtual Systems among the cluster members.
11. On the management execute "vsx_util vsls" to examine the load sharing configuration. *
12. Management Implementation review: Open GuiDBedit and connect to the management server. Examine
Lab_Cluster network object. Identify the Cluster's mode, VS load sharing, list of members and the private
network (funny IPs network).
13. Examine the vsls_parameters, Main_VSX_Server the Virtual System's weights and the priority vector.
See the differences between the interfaces list of VS_USA,VSX-GW1-R80.10_VS_USA & VSX-GW2-
R80.10_VS_USA.
14. ON Main_VSX_Server DMS Look at the object's vs_slots table as well. See the interfaces / routes tables and their
parallel "installed" cluster members (On Provider-1, GuiDBedit connects each DMS separately. The slots are
located on the Main_VSX_Server DMS only!)
15. On the Main_VSX_Server DMS please run vsx_util vsls -s 10.1.1.5 –u admin and select Lab_Cluster as the cluster
that you are about to work, chose option 4 for manually set the priority and weight.
16. Set VS_USA with weight of 30 and VS_IL with weight 40 after enforcing the configuration.
17. Run cphaprob stat.
18. Test VS fail over.

19
Lab 6
In this lab we will handle Clustering.
Objectives: Adding a Cluster Member
Time: 40 min

1. Verify that the SmartConsole is closed.

2. On the Main_VSX_Server_Server run vsx_util add_member-s 10.1.1.5 – u admin
Follow the instruction and fill in the details:
Admin: admin ; pass: vpn123; cluster name:Lab_CLuster; member name: VSX-GW3-R80.10; eth1 (sync):
6.6.6.103; eth0: 10.1.1.103
3. Invoke vsx_util add_member_reconf –s 10.1.1.5 –u admin.
4. During the operation please open ssh, run watch –d “ps auxww | grep VSX” command and look at the operation
that are running on the Gateway.
5. Reboot VSX-GW3-R80.10. At this point we have 3 members in the cluster.
6. Execute "cphaprob state" and see which member is Active.
7. Add licenses to VSX-GW3-R80.10 from C:\VSX_training\tools\newlic.
8. Management Implementation review: Open GuiDBedit and connect to the management server. Examine
Lab_Cluster network object. Identify the Cluster's mode, VS load sharing, list of members and the private
network (funny IPs network).
9. Examine the vsls_parameters, Main_VSX_Server the Virtual System's weights and the priority vector.
10. Look at the object's vs_slots table under Lab_cluster see the reference to the 3 member that was add in the VS
vsx_slot_objects.

Lab 7
In this lab we will demonstrate how VSLS functions in cases of failovers, and how the Management Server decide
about the distribution of VSs. We'll also learn how to use the ClusterXL commands in the context of VSX.

Objectives: VSLS with 3 members

Time: 40 min

Changing the VSLS distribution:

1. Login to one of the modules, and execute "cphaprob state". See that VSX-GW3-R80.10 doesn't have any VSs in in
Active or in Standby mode.
2. Login to the MDS, and execute "vsx_util vsls –s 10.1.1.5" to distribute VSs equally among all 3 members.
(Option 2) Do not assign weights.
3. Check "cphaprob stat" and see that now some of the VSs in VSX-GW3-R80.10 are in Active/Standby state.

Test VSLS:
1. Check connectivity from USA.corp to the server (ping server 10.3.1.104). The traffic will pass through VS_USA.
2. On another member's terminal run: “watch –d "cphaprob stat".
This will display the cluster configuration and will refresh every 2 seconds (you can change the resolution using
the –n <seconds> switch).
3. We'll now perform the connectivity test from USA.corp to Japan.corp.
4. Trigger VS_USA to be down (think how). Quickly change to the window of the "watch".
5. See the changes taking place on the member from clauses 2-3:
Active->Down, Standby->Active, Backup->Standby.

6. See that during the failover process the ping is still running.
7. Return VS_USA back to UP state.
8. Check status of VSLS again by “cphaprob stat” command.

20
Lab 8
In this lab we will demonstrate issues with a corrupted Gateway (Gaia configuration) and how to trace it, and
resolved them.

Objectives: Reconfigure member

Time: 90 min

Part-1
1. On VSX-GW3-R80.10 run reset_gw
2. After reboot run in clish set vsx off
3. Add 2 default routes:
Set static-route default nexthop gateway address 10.1.1.230 on
Set static-route default nexthop gateway address 10.1.1.231 on
Set vsx on
4. Please verify the management IP is configure as 10.1.1.103.
5. On Main_VSX_Server_Server DMS run vsx_util reconfigure –s 10.1.1.5 –u admin.
6. Chose VSX-GW3-R80.10 as the member to be reconfigured.
7. On the Gateway Run watch –n1 –d ps aux | grep vsx reconfigure should fail.
8. Look at the VSX creation document for debugging see which debugs are needed.
9. Start the cpd debug on the Gateway by running the following command:
cpd_admin debug on TDERROR_ALL_FP_dbg=5
In order to print all data to a single file run:
tail -f $CPDIR/log/cpd.elg &> <output file name>
You should see the vsx fetch running in the background, you should see VSX fetch process running.
See the output file, look for Failed to add or route.
Please see the output of cpd.elg file.
*Never start another reconfigure while fw vsx fetch_cpd is running.
10. Fix the problem.
dbset routed:instance:default:static:default:gateway:address:10.1.1.231
dbset routed:instance:default:static:default:gateway:address:10.1.1.230
dbset :save
Or
set vsx off in clish set static-route default off then set vsx on
Complete the reconfigure.
11. On the getaway Verify that configuration is update
fw vsx fetch –n

21
Part-2
1. Run another reset_gw on the Gateway.
2. After system boot run the following commands:
Run mkdir –p $CPDIR/CTX/CTX00002
Run /bin/ln -s /opt/CPshrd-R80/bin /opt/CPshrd-R80/CTX/CTX00002
Enable debug on the Gateway start again from line 9.

3. Run another reconfigure, it should failed

4. Review the output of the debug.

Part-3
1. Please revert in VMware to snapshot Lab_8_C on VSX-GW3-R80.10
2. Enable debug on the gateway from line 9.
3. Run vsx_util reconfigure - it should failed. Review the debug.
4. Shutdown VSX-GW3-R80.10 and remove member from database.

Lab 9

Cluster Troubleshooting:
1. On VSX-GW2-R80.10 Disconnect Ethernet 3 on VS_USA by running ifconfig eth3 down and for the second test
disconnect it from VMware. Now let's troubleshoot the cluster.
2. Vsenv VS_USA Type “cphaprob stat” to see that the cluster is down. “cphaprob stat” will show only the specific
VS stat.
3. Vsenv VS_USA "cphaprob list" output for the problem. Surprisingly enough, it's an Interface Active Check
problem. Look for other possible problems in the output.
4. Search "cphaprob –vs <vsid> -a if" output for the problematic interface this will show the output of the all VSs.
5. Run "fw ctl zdebug –v vsid -m cluster + if " for a brief moment to see the clustering kernel debug messages.
6. Ifconfig Eth3 up on VS_USA.

22
Lab10

In this lab we will demonstrate Reset SIC in VSX environment on Virtual system and VSX Gateway.

Objectives: How to reset SIC on VS and on VSX

Time: 60 min

Part-1
Resetting SIC:
If for some reason (file corruption or some other voodoo) a Virtual Device had lost trust with management do the
following:

1. Log in to VSX-GW1-R80.10.
2. On Gateway: vsenv <vsid Japan> Look at the registry cd $CPDIR/ registry/ vi HKLM_registry.data
of Japan_VS, search for “MySICname” string and look at “CN=….”. Exit the registry.
3. Execute fw vsx sicreset or cpconfig within the VS content for Japan_VS.
Look at the registry again – what was changed?
Examine vsx stat –v as well.
4. On management: Look for the full SIC name of VS_Japan or run vsx stat –l on the Gateway to see the vs name.
cpca_client lscert | grep <member_name>

IMPORTANT: SIC name is per member. The name you are looking for is as displayed in the output of vsx stat –l on
the module (VSX-GW1-R80.10_Japan_VS)

5. Copy the full sic name and execute :

cpca_client revoke_cert –n "CN=<common name>"
6. On management: edit VS_Japanand, look at the Cluster members tab, Press OK. This will create a new certificate
and will trigger the module to pull it.
7. Edit the VS again and look at the Cluster members tab again.
Examine “vsx stat –v” on the module to see that VS_Japan now has trust.
8. Push policy to the VSs.

It is important to note that if there is a simple connectivity problem, or other much simpler problem, the
GUI/Management may say that there is some "SIC error". 99.9% of the time this DOESN'T mean that SIC has been
lost. The procedure above is for rare cases.

Part-2
How to reset SIC Gateway Level on a DMI configuration
1. On vs0 run netstat –na | grep 257 or 18192 and see that sockets are open from the VSs to the DMSs.
2. On vs0 run cpconfig or cp_conf sic int (activation key) this will perform cpstop and cpstart.
3. Establish SIC from the GUI and push policy.
At this point run vsx stat –v all VSs should not have a policy stat.
Please try to push policy to the VSs - does not work!
4. We will need to run another cpstop cpstart since All the VSs are communicating VIA cpsicdimux and all
communication is currently broken, cpstop and cpstart will trigger the communication again.

23