TN NMS Redundancy and Failover RevF 10012008

NMS Redundancy and Failover
iDX Release 2.0 and Earlier
Technical Note
October 1, 2008
Copyright 2008 VT iDirect, Inc. All rights reserved. Reproduction in whole or in part without permission is
prohibited. Information contained herein is subject to change without notice. The specifications and information
regarding the products in this document are subject to change without notice. All statements, information, and
recommendations in this document are believed to be accurate, but are presented without warranty of any kind,
express, or implied. Users must take full responsibility for their application of any products. Trademarks, brand
names and products mentioned in this document are the property of their respective owners. All such references
are used strictly in an editorial fashion with no intent to convey any affiliation with the name or the product's
rightful owner.
Document Name: TN_NMS_Redundancy_and_Failover_Rev F_01012008.pdf

Document Part Number: T00000052
ii
Revision History
The following table shows all revisions for this document. Refer to this information to verify
that you have the latest version.
Rev
Date Released
Reason for Change(s)
Who Updated?
August 7, 2006
Updated for new release.
E Hoffman
August 11, 2006
E Hoffman
September 25, 2006
E Hoffman
November 16, 2007
E Hoffman
April 21, 2008
Updated based on TAC

feedback changes.
E Hoffman
October 1, 2008
Updated based on TAC

feedback changes.
S Murphy
iii
iv
Contents
About This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .vii
Purpose. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Intended Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Contents Of This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Document Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
iDX Related Documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
iDS Related Documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Hardware-Related Documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Getting Help . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
1 NMS Redundancy Levels . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 Process Redundancy
.................................. 1
1.2 RAID-1 Disk Drive Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2 Configuring the dbBackup Script Parameters . . . . . . . . . . 3

2.1 Setting Up SSH between Primary and Backup NMS Servers
........ 3
2.2 Configuring The NMS to Send Database Backups to The Backup NMS . . 4
2.3 Defining dbBackup Script Default Behavior . . . . . . . . . . . . . . . . . . . 4
2.3.1 Changing the Number of Backup Copies to Save . . . . . . . . . . . . . . . . . . . 5
2.3.2 Changing Database Backups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3 Special Provisioning for a Distributed NMS . . . . . . . . . . . . 7

3.1 Configuring the Primary NMS Server . . . . . . . . . . . . . . . . . . . . . . . . 7
3.2 Configuring the Auxiliary NMS Servers . . . . . . . . . . . . . . . . . . . . . . . 8
3.3 Configuring the Backup NMS Server . . . . . . . . . . . . . . . . . . . . . . . . 8
4 Configuring dbRestore Script Parameters . . . . . . . . . . . . .11

4.1 Configuring the Backup NMS to Restore the Databases . . . . . . . . . . 11
5 Changing Over to the Backup NMS Server . . . . . . . . . . . . .13

5.1 Procedures for Moving Component Files to the Backup NMS Server . . 13
Logging Into The Primary NMS Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Compressing the Component Options File Directories . . . . . . . . . . . . . . . . . . . . . 14
Copying the Component Options File Directories from the Primary NMS Server to Backup NMS Server
14
Extracting Component File Directories Onto Backup NMS Server . . . . . . . . . . . . . . 15
5.2 Procedures for Moving pp_controller Files To the Backup NMS Server 16
Logging Into The Primary NMS Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Compressing the pp_controller Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Copying the pp_controller Files from the Primary NMS Server to Backup NMS Server
17
Extracting the pp_controller Files Onto Backup NMS Server . . . . . . . . . . . . . . . . . 17
5.3 Bringing The Backup NMS Online . . . . . . . . . . . . . . . . . . . . . . . . . 19

Changing The Backup NMS Server To The Primary NMS Server . . . . . . . . . . . . . . . 20
vi
About This Guide
Purpose
The purpose of this document is to explain how to set up and automated backup/restore
mechanism for your NMS configuration and/or archve databases.
Intended Audience
This guide is written for network operators using the iDirect iDS iBuilder/iMonitor/iSite
software suite, network architects, and any other personnel who operate or monitor an
iDirect network. A basic knowledge of TCP/IP concepts, satellite communications, Linux and
MS Windows operating systems and tools (including WinSCP and PuTTY) is assumed. Prior
experience with the operation and monitoring of an iDirect network is desirable but not
required.
Contents Of This Guide

This document contains the following:
Document Conventions
This section illustrates and describes the conventions used throughout the manual. Take a
look now, before you begin using this manual, so that youll know how to interpret the
information presented.
Convention Description
Blue
Courier
font
Used when the user is

required to enter a
command at a command
line prompt or in a console)
Example
[SWITCH_PORT_n]
vid = vlan_id
vii
iDX Related Documents
Courier
font
Used when showing

resulting output from a
command that was entered
at a command line or on a
console.
Output similar to the following sample appears:

[SECURITY]
password =
$idi2$/bFMhf$5H8mYAaP1sTZ0m1Ny/dYyLaS40/
admin_password =
$idi2$146rgm$.KtDb4OH5CEBxzH6Ds2xM.ehHCH
os_password =
$1$UTKh0V$cc/UfNThFmBI7sT.zYptQ0
Bold
Trebuchet
font
Used when the user is

required to type
information or values into a
field within a windows-type
interface software.
1. If you are adding a remote to an inroute group,

right-click the Inroute Group and select Add Remote.
The Remote dialog box has a number of userselectable tabs across the top. The Information Tab is
visible when the Dialog opens.
Used when specifying

names of commands,
menus, folders, tabs,
dialogs, list boxes, and
options.
Blue
Trebuchet
font
Used to show all

hyperlinked text within a
document.
For instructions on adding an iSCPC line card to the

network tree and selecting a Hub RFT for the line card,
see Adding an iSCPC Line Card on page 108.
Bold italic
Trebuchet
font
Used to emphasize
information for the user,
such as in notes.
Note:
Red italic
Trebuchet
font
(or see
table
below)
Used when the user needs

to STRICTLY follow the
instructions or have
additional knowledge about
a procedure or action.
Several remote model types can be

configured as iSCPC remotes.
WARNING! The following procedure may cause

a network outage.
iDX Related Documents

Note:
The following list of documents applies to iDX Release 1.0 and up.
The following iDirect documents are available at http://tac.idirect.net and may also contain
information relevant to this release. Please consult these documents as needed or indicated
within this guide.
viii
iDX Release Notes
iDX Software Installation Guide
iDX iBuilder User Guide
iDX iMonitor User Guide
iDX Technical Reference Guide
iDX Software Installation Survey
iDS Related Documents
iDirect Hub Readiness Checklist
Teleport Design Considerations
Hub Installation Guide
iDX Satellite Router Installation & Commissioning Guide
iDirect Hub Installation As-Built Template
iDirect Acceptance Test Plan
iDX 1.0.1 Link Budget Analysis Guide
iDS Related Documents

Note:
The following list of documents applies to iDS and below!!
within this guide.
iDS Release Notes
iDS Network Upgrade Procedure Guide
iDS NMS iBuilder User Guide
iDS NMS iMonitor User Guide
iDirect Technical Reference Guide
iDS Upgrade Survey
iDirect Teleport Design Considerations
iDirect Hub Installation Guide
iDX Remote Installation & Commissioning Guide
Link Budget Analysis Guide
Hardware-Related Documents
Note:
This list is ONLY for the hardware-related Manuals
within this guide.
iBuilder User Guide
iMonitor User Guide
Technical Reference Guide
iDirect Software Installation Guide (iDX) OR Network Upgrade Procedure Guide (iDS)
ix
Getting Help
iDirect Teleport Design Considerations
iDirect Hub Installation Guide
Remote Installation & Commissioning Guide
Getting Help
The iDirect Technical Assistance Center (TAC) is available to help you 24x7x365. iDS Software
users guides, installation procedures, an FAQ page, and other documentation that supports
our products are available on the TAC webpage. Please access our TAC webpage at:
http://tac.idirect.net.
If you are unable to find the answers or information that you need, you can contact the TAC at
(703) 648-8151.
1 NMS Redundancy Levels
The NMS Redundancy and Failover mechanism, when set up on your network, automatically
backs up your primary NMS databases every night and restores them onto the Backup NMS
Server. In the event of a catastrophic NMS Primary Server failure, you can switch to your
Backup NMS Server with the assurance that most, if not all, of your data remains in tact.
The iDirect Technologies Network Management System (NMS) has a number of redundancy
levels that ensure robust operation, each covering a particular point of potential failure.
1.1
Process Redundancy
The standard NMS installation includes a daemon1 process called nms_monitor. This process
monitors all of the other NMS server processes on a constant basis. If any of these processes
terminate abnormally, nms_monitor automatically restarts the process.
To view the status of the NMS monitor and server processes, perform the following:
1. Logon to the NMS as root.
2. At the # prompt, enter the following command:
service idirect_nms status
The output indicates the processes as either stopped or running. The event history of the
NMS monitor can be viewed in the nms_monitor.log file. Each event is listed with a date
and time stamp. This file is located in the /home/nms/utils/ directory. Optionally,
nms_monitor can send email to a designated recipient indicating the restart condition.
The email capability requires Linux sendmail to be configured on the NMS server. iDirect
recommends setting up sendmail only if your NMS Server is behind a firewall.
1.2
RAID-1 Disk Drive Redundancy

The NMS IBM server is configured with redundant RAID-1 disk drives. In a RAID-1
configuration, all data is written simultaneously to both drives. In the event that one drive fails,
the second drive takes over and there is no loss of data. The NMS does not flag this situation,
but a red LED displays on the front panel of the server to indicate the disk problem.
1. A daemon (or service) is a background process that is designed to run autonomously, with little or not user
intervention. For example, The Apache web server http daemon (httpd) is one such example of a daemon. It waits in
the background listening on specific ports, and serves up pages or processes scripts, based on the type of request.
vii
RAID-1 Disk Drive Redundancy
Note: As part of a periodic maintenance schedule, inspect the servers (NMS and Protocol
Processor Blades) for physical damage. Verify LED status during scheduled maintenance
periods.
In the event that you experience a partial disk failure, contact the iDirect TAC for support.
You can also contact IBM technical support directly.
viii
2 Configuring the dbBackup

Script Parameters
The dbBackup script runs on the Primary NMS Server. By default, 30 minutes after midnight,
this script automatically backs up the NMS configuration and archive databases and copies
them to the Backup NMS Server. This script must be specifically configured at each Hub
installation.
2.1
Setting Up SSH1 between Primary and Backup

NMS Servers
The dbBackup script uses Secure Copy (SCP) to copy the backed-up database files from the
Primary NMS Server to the Backup NMS Server using SSH. By default, SCP prompts for a
password. The tasks in this section ensure that SCP can copy the files without prompting for a
password. This is allowed because security key-pairs are preconfigured.
To configure SCP for copying the files without prompting for a password, perform the
following:
1. In the /root directory on the Backup NMS Server, enter:
ssh-keygen -t rsa
2. Press the Enter key at all prompts to choose the defaults.

3. In the /root directory on the Primary NMS Server, enter:
ssh-keygen -t rsa
4. Press the Enter key at all prompts to choose the defaults.

5. Copy the generated file named id_rsa.pub on the primary NMS to the Backup NMS by
entering:
scp ~/.ssh/id_rsa.pub root@ipaddr:/root
Where:
ipaddr = the IP address of the Backup NMS Server.
1. Developed by SSH Communications Security Ltd., Secure Shell is a program that allows you to log into another computer over
a network to execute commands on a remote machine, and to move files from one machine to another. It provides strong
authentication and secure communications over insecure channels. It is a replacement for rlogin, rsh, rcp, and rdist. SSH protects
a network from attacks such as IP spoofing, IP source routing, and DNS spoofing.
vii
Configuring The NMS to Send Database Backups to The Backup NMS
6. Enter the following command to connect to the Backup NMS Server.

ssh root@ipaddr
Where:
7. Enter the user ID and password when prompted.

8. Enter the following command to append the contents of the id_rsa.pub file to the end of
the authorized_keys2 file:
cat id_rsa.pub >> .ssh/authorized_keys2
9. Verify that this prodecure was a success by repeating Step 6. If you are not prompted for
a password, the procedure was successful.
2.2
Configuring The NMS to Send Database Backups to

The Backup NMS
To configure the NMS to send the backup files to the Backup NMS, perform the following:
1. On the Backup NMS Server, create a directory called backup in the /root directory by
entering:
cd /root
mkdir backup
2.
On the Primary NMS Server, backup and restore scripts and .ini files are contained in
/home/nms/utils/db_maint. Using the vi editor, add the following line to the
dbBackup.ini script for each target db backup machine you want to use:
root@ipaddr:/root/backup
Where:
addr = the IP address of each backup server receiving the backup files.
The backup process creates two archive files, one for each DB: nms.##########.tgz and
nrd_archive.##########.tgz, where ### is a random number.
2.3
Defining dbBackup Script Default Behavior

By default, the dbBackup script saves two copies of your databases, and backs up both of your
configuration and archive databases. You can change both of these defaultsby performing the
following procedures.
Note:
viii
Saving more than two copies uses more disk space on the Primary NMS.
2.3.1 Changing the Number of Backup Copies to Save

To change the number of backup copies to save, perform the following:
1. Log onto the Pimary NMS Server as root.
2. Change to the directory where the dbBackup.ini file resides by entering:
cd /home/nms/utils/db_maint.
3. Using vi editor, open the dbBackup.ini file.

4. Change the value in the line KeepVersions=2 to the number of backups you require.
2.3.2 Changing Database Backups

To change which databases the dbBackup script backs up, perform the following:
1. Logon to the Primary NMS Server as root.
2. Enter the following command:
cd /home/nms/utils/db_maint.
3. Using the vi editor, open the dbBackup.ini file.

4. If you only want to back up your configuration database, make the command line
ad=nrd_archive a comment by adding the # character to the beginning of the line, shown
as follows:
#ad=nrd_archive
5. If you only want to backup your archive database, make the command line cd=nms a
comment by adding the # character to the beginning of the line, shown as follows:
#cd=nms
ix
3 Special Provisioning for a

Distributed NMS
Special consideration is required for Hubs that are using a Distributed NMS. If you are using a
Distributed NMS, you must make configuration changes on the Primary NMS Server, Auxiliary
Servers, and Backup Server (event server (evtsvr), latency server (latsvr), and the archive
server (nrdsvr)).
3.1
Configuring the Primary NMS Server

If you are not running a Distributed NMS, skip this section and go to :wq! on page ix.
To configure the Primary NMS Server, perform the following:
1. Logon to the Primary NMS Server as root.
3. Make the following command a comment by adding # at the beginning of the line, shown as
follows:
#ad=nrd_archive
4. Add the following lines to the dbBackup.ini file:

cd=nms
KeepVersions=5
root@backup_nms_ipaddr:/root/backup
Where:
backup_nms_ipaddr = the IP address of the Backup NMS Server.
5. Open the crontab file for editing by entering:

crontab e.
6. Enter the following lines to the crontab file:

30 0 * * * /home/nms/utils/db_maint/dbBackup
>> /home/nms/utils/dbbackup.output 2>&1
0 0 * * * //home/nms/utils/db_maint/cons.pl
>> /home/nms/utils/db_maint/cons.output 2>&1
vii
Configuring the Auxiliary NMS Servers
7. Press Esc to exit Edit Mode.

8. Save and close the crontab file by entering the following command:
:wq!
3.2
Configuring the Auxiliary NMS Servers

To configure the Auxiliary NMS Servers, perform the following:
1. Logon to the Auxiliary NMS Server as root.
3. Make the following command line a comment by adding the #character at the beginning of
the line, as shown:
#cd=nms
4. Add the following lines to the file:

ad=nrd_archive
KeepVersions=5
5. Optionally, copy the database to the Backup NMS Server by adding the following command
line to the dbBackup.ini file:
root@backup_nms_ipaddr:/root/backup-server_name
Where:
backup_nms_ipaddr = the IP address of the Backup NMS Server
server_name = the name of the Auxiliary Server.
Note:
If the line already exists in the file and you do not want to backup the database,
comment out this line by inserting a # at the beginning of the line, as shown:
#root@backup_nms_ipaddr:/root/backup-server_name
6. Open the crontab file for editing by entering the following command:
crontab e
7. Enter the following lines:

0 */6 * * * /home/nms/utils/db_maint/cons.pl
>> /home/nms/utils/db_maint/cons.output 2>&1

:wq!
viii
Configuring the Backup NMS Server
3.3

To configure the Backup NMS Server, perform the following:
1. Logon to the Backup NMS Server as root.
2. Using the vi editor, open the dbRestore.ini file.
3. Make the following command line a comment by adding # at the beginning of the line, as
shown:
#ad=nrd_archive
4. Add the following lines to the file:

cd=nms
KeepVersions=5
Location=/root/backup
5. Create the backup directories by entering the following commands:

mkdir root/backup
mkdir root/backup -server_name
Where:
server_name = the name of the Backup NMS Server.
crontab e
7. Enter the following lines:

0 4 * * * /home/nms/utils/db_maint/dbRestore
>> /home/nms/utils/db_maint/dbrestore.output 2>&1

:wq!
ix
4 Configuring dbRestore
Script Parameters
This section provides the procedure to configure the dbRestore script parameters.
4.1
Configuring the Backup NMS to Restore the

Databases
To configure the Backup NMS to restore the databases, perform the following:
1. On the Backup NMS Server, change to the /home/nms/utils/db_maint/ directory.
2. Using the vi editor, open the dbrestore.ini script file.
3. Make the following command line a comment a command by deleting the # character and
adding the backup location by changing the Location line to read:
Location=/root/BACKUP
Note:
The cron job1 performs the restore on the backup mahine nightly.
To modify the cron job, perform the following:

crontab -e
2. Open the vi editor and press the I key to turn on Insert Mode.
3. Add the following line:
30 1 * * * /home/nms/utils/db_maint/dbRestore>>/home/nms/utils/db_maint/dbrestore.output
2>&1
4. Press Esc to exit Insert Mode.

:wq!
1. cron is a Linux system process (daemon) that will execute a program at a preset time. To use
cron you must prepare a text file that describes the program that you want executed and the
times that cron should execute them. Then you use the crontab program to load the text file that
describes the cron jobs into cron
vii
Configuring the Backup NMS to Restore the Databases
viii
5 Changing Over to the

Backup NMS Server
If you need to perform maintenance on the Primary NMS, you can do a change over to the
Backup NMS Server during a scheduled maintenance window or during a catastrophic failure
situation.
In catastrophic failure situation, the Primary NMS Server may become inaccessible.
There are three procedures required to move a file from the Primary NMS Server to the
Backup NMS Server: Compress, Copy, and Extract.
When you are changing from the Primary NMS Server to the Backup NMS Server, there are
major processes that need to be followed, as outlined in this guide:
Section 2.1, Procedures for Moving Component Files to the Backup NMS Server
Section 2.2, Procedures for Moving pp_controller Files To the Backup NMS Server
Section 2.3, Bringing The Backup NMS Online
5.1
Procedures for Moving Component Files to the

Backup NMS Server
The required procedures consist of:
Logging Into The Primary NMS Server
Compressing the Component Options File Directories
Copying the Component Options File Directories from the Primary NMS Server to
Backup NMS Server
Extracting Component File Directories Onto Backup NMS Server

Before you can begin to move from your Primary NMS Server to the Backup NMS Server, logon
to the Primary NMS as root.
Now you are ready to begin moving files. Perform the procedures in the order shown for the
best results.
Compressing the Component Options File Directories

To compress the component options file directories that reside on the Primary NMS Server,
perform the following:
vii
Procedures for Moving Component Files to the Backup NMS Server
1. Record any static routes that are configured on the Primary NMS Server.
2. On the primary NMS, run the backup script to ensure that you have an up-to-date
database on the Backup NMS Server. If the database is not up-to-date, copy any necessary
files over as required.
3. In addition to the database files, additional component files must be copied, shown in
their appropriate directories, as follows:
/home/nms/cfg/options_files/modems
/home/nms/cfg/options_files/rmtdefs
/home/nms/cfg/options_files/chassis
/home/nms/cfg/options_files/pp_globals
/home/nms/cfg/options_files/netdefs
4. To compress each of these directories, enter the following command:

4a. Change the the appropriate directory by entering the following command:
cd /home/nms/cfg/options_files/
4b. Enter the following command to compress each directory:
tar cvf filename.tar source file name or directory
Where:
filename.tar = is the name of the compressed file that contains the component name.
(for example: tar -cvf modems.tar modems).
source file name or directory = is the original name of the file, as displayed in
Step 3. (For example, the first line shows the directory and file name of
/home/nms/cfg/options_files/modems. The source file name is modems).
5. Repeat Step 4 for each component directory, using the commands as follows:
5a. Enter tar cvf modems.tar modem
5b. Enter tar cvf rmtdefs.tar rmtdefs
5c. Enter tar cvf chassis.tar chassis
5d. Enter tar cvf pp_globals.tar pp_globals
5e. Enter tar cvf netdefs.tar netdefs
You are now ready to copy the files you compressed from the Primary NMS Serer to the Backup
NMS Server.
Copying the Component Options File Directories from the Primary

NMS Server to Backup NMS Server
1. Copy the tar file to the Backup NMS Server using the SCP command. For example,
viii
Procedures for Moving Component Files to the Backup NMS Server
scp ~/ filename.tar root@ipaddr:/home/nms/cfg/options_files/
Where:
filename.tar = the name of the compressed file that you created in Step 4.
(for example: modems.tar)
2. Repeat Step 1 for each component options file, using the commands as follows (you can
refer to Step 1 to understand the elements of this command):
2a. Enter:
scp / modems.tar root@nnn.nnn.nnn:/home/nms/cfg/options_files/
2b. Enter:
scp / rmtdefs.tar root@nnn.nnn.nnn:/home/nms/cfg/options_files/
2c. Enter:
scp / chassis.tar root@nnn.nnn.nnn:/home/nms/cfg/options_files/
2d. Enter:
scp / pp_globals.tar root@nnn.nnn.nnn:/home/nms/cfg/options_files/
2e. Enter:
scp / netdefs.tar root@nnn.nnn.nnn:/home/nms/cfg/options_files/
Now that you have compressed and copied all of the required files, you are ready to extract
these files onto the Backup NMS Server.
Extracting Component File Directories Onto Backup NMS Server

To extract the component files onto the Backup NMS Server, perform the following:
1. Login to the Backup NMS Server as root.
2. Optionally, if you want to display the contents of this directory so that you can list all of the
component files you need to extract, enter the following command to change to the
appropriate directory:
cd /home/nms/cfg/options_files/
3. Enter the following command to list the contents:
ls
An example of the output that displays is as follows:
chassis
modems
netdefs
pp_globals
rmtdefs
ix
Procedures for Moving pp_controller Files To the Backup NMS Server
4. Enter the following command:

tar xvf filename.tar
Where:
(for example: modems.tar)
5. Repeat Step 4 for each component file, using the commands as follows:
5a. Enter tar xvf modems.tar
5b. Enter tar xvf rmtdefs.tar
5c. Enter tar xvf chassis.tar
5d. Enter tar xvf pp_globals.tar
5e. Enter tar xvf netdefs.tar
Now that you have moved the component files to the Backup NMS Server successfully, you are
ready to move the pp_controller files.
5.2
Procedures for Moving pp_controller Files To the

Backup NMS Server
The required procedures for moving the pp_controller files to the Backup NMS Server consist
of:
Compressing the pp_controller Files
Copying the pp_controller Files from the Primary NMS Server to Backup NMS Server\
Extracting the pp_controller Files Onto Backup NMS Server

Before you can begin to move from your Primary NMS Server to the Backup NMS Server, logon
to the Primary NMS root.
Now you are ready to begin moving files. Perform the procedures in the order shown for the
best results.
Compressing the pp_controller Files

To compress the pp_controller files that reside on the Primary NMS Server, perform the
following:
1. Record any static routes that are configured on the Primary NMS Server.
2. On the primary NMS, run the backup script to ensure that you have an up-to-date
database on the Backup NMS Server. If the database is not up-to-date, copy any necessary
files over as required.
3. Change to the appropriate directory by entering the following command:

cd /etc/idirect/
4. Display the contents of this directory by entering the following command:

ls -la

total 28
drwxr-xr-x
6 root
root
4096 Sep
4 23:18 .
drwxr-xr-x
82 root
root
8192 Sep
5 14:17 ..
drwx------
3 root
root
4096 Jun 19 11:45 ca
drwxr-xr-x
2 root
root
4096 Sep
4 23:18 map
drwxr-xr-x
3 root
root
4096 Sep
5 20:35 pp_controller_15001
drwxr-xr-x
3 root
root
4096 May 29 21:56 pp_controller_47767
5. Compress all of the pp_controller files (as displayed in previous step) in the 15000 range
by entering the following command for each pp_controller that is listed above
tar -cvf pp_controller_1500n.tar pp_controller_1500n
Where:
n = the number of the pp_controller for which you are extracting files, and is listed when
you list the contents of the /etc/idirect directory.
Note:
Referring to the above output display as an example, the command for

compressing a pp_controller file listed is: pp_controller_15001.tar
6. Repeat Step 5 for each pp_controller on your network.

You have successfully compressed all of the pp_controller files and are now ready to copy the
files you compressed from the Primary NMS Serer to the Backup NMS Server.
Copying the pp_controller Files from the Primary NMS Server to

Backup NMS Server
1. Copy the compressed pp_controller tar file to the Backup NMS Server by entering the
following command:
scp ~/ filename.tar root@ipaddr:/etc/idirect
Where:
filename.tar = the name of the compressed file that you created in the previous
procedure, Step .
(for example: pp_controller_15002.tar)
xi
Now that you have compressed and copied all of the required files, you are ready to extract
these files onto the Backup NMS Server.
Extracting the pp_controller Files Onto Backup NMS Server

The following procedure assumes you are still logged into the Backup NMS Server from
performing the previous procedure. If not, do so now.
Extract the pp_controller files onto the Backup NMS Server by performing the following:
1. Change to the appropriate directory into which the pp_controller files being extracted by
entering the following command:
cd /etc/idirect/
2. Display the contents of this directory by entering the following command:
ls -la

total 104
drwxr-xr-x
6 root
root
4096 Sep 12 15:31 .
drwxr-xr-x
82 root
root
8192 Sep
drwx------
3 root
root
4096 Jun 19 11:45 ca
drwxr-xr-x
2 root
root
4096 Sep
5 14:17 ..
4 23:18 map
-rwxr--r-1 root
pp_controller_15001.tar
root
71680 Sep 12 15:27
drwxr-xr-x
3 root
root
4096 May 29 21:56 pp_controller_47767
drwxr-xr-x
3 root
root
4096 Sep
5 20:35 pp_controller_test
3. Extract the pp_controller files by entering the following command:

tar -xvf filename_1500n.tar
Where:
filename= pp_controller
n= the number of the pp_controller for which you are extracting files, and is listed when
you execute the ps -ef | grep pp_ as shown in Step 1 results.
pp_controller_15001/
pp_controller_15001/networks/
pp_controller_15001/networks/1.net/
xii
Bringing The Backup NMS Online
pp_controller_15001/networks/1.net/netdef.opt
pp_controller_15001/networks/1.net/remote_5035562.opt
pp_controller_15001/controller.conf
pp_controller_15001/global.opt
pp_controller_15001/x509_local_key.txt
pp_controller_15001/x509_local_cert.txt
4. Repeat Step 3 for each port number.

You have successfully moved all of the required files to your Backup NMS Server.
5.3

Now that you have moved all of the files to the Backup NMS Server, you need to bring this
server into your network as your Primary NMS Server. This requires several tasks, as indicated
in the following procedure.
Perform the following steps:
1. If the primary NMS is still accessible, reconfigure the primary NMS eth0 interface with
another unused IP address or preferably, physically unplug the eth0 interface. If the
interface has been reconfigured, the NMS server must be rebooted. You can also restart
the network services by entering the follow command:
service network restart
2. If the primary NMS remains connected to the network, the iDirect NMS services MUST be
shutdown with the command below and the IP address must be different. Additionally,
these services MUST remain down while the other new (backup) NMS server is online.
service idirect_nms stop
3. Verify that the routing configuration on the Backup NMS Server is correct. If RIP v2 was
running on the Primary NMS Server, it has to be running on the backup. Any static routes
that were configured on the Primary NMS Server need to be configured on the Backup NMS
Server. Make sure they are configured as persistent routes so they remain in effect after a
reboot.
xiii
4. Configure Backup NMS Server eth0 interface with the primary NMS original eth0 interface
address. The reason for this is to match the original primary NMS IP address that is
configured in the configuration (options) files of the iDirect network elements.
5. Reboot the Backup NMS Server machine or restart the network services entering the
following command:
service network restart
6. Verify the new Primary NMS Server routing table by entering the following command:
netstat r -n
7. If network has secure mobile and or encrypted remotes, make sure

/home/nms/cfg/nms_cfg.opt file has the correct parameters (applies to iDS version
6.0.x).
8. Start the NMS services by entering the following command:
service idirect_nms start
9. The network should now be running on the Backup NMS Server. The status on the Line
Cards and the Remotes should be normal in iMonitor.
10. If needed, apply the Protocol Processor level and network level configuration to all
networks via iBuilder.
Changing The Backup NMS Server To The Primary NMS Server

If the Backup NMS Server will remain as the new Primary NMS Server, the crontab file and the
config table must be modified in order for the new NMS to act as the Primary NMS Server. The
modification of the crontab is to allow the new Primary NMS Server to execute the daily
backup and consolidation scripts.
To change the Backup NMS Server to be your Primary NMS Server, perform the following:
1. Enter the command to open the crontab file for editing:
crontab e
2. Make the command lines in the crontab file comments by adding a # in front of the each
command line.
3. Adding the following lines:
30 0 * * * /home/nms/utils/db_maint/dbBackup >>
/home/nms/utils/db_maint/dbbackup.output 2>&1
30 1 * * * /home/nms/utils/db_maint/cons.pl >>
/home/nms/utils/db_maint/cons.output 2>&1
xiv
wq!
6. To configure the new (Backup) NMS Server to start the NMS services every time at bootup, enter the following command:
chkconfig --level 2345 idirect_nms on
The backup is a success if there is no data output after entering this command.
xv

TN NMS Redundancy and Failover RevF 10012008

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

TN NMS Redundancy and Failover RevF 10012008

Caricato da

Copyright:

Formati disponibili

NMS Redundancy and Failover

iDX Release 2.0 and Earlier

Document Name: TN_NMS_Redundancy_and_Failover_Rev F_01012008.pdf

NMS Redundancy and Failover

Reason for Change(s)

Updated for new release.

August 11, 2006

Updated for new release.

September 25, 2006

Updated for new release.

November 16, 2007

Updated for new release.

April 21, 2008

Updated based on TAC

Updated based on TAC

NMS Redundancy and Failover

NMS Redundancy and Failover

1 NMS Redundancy Levels . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 RAID-1 Disk Drive Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2 Configuring the dbBackup Script Parameters . . . . . . . . . . 3

3 Special Provisioning for a Distributed NMS . . . . . . . . . . . . 7

NMS Redundancy and Failover

4 Configuring dbRestore Script Parameters . . . . . . . . . . . . .11

5 Changing Over to the Backup NMS Server . . . . . . . . . . . . .13

Extracting the pp_controller Files Onto Backup NMS Server . . . . . . . . . . . . . . . . . 17

5.3 Bringing The Backup NMS Online . . . . . . . . . . . . . . . . . . . . . . . . . 19

NMS Redundancy and Failover

About This Guide

Contents Of This Guide

Used when the user is

NMS Redundancy and Failover

iDX Related Documents

Used when showing

Output similar to the following sample appears:

Used when the user is

1. If you are adding a remote to an inroute group,

Used when specifying

Used to show all

For instructions on adding an iSCPC line card to the

Used when the user needs

Several remote model types can be

WARNING! The following procedure may cause

iDX Related Documents

iDX Release Notes

iDX Software Installation Guide

iDX iBuilder User Guide

iDX iMonitor User Guide

iDX Technical Reference Guide

iDX Software Installation Survey

NMS Redundancy and Failover

iDS Related Documents

iDirect Hub Readiness Checklist

Teleport Design Considerations

Hub Installation Guide

iDX Satellite Router Installation & Commissioning Guide

iDirect Hub Installation As-Built Template

iDirect Acceptance Test Plan

iDX 1.0.1 Link Budget Analysis Guide

iDS Related Documents

The following list of documents applies to iDS and below!!

iDS Release Notes

iDS Network Upgrade Procedure Guide

iDS NMS iBuilder User Guide

iDS NMS iMonitor User Guide