Symmetry Nms Database

SYMMETRY NMS Database
Maintenance Guide (v2.40)

033-100692-240001, Issue 1A
The information in this manual is subject to change without notice. All statements, information and recommendations in
this manual are believed to be accurate, but are presented without warranty of any kind, expressed or implied. Users
must take full responsibility for their use of any products.
No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical,
including photocopying and recording, or by any information storage or retrieval system, without prior written consent
from SR Telecom Inc.
SR TELECOM, AIRSTAR, ANGEL, INSIGHT NMS, METROFLEX, METROPOL, SR500, SR500IP, SWING, SYMMETRY
and WL500 are trademarks of SR Telecom Inc. All rights reserved 2005. All other trademarks are property of their
owners. Information subject to change without notice.
2005, SR Telecom Inc.
All rights reserved. 5/30/05
Printed in Canada
Head Office
SR Telecom Inc.
8150 Trans-Canada Hwy.
Montreal, Quebec
Canada H4S 1M5
Tel.: +1 514 335 1210
Fax: +1 514 334 7783
1 888 SRTELECOM (778 3532)
(U.S. and Canada)
www.srtelecom.com
Table of Contents
Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Introduction .............................................................................................................................................. 5
1.1
Introduction ......................................................................................................................... 6
1.2
Database Data Files............................................................................................................ 6
1.3
Database Redo Logs .......................................................................................................... 7
1.4
Database Archive Logs ....................................................................................................... 7
1.5
Database Control Files........................................................................................................ 7
1.6
Database Cold Backup ....................................................................................................... 8
1.7
Database Hot Backup ......................................................................................................... 8
NMS Database Monitor ........................................................................................................................... 9

2.1
Installing the NMS Database Monitor................................................................................ 10
2.2
Scheduling NMS Database Monitor .................................................................................. 11
2.3
2.3.1
2.3.2
2.3.3
2.3.4
2.3.5
Reviewing NMS Database Monitor Reports...................................................................... 12

Error Alerts ........................................................................................................................ 12
Tablespaces with < 15% and < 1Gb Free Space.............................................................. 12
Segments Cant Allocate NEXT Extent ............................................................................. 13
Tablespaces with Over 20 Free Fragments ...................................................................... 13
Segments Nearing Max Extents........................................................................................ 14
2.4
2.4.1
2.4.2
2.4.3
2.4.4
Preventive Actions ............................................................................................................ 15

Resizing a Data File .......................................................................................................... 15
Coalescing a Tablespace .................................................................................................. 17
Increasing a Segment Maximum Number of Extents ........................................................ 18
Rebuilding a Large Raphael Tablespace .......................................................................... 18
NMS Database Cold Backup ................................................................................................................. 21

3.1
Installing the NMS Database Cold Backup Tool ............................................................... 22
3.2
Performing an NMS Database Cold Backup ..................................................................... 23
NMS Database Hot Backup................................................................................................................... 25

4.1
Installing the NMS Database Hot Backup Tool ................................................................. 26
4.2
Performing an NMS Database Hot Backup ...................................................................... 28
NMS Database Restore......................................................................................................................... 29

5.1
Restoring the NMS Database from a Cold Backup ........................................................... 30
SYMMETRY NMS Database Maintenance Guide
Chapter 6
5.2
Restoring the NMS Database from a Hot Backup............................................................. 32
5.3
Reducing the NMS Database Recovery Time................................................................... 36
5.4
Restoring the NMS Database Completely......................................................................... 37
NMS Standby Database ........................................................................................................................ 39

6.1
Standby NMS Configuration.............................................................................................. 40
6.2
Oracle Standby Database ................................................................................................. 41
6.3
Setting Up an NMS Standby Database ............................................................................. 43
6.4
6.4.1
6.4.2
6.4.3
6.4.4
Monitoring and Managing the NMS Standby Database .................................................... 46

Using the NMS Database Monitor Tool............................................................................. 46
Determining the Most Recent Archived Log...................................................................... 47
Reviewing the Archive Trace Files .................................................................................... 47
Resolving a Gap Sequence .............................................................................................. 48
6.5
Opening the Standby Database as Read-only .................................................................. 51
6.6
6.6.1
6.6.2
Activating the NMS Standby Database ............................................................................. 53

Activating the Standby NMS Database ............................................................................. 53
Recreating the Primary After Standby Activation .............................................................. 55
Chapter 1
Introduction
This chapter introduces some basic database concepts that you need to understand in
order to effectively use the tools. Chapter 2 describes a monitor tool that checks the
health and state of the NMS database and sends out reports using e-mail. Chapter 3
describes how to install and perform NMS database cold backup, and Chapter 4
describes how to install and perform NMS database hot backup. Chapter 5 describes
how to restore an NMS database from backups and suggests the preparation steps
that you should follow to reduce the database recovery time as well as restore
database 100%. Chapter 6 describes how to prepare, manage, and maintain an
Oracle standby database using SYMMETRY NMS.
1.1
Introduction
One of the most important tasks for SYMMETRY NMS system administrators is to protect the NMS
database from failures. Particularly, the database must be backed up frequently so that the database
can be restored in the event of media failure. SYMMETRY NMS provides tools to help system
administrators maintain database operations. This document describes these tools and how to use
them.
The SYMMETRY NMS Database Maintenance Guide also contains some information about common
database maintenance tasks, including how to purge old Raphael data records, how to purge old NMS
alarm history records and event records. It also explains how to schedule these tasks to run
automatically. It is important for a system administrator to perform these tasks to ensure proper NMS
operations.
d
1.2
All database system files described in this chapter must not be modified by anyone. Doing so
will cause the NMS database to be corrupted.
Database Data Files

The database writes its modified buffer cache to files on disk. The files containing user data are called
data files. For the NMS database, these files are located in /data/oradata and are updated by NMS
applications. If you have installed the entire NMS including Gabriel, Raphael, and Michael Order
Manager (MOM), you will see the following subdirectories, corresponding to each of the systems: GMS,
RAPHAEL, and MICHAEL.
Introduction
1.3
Database Redo Logs

All changes made to data are also recorded as redo entries in the redo log buffer. A change is not
committed until its redo entries in the buffer are written to a redo log file on the disk. Redo log files (or
simply redo logs) are sometimes called online redo logs (or simply online logs) since the redo entries in
these files are not archived.
For the NMS database, the redo log files for Gabriel, Raphael, and MOM are located in
/data/oradata/GMS.
1.4
Database Archive Logs

An optional background process called Archiver only starts when the database is run in ARCHIVELOG
mode with automatic archiving enabled. A redo log file is archived when the log file is full and the
database needs to switch to another redo log file. When this occurs, the Archiver copies the filled redo
log file to a specified location. These copied log files are called archive logs and they can be used to
recover data when the media fails.
By default, the database runs in NON_ARCHIVELOG mode. In this mode, a redo log file is not archived
before it is overwritten. In addition, the database can only be backed up after you have shutdown the
database. The archive logs generated by the database running in ARCHIVELOG mode make database
online backup possible.
For the NMS database, the archive log files are located in /data/oradata/GMS/archive directory.
1.5
Database Control Files

The control files help the database synchronize the write operations between the redo log files and the
data files. At the synchronization point (check point), all modified data buffers are written to disk. After
the check point process is completed, the file headers are updated and marked as current, and a check
point sequence number is recorded in the control file. During the database startup, the control file
confirms that all the files are at the same check point number.
Control files can not be directly backed up when the database is online. The database is constantly
being updated. A simple UNIX copy of the file may not give you a valid copy.
For the NMS database, the control files are located in /data/oradata/GMS.
1.6
Database Cold Backup

Database cold backup is also called closed database backup. You need to shutdown the database
before performing the cold backup. Although cold backup makes the database unavailable during the
backup, cold backup should still be performed from time to time during maintenance windows. A cold
backup can backup the entire database, including data files, redo logs, control files, archive logs, and
parameter files. Since the database is not available for use during the backup, all the database files are
consistent to a specific point in time. This makes a cold backup extremely reliable and, as a result, you
can always restore your database from a good cold backup.
1.7
Database Hot Backup

Although the cold database backup is reliable and can backup the entire database, the NMS database
can not be shut down often to perform backups. If the database runs in the ARCHIVELOG mode with
automatic archiving enabled, then you can perform a database hot backup while it is online. Therefore,
hot backups should be performed often and cold backups can be performed less frequently.
Hot backup also cleans out the archive log files so that the archive log directory does not become full.
The hot backup will copy the archive log files to the backup location and delete them from the original
location.
Chapter 2
NMS Database Monitor

SYMMETRY NMS provides a script that checks the health and state of the NMS
database and generates a report. The report is then e-mailed to system
administrators. The script can be scheduled as a system task which will enable system
administrators to monitor the NMS database activities so that potential issues can be
identified and addressed before they become problems. This chapter describes how to
install, run, and schedule the SYMMETRY NMS database monitor tool.
2.1
Installing the NMS Database Monitor

To install the NMS database monitor
1.
Login NMS server as user oracle and copy the supplied monitor.tar file to /tmp.
2.
Extract the tar file with the UNIX tar command.

%cd /tmp
%tar xvf monitor.tar
This will produce two files, installmonitor.ksh and monitor_app.tar.

3.
Run the install script to install and configure the NMS database monitor.
%/tmp/installmonitor.ksh
When prompted, please be sure to enter correct e-mail address(es). NMS database monitor will
send reports to the specified e-mail address(es). Multiple e-mail addresses must be separated by a
space.
4.
To verify the installation, run the DB checkup script manually.

%/export/apps/oracle/admin/db_monitor/scripts/db_CHECKUP.sh
If the installation is successful, you should receive an e-mailed database report.
10
2.2
Scheduling NMS Database Monitor

Schedule the database monitor so that it runs automatically and periodically, for example, once every
day. This way, you can be proactive and address the issues before they become problems.
To schedule the NMS database monitor as a periodic UNIX task
1.
Login NMS server as user oracle.
2.
Open the user oracle cron table with these commands:

%export EDITOR=vi
%crontab -e
This opens a UNIX vi editor session.

3.
Using vi, add this line on its own line (case-sensitive).

0 1 * * * /export/apps/oracle/admin/db_monitor/scripts/db_CHECKUP.sh 2>&1
4.
Exit the vi session to complete the crontab -e command.

This entry will run the NMS database monitor at 0 minute, 1 hour (1 AM), every day. Please consult
the Software Installation Setup Guide provided with your SYMMETRY NMS system for details on
how to use default UNIX editor vi to add and save the line above.
To verify the entry

Enter the following command as user oracle:
%crontab -l
If you have performed the above steps correctly, you should automatically receive NMS database
reports via e-mail, as scheduled.
11
2.3
Reviewing NMS Database Monitor Reports

The following sections describe some important items that you will find in NMS database monitor
reports.
2.3.1
Error Alerts
The error alert section lists all Oracle errors that have been generated since yesterday. For example, the
following error alerts indicate that the Oracle database was not able to insert new alarm history records
because it was unable to extend the tablespace. Therefore, old alarm history records need to be purged
to make room for new ones, or the tablespace needs to made larger.
<< Alert Errors Since Yesterday >>
ORA-1653: unable to extend table GMS_S1.ALARMHISTORY by 1024 in tablespace ALMA_DATA
ORA-1653: unable to extend table GMS_S1.ALARMHISTORY by 1024 in tablespace ALMA_DATA
All errors in the error alerts section should be investigated and addressed immediately.
2.3.2
Tablespaces with < 15% and < 1Gb Free Space

All tablespaces with less than 15% and less than 1 GB space will be listed in this section of the report.
For example:
<< Tablespaces with < 15% and < 1Gb Free Space >>
GMS_DATA (3645440 bytes or 2.31% free)
This is a warning that these tablespaces may need to be given more space soon so that they will not run
over the currently allocated space limits.
For information about growing tablespaces, refer to Section 2.4.1 Resizing a Data File on page 15.
12
2.3.3
Segments Cant Allocate NEXT Extent

The first field is the tablespaces name, and the second field is the table name. For example:
<< Segments Can't Allocate NEXT Extent >>
GMS_DATA: GMS_S1.HSDRESOURCE (TABLE, next_extent 8388608)
GMS_DATA : GMS_S1.LOG_RECORD (TABLE, next_extent 8388608)
This means that if the Oracle database needs to extend the spaces used by these tables, it will not be
able to because there is not going to be enough contiguous free space for the table to grow.
For information about growing tablespaces, refer to Section 2.4.1 Resizing a Data File on page 15.
f
2.3.4
You can ignore the warnings related to tablespace raphael_dimension_data, for example,
RAPHAEL_DIMENSION_DATA : RAPHAEL.DAY (TABLE, next_extent 8388608). This
tablespace is no longer used.
Tablespaces with Over 20 Free Fragments

The first field is the tablespace name, and the second field is the number of free fragments. For
example:
<< Tablespaces with Over 20 Free Fragments >>
RAPHAEL_CDR_DATA_P1 has 89 Free Fragments
The available free space for a tablespace can become fragmented. When a segment is dropped, its
extents are deallocated and marked as free, but they are not combined to be a larger, contiguous
extent. Unless these free extents are coalesced, they may not be used within the tablespace during the
next space request.
For information about combining neighbouring free extents, refer to Section 2.4.2 Coalescing a
Tablespace on page 17.
13
2.3.5
Segments Nearing Max Extents

<< Segments Nearing Max Extents >>
RAPHAEL.BASE_CALL_UTIL_MEASUREMENT (TABLE, 90% extents used)
Each segment has a maximum allowable number of extents. Any transaction that causes the table to
attempt to exceed its maximum extent number will fail. This section of the monitor report warns you that
you need to increase the maximum extent number for a table.
The example report above warns that the table BASE_CALL_UTIL_MEASUREMENT in schema
RAPHAEL has reached 90% of its maximum number of extents.
For information about changing a maximum extent number for a segment, refer to Section 2.4.3
Increasing a Segment Maximum Number of Extents on page 18.
14
2.4
Preventive Actions
The following sections describe the common maintenance tasks that you may need to perform when
addressing the issues reported in the NMS database monitor report.
2.4.1
Resizing a Data File

Assume that the NMS database monitor report has entries as described in Section 2.3.2 on page 12 or
in Section 2.3.3 on page 13.
To increase the size of the correct data file
1.
Login as user oracle.
2.
Invoke Oracle server manager tool by typing svrmgrl to get SVRMGR prompt.
3.
At SVRMGR prompt, type in the following commands. Press the return key or ENTER after each
command.
SVRMGR>connect internal
SVRMGR>select file_name from dba_data_files where tablespace_name='GMS_DATA';
Note the semicolon at the end of the select command above. The select command gives you a file
name. This is the name of the data file for the tablespace. In this example, for tablespace
GMS_DATA, the data file would be /data/oradata/GMS/gms_data01.dbf.
4.
On a different terminal window, issue the following UNIX command:

ls -l /data/oradata/GMS
? Take note of the file size for /data/oradata/GMS/gms_data01.dbf file. For example, let us assume
that it is 150 MB.
15

5.
Return to the terminal window with the SVRMGR prompt and type in the following command at the
SVRMGR prompt:
SVRMGR>alter database datafile '/data/oradata/GMS/gms_data01.dbf' resize 210M;
Note the single quotes between the file name. The above command will take a short while and it will
increase the file size from 150 MB to 210 MB.
6.
Type exit to close the svrmgrl tool.

SVRMGR>exit
7.
Verify the file size change for /data/oradata/GMS/gms_data01.dbf by entering the following
command:
ls -l /data/oradata/GMS
16
For Raphael tablespaces such as RAPHAEL_CDR_DATA_P1, you may have to keep

increasing the data file sizes as your network deployment grows when CPEs and base
stations are added. This is true even you are purging the database to only keep the CDR
records for the last 30 days, as described in the Software Installation Setup Guide provided
with your SYMMETRY NMS system. If a data file size is close to 1 GB, you should rebuild
the tablespace; refer to Section 2.4.4 Rebuilding a Large Raphael Tablespace on page 18.
2.4.2
Coalescing a Tablespace
Assume that the NMS database monitor report has entries as described in Section 2.3.4 on page 13.
Coalesce a tablespace to try to reduce space fragments.
To coalesce the tablespace RAPHAEL_CDR_DATA_P1
1.
2.
3.
command.
SVRMGR>set heading off
SVRMGR>set pagesize 0
SVRMGR> select 'alter table ' || owner || '.' || segment_name || ' deallocate
unused;' from dba_segments where tablespace_name = 'RAPHAEL_CDR_DATA_P1;
The above select command provides one or more alter table statements for each table in the
tablespace RAPHAEL_CDR_DATA_P1:
alter table RAPHAEL.VOICE_CDR_SEGMENT deallocate unused;
alter table RAPHAEL.RU_CDR_SEGMENT deallocate unused;
alter table RAPHAEL.VOICE_V52_MEASUREMENT deallocate unused;
4.
Cut and paste the above alter table commands at the prompt, one by one, to execute them.
5.
Coalesce the tablespace by entering the following commands:

SVRMGR>alter tablespace RAPHAEL_CDR_DATA_P1 coalesce;
SVRMGR>exit
17
2.4.3
Increasing a Segment Maximum Number of Extents

Assume that you have entries in the NMS database monitor report as described in Section 2.3.5 on
page 14.
To set the maximum extent number to be unlimited for table BASE_CALL_UTIL_MEASUREMENT
1.
2.
3.
command.
SVRMGR>alter table raphael.base_call_util_measurement storage (maxextents
unlimited);
SVRMGR>exit
2.4.4
Rebuilding a Large Raphael Tablespace

Raphael uses SQL*Loader direct path load to insert new records. Direct path load bypasses normal
database processing by adding new records after the high-water-mark. This dramatically increases the
insertion speed, and it is necessary for Raphael. However, the free space below the high-water-mark is
not searched and reused. This is why you may have to keep increasing the size of the datafile, even
though the records older than 30 days are being deleted everyday by the purge script.
If a tablespace is becoming too large (close to 1 GB), you should rebuild the tablespace by truncating
the tables. Truncating a table removes all records of the table and resets the high-water-mark to the
beginning. Of course, the existing records need to be exported first before truncating and imported back
after truncating. This section describes the steps for performing this task.
The steps below use Raphael tablespace RAPHAEL_CDR_DATA_P1 as an example. For a

different tablespace, you will need different export and import parameter files, and the truncate
and alter table commands below will also be different. Contact your SR Telecom support
personnel first.
Before you rebuild a tablespace, you must stop the Raphael loading processes.
18

To stop the Raphael loading processes
1.
Login as user gabriel and invoke stopraphael to stop the Raphael loading processes.
2.
Confirm that the Raphael parser (dta) and loader processes (rdm) are stopped using the following
UNIX command:
ps -ef | grep dta and ps -ef | grep rdm
To rebuild a large Raphael tablespace

1.
Login as oracle and create a directory exp_imp.
2.
Place the export parameter file (exp_rap_cdr_data_p1) and import parameter file
(imp_rap_cdr_data_p1) in the new directory exp_imp.
3.
Export the three tables within the RAPHAEL_CDR_DATA_P1 tablespace.

%cd /export/home/oracle/exp_imp
%exp sys/live parfile=exp_rap_cdr_data_p1
After the export is completed, there should be a export dump file called cdr_data_p1.dmp and a log
file called exp_cdr_data_p1.log.
4.
Review the log file to make sure that the export has been finished before you continue. Do not
attempt to view the dump file.
5.
Truncate the tables and coalesce the tablespace. Truncating a table deletes all the records and also
reset its storage parameter to the initial values when the table was created. This will cause the new
records to be inserted from the beginning of the data file.
%svrmgrl
SVRMGR>truncate table RAPHAEL.VOICE_CDR_SEGMENT;
SVRMGR>truncate table RAPHAEL.RU_CDR_SEGMENT;
SVRMGR>truncate table RAPHAEL.VOICE_V52_MEASUREMENT;
SVRMGR>alter tablespace RAPHAEL_CDR_DATA_P1 coalesce;
SVRMGR>exit
6.
Import back.
%imp sys/live parfile=imp_rap_cdr_data_p1
19

Exporting tables does not take long. Importing takes some time. For example, importing approximately
600,000 records for each of the three tables in the raphael_cdr_data_p1 tablespace took roughly 45
minutes in our testing.
The following are the contents of the export and import parameter files used in the steps above:
File exp_rap_cdr_data_p1:
DIRECT=Y
ROWS=Y
CONSISTENT=Y
COMPRESS=N
STATISTICS=NONE
RECORDLENGTH=57344
#THE FOLLOWING PARAMETERS ARE TABLE OR TABLESPACE SPECIFIC
FILE=cdr_data_p1.dmp
LOG=exp_cdr_data_p1.log
TABLES=(RAPHAEL.VOICE_CDR_SEGMENT, RAPHAEL.RU_CDR_SEGMENT,
RAPHAEL.VOICE_V52_MEASUREMENT)
File imp_rap_cdr_data_p1:
FROMUSER=RAPHAEL
TOUSER=RAPHAEL
IGNORE=Y
#THE FOLLOWING PARAMETERS ARE TABLE OR TABLCESPACE SPECIFIC
FILE=cdr_data_p1.dmp
LOG=imp_cdr_data_p1.log
20
Chapter 3
NMS Database Cold Backup

Cold backup should be a part of an overall NMS database backup strategy. This
chapter describes how to install and perform an SYMMETRY NMS database cold
backup.
21
3.1
Installing the NMS Database Cold Backup Tool

You must have already installed the NMS database monitor tool. If not already installed, follow the steps
in Chapter 2 to do so before installing cold backup tool.
To install the SYMMETRY NMS database cold backup tool
1.
Login NMS server as user oracle and copy the supplied backup_cold.tar file to /tmp.
2.
Extract the tar file using the following UNIX tar command:
%cd /tmp
%tar xvf backup_cold.tar
This will produce two files, installcoldbackup.ksh and backup_cold_app.tar.

3.
Run the install script to install and configure the NMS database cold backup tool.
%/tmp/installcoldbackup.ksh
This installs the cold backup tool.
22
NMS Database Cold Backup
3.2
Performing an NMS Database Cold Backup

To perform a cold backup of the NMS database
1.
Login NMS server as user gabriel and shutdown all NMS applications.
%stopgabriel
%stopraphael
%stopmichael
2.
Ensure that all NMS GUI applications are closed, particularly any NMS Admin Viewer and Raphael
PM Viewer since they access database directly. Do not shutdown the Oracle database.
f
3.
You must leave the database up and running as the cold backup scripts will need to access
the database before it starts the backup. The scripts will shutdown the database during the
cold backup and will restart the database after the cold backup is completed.
Login as user oracle and perform the cold backup.

%su - oracle
%cd /export/apps/oracle/admin/backups
% ./coldbackup.sh
The time it takes to complete the cold backup depends on the size of the NMS database. You can
monitor the backup progress by tailing the backup log file.
23

To monitor backup progress
1.
Open another terminal window.
2.
Type in the following UNIX commands:

%cd /export/apps/oracle/admin/GMS/bkup/cold
% tail -f coldbp_GMS_11Feb04.log
The actual log file name for the tail command will be different. The above example assumes that
you are performing the cold backup on Feb 11, 2004. When the cold backup completes, the log file
will print lines to inform you. You should receive an e-mail at the e-mail address(es) that you have
specified when you installed the NMS database monitor tool.
The backup files are located in /export/apps/oracle/admin/GMS/bkup/cold. This directory also contains
the restore scripts, that were generated during the backup, that will be required to restore the database
from these backup files. This is the directory that you should tar up and save a copy on another disk or
on tape every time that you complete a cold backup.
24
Chapter 4
NMS Database Hot Backup

A hot backup can be performed while the database is online. Therefore, hot backups
should be performed in addition to cold backups to allow the database to be backed up
more frequently. This chapter describes how to install and perform an ANGLE NMS
database hot backup.
25
4.1
Installing the NMS Database Hot Backup Tool

You must have already installed the NMS database monitor tool. If not, follow the steps in Chapter 2 to
do so before you continue with the steps below.
You should perform a database cold backup before attempting the steps listed here. This
ensures that you can always restore the database to the current state using the cold backup, if
necessary. For information about performing a database cold backup, refer to the steps in
Chapter 3.
To install the SYMMETRY NMS database hot backup tool

1.
Login NMS server as user gabriel and shutdown all NMS applications.
%stopgabriel
%stopraphael
%stopmichael
2.
Ensure that all NMS GUI applications are closed, particularly any NMS Admin Viewer and Raphael
PM Viewer, since they access the NMS database directly. Do not shutdown the Oracle database.
You must leave the database up and running. The hot backup installation scripts need to
access to the database. The database will be restarted automatically when the installation
completes.
3.
Login NMS server as user oracle and copy the supplied backup.tar file to /tmp.
4.
Extract the tar file with the UNIX tar command.

%cd /tmp
%tar xvf backup.tar
This will produce two files, installbackup.ksh and backup_app.tar.
26
NMS Database Hot Backup

5.
Run the install script to install and configure the NMS database hot backup tool.
%/tmp/installbackup.ksh
This script will shutdown the NMS database and restart it in archive log mode to allow for the
backing up of the database while it is online. Please be patient while waiting for the script to
complete, as a proper database shutdown may take a little while.
6.
Verify that the database is in archive log mode.

%svrmgrl
SVRMGR>archive log list
The output of the last SVRMGR command should show that: the database is in archive mode, the
automatic archival is enabled, the archive destination is set to
/export/apps/oracle/admin/GMS/multiplex/archives/.
7.
Type exit to quit svrmgrl.

SVRMGR>exit
%
You have just installed the NMS database hot backup tool. You can now restart NMS applications. Login
as user gabriel and follow the steps described in the Software Installation Setup Guide to start Gabriel,
Michael Order Manager, and Raphael.
To confirm the successful installation, you should perform a NMS database hot backup right away.
27
4.2
Performing an NMS Database Hot Backup

To perform a hot backup of the NMS database
1.
Login the NMS server as user oracle.
2.
Execute the hot backup script.

%cd /export/apps/oracle/admin/backups
% ./hotbackup.sh
The time it takes to complete the hot backup will depend on the size of the NMS database. You can
monitor the backup progress by tailing the backup log file.
To monitor backup progress
1.
Open another terminal window.
2.
Type in the following UNIX commands:

%cd /export/apps/oracle/admin/GMS/bkup/hot
% tail -f hotbp_GMS_11Feb04.log
The actual log file name for the above tail command will be different. The above example assumes
that you are performing the hot backup on Feb 11, 2004. When the hot backup completes, the log
file will print lines to inform you. You should also get an e-mail at the e-mail address(es) that you
have specified when you installed the NMS database monitor tool.
The backup files are located in /export/apps/oracle/admin/GMS/bkup/hot. This directory also
contains the restore scripts, that were generated during the backup, that will be required to restore
the database from these backup files. This is the directory that you should tar up and save a copy
on another disk or on tape every time you that complete a hot backup. This step can be automated
by using UNIX cron to schedule daily hot backups, followed by simple UNIX tar and move
commands.
28
Chapter 5
NMS Database Restore

This chapter describes how to restore the entire NMS database from backups. It also
makes suggestions on what you should do to reduce database restore time so that the
NMS database downtime can be minimized.
29
5.1
Restoring the NMS Database from a Cold Backup

To restore the entire NMS database from a cold backup
1.
If necessary, reinstall the entire SYMMETRY NMS. This is required if the entire NMS is lost due to a
disk crash. You may also need to install and configure Solaris if the damaged disk also hosted the
OS. Refer to the Software Installation Setup Guide provided with your SYMMETRY NMS system to
install the OS and SYMMETRY NMS. Installing the OS alone may take several hours. To reduce
NMS downtime, you should have a second Sun server available, with the OS and the NMS installed
and ready to use.
2.
If necessary, reinstall the NMS database monitor and backup tools. Follow the installation
instructions in Chapter 2 and Chapter 3, respectively. Note that you may need to start up the
database first before installing the NMS database monitor and backup tools. This step can be
avoided by having another server available with the NMS database monitor and backup tools
already installed.
3.
Ensure that all NMS applications have been stopped. Login as user gabriel and follow the steps in
the Software Installation Setup Guide to shutdown the Gabriel, Raphael, and MOM applications.
4.
Login in as user oracle and make sure that the database and its listener have all been stopped.
Follow the steps in the Software Installation Setup Guide to check this and issue the proper stop
commands. If you are not able to shutdown the database using the SVRMGR shutdown immediate
command described in the Software Installation Setup Guide, you will have to use the shutdown
abort command.
5.
Get the last cold backup files and place them in the following directory:
/export/apps/oracle/admin/GMS/bkup/cold. In addition to other files, there are two restore scripts in
this directory and their names depend upon the date when the cold backup was taken. Make sure
that you are still logged in as user oracle and perform the following two commands to change them
to be executable:
%cd /export/apps/oracle/admin/GMS/bkup/cold
%chmod u+x restore_cold_backup_11Feb04.sh
%chmod u+x restore_cold_backup_GMS_11Feb04.sh
The above commands assume that the cold backup was taken on Feb 11, 04.
30

6.
Run the restore scripts in the following order:

%./restore_cold_backup_11Feb04.sh
%./restore_cold_backup_GMS_11Feb04.sh
The first script restores the database configuration parameter files and takes very little time. The
second script restores the database data files; the time it takes depends on the size of the
database. As the second script executes, it reports the progress status and you should start to see
the data files appearing in the directories under /data/oradata.
7.
After the restore scripts complete the execution, verify that the Oracle files tnsnames.ora and
listener.ora have the correct host name. This is particularly important if you are using a cold backup
file taken on a different host.
% cd $TNS_ADMIN
You will see the two files in this directory. Make proper changes if the values for variable HOST in
these two files do not match the correct name of the host.
8.
You can proceed to start up the NMS database and applications. Please follow the instructions in
the Software Installation Setup Guide provided with your SYMMETRY NMS system.
31
5.2
Restoring the NMS Database from a Hot Backup
If you have copies of the latest online redo logs, archive logs, and control files, the hot backup
restore is also easier and you can recover the changes made after the hot backup was taken. If
this is the case, ensure that you make and store a copy of these file somewhere. Do not put
them in /data/oradata/GMS and /export/apps/oracle/admin/GMS/multiplex, since these
locations will be overwritten during the steps below. Refer to Section 5.4 Restoring the NMS
Database Completely on page 37.
To restore the entire NMS database from a hot backup
32
1.
If necessary, reinstall the entire SYMMETRY NMS This is required if the entire NMS is lost due to a
disk crash. You may also need to install and configure Solaris if the damaged disk also hosted the
OS. Refer to the Software Installation Setup Guide provided with your SYMMETRY NMS system to
install the OS and SYMMETRY NMS. Installing the OS alone may take several hours. To reduce
NMS downtime, you should have a second Sun server available, with OS and NMS installed and
ready to use.
2.
If necessary, reinstall the NMS database monitor and backup tools. Follow the installation
instructions Chapter 2 and Chapter 4, respectively. Note that you may need to start up the
database first before installing the NMS database monitor and backup tools. This step can be
avoided by having another server available with the NMS database monitor and backup tools
already installed.
3.
Ensure that all NMS applications have been stopped. Login as user gabriel and follow the steps in
the Software Installation Setup Guide to shutdown the Gabriel, Raphael, and MOM applications.
4.
Login in as user oracle and make sure that the database and its listener have all been stopped.
Follow the steps in the Software Installation Setup Guide to check this and issue the proper stop
commands. If you are not able to shutdown the database using the SVRMGR shutdown immediate
command described in the Software Installation Setup Guide, you will have to use the shutdown
abort command.

5.
Get the last hot backup files and place them in the following directory:
/export/apps/oracle/admin/GMS/bkup/hot. In addition to other files, there is a restore script in this
directory and its name depends upon the date when the hot backup was taken. Make sure that you
are still logged in as user oracle and perform the following command to change it to be executable:
%chmod u+x restore_GMS_11Feb04.sh
The above command assumes that the hot backup was taken on Feb 11, 04.
6.
If you have copies of the latest control file, redo log files, and archived redo log files, stop
here and do not perform the steps below. Refer to the steps in Section 5.4 Restoring the
NMS Database Completely on page 37 to continue so that even the changes after the hot
backup was taken can be restored.
Run the restore script to restore database data files, archive logs, and database parameter files.
You must specify the database name GMS on the command line.
%./restore_GMS_11Feb04.sh GMS
7.
The hot backup cannot directly copy the database control files because hot backup is performed
while database is open and running. The steps given here will create the control files back from a
control file trace created during the hot backup. You can avoid losing control files so that these
steps here will not be necessary. Having a most recent control file also ensures a safe database
recovery. There are ways for maintaining a copy of a recent control file and they will be discussed in
a later section. Follow the steps below to recreate the database control files from the trace file.
%cp control_GMS_11Feb04.sql /data/oradata/GMS
This copies the control trace file created during the hot backup to the original location of the control
file.
33

8.
Edit the file control_GMS_11Feb04.sql in /data/oradata/GMS to comment out the last line by adding
-- (two dashes) to the beginning of the line. After this is done, the last line should read as:
-- ALTER DATABASE OPEN RESETLOGS;
It is important to comment out this line before you continue with the next step.
9.
Perform the following commands to start the recovery of control files:

%cd /data/oradata/GMS
%svrmgrl
SVRMGR>@control_GMS_11Feb04.sql
SVRMGR>recover database until cancel using backup controlfile;
You will be prompted with a message like:

ORA-00279: change 733543036 generated at 02/11/2004 00:00:10 needed for thread 1
ORA-00289: suggestion : /export/apps/oracle/admin/GMS/multiplex/archives/1_99.gms
ORA-00280: change 733543036 for thread 1 is in sequence #99
Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
10. Type and enter AUTO and all of the archive logs which are present in the backup will be applied.
After all of the available logs have been applied, you may see some error message like:
ORA-00308: cannot open archived log
'/export/apps/oracle/admin/GMS/multiplex/archives/1_101.gms'
ORA-27037: unable to obtain file status
SVR4 Error: 2: No such file or directory
Additional information: 3
This means that archive log file it is looking for is not there. If this number is higher than the
numbers in the archive log directory, it is acceptable because it has not been written to disk yet.
At this point, the database has been recovered. Because the control file was recreated, you must
synchronize the data files with the control files and redo log files.
34

11. Enter the following database alter command to start up the database and synchronize files:
SVRMGR>alter database open resetlogs;
SVRMGR>exit
%
12. At this point you should perform a cold backup of the database before launching the NMS. This
ensures that you can easily restore the database up to this point using the cold backup. Follow the
instructions provided in Chapter 3 to perform a cold backup.
At the end of the cold backup, the database will be restarted automatically.
13. Restart the listener as user oracle.
%lsnrctl start
14. Verify that you can login database using sqlplus.

%sqlplus gms_s1/gms_s1@GMS
You may need to wait for a minute or so for the above command to work, since Oracle listener takes
a little while to start up.
15. Follow the Software Installation Setup Guide provided with your SYMMETRY NMS system to start
up the NMS applications.
35
5.3
Reducing the NMS Database Recovery Time

Backing up the NMS database is critical to prevent loss of your network and customer data. Nothing can
replace a good backup in the event of an NMS server hardware failure. However, you will not be able to
start restoring the NMS database unless the NMS and its database software are properly installed and
configured. If you do not have a spare server, in the event of an NMS server hardware failure, you may
need to first repair the server and you may also need to reinstall the OS and the NMS. This will incur
substantial NMS downtime.
You should have another server to act as an emergency standby. On this server, you should install the
same version of the NMS, including the NMS database monitor and backup tools. If your active NMS
server failed and could not be repaired quickly, you can immediately start restoring the NMS database
on the standby server by invoking the restore script. This will dramatically reduce the NMS downtime.
36
5.4
Restoring the NMS Database Completely

Although it is reliable and complete, the cold backup cannot be performed often because it requires
shutting down the database. Hot backup allows the database to stay online and available to users and
therefore can be performed more frequently, even daily. However, unless you have redundant copies of
the latest online redo logs and archive logs, you may still lose the database updates that have taken
place since your last hot backup.
Another factor to consider is the database control file. As mentioned in the hot backup restore section,
database control files are key, but they can not be backed up by hot backup since the database is open
and running. Therefore the hot backup restore process described above has one step for recreating the
control files. This step would not be needed and the restore would be much safer if you have redundant
copies of the control files.
To be able to fully restore the NMS database, you need to multiplex the database redo logs, archive
logs, and control files.
As soon as you install the NMS database hot backup tool and the database is put in ARCHIVELOG
mode, the NMS database will start maintaining two copies of these files. The first set of copies of these
files are located in /data/oradata as described in Chapter 1, and the default location of the second set of
copies of these files are saved in /export/apps/oracle/admin/GMS/multiplex. These two locations should
reside on two different disks so that the failure of one disk does not cause both sets to be lost.
If you have copies of the latest redo logs, archive logs, and control files, the hot backup restore is also
easier.
37

To start the database with automatic recovery
1.
Make sure that you make and store a copy of the latest redo logs, archived redo logs, and control
file somewhere. Do not put them in /data/oradata/GMS and
/export/apps/oracle/admin/GMS/multiplex, since these locations will be overwritten during steps
below.
2.
Refer to Section 5.2 on page 32 and complete Step 1 on page 32 to Step 5 on page 33.
3.
Run the restore script to restore database the data files, archive logs, and database parameter files
from the hot backup. You must specify the database name GMS on the command line.
%./restore_GMS_11Feb04.sh GMS
4.
Place a copy of the control files, latest redo log files, and archive log files in both locations,
/data/oradata/GMS and /export/apps/oracle/admin/GMS/multiplex.
5.
Do the following to start up the database with automatic recovery:

%svrmgrl
SVRMGR>startup mount
SVRMGR>recover automatic database
SVRMGR>alter database open;
SVRMGR>exit
This restores the database including the changes that have taken place since your last hot backup. After
this step, continue with the remaining steps described Section 5.2 Restoring the NMS Database from a
Hot Backup on page 32 to perform a cold backup, to start the database listener, and to start NMS
applications (Step 12 on page 35 to Step 15 on page 35).
38
Chapter 6
NMS Standby Database

SYMMETRY NMS redundancy architecture recommends the use of a standby
database. The standby database is synchronized with the primary database and
therefore can be activated at any time, as required. This chapter describes how to
prepare, manage, and maintain an Oracle standby database with SYMMETRY NMS.
39
6.1
Standby NMS Configuration

SYMMETRY NMS redundancy architecture recommends using one host as the standby NMS
regardless of how many hosts are used for the primary NMS system (see Figure 6.1). On the standby
host, you must install the same version of SYMMETRY NMS software as the primary, start the standby
database and synchronize it with the active database, but keep the NMS background processes down.
To failover to the standby NMS, you first activate the standby database, and then launch all NMS
backgrounds. After all the processes come up, the standby system becomes the new primary and it
connects all existing base stations.
Figure 6.1
40
SYMMETRY NMS Redundancy Configuration
6.2
Oracle Standby Database

An Oracle standby database is created from a backup of the primary database and is synchronized by
applying the redo logs archived by the primary database. After it is configured for the managed standby
environment and managed recovery mode, the primary database will automatically archive the redo
logs to the standby host and they are automatically applied to the standby database. See Figure 6.2 and
Figure 6.3 on page 42.
Figure 6.2
SYMMETRY NMS Primary Database and Standby Database
41
Figure 6.3
42
Failover to Standby Database
6.3
Setting Up an NMS Standby Database
To use the standby NMS feature, you must install the NMS database monitor, cold backup, and
hot backup tools on the primary database. If you have not done so, you should first install them
on the primary host and perform some monitor and backup operations before continuing to set
up the standby database. Refer to Chapter 2, Chapter 3, and Chapter 4.
To prepare the NMS standby database

1.
Install a complete SYMMETRY NMS system on the standby host, including the NMS database
maintenance packages described in previous chapters.
2.
Take a complete backup of the primary database
? Shutdown the primary NMS background processes.

? Perform a cold backup.
? Keep the processes down.
3.
From the primary database, create a control file for the standby database and then shutdown the
primary database.
% svrmgrl
SVRMGR>alter database create standby controlfile as /tmp/control01.ctl;
SVRMGR>shutdown
SVRMGR>exit
%
43

4.
Restore the standby database using the primary database backup.
? Place the backup file in /export/apps/oracle/admin/GMS/bkup/cold on the standby host.

? Run the restore scripts. Refer to the procedure in Section 5.1 on page 30.
? Since you are using a backup from one host and restoring it on another, remember to change the
Oracle TNS files tnsnames.ora and listener.ora to use the standby host name in these two files,
as described in Section 5.1 on page 30.
? Do not start the standby database at this time.
5.
Place the standby control file that you created above into the following two directories on the
standby host: /data/oradata/GMS and /export/apps/oracle/admin/GMS/multiplex/control.
6.
Configure the standby database to automatically apply the archived redo logs to be received from
the primary database. As user oracle on the standby host, place the standby package standby.tar
that you are given in the Oracle home directory /export/home/oracle, extract the tar file, and then
run the setup script for the standby database.
%
%
%
%
44
cd /export/home/oracle
tar xvf standby.tar
cd stdby_config/scripts/setup
./setupstandby.ksh

7.
Start the standby database listener, mount the database as standby, and then place it in managed
recovery mode. As user oracle on the standby host:
% lsnrctl start listener_standby_gms
% svrmgrl
SVRMGR>startup nomount
SVRMGR>alter database mount standby database;
SVRMGR>recover managed standby database
8.
On the primary host, configure the primary database to automatically archive the redo logs to the
standby database. As user oracle, place the standby package standby.tar that you are given in the
Oracle home directory /export/home/oracle, extract the tar file, and then run the setup script for the
primary database.
%
%
%
%
9.
After you execute the above command, the SVRMGR prompt does not return and stay on
the next line. This is normal. Do not press CTRL+C. Pressing CTRL+C will cancel the
managed recovery, and the logs from the primary database will not be applied. Closing the
terminal window does not affect the managed recovery; the recovery continues in the
background.
cd /export/home/oracle
tar xvf standby.tar
cd stdby_config/scripts/setup
./setupprimary.ksh
Start the primary database, its listener, and all NMS processes. Refer to the Software Installation
Setup Guide provided with your SYMMETRY NMS system for the correct steps.
This completes the setup steps for both the primary and the standby database. When it is time to
archive a redo log file on the primary database, it will also be archived and applied to the standby
database.
45
6.4
Monitoring and Managing the NMS Standby Database

You should monitor the archiving status of your primary and standby database to check that the standby
database is indeed synchronized with the primary database. This will help make sure that no data is lost
if the primary host fails and you need to perform a failover. This section describes how to check the
synchronization status and what to do if the standby database is no longer synchronized with the
primary database.
6.4.1
Using the NMS Database Monitor Tool

Recall that the NMS database monitor generates a database report that you can configure to
automatically send you the report daily. If the primary database is not able to archive redo logs to the
standby, there will be errors in the reports Alert section. For example:
<< Alert Errors Since Yesterday >>
ARC0:Error 12543 connecting to standby host STANDBY_GMS
ARC0: Error 12543 creating archive log file STANDBY_GMS
These errors inform you that the primary database is not able to connect to the standby database,
possibly due to some network issues.
46
6.4.2
Determining the Most Recent Archived Log

As user oracle, run the script maxseq.ksh to query the database for the most recent archived redo log.
% cd /export/home/oracle/stdby_config/scripts/utils
% ./maxseq.ksh
If the archiving of redo logs from the primary to the standby databases is working properly, the value
you get from both the primary and standby databases should match.
6.4.3
Reviewing the Archive Trace Files

There is a trace file on both the primary and standby hosts that records the redo log archiving activities.
On the primary host, they are in directory /export/oracle/admin/GMS/bdump and have names like
gms_arc0_20210.trc. On the standby host, they are in directory /export/oracle/admin/GMS/udump and
have names like gms_rfs_4361.trc. You may want to review recent files when you are troubleshooting as
they contain detailed information generated from the databases archiving background process (ARC).
47
6.4.4
Resolving a Gap Sequence

The Oracle database assigns a sequence number for each redo log file. When a primary database
archives a redo log file but it is not able to transfer it to the standby host, the standby database will miss
a set of archived redo log files. The sequence numbers for the these missed archived logs are called
gap sequences. A gap sequence is usually caused by some network problem between the primary host
and the standby host.
Another common reason that can cause a gap sequence is when you shutdown the standby database
while the primary database is still up and open. Therefore, you should avoid shutting down the standby
database without first shutting down the primary database.
Even after you fix the network problems and the transfer of the archived logs resumes, the newly
archived logs are not going to be automatically applied to the standby database unless you resolve the
gap sequence manually. Therefore, when a gap sequence occurs, the standby database is no longer
synchronized with the primary database. You should resolve the gap sequence as soon as possible so
that the log files received from the primary database can again be applied automatically.
To identify and resolve the gap sequence
1.
2.
Identify the gaps by running the script gap.ksh on the standby host.
% cd /export/home/oracle/stdby_config/scripts/utils
% ./gap.ksh
Let us assume the script produces the following result:

THREAD#
LowSeq# HighSeq#
---------------------------1 460 463
If there is no gap, then the script should return either no result, or the LowSeq# will be equal to the
HighSeq#.
48

3.
Using FTP or making a CD, transfer the archived log files from seq# 460 to seq# 463 in
/data/oradata/GMS/archive on the primary host to the standby host and place them in both of the
following directories: /data/oradata/GMS/archive and
/export/apps/oracle/admin/GMS/multiplex/archives.
f
4.
You must place these log files in both directories, otherwise the step below will fail and you
will receive error messages about not finding a log file in the directory.
On the standby host, cancel the managed recovery that is still in progress and then apply the log
files within the gap sequences.
% svrmgrl
SVRMGR> recover managed standby database cancel
SVRMGR> recover automatic standby database
5.
After recovering the available logs, Oracle prompts for the name of a log that does not exist. The
reason is that the recovery process does not know about the logs archived to the standby site by
the primary database. For example, you might see:
ORA-00308: cannot open archived log /oracle/standby/standby_logs/arcr_1_464.arc
ORA-27037: unable to obtain file status
SVR4 Error: 2: No such file or directory
Additional information: 3
Specify log: {<RET>=suggested | filename | AUTO | CANCEL}
49

6.
Cancel the recovery.

SVGMGR> cancel
You may see these messages (they are acceptable):

Media recovery cancelled.
ORA-01547: warning: RECOVER succeeded but OPEN RESETLOGS would get error below
ORA-01194: file 1 needs more recovery to be consistent
ORA-01110: data file 1: some_filename
ORA-01112: media recovery not started
7.
You are now ready to put the managed recovery mode back on.
SVRMGR> recover managed standby database
The standby database will now again be able to automatically apply the archived logs to be received
from the primary database.
50
6.5
Opening the Standby Database as Read-only

When you place the standby database into recovery mode by executing the recover managed standby
database command, the database does not allow users to connect to the database to perform queries.
Sometimes you may want to query the standby database to make sure that the records match the
primary database. You can do this by placing the standby database in read-only mode. But you must
first cancel the managed recovery mode before you can place the database in read-only mode.
The log files received from the primary database while the standby database is in read-only mode will
not be applied, and this will create gap sequence situation described in Section 6.4.4 Resolving a Gap
Sequence on page 48. Therefore, you should avoid placing the standby database in read-only mode
while the NMS background processes on the primary host are up, and may cause the updating of the
primary database.
To place the standby database in read-only mode
1.
2.
Cancel the managed recovery.

% svrmgrl
SVRMGR> connect internal
3.
Open the database in read-only mode.

SVRMGR> alter database open read only;
SVRMGR> exit
%
Now you can start SQL*Plus sessions on the standby host the same way you do on the primary
host, and perform SQL select commands against any Gabriel, Raphael, and MOM tables.
You should put the standby database back to managed recovery mode as soon as you have completed
your queries.
51

To place the standby database back to managed recovery mode
1.
Make sure that you exit all SQL*Plus sessions that you opened during the read-only mode.
2.
Issue the following statements:

% svrmgrl
SVRMGR> recover managed standby database
SVRMGR> exit
%
If you have not caused a gap sequence while the database was in read-only mode, the managed
recovery will continue when a new log file is received from the primary database. If there is any gap
sequence, you must resolve the gap; refer to Section 6.4.4 Resolving a Gap Sequence on
page 48.
52
6.6
Activating the NMS Standby Database

Activating a standby database is a permanent action and it can not be undone. As soon as it is
activated, the standby database will be completely destroyed, and you would have to go through all
the steps described in Section 6.3 Setting Up an NMS Standby Database on page 43 if you want to
create the standby database again. Therefore, you should not activate your standby database just to
verify if your standby database is being synchronized with the primary database.
You should also make a complete cold backup as soon as possible after you activate your standby
database. All of your previous hot backups of your old primary database are no longer valid after you
have activated your standby database.
6.6.1
Activating the Standby NMS Database

Follow these steps to activate the standby NMS database.
To activate the standby NMS database
1.
Cancel the ongoing managed recovery, activate the standby, and then shut down all standby
database background processes. As user oracle on the standby host:
% lsnrctl stop listener_standby_gms
% svrmgrl
SVRMGR> alter database activate standby database;
If you receive an error message from the above command, please stop here and follow the
steps described in Step 2 on page 54. If not, continue with Step 3 on page 54.
SVRMGR> shutdown immediate

SVRMGR> exit
%
53

2.
You may get error messages when you execute the above alter database activate standby
database command. If you do, please follow these steps to restart the standby database, try the
alter database command again, and then shutdown the standby database, before you proceed to
Step 3 on page 54 for starting up the new primary database:
SVRMGR> exit
% svrmgrl
SVRMGR>startup nomount
SVRMGR>alter database mount standby database;
SVRMGR> alter database activate standby database;
SVRMGR> exit
%
3.
Start the new primary database. As user oracle on the activated standby (the new primary) host
% lsnrctl start listener_gms
% svrmgrl
SVRMGR> startup mount
SVRMGR> alter database open;
SVRMGR> exit
%
You can now proceed to start all NMS background processes on the new primary host. As soon as you
have done this, the new primary NMS systems are ready for use by users.
54
If your Raphael system is standalone (i.e., the Raphael system is running on its own host), you
may also need to run the script gabrielhost_script to configure the new primary database so
that Gabriel will continue to send CDR data to the Raphael system. Please consult the Software
Installation Setup Guide provided with your SYMMETRY NMS system.
6.6.2
Recreating the Primary After Standby Activation

You may want to recreate the primary NMS system on a new host, or on the original primary host after
the problems which caused the failover have been resolved. Please refer to proper NMS manuals for
each of the steps listed below.
To recreate the primary NMS system on a new host
1.
On the new primary host, you may need to reinstall all SYMMETRY NMS software, such as Gabriel,
Raphael, MOM, as well as database maintenance packages.
2.
Make a cold backup of the activated standby database (the current primary system).
3.
Restore the cold backup to the original primary database by following the steps in Section 5.1
Restoring the NMS Database from a Cold Backup on page 30.
4.
Shut down the activated standby database and NMS processes on the current primary host.
5.
Start up the restored database on the original primary host and NMS processes.
Users can now access the original primary host as their SYMMETRY NMS systems.
6.
Make a cold backup of the primary database and use the backup to recreate the standby database.
55

Symmetry Nms Database

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Symmetry Nms Database

Caricato da

Copyright:

Formati disponibili

SYMMETRY NMS Database

Maintenance Guide (v2.40)

Database Data Files............................................................................................................ 6

Database Redo Logs .......................................................................................................... 7

Database Archive Logs ....................................................................................................... 7

Database Control Files........................................................................................................ 7

Database Cold Backup ....................................................................................................... 8

Database Hot Backup ......................................................................................................... 8

NMS Database Monitor ........................................................................................................................... 9

Installing the NMS Database Monitor................................................................................ 10

Scheduling NMS Database Monitor .................................................................................. 11

Reviewing NMS Database Monitor Reports...................................................................... 12

Preventive Actions ............................................................................................................ 15

NMS Database Cold Backup ................................................................................................................. 21

Installing the NMS Database Cold Backup Tool ............................................................... 22

Performing an NMS Database Cold Backup ..................................................................... 23

NMS Database Hot Backup................................................................................................................... 25

Installing the NMS Database Hot Backup Tool ................................................................. 26

Performing an NMS Database Hot Backup ...................................................................... 28

NMS Database Restore......................................................................................................................... 29

Restoring the NMS Database from a Cold Backup ........................................................... 30

SYMMETRY NMS Database Maintenance Guide

Restoring the NMS Database from a Hot Backup............................................................. 32

Reducing the NMS Database Recovery Time................................................................... 36

Restoring the NMS Database Completely......................................................................... 37

NMS Standby Database ........................................................................................................................ 39

Standby NMS Configuration.............................................................................................. 40

Oracle Standby Database ................................................................................................. 41

Setting Up an NMS Standby Database ............................................................................. 43

Monitoring and Managing the NMS Standby Database .................................................... 46

Opening the Standby Database as Read-only .................................................................. 51

Activating the NMS Standby Database ............................................................................. 53

SYMMETRY NMS Database Maintenance Guide

Database Data Files

Database Redo Logs

Database Archive Logs

Database Control Files

SYMMETRY NMS Database Maintenance Guide

Database Cold Backup

Database Hot Backup

NMS Database Monitor

SYMMETRY NMS Database Maintenance Guide

Installing the NMS Database Monitor

Extract the tar file with the UNIX tar command.

This will produce two files, installmonitor.ksh and monitor_app.tar.

To verify the installation, run the DB checkup script manually.

If the installation is successful, you should receive an e-mailed database report.

NMS Database Monitor

Scheduling NMS Database Monitor

Login NMS server as user oracle.

Open the user oracle cron table with these commands:

This opens a UNIX vi editor session.

Using vi, add this line on its own line (case-sensitive).

Exit the vi session to complete the crontab -e command.

To verify the entry

SYMMETRY NMS Database Maintenance Guide

Reviewing NMS Database Monitor Reports

Tablespaces with < 15% and < 1Gb Free Space

NMS Database Monitor

Segments Cant Allocate NEXT Extent

Tablespaces with Over 20 Free Fragments

SYMMETRY NMS Database Maintenance Guide

Segments Nearing Max Extents

NMS Database Monitor