EM Migration Scenarios

Merck Standards and Naming Convention..........................................................................2

Group naming convention...............................................................................................2
Job naming convention....................................................................................................3
Blackout naming convention...........................................................................................3
Scenario 1: Add Database Instance to EM.........................................................................4
1. Add Database Instance to a New Group...................................................................4
2. Add Database Instance to a Existing Group.............................................................4
Scenario 2: Change Database Instance from Cold to Hot or from Hot to Cold Backups..5
1. Add Database Instance to a New Group...................................................................5
2. Add Database Instance to a Existing Group.............................................................5
Scenario 3: Move Database Instance from Server A to Server B.......................................5
Scenario 4: Upgrade Database to a higher release...........................................................6
Section 1: Update 9i or higher database to higher oracle version (12c)..........................6
Section 2: Update 9i or higher database to higher oracle version (10g or 11g)...............6
Section 3: Update 8i Database to 10g Database..............................................................7
Scenario 5: Remove Database Instance from EM..............................................................7
Scenario 6: Add Archivelog Repository in EM...................................................................7
Appendix B: Administrator Configuration.........................................................................8
Section 1: Add Database Instance and ASM Instance to EM..........................................8
Section 2: Add database target to existing Group............................................................8
Section 3: Create Appropriate Group..............................................................................9
Appendix C: Monitoring Setup.........................................................................................10
Appendix D: SOX Monitoring...........................................................................................11
Appendix E: Backup Jobs and Blackouts.........................................................................12
Appendix F: RMAN, TSM Configuration.........................................................................14
Appendix G: RMAN, Data Domain Boost Configuration................................................15
Appendix H: RMAN, Commvault Configuration..............................................................16
Appendix I: Oracle Wallet Monitoring.............................................................................18
Appendix J: EM Archivelog Repository job......................................................................18
Appendix K – Add Database To Backup Crawler..............................................................19
Appendix L – Update RMAN for New Oracle Home (DDBoost)......................................19

452379753.doc Page 1 of 20
Merck Standards and Naming Convention

1. Login to NONPROD in http://emnonprod.merck.com:7778/em if database is non

production (non-Exadata) and less than 12c

Login to PROD in http://emprod.merck.com:7778/em if database is production (non-

Exadata) and less than 12c

Login to EMOPER in http://emxprod.merck.com:7788/em if database is on Exadata

and less than 12c

Login to EMOPER in http://emnonprod2.merck.com:7788/em if database is 12c Non-

production

Login to EMOPER in http://emprod2.merck.com:7788/em if database is 12c

Production

2. ARCHIVELOG databases are always configured for HOT FULL weekly and HOT
INCR daily backups

3. NOARCHIVELOG databases are configured for COLD backups weekly

4. “ARCHIVELOG REPOSITORY” databases are standby databases and only cleanup

of archive jobs are created.

Group naming convention

Regular Database Instance
<Server_Name>_<Media_Manager>_<Backup Type>
Cluster Database
<Cluster_Name>_<Backup Type>
ARCHIVELOG REPOSITORY
<Server_Name>_<Media_Manager>_ARCHREPO

<Media_Manager> values are <NULL> for TSM, “DDB” for DDBoost, “DDR”
for Direct Data Domain Disks, “CV” for Comvault

<Backup Type> values are HOT, HOT2, COLD, COLD2, NB,

HOT_ARCHREPO
Examples:
UXWSDV01_HOT
UXWSDV01_DDB_HOT2
UXWSDV01_CV_COLD
UXWSDV01_NB

452379753.doc Page 2 of 20
 UXWSDV01_ARCHREPO

Job naming convention

<Group_Name>_<Backup Level>
Examples:
UXWSDV01_HOT_FULL
UXWSDV01_DDB_HOT2_INCR
UXWSDV01_CV_COLD_FULL
UXWSDV01_NB_CONTROLFILE
UXWSDV01_ARCHREPO_CLEANUP

Blackout naming convention

Blackout names are same as group names
<Group_Name>
Examples:
UXWSDV01_COLD
UXWSDV01_DDB_COLD
UXWSDV01_CV_COLD2

452379753.doc Page 3 of 20
Scenario 1: Add Database Instance to EM
1. Add Database Instance to a New Group
o Verify passwords for userid dbsnmp and oemmon in the database
o Execute Sections 1, and 3 in Appendix B – EM Administration Configuration
o Execute steps in Appendix C – EM Monitoring Setup
o Execute steps in Appendix E – EM Backup Jobs
o Execute steps in Appendix F, G or H – RMAN, DDB/TSM/Commvault
Configuration. (Refer to Table 1.1 for applicable configuration)
o 12c Only: Execute steps in Appendix K – Add Database To Backup Crawler
Note: This is a temporary requirement, that will not be necessary once the backup
dashboard application is updated.

OR

2. Add Database Instance to a Existing Group

o Verify passwords for userid dbsnmp and oemmon in the database
o Execute Sections 1, and 2 in Appendix B – EM Administration Configuration
o Execute steps in Appendix C – EM Monitoring Setup
o Execute steps in Appendix F, G or H – RMAN, DDB/TSM/Commvault
Configuration. (Refer to Table 1.1 for applicable configuration)
o 12c Only: Execute steps in Appendix K – Add Database To Backup Crawler
Note: This is a temporary requirement, that will not be necessary once the backup
dashboard application is updated.

Table 1.1
DB OS Backup Media
Version Platform Version Manager Appendix
<10 All All TSM Appendix F
>=10 HPUX 11.11 TSM Appendix F
(1)
>=10 HPUX 11.31 DDBoost Appendix G
>=10 LINUX RHEL4 TSM Appendix F
(1)
>=10 LINUX RHEL5 DDBoost Appendix G
(1)
>=10 LINUX RHEL6 DDBoost Appendix G
>=10 AWS-EC2 RHEL6 CommVault Appendix H
>=10 Other All TSM Appendix F
(1)
DDBoost only available in CTC and WHS data centers, otherwise use TSM

452379753.doc Page 4 of 20
Scenario 2: Change Database Instance from Cold to Hot or from Hot to
Cold Backups

1. Add Database Instance to a New Group

o Execute Appendix G – Remove Database Instance from Group
o Execute Sections 1, and 2 in Appendix B – EM Administration Configuration
o Execute steps in Appendix C – EM Monitoring Setup
o Execute steps in Appendix E – EM Backup Jobs

2. Add Database Instance to a Existing Group

o Execute Appendix G – Remove Database Instance from Group
o Execute Section 2 in Appendix B – EM Administration Configuration
o Execute steps in Appendix C – EM Monitoring Setup

Scenario 3: Move Database Instance from Server A to Server B

1. Gather Job information before removing Database Instance

o Click Jobs tab
o Click Advanced Search
o Type Database Instance in Target Name field
o Click GO
o Print screen to be used later when adding instance to Server B

2. Remove Database Instance from EM

o Perform Scenario 5: Remove Database Instance from EM

3. Add Database Instance to EM

o Execute steps in "Scenario 1: Add Database Instance to Existing Server"
Note: Use the Job information gathered in step 1

452379753.doc Page 5 of 20
Scenario 4: Upgrade Database to a higher release

Section 1: Update 9i or higher database to higher oracle version (12c)

1. From pre-12c EM console, gather Job information before removing Database

Instance
o Click Jobs tab
o Click Advanced Search
o Type Database Instance in Target Name field
o Click GO
o Print screen to be used later when adding instance to Server B

2. From pre-12c EM console, Remove Database Instance:

o Perform Scenario 5: Remove Database Instance from EM

3. From 12c EM console, Add Database Instance

Make sure you are logged into the correct 12c EM console
o Execute steps in "Scenario 1: Add Database Instance to Existing Server"
Note: Use the Job information gathered in step 1

Section 2: Update 9i or higher database to higher oracle version (10g or

11g)
1. Configure databases
o Click Targets tab
o Click Database tab
o Choose Database
o Click Configure
o Update the Oracle Home Patch field with the new ORACLE_HOME
o Click Test Connection
o Click Next
o Click Submit
o Click OK

2. If target server backup is using DDBoost

o Execute steps in "Appendix L – Update RMAN for New Oracle Home
(DDBoost)"

452379753.doc Page 6 of 20
Section 3: Update 8i Database to 10g Database

1. Remove Database Instance from GRID Control

o Execute Scenario 5: Remove Database Instance from EM

2. Add Database Instance Setup

o Execute steps in Scenario 1 – Add Database Instance to an existing server.

Scenario 5: Remove Database Instance from EM

1. Remove Database Instance

o Click Targets
o Click Databases
o Choose Database
o Click Remove
o Click YES

Scenario 6: Add Archivelog Repository in EM

 Verify passwords for sys in the primary database

 Execute Sections 1, and 3 in Appendix B – EM Administration Configuration
(Use sys and sysdba)
 Execute steps in Appendix J – EM Archivelog Repository Job

452379753.doc Page 7 of 20
Appendix B: Administrator Configuration

Section 1: Add Database Instance and ASM Instance to EM

o Choose Agent in Target Search, Type server name in Target Search Field, Click Go
o Click Go in front of Add Database Instance
o Under Databases – Select the only the databases you want to add
o Under Listeners - Click "Select None"
o Under ASM Instances – Click “Select All” if not selected
o For Databases
o Choose Higher Level Group in the Group Field (<Division>_P or
<Divison>_NP)
o Click ICON in Configure column to enter dbnmp password
o Type “dbsnmp” password in the monitor password field
o Click “Test Connection”
o Click Next
o Click OK
o For ASM Instance
o Click ICON in Configure column to enter SYS password
o Type SYS password in the “Password” field
o Type SYS password in the “Confirm Password” field
o Click “Test Connection”
o Click OK
o Click OK.

Section 2: Add database target to existing Group

o Click Targets
o Click Groups
o Choose Group name from the list
o Click Configure
o Click Search
o Choose "Database Instance" in the "Target Type" drop down
o Type Database name in the “Target Name” field
o Click GO
o Select database for the above group
o Click Select
o Click OK

452379753.doc Page 8 of 20
Section 3: Create Appropriate Group
Add New Group
o (Use Standardized EM Group Name in Merck Standards)
o Click Targets
o Click Groups
o Click Add
o Type group name in the Name field
o Click Search below to add members
o Choose “Database Instance” in the “Target Type” drop down
o Type hostname in the “On Host” field
o Click Go
o Select databases for the above group
o Click Select
o Click OK

452379753.doc Page 9 of 20
Appendix C: Monitoring Setup

Note: Monitoring Template is applied by default for all database targets

1. Apply Corrective action to Database

o Click Setup
o Click Monitoring Templates
o Select appropriate template
Select "CA_ARCH_FRA_TSM" for databases using ASM
OR
Select "CA_ARCH_TSM" for databases greater than version 8i and using
regular archivelog filesystem
OR
Select "CA_ARCH_8I_TSM" for 8i databases.
OR
o Click Apply
o Click Add
o Choose the Databases
o Click Select
o Click Next
o For CA_ARCH_FRA_TSM
Type oemmon in “Username” field
Enter password for oemmon in “Password” field
o Click Finish

2. Exclude UNDO and TEMP only on Databases greater than version 9i

o Click Jobs
o Click Job Library
o Select <TEMPLATE>_EXCLUDE_UNDO_TEMP_MONITORING
o Click Submit
o Click Add
o Choose Databases
o Click Submit

452379753.doc Page 10 of 20
Appendix D: SOX Monitoring
1. Create Failed Login Job (One Job per target)
o Click Jobs
o Click Job Library
o Select <TEMPLATE>_SOX_AUDIT_FAILED_LOGINS
o Click Create Like
o Enter Name for the job “<SID>_SOX_AUDIT_FAILED_LOGINS”
o Click Add
o Choose Database
o Click Save
o Click Job Library
o Select the appropriate jobs created
o Click Submit
o Delete all the characters after the Period (".") including the Period in the job name
(Example: FINMCRP_SOX_AUDIT_FAILED_LOGINS.1)
o Click Submit

2. Create Critical Table Audit Job (One Job per target)

o Click Jobs
o Click Job Library
o Select <TEMPLATE>_SOX_AUDIT_CRITICAL_TABLES
o Click Create Like
o Enter Name for the job “<SID>_SOX_AUDIT_CRITICAL_TABLES”
o Click Add
o Choose Database
o Click Save
o Click Job Library
o Select the appropriate jobs created
o Click Submit
o Delete all the characters after the Period (".") including the Period in the job name
(Example: FINMCRP_SOX_AUDIT_CRITICAL_TABLES.1)
o Click Submit

3. Add database to SOX_DATABASES_P group.

o Click Targets
o Click Groups
o Select SOX_DATABASES_P group
o Click Configure
o Click Add
o Choose Databases
o Click Go
o Select Database
o Click Select

452379753.doc Page 11 of 20
 o Click Ok.

452379753.doc Page 12 of 20
Appendix E: Backup Jobs and Blackouts
ARCHIVELOG : Configured for HOT FULL weekly, HOT INCR daily backups
NOARCHIVELOG : Configured for COLD backups weekly

Templates TSM : <TEMPLATE>_HOT_FULL,

<TEMPLATE>_COLD_FULL, <TEMPLATE>_HOT_INCR

Templates DDBoost : <TEMPLATE>_DDB_HOT_FULL,

<TEMPLATE>_DD_COLD_FULL, <TEMPLATE>_DDB_HOT_INCR

1. Create FULL backup jobs in library

o Click Jobs
o Click Job Library
o Select job template <TEMPLATE>_HOT_FULL or
<TEMPLATE>_COLD_FULL
o Click Create Like
o Enter Name for the job. Job name is <Group Name>_FULL (Example:
UXWSDV01_HOT_FULL or UXWSDV01_COLD_FULL).
o Click Add
o Choose Group in the Target Type drop down
o Click appropriate group
o Click Select
o Click Schedule
o Choose appropriate schedule
o Click Save to Library

2. Create INCR backup jobs in library

o Click Jobs
o Click Job Library
o Select job template <TEMPLATE>_HOT_INCR
o Click Create Like
o Enter Name for the job. Job name is <Group Name>_INCR (Example:
UXWSDV01_HOT_INCR or UXWSDV01_COLD_INCR)
o Click Add
o Choose Group in the Target Type drop down
o Click appropriate group
o Click Select
o Click Schedule
o Choose appropriate schedule
o Click Save to Library

3. Submit backup jobs from library

o Click Job Library

452379753.doc Page 13 of 20
 o Select the appropriate jobs created
o Click Submit
o Delete all the characters after the period (".") including the period in the job name
(UXWSDV01_HOT_FULL.1)
o Click Submit
o Repeat above steps for all jobs created

4. Create Blackouts for cold backups

o Click Setup
o Click Blackouts
o Click Create
o Enter name for the blackout. Blackout name is same as Group Name.
(Example: UXWSDV01_COLD)
o In the "Available Targets" choose Group
o Choose the appropriate Group
o Click Select
o Click Next
o Click Later under Start
o Choose Time (15 minutes from the backup time. If backup at 10:00PM choose
9:45PM).
o Click Length under Duration
o Type hours and minutes (Number of hours and minutes until 8:00AM )
o Choose Weekly in Repeating.
o Click on appropriate Day of Week
o Click No End Date
o Click Next
o Click Finish

452379753.doc Page 14 of 20
Appendix F: RMAN, TSM Configuration

1. Register and configure database for RMAN

 Login to the host as the database owner
 Set the current environment to the new database. Use .oraenv
For Production databases
Unix> rman target /
configure retention policy to recovery window of 28 days;
configure default device type to 'sbt_tape';
configure device type sbt_tape parallelism 2;
configure controlfile autobackup format for device type 'sbt_tape' to 'ctl_%d_%F';
configure channel device type 'sbt_tape' parms
'ENV=(TDPO_OPTFILE=/opt/oracle/<owner>/admin/<SID>/pfile/tdpo<SID>.opt)';

For NonProduction databases

Unix> rman target /
configure retention policy to recovery window of 7 days;
configure default device type to 'sbt_tape';
configure device type sbt_tape parallelism 2;
configure controlfile autobackup format for device type 'sbt_tape' to 'ctl_%d_%F';
configure channel device type 'sbt_tape' parms
'ENV=(TDPO_OPTFILE=/opt/oracle/<owner>/admin/<SID>/pfile/tdpo<SID>.opt)';

2. Setup TSM connectivity

 cd $ORACLE_BASE/admin/<SID>/pfile
 Create a tdpo<SID>.opt file in the database pfile directory. File contents below:
DSMI_ORC_CONFIG /opt/tivoli/tsm/client/config/dsmc/dsm.opt
DSMI_LOG /var/opt/tivoli/tsm/oracle/<SID>
TDPO_PSWDPATH /opt/tivoli/tsm/client/config/oracle/password
TDPO_FS <SID_DBID>
TDPO_NODE <SID_DBID>
TDPO_DATE_FMT 1
TDPO_NUM_FMT 1
TDPO_TIME_FMT 1

3. Register Database to EM for RMAN jobs

o Go to Database Home Page
o Click Availability
o Click Recovery Catalog Settings
o Select Use Recovery Catalog
o Click OK (Register database takes couple of minutes)
o Click OK

4. Open a Remedy SRM to request a TSM password.

452379753.doc Page 15 of 20
Appendix G: RMAN, Data Domain Boost Configuration

1. HPUX IA64 Only: Check for links in /usr/lib/hpux64:

> ls -la /usr/lib/hpux64 | grep oracle

Expected Results:
libDDBoost.so@ -> /opt/oracle/libs/DDBoost/v1.2/lib/libDDBoost.so
libddobk.so@ -> /opt/oracle/libs/DDBoost/v1.2/lib/libddobk.so

If the above links do not exist on HPUX IA64, open a Remedy WO with the UNIX
team to create them:

cd /usr/lib/hpux64

ln -s /opt/oracle/libs/DDBoost/v1.2/lib/libDDBoost.so .
ln -s /opt/oracle/libs/DDBoost/v1.2/lib/libddobk.so .

2. Open a Remedy SRM to request a DDBoost storage unit and backup host (primary &
replicated).

3. Register ORACLE_HOME with DDBoost storage unit and backup host. NOTE: For
Oracle 10g on Linux, use “v1.1” in SBT_LIBRARY path.
a. First DB in the Oracle Home:
run {
allocate channel dd0 type 'SBT_TAPE'
parms='BLKSIZE=1048576,SBT_LIBRARY=/opt/oracle/libs/DDBoost/v1.2/lib/libd
dobk.so ';
send 'set username ddboost password DDb00st servername <backup-
host>.merck.com';
release channel dd0;
}
b. Existing Oracle Home: N/A - Skip to next step

4. Register and configure database for RMAN

 Login to the host as the database owner
 Set the current environment to the new database. Use .oraenv
 Determine parallelism (i.e. channels) based on database size
Table 4.1
Platform Channel Formula Example(s)
HPUX/AIX 1 per 1.2TB 500 GB = 1
GB/1200 1.5TB=2
Round UP 5TB=3
LINUX 1 per 2TB 500 GB = 1
GB/2000 1.5TB=1
(Round UP) 5TB=3

452379753.doc Page 16 of 20
For Production databases
Unix> rman target /
configure retention policy to recovery window of 28 days;
configure default device type to 'sbt_tape';
configure device type sbt_tape parallelism <See Table 4.1>;
configure controlfile autobackup format for device type 'sbt_tape' to 'ctl_%d_%F';
configure channel device type 'sbt_tape' parms
'BLKSIZE=1048576,SBT_LIBRARY=<ORACLE_HOME> /lib/libddobk.so,
ENV=(STORAGE_UNIT=<storage-unit>,BACKUP_HOST=<backup-
host>.merck.com)';

For NonProduction databases

Unix> rman target /
configure retention policy to recovery window of 7 days;
configure default device type to 'sbt_tape';
configure device type sbt_tape parallelism <See Table 4.1>;
configure controlfile autobackup format for device type 'sbt_tape' to 'ctl_%d_%F';
configure channel device type 'sbt_tape' parms
'BLKSIZE=1048576,SBT_LIBRARY=<ORACLE_HOME> /lib/libddobk.so,
ENV=(STORAGE_UNIT=<storage-unit>,BACKUP_HOST=<backup-
host>.merck.com)';

5. Register Database to EM for RMAN jobs

o Go to Database Home Page
o Click Availability
o Click Recovery Catalog Settings
o Select Use Recovery Catalog
o Click OK (Register database takes couple of minutes)
o Click OK

Appendix H: RMAN, Commvault Configuration

1. Determine if this server is hosted in AWS-EC2 Platform in a region which supports

Commvault backups. If it is then proceed with below steps.
a. First DB on server: Open a WO in remedy to install Commvault agent on the
server and assign it to Backup & Recovery Team.

2. Register ORACLE Database in RMAN catalog with Commvault settings.

run {
CONFIGURE CHANNEL DEVICE TYPE 'SBT_TAPE' PARMS
"SBT_LIBRARY=/opt/simpana/Base/libobk.so,BLKSIZE=1048576";
}

3. Register and configure database for RMAN

452379753.doc Page 17 of 20
  Login to the host as the database owner
 Set the current environment to the new database. Use .oraenv
For Production databases
Unix> rman target /
configure retention policy to recovery window of 28 days;
configure default device type to 'sbt_tape';
configure controlfile autobackup format for device type 'sbt_tape' to 'ctl_%d_%F';
configure channel device type 'sbt_tape' parms
"SBT_LIBRARY=/opt/simpana/Base/libobk.so,BLKSIZE=1048576";

For NonProduction databases

Unix> rman target /
configure retention policy to recovery window of 7 days;
configure default device type to 'sbt_tape';
configure device type sbt_tape parallelism 2;
configure controlfile autobackup format for device type 'sbt_tape' to 'ctl_%d_%F';
configure channel device type 'sbt_tape' parms
"SBT_LIBRARY=/opt/simpana/Base/libobk.so,BLKSIZE=1048576";
4. Register Database to EM for RMAN jobs
o Go to Database Home Page
o Click Availability
o Click Recovery Catalog Settings
o Select Use Recovery Catalog
o Click OK (Register database takes couple of minutes)
Click OK

452379753.doc Page 18 of 20
Appendix I: Oracle Wallet Monitoring

1. Setup database for wallet monitoring

a. Run script Oracle_Data_Monitoring.sql as oemmon in the target database.
The script is in the Scripts directory under OEM in documentum

2. Apply Oracle Wallet Monitoring Templates

o Click setup
o Click Monitoring Templates
o Select "Oracle_Wallet_Monitoring"
o Click Apply
o Click Add
o Choose Databases
o Click Select
o Click Continue
o For username and password prompt, enter the OEMMON and its password
o Click Finish

Appendix J: EM Archivelog Repository job

Templates : <TEMPLATE>_ARCHREPO_CLEANUP

1. Create FULL backup jobs in library

o Click Jobs
o Click Job Library
o Select job template <TEMPLATE>_ ARCHREPO_CLEANUP
o Click Create Like
o Enter Name for the job. Job name is <Group Name>_CLEANUP (Example:
UXWSDV01_ARCHREPO_CLEANUP)
o Click Add
o Choose Group in the Target Type drop down
o Click Save to Library

2. Submit backup jobs from library

o Click Job Library
o Select the appropriate jobs created
o Click Submit
o Delete all the characters after the period (".") including the period in the job name
(UXWSDV01_ARCHREPO_CLEANUP.1)
o Click Submit
o Repeat above steps for all jobs created

452379753.doc Page 19 of 20
Appendix K – Add Database To Backup Crawler

1. Sign into the SSDBA Portal with your ISID

SSDBA Portal (Select "dbCrawler")

2. Click on “dbCrawler”

3. Click on New Entry – Wizard and provide requested information

Note: If the database is not registered in REMEDY, you will not be able to add it.

Appendix L – Update RMAN for New Oracle Home (DDBoost)

1. If migrating to a new HOST, Open a Remedy SRM to request a DDBoost storage unit
and backup host (primary & replicated).

2. Update RMAN channel configuration with new ORACLE_HOME and new Storage
Unit/Backup Host (if applicable)

> rman target /

configure channel device type 'sbt_tape' parms
'BLKSIZE=1048576,SBT_LIBRARY=<ORACLE_HOME> /lib/libddobk.so,
ENV=(STORAGE_UNIT=<storage-unit>,BACKUP_HOST=<backup-
host>.merck.com)';

452379753.doc Page 20 of 20