Installing Oracle Database 11 g on HP-UNIX V3i 64 by Using Real Application Clusters (RAC) and Automated Storage Management (ASM)

Purpose This document describes how to install Oracle Database 10 g on a Solaris x86-64 bit twonode cluster by using Real Application Clusters (RAC) and Automatic Storage Management (ASM). This document assumes installation of Oracle Database 10 g Enterprise Edition. Overview The installation has the following phases. 1. Pre-requisites 2. Verifying Oracle Clusterware Requirements with CVU 3. Installing Oracle Clusterware with OUI 4. Installing Oracle Clusterware patch 5. Installing Oracle ASM software files 6. Installing Oracle ASM patch 7. Installing Oracle database files 8. Installing Oracle database patch 9. Configuring ASM 10. Creating database using dbca 11. Oracle net configuration 12. RAC Commands 1. PRE-REQUISITES SYSTEM REQUIREMENTS Check the Physical Memory Size # /usr/contrib/bin/machinfo | grep -i Memory (Minimum requirement of RAM 1GB.) Check the SWAP space # bdf (Minimum requirement of SWAP 1GB.) Check the Temp Space. # bdf /tmp (Minimum requirement of temp space 650MB.) NETWORK REQUIREMENTS Must require 3 IP addresses Public IP Address : Must reside on the same port on all the RAC nodes. Virtual IP Address Private IP Address : Must reside on the same port on all the RAC nodes.

SOFTWARE REQUIREMENTS Compiler Requirements HP C/ANSI C Compiler (B11.11.16) HP ANSI C compiler (B.11.11.16) - June (AR0606) release HP aC++ Compiler (A.03.73) C++ (aCC) compiler (A.03.70) - June 2006 (AR0606) release Patch Requirement PHKL_35900: 11.31 evacd performance, kvaddr leak panic PHKL_36248: 11.31 esctl cumulative patch PHKL_36249: 11.31 esdisk cumulative patch PHKL_35936: 11.31 call to read(2) or write(2) may incorrectly return -1 PHKL_38038: ABORT CORRUPTION HANG OTHER PANIC PHSS_37959: 11.31 Libcl patch Following patches are required for OCI, C++ Call Interface, Pro C/C++ HP C/aC++ B.11.31.01 (Swlist Bundle - C.11.31.01) February 2007 for HP C/ANSI C Compiler HP C/aC++ A.03.74 (Swlist Bundle - C.11.31.01) February 2007 for HP aC++ Compiler
HP C/aC++ A.06.12 (Swlist Bundle - C.11.31.01) February 2007

http://itresourcecenter.hp.com # /usr/sbin/swlist -l patch | grep PHKL_35900:11.31 # /usr/sbin/swlist -l patch | more Additional Requirements gcc 3.4.5 or later JDK 5.0 KERNEL PARAMETERS ksi_alloc_max executable_stack max_thread_proc maxdsiz maxdsiz_64bit maxssiz maxssiz_64bit maxuprc msgmap msgmni msgseg msgtql ncsize `nfile nflocks ninode nkthread nproc semmni semmns (nproc*8) 0 1024 1073741824 (1 GB) 2147483648 (2 GB) 134217728 (128 MB) 1073741824 (1 GB) ((nproc*9)/10) (msgtql+2) (nproc) 32767 (nproc) (ninode+1024) (15*nproc+2048) (nproc) (8*nproc+2048) (((nproc*7)/4)+16) 4096 (nproc) (semmni*2) 32768 0 1024 1073741824 2147483648 134217728 1073741824 3687 4098 4096 32767 4096 36840 63488 4096 34816 7184 4096 4096 8192

semmnu semvmx shmmax shmmni shmseg vps_ceiling

(nproc-4) 32767 40% of RAM 4096 512 64

4092 32767 4096 64 512

MAKE THE HOSTFILE ENTRIES ( ON BOTH THE RAC NODES ) # vi /etc/hosts Make the following entries ################## Public Interfaces ################### <RAC Node 1 Public IP Address> <RAC Node 2 Public IP Address> <RAC Node 1 Hostname> <RAC Node 2 Hostname>

################## Private Interfaces ################### <RAC Node 1 Private IP Address> <RAC Node 2 Private IP Address> <RAC Node 1 Hostname>-priv <RAC Node 2 Hostname>-priv

################## Virtual Interfaces ################### <RAC Node 1 Virtual IP Address> <RAC Node 2 Virtual IP Address> <RAC Node 1 Hostname>-vip <RAC Node 2 Hostname>-vip

Test the entries by pinging the hostnames for public and private interfaces from both the nodes. It must able to ping. # # # # # # # # ping ping ping ping ping ping ping ping <RAC <RAC <RAC <RAC <RAC <RAC <RAC <RAC Node1 Node1 Node1 Node1 Node2 Node2 Node2 Node2 Hostname> Public IP Address> Hostname>-priv Private IP Address> Hostname> Public IP Address> Hostname>-priv Private IP Address>

CONFIGURING PASSWORD-LESS SSH Check If the .ssh is present in the users home directory or not. If not then create the same. # # # # cd< Space > ( This will take you to the home directory of the user. ) mkdir .ssh chmod R 640 to .ssh cd .ssh

#ssh-keygen t dsa To Accept the default options press enter. This will the generate the DSA key and store in the file id_dsa.pub file under the .ssh directory. #ssh-keygen t rsa To Accept the default options press enter. This will the generate the DSA key and store in

the file id_rsa.pub file under the .ssh directory. Repeat the above steps on both the RAC nodes. Perform the following steps on node1 # # # # # # touch authorized_keys ssh <RAC node1 hostname> cat <Home Directory>/.ssh/id_dsa.pub >> authorized_keys ssh <RAC node1 hostname> cat <Home Directory>/.ssh/id_rsa.pub >> authorized_keys ssh <RAC node2 hostname> cat <Home Directory>/.ssh/id_dsa.pub >> authorized_keys ssh <RAC node2 hostname> cat <Home Directory>/.ssh/id_rsa.pub >> authorized_keys scp authorized_keys rac2: <Home Directory>/.ssh

Test the configuration from both the nodes by trying to do ssh to the other. It will not ask for password. CREATE SYMBOLIC LINKS If X11MotifDevKit.MOTIF21-PRG package is not installed then create the following symbolic links. # # # # # # # # # ln ln ln ln ln ln ln ln ln -s -s -s -s -s -s -s -s -s libX11.3 libX11.sl libXIE.2 libXIE.sl libXext.3 libXext.sl libXhp11.3 libXhp11.sl libXi.3 libXi.sl libXm.4 libXm.sl libXp.2 libXp.sl libXt.3 libXt.sl libXtst.2 libXtst.sl

CREATING REQUIRED OS USER & GROUPS FOR DB Create the following groups Oracle software installation group ( oinstall ) /usr/sbin/groupadd g 301 oinstall Oracle DBA group /usr/sbin/groupadd g 302 dba Create oracle software owner user account /usr/sbin/useradd u 201 g 301 G 302 d /home/oracle m oracle Set the password for oracle user passwd oracle CREATING MOUNT POINTS Create the desired mount points for ORACLE_HOME, ASM_HOME, CRS_HOME MAKE THE PROFILE ENTRIES # cd <space> # vi .profle Make the following entries

CRS_HOME=< Path to the folder where oracle clusterware software files will reside. > ORACLE_HOME=< Path to the folder where oracle database software files will reside. >

RAW DISK IDENTIFICATION

Detect the raw disks and create soft links to configure batter identification of the disks.
# ioscan -m dsf Execute the above command on both the nodes and check the disk details on both the nodes, they must be same on both. Then create soft links for batter identification of the disks. While configuring ASM provide the rad disk path in the ASM_DISKTRACKING parameter. Path : /dev/rdsk/ 2. VERIFYING ORACLE CLUSTERWARE REQUIREMENTS WITH CVU Check the node reachability using the following command $ ./runcluvfy.sh comp nodereach -n node1,node2 -verbose Check the node connectivity $cluvfy comp nodecon -n node_list -i interface_list [-verbose] $cluvfy comp nodecon n node1,node2 I eth0,eth1 verbose Check the equivelence between the nodes using the following command $ ./runcluvfy.sh stage -pre crsinst -n node1,node2 3. INSTALLING ORACLE CLUSTEERWARE WITH OUI Initialize the devices that you want to use for Oracle Cluster Registry (OCR) and Voting Disk (VD). Run the installer from the Oracle Cluster Ready Service (from the location where the Clusterware binaries are stored) # ./runInstaller Welcome

1. Click on Next
Specify Inventory Directory and Credentials 1. The defaults should be correct

2. Click on Next
Specify Home Details 1. Name: OraCRS_Home 2. Path: 3. <CRS Mount point name>/crs

Product-Specific Prerequisite Checks 1. Correct any problems found before proceeding.

2. Click on Next
Specify Cluster Configuration 1. Enter the cluster name (or accept the default of "crs"); Specify Network Interface Usage - Specify the Interface Type (public, private, or "do no use") for each interface Specify Oracle Cluster Registry (OCR) Location Choose Normal Redundancy and enter the full pathname of the OCR file. Specify Voting Disk Location Choose Normal Redundancy and enter the full pathname of the voting disk file. Summary

1. Click on Install
Execute Configuration Scripts 1. Execute the scripts as root on each node, one at a time, starting with the installation node.

2. Do not run the scripts simultaneously. Wait for one to finish before
starting another.

3. Click on OK to dismiss the window when done.

End of Clusterware Installation Verify that the installation succeeded by running olsnodes from the $ORACLE_BASE/crs/bin directory. for example: $ /ora_crs/crs/bin/olsnodes This will list the name of the RAC nodes. 4. INSTALLING ORACLE CLUSTERWARE PATCH As Oracle OS user go to the location where the patch is present. Go to Disk1 directory and execute ./runInstaller #./runInstaller Welcome

1. Click on Next
Specify Home details 1. The defaults should be correct

2. Click on Next
Specify Hardware Cluster installation node

1. Select both the nodes (try5ieccmdb1, try5ieccmdb2)

2. Click on Next
Summary 1. Click on Install End of Installation 1. Execute the scripts as root on each node, one at a time, starting with the installation node. 2. Do not run the scripts simultaneously. Wait for one to finish before starting another.

3. Click on EXIT then on YES to dismiss the window when done.

5. INSTALLING ORACLE ASM SOFTWARE FILES Go to the location where the database binaries are located and execute ./runInstaller

# export ORACLE_HOME=<ASM HOME location> # ./runInstaller Welcome

1. Click on Next
Select Installation Type

2. Select Enterprise Edition 3. Click on Next

Specify Home Details

1. Name: OraASM10g_home1 2. Path:

<Oracle ASM BASE>/ASM

3. Click on Next
Specify Hardware Cluster Installation Mode

1. Select Cluster Installation 2. Click on Select All 3. Click on Next

Product-specific Prerequisite Checks 1. If you've been following the steps in this guide, all the checks should pass without difficulty. If one or more checks fail, correct

the problem before proceeding.

2. Click on Next
Select Configuration Option

1. Select Install Database Software only 2. Click on Next

Summary 1. A summary of the products being installed is presented.

2. Click on Install.
Execute Configuration Scripts

1. At the end of the installation, a pop up window will appear

indicating scripts that need to be run as root. Login as root and run the indicated scripts.

2. Click on OK when finished.

End of Installation

Make note of the URLs presented in the summary, and click on Exit when ready. 6. INSTALLING ORACLE ASM PATCH As Oracle OS user go to the location where the patch is present. Go to Disk1 directory and execute ./runInstaller #./runInstaller Welcome

2. Click on Next
Specify Home details 3. The defaults should be correct

4. Click on Next
Specify Hardware Cluster installation node 3. Select both the nodes

4. Click on Next
Summary 2. Click on Install End of Installation 4. Execute the scripts as root on each node, one at a time, starting with the installation node.

5. Do not run the scripts simultaneously. Wait for one to finish before starting another. Click on EXIT then on YES to dismiss the window when done. 7. INSTALLING ORACLE DATABASE FILES Go to the location where the database binaries are located and execute ./runInstaller # export ORACLE_HOME=<DATABASE HOME location> # ./runInstaller Welcome

4. Click on Next
Select Installation Type

5. Select Enterprise Edition 6. Click on Next

Specify Home Details

4. Name: OraDB10g_home1 5. Path:

<Oracle DATABASE BASE>/ASM

6. Click on Next
Specify Hardware Cluster Installation Mode

4. Select Cluster Installation 5. Click on Select All 6. Click on Next

Product-specific Prerequisite Checks 3. If you've been following the steps in this guide, all the checks should pass without difficulty. If one or more checks fail, correct the problem before proceeding.

4. Click on Next
Select Configuration Option

3. Select Install Database Software only 4. Click on Next

Summary 3. A summary of the products being installed is presented.

4. Click on Install.

Execute Configuration Scripts

3. At the end of the installation, a pop up window will appear

indicating scripts that need to be run as root. Login as root and run the indicated scripts.

4. Click on OK when finished.

End of Installation

Make note of the URLs presented in the summary, and click on Exit when ready. 8. INSTALLING ORACLE DATABASE PATCH As Oracle OS user go to the location where the patch is present. Go to Disk1 directory and execute ./runInstaller #./runInstaller Welcome

3. Click on Next
Specify Home details 5. The defaults should be correct

6. Click on Next
Specify Hardware Cluster installation node 5. Select both the nodes

6. Click on Next
Summary 3. Click on Install End of Installation 6. Execute the scripts as root on each node, one at a time, starting with the installation node. 7. Do not run the scripts simultaneously. Wait for one to finish before starting another. Click on EXIT then on YES to dismiss the window when done.

9. CONFIGURING ASM
#export ORACLE_HOME=<ASM Home Location> # dbca Welcome 2. Choose Real Application Cluster database

3. Click on Next
Select the operation you want to perform 3. Configure Automatic Storage Management

4. Click on Next
Select the node for which you want to manage ASM diskgroups 4. Choose both the nodes 5. Click next In the next screen 3. Provide the sys password

4. Click on Next
In the popup window click OK then click Next ASM Diskgroup page

1. Click on Create New 2. Provide the Diskgroup name, redundancy(external), checkmark the two disk from the show candidate list. 3. Click on OK. 4. Similarly create all required diskgroups as per the disk details provided below.

10.CREATING DATABASE WITH USING DBCA

#export ORACLE_HOME=<ORACLE DATABASE HOME Location> # dbca Welcome 4. Choose Real Application Cluster database

Then click next and in the popup window click NO.

5. Click on Next
Select the operation you want to perform 5. Create a database

6. Click on Next

Select the node for which you want to manage ASM diskgroups 6. Choose both the nodes 7. Click next

In the next screen 5. Choose general purpose

6. Click on Next
Provide the global database name as click Next In the next screen Disable the Configure Database with Enterprise Manager option and click Next In the next screen provide the Password and click next. In the select Storage Mechanism page select Automatic Storage Management and click Next. In the ASM Credentials page provide the SYS password for ASM instance and click OK. Choose both the data diskgroups and then click Next. In the Specify location for datafile page choose Oracle Managed Files option then click on Browse. In the Available Diskgroup page choose diskgroup and click OK. Then click Next. Choose recovery options for Database 1. Under the option specify Flash Recovery area click the checkbox and untikc the same. 2. Select the checkbox enable Archiving and click on Edit Archive Mode Parameters 3. Set the Archive Log File Format to %t%s %r.dbf 4. Set the Archive Log Destinations. 5. Click OK. In the Database Content screen click Next. In the Database Service page click Next In the Initilization Paramater screen 1. Set PGA and SGA 2. In the sizing tab set Processes

3. Click on Next. 4. Choose the Show Advance Parameters and set the value of the parameters as required. In the Database Storage page Select the Redo Log Groups option in the left pane. 1. Set the size of the Redolog Files, data files and control files as required and click on next. Click on ok.

11.ORACLE NET CONFIGURATION # export ORACLE_HOME=<ORACLE DATABASE HOME path> #netca Select the type of Oracle Net Service configuration 1. Select cluster configuration

2. Click on Next
Select nodes to configure 1. Select all the nodes

2. Click on Next
Choose the configuration you like to do 1. Select Listener Configuration

2. Click on Next
Select what you want to do 1. Select Add 2. Click on Next In the next page select TCP and click next. In the next page provide the Listener Name (LISTENER) and click Next. In the next page choose NO and click next.