Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Installation Manual
(with ASM+Rawdevice)
Author 정철우
Last Updated
Version 1.0
1 2010.04.01 정철우 최초 작성
11gR2 RAC Installation Manual
Contents
1. 개요 ..................................................................................................................................... 4
6. 참고문서 ............................................................................................................................. 12
7. System Overview.............................................................................................................. 13
- 2 –
11gR2 RAC Installation Manual
8.1.5. NPT Service Configuration .....................................................................................16
8.1.6. User OS enviropment Configuration .......................................................................16
8.2. Grid Infrastructure Software Installation .............................................................17
8.2.1. Cluvfy.sh 수행 .......................................................................................................17
8.2.2. Rootpre.sh(root 유저로 반드시 양 노드 모두 수행) .............................................23
8.2.3. GRID Infrastructure Software Installation................................................................25
8.3. Database Software Installation.............................................................................41
8.4. Check Public/Inter-connect Configuration ...........................................................48
8.5. OCR & VOTE Disk Migration ASM To Rawdevice ...................................................49
- 3 –
11gR2 RAC Installation Manual
1. 개요
10g 또는 11gR1 홖경에서 기존에 RAC 를 운영하던, DBA 들의 11gR2 RAC 에 대한 가지고 있는 불만 중
하나는, 기존에 Storage 영역으로 가장 많이 사용하고 있는 RAWDEVICE 에 대한 미 지원이라고 할 수
있습니다.
(정확히 이야기 하면, Software 상에서는 지원하나, OUI or DBCA 등의 GUI 에선 미지원.)
본 문서는 AIX5L 기반의 장비에서 작성 되었으며, 다른 unix or linux 플랫폼에도 동일한 방법으로 적용
가능 합니다.
2. Hardware Requirements
1.1 Shared Storage Option
RAC 구성시 사전에 결정해야 될 가장 중요한 부분은 Data Storage 입니다.
RAC 에서 구성 가능한 (Shared Access) 방법은 ASM, Shared File System(CFS, NFS), RAWDEVICE 입니다.
그러나 앞서 이야기 한 것처럼 11gR2 부터는 rawdevice 를 GUI 메뉴 상에서 지원하지 않으며, 특히
Grid Install 시에는, 반드시 ASM 또는 Shared File System 으로만 OCR 과 Voting Disk 생성이 가능합니다.
11gR1 까지는 ASM Software 는 Database Software 를 공유하여 사용하였으나, 11gR2 부터는 Oracle
clusterware 와 ASM 이 통합되었습니다.
이는 기존 RAC 홖경에서 ASM 을 Storage 영역으로 사용하더라도, CRS 구성 후 ASM Instance 를 구성해
야 했기 때문에 CRS 구성시 필요한 OCR 과 Voting Disk 는 ASM 영역을 사용할 수 없어, RAWDEVICE 나
Shared File System(CFS or NFS)를 사용해야 했으나, 11gR2 부터는 Grid Infrastructer(이하 Grid) 구성시에
ASM Diskgroup 를 생성할 수 있기 때문에 OCR 과 Voting Disk 도 ASM 영역에 생성할 수 있도록 변경
되었습니다.
즉 Oracle 에서 11gR2 RAC 구성시 권고하는 대로 ASM 을 사용하거나 Shared File System (CFS or NFS)을
사용하게 되면, 더 이상 OS Vendor Cluster 없이 RAC 구성이 가능하다는 의미 이기도 합니다.
그러나 본 문서에서는 RAWDEVICE 를 Data Storage 영역으로 사용할 것이므로, Shraed Access 가능한
- 4 –
11gR2 RAC Installation Manual
RAWDEVICE 생성을 위해 IBM 에서 지원하는 OS Vendor clusterware(HACMP)를 사용할 것 입니다.
*10g R2 부터는 Cluster Ready Service(CRS)가 Oracle Clusterware 로 명칭이 변경되었으나, 본 문서에서는
편의상 CRS 로 명칭
ASM(Automatic Storage Management) - strongly recommended
CFS(cluster file system) - GPFS on IBM platforms
NFS(Network file system) - Not supported on AIX Based Systems
Raw Devices - Upgrades only (OUI, DBCA Not Supported)
Note :
You cannot place ASM files on GPFS.
Oracle does not recommend the use of GPFS for voting disks if HACMP is used.
Disk lock속성을 reserve_policy=no_reserve로 설정해야 양 노드에서 Access 가능.
AIX에서 ASM Disk는 /dev/rhdiskN을 Default Path로 찾음
(The response is either a reserve_lock setting, or a reserve_policy setting. If the attribute is reserve_lock,
then ensure that the setting is reserve_lock = no. If the attribute is reserve_policy, then ensure that the
setting is reserve_policy = no_reserve.)
- 5 –
11gR2 RAC Installation Manual
3. Network Requirements(IP address requirements)
Cache Fusion 에 의한 부하를 최소화시키기 위해 InterConnect 용 NIC 에 최소한 Gigabit Line 을 사용할
것을 권고합니다.
또한 11gR2 RAC 에서는 Interconnect Line 을 Cross-Cable 을 이용하여 양 서버를 Direct 로 연결 할 수
없으며, 반드시 양 노드 중간에 Switch 를 사용하여 Interconnect 를 구성해야 합니다.
GNS 는 기존의 2node RAC 홖경을 위한 개념이라기 보다는, Grid 홖경에서 다수의 Node 를 RAC
Cluster 상에서 add 하거나 Remove 하는 과정을 갂단하게 하는 Grid plug and play (GPnP)를 위한
개념이라고 볼 수 있으며, GNS 를 통해 추가된 node 에 IP 할당을 동적으로 수행할 수 있습니다.
참고로 11gR2 RAC 는 최대 100 대의 노드를 Cluster 홖경으로 구축할 수 있습니다.
- 6 –
11gR2 RAC Installation Manual
본 문서에서는 2Node RAC 를 구축할 것 이므로, GNS 사용을 설정하지 않았습니다.
Virtual IP address 사용
DHCP 사용
GPnP(Grid Plug and Play) 사용시에 구성
SCAN 은 Grid 설치 시 반드시 구성해야 하며, GNS 을 사용하지 않는다면, SCAN NAME 을 아래와
같이 DNS 상에 등록해줘야 합니다.
기본적으로 3개의 Public IP 를 동일한 이름으로 등록해줘야 하며, Public 와 SCAN Vip 의 subset 은
동일하여야 합니다. (최소 1개의 IP 등록)
SCAN 은 Grid 설치 후에 제거는 가능하지만, 11gR2 RAC 에서는 EM 구성시 Scan Listener 를
이용해서 접속하기 때문에 EM 을 사용하기 위해서는 SCAN 을 제거하면 안됩니다.
SCAN 을 제거하게 되면, Client 에서 접속시 기존 RAC 와 동일하게 VIP 를 사용해서 접속하면
됩니다.
- 7 –
11gR2 RAC Installation Manual
- 8 –
11gR2 RAC Installation Manual
ioo -o aio_maxreqs
aio_maxreqs = 65536
AIX 5.3
lsattr -El aio0 -a maxreqs
maxreqs 65536 Maximum number of REQUESTS True
Parameter Value
minperm% 3(default is 20)
maxperm% 90(default is 80)
maxclient% = 90(default is 80)
lru_file_repage 0(default is 1)
strict_maxclient 1(default is 1)
strict_maxperm 0(default is 0)
Example
vmo -p -o minperm%=3
vmo -p -o maxperm%=90
vmo -p -o maxclient%=90
vmo -p -o lru_file_repage=0
vmo -p -o strict_maxclient=1
vmo -p -o strict_maxperm=0
You must restart the system for these changes to take effect.
Item in
Shell Limit Hard limit
limits.conf
- 9 –
11gR2 RAC Installation Manual
Maximum number of processes available to a single user maxuproc 16384
-1
Maximum size of the stack segment of the process stack
(unlimited)
ipqmaxlen 512
rfc1323 1
sb_max 2*65536
tcp_recvspace 65536
tcp_sendspace 65536
udp_recvspace 655360
udp_sendspace 65536
/etc/rc.net
if [ -f /usr/sbin/no ] ; then
/usr/sbin/no -o udp_sendspace=65536
/usr/sbin/no -o udp_recvspace=655360
/usr/sbin/no -o tcp_sendspace=65536
/usr/sbin/no -o tcp_recvspace=65536
/usr/sbin/no -o rfc1323=1
- 10 –
11gR2 RAC Installation Manual
/usr/sbin/no -o sb_max=2*655360
/usr/sbin/no -o ipqmaxlen=512
fi
5. Software Requirements
5.1. Operating system
AIX 5L V5.3 TL 09 SP1 ("5300-09-01") 64 bit kernel (Part Number E10854-01)
AIX 6.1 TL 02 SP5 ("6100-02-05") 64-bit kernel
HACMP 5.4.1
HACMP filesets (cluster.es.*)
rsct.hacmp.rte
rsct.compat.basic.hacmp.rte
rsct.compat.clients.hacmp.rte
GPFS
AIX 6L: gpfs.base 3.2.1.8 or later.
AIX 5L: gpfs.base 3.2.1.8 or later
Pro*COBOL
IBM COBOL for AIX version 3.1
Micro Focus Server Express 5.1
6. 참고문서
# E10814-01 : Oracle Grid Infrastructure Installation Guide, 11gR2 for IBM AIX on POWER Systems (64-Bit)
- 12 –
11gR2 RAC Installation Manual
7. System Overview
7.1. System Architecture.
11gRAC
IBM P590
2Nodes RAC * 1Set
7.2.2. Software
NAME Value Remark
OS Kernel AIX 5.3 TL11
SWAP 30G
/tmp size 4GB
HACMP 5.4
- 13 –
11gR2 RAC Installation Manual
7.3. IP 구성정보
Host Name Type Value
# /usr/es/sbin/cluster/clstat
clstat - HACMP Cluster Status Monitor
-------------------------------------
8. Installation Steps
8.1. Preparing the system
8.1.1. Host file Configuration
/etc/hosts 파일에 각노드의 Pulic IP 주소 및 VIP, Interconnect 주소를 등록합니다.
Scan vip 의 경우 설치 후 제거를 할 예정으로, DNS 가 아닌 hosts 파일에 등록 (양 노드 동일한 ip 설정)
(root 사용자로 양 노드 수행합니다.)
[oracle@goodus1:/oracle]$ cat /etc/hosts
127.0.0.1 loopback localhost # loopback (lo0) name/address
20.20.20.1 goodus1
20.20.20.2 goodus2
# IP For Service
20.20.20.3 goodus1-vip
20.20.20.4 goodus2-vip
# SCAN IP
20.20.20.5 goodus-scan
# Interconnect IP
10.10.10.1 goodus1-ipc
10.10.10.2 goodus2-ipc
- 15 –
11gR2 RAC Installation Manual
8.1.4. SSH Configuration
설치간 양 노드에 oracle 및 grid 유저로 ssh 접속이 가능해야 함.
(grid 설치 단계에서 자동 설정 가능하나, cluster Verify 수행을 위해 사전 구성)
/usr/bin/ssh-keygen -t rsa
/usr/bin/ssh-keygen -t dsa
set -o vi
stty -istrip erase
export ORACLE_BASE=/oracle/db
export GI_HOME=/oracle/grid
export ORACLE_HOME=/oracle/db/11g
export ORACLE_SID=GOODUS1
- 16 –
11gR2 RAC Installation Manual
export NLS_LANG=AMERICAN_AMERICA.UTF8
export AIXTHREAD_SCOPE=S
PATH=$PATH:$ORACLE_HOME/bin:/oracle/dba:$GI_HOME/bin:$ORACLE_HOME/OPatch
- 17 –
11gR2 RAC Installation Manual
Source Destination Connected?
------------------------------ ------------------------------ ----------------
goodus1:en14 goodus2:en14 yes
Result: Node connectivity passed for subnet "20.20.20.0" with node(s) goodus1,goodus2
Interfaces found on subnet "20.20.20.0" that are likely candidates for VIP are:
goodus1 en14:20.20.20.1
goodus2 en14:20.20.20.2
Interfaces found on subnet "10.10.10.0" that are likely candidates for a private interconnect are:
goodus1 en15:10.10.10.1
goodus2 en15:10.10.10.2
- 18 –
11gR2 RAC Installation Manual
Path Node Name Mount point Available Required Comment
---------------- ------------ ------------ ------------ ------------ ------------
/tmp/ goodus1 /tmp 2.5644GB 1GB passed
Result: Free disk space check passed for "goodus1:/tmp/"
- 19 –
11gR2 RAC Installation Manual
goodus2 hard 9223372036854775807 65536 passed
Result: Hard limits check passed for "maximum open file descriptors"
- 20 –
11gR2 RAC Installation Manual
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
goodus1 cluster.license-5.4.1.1-0 cluster.license-5.4.1 passed
goodus2 cluster.license-5.4.1.1-0 cluster.license-5.4.1 passed
Result: Package existence check passed for "cluster.license-5.4.1"
- 21 –
11gR2 RAC Installation Manual
Result: Package existence check passed for "rsct.compat.clients.rte-..."
- 22 –
11gR2 RAC Installation Manual
Check: Liveness for "xntpd"
Node Name Running?
------------------------------------ ------------------------
goodus1 yes
goodus2 yes
Result: Liveness check passed for "xntpd"
Checking NTP daemon's boot time configuration, in file "/etc/rc.tcpip", for slewing option "-x"
- 23 –
11gR2 RAC Installation Manual
Configuring HACMP group services socket for possible use by Oracle.
[root@goodus1:/oracle/INST/grid]$ vi /etc/group
hagsuser:!:205:oragrid:oracle -> oracle, oragrid 유저 추가
rootpre 재수행
[root@goodus1:/oracle/INST/grid]$ sh rootpre.sh
rootpre.sh output will be logged in /tmp/rootpre.out_10-03-18.15:53:42
check_prereq[11]: test: argument expected
Saving the original files in /etc/ora_save_10-03-18.15:53:42....
Copying new kernel extension to /etc....
rootpre.sh[348]: test: argument expected
Loading the kernel extension from /etc
2 번 노드도 동일한 방법으로 수행 ( rootpre 는 1 번노드의 rootpre dir 을 tar 로 묶어서 전송)
정상적으로 수행되면, /opt/ORCLcluster 디렉토리가 생성되며 , 수행시
Checking if group services should be configured....
Nothing to configure. 으로 종료되면, Install 시 node information 단계에서 양 노드가 안보이는 문제 발생함.
HACMP 와 ORACLE CLUSTERWARE 를 연동해주는 Library 생성됨
[oracle@goodus1:/oracle/db]$ ls -l /opt/ORCLcluster/lib
lrwxrwxrwx 1 root system 33 Mar 18 15:53 libskgxn2.so@ -> /opt/ORCLcluster/lib/libskgxnr.so*
-rwxr-xr-x 1 root system 208646 Mar 18 15:53 libskgxnr.so*
- 24 –
11gR2 RAC Installation Manual
8.2.3. GRID Infrastructure Software Installation
11gR2 부터는 Clusterware 가 Grid infrastructure 내에 포함되어 있다.
(oragrid 사용자로 goodus1 에서 수행합니다) .
[oracle@goodus1:/oracle/INST/grid/install]$ ./lsnodes -n
goodus1 1
goodus2 2
Answer 'y' if root has run 'rootpre.sh' so you can proceed with Oracle
installation.
Answer 'n' to abort installation and then ask root to run 'rootpre.sh'.
********************************************************************************
Checking Temp space: must be greater than 190 MB. Actual 3778 MB Passed
Checking swap space: must be greater than 150 MB. Actual 30720 MB Passed
Checking monitor: must be configured to display at least 256 colors
- 25 –
11gR2 RAC Installation Manual
- 26 –
11gR2 RAC Installation Manual
- 27 –
11gR2 RAC Installation Manual
- 28 –
11gR2 RAC Installation Manual
ASM Instance Password 는 오라클에서 요구하는 복잡성을 만족할 것을 권고 (8 자리, 특수문자, 숫자 포함)
- 29 –
11gR2 RAC Installation Manual
- 30 –
11gR2 RAC Installation Manual
- 31 –
11gR2 RAC Installation Manual
- 32 –
11gR2 RAC Installation Manual
- 33 –
11gR2 RAC Installation Manual
goodus1:/root># /oracle/grid/root.sh
Running Oracle 11g root.sh script...
- 36 –
11gR2 RAC Installation Manual
CRS-2672: Attempting to start 'ora.crsd' on 'goodus1'
CRS-2676: Start of 'ora.crsd' on 'goodus1' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'goodus1'
CRS-2676: Start of 'ora.evmd' on 'goodus1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'goodus1'
CRS-2676: Start of 'ora.asm' on 'goodus1' succeeded
CRS-2672: Attempting to start 'ora.DGDATA.dg' on 'goodus1'
CRS-2676: Start of 'ora.DGDATA.dg' on 'goodus1' succeeded
Checking swap space: must be greater than 500 MB. Actual 30464 MB Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /oracle/oraInventory
'UpdateNodeList' was successful.
[root@goodus2:/oracle]$ /oracle/grid/root.sh
Running Oracle 11g root.sh script...
- 37 –
11gR2 RAC Installation Manual
CRS-2672: Attempting to start 'ora.ctssd' on 'goodus2'
CRS-2676: Start of 'ora.ctssd' on 'goodus2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'goodus2'
CRS-2676: Start of 'ora.asm' on 'goodus2' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'goodus2'
CRS-2676: Start of 'ora.crsd' on 'goodus2' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'goodus2'
CRS-2676: Start of 'ora.evmd' on 'goodus2' succeeded
Checking swap space: must be greater than 500 MB. Actual 30464 MB Passed
The inventory pointer is located at /etc/oraInst.loc
The inventory is located at /oracle/oraInventory
'UpdateNodeList' was successful.
- 38 –
11gR2 RAC Installation Manual
설치를 종료합니다.
- 39 –
11gR2 RAC Installation Manual
[oracle@goodus1:/oracle/grid/bin]$ ps -ef | grep d.bin
root 102490 1 6 11:26:07 - 0:00 /oracle/grid/bin/crsd.bin reboot
root 168070 1 3 11:26:08 - 0:00 /oracle/grid/bin/oclskd.bin
oragrid 172222 1 0 11:25:30 - 0:00 /oracle/grid/bin/diskmon.bin -d -f
oragrid 188574 1 0 11:25:55 - 0:00 /oracle/grid/bin/oclskd.bin
oragrid 237576 1 0 11:25:25 - 0:00 /oracle/grid/bin/mdnsd.bin
root 254100 1 0 11:25:49 - 0:00 /oracle/grid/bin/octssd.bin reboot
oragrid 274678 1 2 11:25:27 - 0:00 /oracle/grid/bin/gpnpd.bin
root 282714 1 0 11:25:28 - 0:00 /oracle/grid/bin/cssdmonitor
oragrid 291002 1 0 11:25:24 - 0:00 /oracle/grid/bin/gipcd.bin
root 115108 1 0 11:25:30 - 0:00 /oracle/grid/bin/cssdagent
oragrid 180648 127154 0 11:26:10 pts/1 0:00 grep d.bi
root 229850 1 0 11:25:30 - 0:00 /oracle/grid/bin/orarootagent.bin
oragrid 368946 434436 1 11:26:10 - 0:00 /oracle/grid/bin/evmlogger.bin -o
/oracle/grid/evm/log/evmlogger.info -l /oracle/grid/evm/log/evmlogger.log
oragrid 373232 409880 4 11:25:30 - 0:00 /oracle/grid/bin/ocssd.bin
oragrid 397654 1 0 11:25:24 - 0:00 /oracle/grid/bin/oraagent.bin
root 409880 1 0 11:25:30 - 0:00 /bin/sh /oracle/grid/bin/ocssd
root 426386 1 0 11:25:21 - 0:02 /oracle/grid/bin/ohasd.bin reboot
oragrid 434436 1 13 11:25:50 - 0:00 /oracle/grid/bin/evmd.bin
[oracle@goodus1:/oracle/grid/bin]$ ifconfig -a
en14:
flags=5e080863,c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM
_OFFLOAD(ACTIVE),PSEG,LARGESEND,CHAIN>
inet 20.20.20.1 netmask 0xffffff00 broadcast 20.20.20.15 -> public
inet 20.20.20.3 netmask 0xffffff00 broadcast 20.20.20.15 -> vip
inet 20.20.20.5 netmask 0xffffff00 broadcast 20.20.20.15 -> scan-vip
tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
en15:
flags=5e080863,c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM
_OFFLOAD(ACTIVE),PSEG,LARGESEND,CHAIN>
inet 10.10.10.1 netmask 0xffffff00 broadcast 10.10.10.255 -> private
tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
lo0: flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT>
inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255
inet6 ::1/0
tcp_sendspace 131072 tcp_recvspace 131072 rfc1323 1
goodus1:/root># /oracle/grid/bin/crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.DATA.dg ora....up.type ONLINE ONLINE goodus1
ora....N1.lsnr ora....er.type ONLINE ONLINE goodus1 -> SCAN_LISTENER
ora.asm ora.asm.type ONLINE ONLINE goodus1
ora.eons ora.eons.type ONLINE ONLINE goodus1
ora.gsd ora.gsd.type OFFLINE OFFLINE
ora....SM1.asm application ONLINE ONLINE goodus1
ora.goodus1.gsd application OFFLINE OFFLINE
ora.goodus1.ons application ONLINE ONLINE goodus1
ora.goodus1.vip ora....t1.type ONLINE ONLINE goodus1
ora....SM2.asm application ONLINE ONLINE goodus2
ora.goodus2.gsd application OFFLINE OFFLINE
ora.goodus2.ons application ONLINE ONLINE goodus2
ora.goodus2.vip ora....t1.type ONLINE ONLINE goodus2
ora....network ora....rk.type ONLINE ONLINE goodus1
ora.oc4j ora.oc4j.type OFFLINE OFFLINE
ora.ons ora.ons.type ONLINE ONLINE goodus1
ora.scan1.vip ora....ip.type ONLINE ONLINE goodus1->1 개의 SCAN IP 만 지정함으로 SCAN VIP 도 1 개 생성
- 40 –
11gR2 RAC Installation Manual
8.3. Database Software Installation
(oracle 사용자로 goodus1 에서 수행합니다.)
[oracle@goodus1:/oracle/INST/database]$ id
uid=300(oracle) gid=300(dba) groups=0(system),1(staff),3(sys),4(adm),205(hagsuser)
[oracle@goodus1:/oracle/INST/database]$ export DISPLAY=10.25.235.120:1
[oracle@goodus1:/oracle/INST/database]$ chmod +x run*
[oracle@goodus1:/oracle/INST/database]$ chmod +x ./install/*
[oracle@goodus1:/oracle/INST/database]$ chmod +x ./install/.oui
[oracle@goodus1:/oracle/INST/database]$ ./runInstaller
********************************************************************************
Your platform requires the root user to perform certain pre-installation
OS preparation. The root user should run the shell script 'rootpre.sh' before
you proceed with Oracle installation. rootpre.sh can be found at the top level
of the CD or the stage area.
Answer 'y' if root has run 'rootpre.sh' so you can proceed with Oracle
installation.
Answer 'n' to abort installation and then ask root to run 'rootpre.sh'.
********************************************************************************
Has 'rootpre.sh' been run by root? [y/n] (n)
y
Starting Oracle Universal Installer...
Checking Temp space: must be greater than 190 MB. Actual 3465 MB Passed
Checking swap space: must be greater than 150 MB. Actual 30720 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 65536 Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2010-03-18_04-36-49PM. Please wait ...
- 41 –
11gR2 RAC Installation Manual
- 42 –
11gR2 RAC Installation Manual
반드시 양 노드를 모두 선택
- 43 –
11gR2 RAC Installation Manual
- 44 –
11gR2 RAC Installation Manual
- 45 –
11gR2 RAC Installation Manual
- 46 –
11gR2 RAC Installation Manual
- 47 –
11gR2 RAC Installation Manual
root 사용자로 goodus1 goodus2 순서로 /oracle/db/11g/root.sh 수행 완료 후 OK 버튼을 Click 합니다.
Goodus1:/oracle/db/11g># sh root.sh
Running Oracle 11g root.sh script...
설치 완료
- 49 –
11gR2 RAC Installation Manual
9. Create Database
9.1. DBCA
DBCA를 통해 DATABASE를 생성합니다. (oracle 사용자로 goodus1에서 수행합니다.)
DBCA 상에서는 RAWDEVICE 구성을 지원하지 않음으로, 여기선 최종 단계에서 Create Script만 수행하여,
생성된 Script에서 File Path를 Rawdevice로 수정
[goodus1(oracle):/oracle/db/11g/bin>dbca
- 50 –
11gR2 RAC Installation Manual
- 51 –
11gR2 RAC Installation Manual
- 52 –
11gR2 RAC Installation Manual
- 53 –
11gR2 RAC Installation Manual
- 54 –
11gR2 RAC Installation Manual
- 55 –
11gR2 RAC Installation Manual
- 56 –
11gR2 RAC Installation Manual
- 57 –
11gR2 RAC Installation Manual
- 58 –
11gR2 RAC Installation Manual
좌측 프레임의 Controlfile을 클릭하여 Maximum Datafiles 값을 2048, Maximum Log Members 5로 조정합니다.
- 59 –
11gR2 RAC Installation Manual
- 60 –
11gR2 RAC Installation Manual
좌측 프레임의 Redo Log Groups을 클릭하여 각 Thread 당 Redo Log Group을 5개씩 할당 합니다.
(Thead# 1 Redo Log Group 1~5, Thead#2 Redo Log Group 6~10) Redo Log Goup의 FileSize를 500M로
조정합니다.
- 61 –
11gR2 RAC Installation Manual
9.2. Manual DB Creation
생성된 Script를 수정해서, 전체 datafile, controlfile, spfile의 경로를 할당된 Rawdevice로 변경 후
GOODUS1.sh를 수행합니다. (적색으로 표시된 부분을 ASM에서 RAWDEVICE명으로 수정)
GOODUS1.sh
#!/bin/sh
OLD_UMASK=`umask`
umask 0027
mkdir -p /archive/datafile/GOODUS
mkdir -p /oracle/db/admin/GOODUS/adump
mkdir -p /oracle/db/admin/GOODUS/dpdump
mkdir -p /oracle/db/admin/GOODUS/hdump
mkdir -p /oracle/db/admin/GOODUS/pfile
mkdir -p /oracle/db/cfgtoollogs/dbca/GOODUS
umask ${OLD_UMASK}
ORACLE_SID=GOODUS1; export ORACLE_SID
PATH=$ORACLE_HOME/bin:$PATH; export PATH
echo You should Add this entry in the /etc/oratab: GOODUS:/oracle/db/11g:Y
/oracle/db/11g/bin/sqlplus /nolog @/oracle/db/admin/GOODUS/scripts/GOODUS1.sql
CreateDB.sql
SET VERIFY OFF
connect "SYS"/"&&sysPassword" as SYSDBA
set echo on
spool /oracle/db/admin/GOODUS/scripts/CreateDB.log append
- 62 –
11gR2 RAC Installation Manual
startup nomount pfile="/oracle/db/admin/GOODUS/scripts/init.ora";
CREATE DATABASE "GOODUS"
MAXINSTANCES 32
MAXLOGHISTORY 1
MAXLOGFILES 320
MAXLOGMEMBERS 5
MAXDATAFILES 2048
DATAFILE '/dev/rsystem01_lv' SIZE 1000M AUTOEXTEND OFF
EXTENT MANAGEMENT LOCAL
SYSAUX DATAFILE '/dev/rsysaux_lv' SIZE 1000M AUTOEXTEND OFF
DEFAULT TEMPORARY TABLESPACE TEMP TEMPFILE '/dev/rtemp01_lv' SIZE 2000M AUTOEXTEND OFF
UNDO TABLESPACE "UNDOTBS1" DATAFILE '/dev/rudo01_lv' SIZE 2000M AUTOEXTEND OFF
CHARACTER SET UTF8
NATIONAL CHARACTER SET AL16UTF16
LOGFILE GROUP 1 ('/dev/rcoredo01_lv') SIZE 500M,
GROUP 2 ('/dev/rcoredo02_lv') SIZE 500M,
GROUP 3 ('/dev/rcoredo03_lv') SIZE 500M,
GROUP 4 ('/dev/rcoredo04_lv') SIZE 500M,
GROUP 5 ('/dev/rcoredo05_lv') SIZE 500M
USER SYS IDENTIFIED BY "&&sysPassword" USER SYSTEM IDENTIFIED BY "&&systemPassword";
spool off
CreateDBFiles.sql
SET VERIFY OFF
connect "SYS"/"&&sysPassword" as SYSDBA
set echo on
spool /oracle/db/admin/GOODUS/scripts/CreateDBFiles.log append
CREATE UNDO TABLESPACE "UNDOTBS2" DATAFILE '/dev/rundo02_lv' SIZE 2000M AUTOEXTEND OFF;
CREATE TABLESPACE "USERS" LOGGING DATAFILE '/dev/rusers_lv' SIZE 1000M AUTOEXTEND OFF
EXTENT MANAGEMENT LOCAL SEGMENT SPACE MANAGEMENT AUTO;
ALTER DATABASE DEFAULT TABLESPACE "USERS";
spool off
CreateDBCatalog.sql
SET VERIFY OFF
connect "SYS"/"&&sysPassword" as SYSDBA
set echo on
spool /oracle/db/admin/GOODUS/scripts/CreateDBCatalog.log append
@/oracle/db/11g/rdbms/admin/catalog.sql;
@/oracle/db/11g/rdbms/admin/catblock.sql;
@/oracle/db/11g/rdbms/admin/catproc.sql;
@/oracle/db/11g/rdbms/admin/catoctk.sql;
@/oracle/db/11g/rdbms/admin/owminst.plb;
connect "SYSTEM"/"&&systemPassword"
@/oracle/db/11g/sqlplus/admin/pupbld.sql;
connect "SYSTEM"/"&&systemPassword"
set echo on
spool /oracle/db/admin/GOODUS/scripts/sqlPlusHelp.log append
@/oracle/db/11g/sqlplus/admin/help/hlpbld.sql helpus.sql;
spool off
spool off
CreateClustDBViews.sql
SET VERIFY OFF
connect "SYS"/"&&sysPassword" as SYSDBA
set echo on
- 63 –
11gR2 RAC Installation Manual
spool /oracle/db/admin/GOODUS/scripts/CreateClustDBViews.log append
@/oracle/db/11g/rdbms/admin/catclust.sql;
spool off
lockAccount.sql
SET VERIFY OFF
set echo on
spool /oracle/db/admin/GOODUS/scripts/lockAccount.log append
BEGIN
FOR item IN ( SELECT USERNAME FROM DBA_USERS WHERE ACCOUNT_STATUS IN ('OPEN', 'LOCKED',
'EXPIRED') AND USERNAME NOT IN (
'SYS','SYSTEM') )
LOOP
dbms_output.put_line('Locking and Expiring: ' || item.USERNAME);
execute immediate 'alter user ' ||
sys.dbms_assert.enquote_name(
sys.dbms_assert.schema_name(
item.USERNAME),false) || ' password expire account lock' ;
END LOOP;
END;
/
spool off
postDBCreation.sql
SET VERIFY OFF
connect "SYS"/"&&sysPassword" as SYSDBA
set echo on
spool /oracle/db/admin/GOODUS/scripts/postDBCreation.log append
execute DBMS_AUTO_TASK_ADMIN.disable();
select 'utl_recomp_begin: ' || to_char(sysdate, 'HH:MI:SS') from dual;
execute utl_recomp.recomp_serial();
select 'utl_recomp_end: ' || to_char(sysdate, 'HH:MI:SS') from dual;
select group# from v$log where group# =6;
select group# from v$log where group# =7;
select group# from v$log where group# =8;
select group# from v$log where group# =9;
select group# from v$log where group# =10;
ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 6 ('/dev/rcoredo06_lv') SIZE 500M,
GROUP 7 ('/dev/rcoredo07_lv') SIZE 500M,
GROUP 8 ('/dev/rcoredo08_lv') SIZE 500M,
GROUP 9 ('/dev/rcoredo09_lv') SIZE 500M,
GROUP 10 ('/dev/rcoredo10_lv') SIZE 500M;
ALTER DATABASE ENABLE PUBLIC THREAD 2;
host echo cluster_database=true >>/oracle/db/admin/GOODUS/scripts/init.ora;
host echo remote_listener=goodus-scan:1521>>/oracle/db/admin/GOODUS/scripts/init.ora;
connect "SYS"/"&&sysPassword" as SYSDBA
set echo on
create spfile='/dev/rspfile_lv' FROM pfile='/oracle/db/admin/GOODUS/scripts/init.ora';
shutdown immediate;
host /oracle/db/11g/bin/srvctl enable database -d GOODUS;
host /oracle/db/11g/bin/srvctl start database -d GOODUS;
connect "SYS"/"&&sysPassword" as SYSDBA
#host /oracle/db/11g/bin/emca -config dbcontrol db -silent -cluster -DB_UNIQUE_NAME GOODUS -PORT 1521 -
EM_HOME /oracle/db/11g -LISTENER LISTENER -CLUSTER_NAME goodus-cluster -SERVICE_NAME GOODUS -SID
GOODUS -ORACLE_HOME /oracle/db/11g -HOST goodus1 -LISTENER_OH /oracle/grid -LOG_FILE
/oracle/db/admin/GOODUS/scripts/emConfig.log;
spool off
- 64 –
11gR2 RAC Installation Manual
exit;
Init.ora
##############################################################################
# Copyright (c) 1991, 2001, 2002 by Oracle Corporation
##############################################################################
###########################################
# Cache and I/O
###########################################
db_block_size=8192
###########################################
# Cluster Database
###########################################
#cluster_database=true
#remote_listener=goodus-scan:1521
###########################################
# Cursors and Library Cache
###########################################
open_cursors=300
###########################################
# Database Identification
###########################################
db_domain=""
db_name=GOODUS
###########################################
# File Configuration
###########################################
control_files=("/dev/rcontrol01_lv", "/dev/rcontrol02_lv")
###########################################
# Miscellaneous
###########################################
compatible=11.2.0.0.0
diagnostic_dest=/oracle/db
memory_target=52935262208
###########################################
# Processes and Sessions
###########################################
processes=1000
sessions=1105
###########################################
# Security and Auditing
###########################################
audit_file_dest=/oracle/db/admin/GOODUS/adump
audit_trail=db
remote_login_passwordfile=exclusive
###########################################
# Shared Server
###########################################
dispatchers="(PROTOCOL=TCP) (SERVICE=GOODUSXDB)"
GOODUS1.instance_number=1
GOODUS2.instance_number=2
GOODUS2.thread=2
GOODUS1.thread=1
GOODUS1.undo_tablespace=UNDOTBS1
GOODUS2.undo_tablespace=UNDOTBS2
- 65 –
11gR2 RAC Installation Manual
9.3. DATABASE Status
COMP_NAME VERSION STATUS
---------------------------------------- ------------------------------ ---------------------------------
Oracle Enterprise Manager 11.2.0.1.0 VALID
Oracle Workspace Manager 11.2.0.1.0 VALID
Oracle Database Catalog Views 11.2.0.1.0 VALID
Oracle Database Packages and Types 11.2.0.1.0 VALID
Oracle Real Application Clusters 11.2.0.1.0 VALID
9.5.3. LISTENER.ORA
11gR2 부터는 listener.ora 는 $GRID_HOME/network/admin 에 위치하며, tnsnames.ora 는
$ORACLE_HOME/network/admin 의 것을 사용함. Listener Resource 는 grid 유저를 owner 로 함.
- 66 –
11gR2 RAC Installation Manual
[oracle@goodus1:/oracle/grid/network/admin]$ cat listener.ora
LISTENER=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER))))
ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER=ON
Tnsnames.ora
GOODUS =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = 20.20.20.3)(PORT = 1526))
(ADDRESS = (PROTOCOL = TCP)(HOST = 20.20.20.4)(PORT = 1526))
(FAIL_OVER=ON)
(CONNECT_DATA =
(SERVER = DEDICATED)
(SERVICE_NAME = GOODUS)
)
)
- 67 –
11gR2 RAC Installation Manual
SQL> alter system set cluster_database=true scope=spfile sid='*';
SQL> shutdown immediate;
9.8. DB Parameter
show parameter
- 68 –
11gR2 RAC Installation Manual
audit_sys_operations boolean FALSE
audit_syslog_level string
audit_trail string DB
background_core_dump string partial
background_dump_dest string /oracle/db/diag/rdbms/goodus/GOODUS1/trace
backup_tape_io_slaves boolean FALSE
bitmap_merge_area_size integer 1048576
blank_trimming boolean FALSE
buffer_pool_keep string
buffer_pool_recycle string
cell_offload_compaction string ADAPTIVE
cell_offload_decryption boolean TRUE
cell_offload_parameters string
cell_offload_plan_display string AUTO
cell_offload_processing boolean TRUE
cell_partition_large_extents string DEFAULT
circuits integer
client_result_cache_lag big integer 3000
client_result_cache_size big integer 0
cluster_database boolean TRUE
cluster_database_instances integer 2
cluster_interconnects string
commit_logging string
commit_point_strength integer 1
commit_wait string
commit_write string
compatible string 11.2.0.0.0
control_file_record_keep_time integer 7
control_files string /dev/rk2coredo39_lv, /dev/rk2coredo40_lv
control_management_pack_access string DIAGNOSTIC+TUNING
core_dump_dest string /oracle/db/diag/rdbms/goodus/GOODUS1/cdump
cpu_count integer 8
create_bitmap_area_size integer 8388608
create_stored_outlines string
cursor_sharing string EXACT
cursor_space_for_time boolean FALSE
db_16k_cache_size big integer 0
db_2k_cache_size big integer 0
db_32k_cache_size big integer 0
db_4k_cache_size big integer 0
db_8k_cache_size big integer 0
db_block_buffers integer 0
db_block_checking string FALSE
db_block_checksum string TYPICAL
db_block_size integer 8192
db_cache_advice string ON
db_cache_size big integer 10G
db_create_file_dest string
db_create_online_log_dest_1 string
db_create_online_log_dest_2 string
db_create_online_log_dest_3 string
db_create_online_log_dest_4 string
db_create_online_log_dest_5 string
db_domain string
db_file_multiblock_read_count integer 16
db_file_name_convert string
db_files integer 1000
db_flash_cache_file string
db_flash_cache_size big integer 0
- 69 –
11gR2 RAC Installation Manual
db_flashback_retention_target integer 1440
db_keep_cache_size big integer 0
db_lost_write_protect string NONE
db_name string GOODUS
db_recovery_file_dest string
db_recovery_file_dest_size big integer 0
db_recycle_cache_size big integer 0
db_securefile string PERMITTED
db_ultra_safe string OFF
db_unique_name string GOODUS
db_writer_processes integer 1
dbwr_io_slaves integer 0
ddl_lock_timeout integer 0
deferred_segment_creation boolean TRUE
dg_broker_config_file1 string /oracle/db/11g/dbs/dr1GOODUS.dat
dg_broker_config_file2 string /oracle/db/11g/dbs/dr2GOODUS.dat
dg_broker_start boolean FALSE
diagnostic_dest string /oracle/db
disk_asynch_io boolean TRUE
dispatchers string
distributed_lock_timeout integer 60
dml_locks integer 33120
dst_upgrade_insert_conv boolean TRUE
enable_ddl_logging boolean FALSE
event string
fal_client string
fal_server string
fast_start_io_target integer 0
fast_start_mttr_target integer 240
fast_start_parallel_rollback string LOW
file_mapping boolean FALSE
fileio_network_adapters string
filesystemio_options string asynch
fixed_date string
gcs_server_processes integer 2
global_context_pool_size string
global_names boolean FALSE
global_txn_processes integer 1
hash_area_size integer 131072
hi_shared_memory_address integer 0
hs_autoregister boolean TRUE
ifile file
instance_groups string
instance_name string GOODUS1
instance_number integer 1
instance_type string RDBMS
java_jit_enabled boolean TRUE
java_max_sessionspace_size integer 0
java_pool_size big integer 320M
java_soft_sessionspace_limit integer 0
job_queue_processes integer 1
large_pool_size big integer 320M
ldap_directory_access string NONE
ldap_directory_sysauth string no
license_max_sessions integer 0
license_max_users integer 0
license_sessions_warning integer 0
listener_networks string
local_listener string GOODUS
- 70 –
11gR2 RAC Installation Manual
lock_name_space string
lock_sga boolean FALSE
log_archive_config string
log_archive_dest string
log_archive_dest_1 string location=/arch1
log_archive_dest_state_1 string enable
log_archive_duplex_dest string
log_archive_format string arch_%t_%r_%s.arc
log_archive_local_first boolean TRUE
log_archive_max_processes integer 4
log_archive_min_succeed_dest integer 1
log_archive_start boolean FALSE
log_archive_trace integer 0
log_buffer integer 22716416
log_checkpoint_interval integer 0
log_checkpoint_timeout integer 1800
log_checkpoints_to_alert boolean FALSE
log_file_name_convert string
max_dispatchers integer
max_dump_file_size string unlimited
max_enabled_roles integer 150
max_shared_servers integer
memory_max_target big integer 0
memory_target big integer 0
nls_calendar string
nls_comp string BINARY
nls_currency string
nls_date_format string
nls_date_language string
nls_dual_currency string
nls_iso_currency string
nls_language string AMERICAN
nls_length_semantics string BYTE
nls_nchar_conv_excp string FALSE
nls_numeric_characters string
nls_sort string
nls_territory string AMERICA
nls_time_format string
nls_time_tz_format string
nls_timestamp_format string
nls_timestamp_tz_format string
object_cache_max_size_percent integer 10
object_cache_optimal_size integer 102400
olap_page_pool_size big integer 0
open_cursors integer 3500
open_links integer 4
open_links_per_instance integer 4
optimizer_capture_sql_plan_baselines boolean FALSE
optimizer_dynamic_sampling integer 2
optimizer_features_enable string 11.2.0.1
optimizer_index_caching integer 0
optimizer_index_cost_adj integer 100
optimizer_mode string ALL_ROWS
optimizer_secure_view_merging boolean TRUE
optimizer_use_invisible_indexes boolean FALSE
optimizer_use_pending_statistics boolean FALSE
optimizer_use_sql_plan_baselines boolean TRUE
os_authent_prefix string ops$
os_roles boolean FALSE
- 71 –
11gR2 RAC Installation Manual
parallel_adaptive_multi_user boolean TRUE
parallel_automatic_tuning boolean FALSE
parallel_degree_limit string CPU
parallel_degree_policy string MANUAL
parallel_execution_message_size integer 16384
parallel_force_local boolean FALSE
parallel_instance_group string GOODUS1
parallel_io_cap_enabled boolean FALSE
parallel_max_servers integer 160
parallel_min_percent integer 0
parallel_min_servers integer 0
parallel_min_time_threshold string AUTO
parallel_server boolean TRUE
parallel_server_instances integer 2
parallel_servers_target integer 64
parallel_threads_per_cpu integer 2
permit_92_wrap_format boolean TRUE
pga_aggregate_target big integer 5G
plscope_settings string IDENTIFIERS:NONE
plsql_ccflags string
plsql_code_type string INTERPRETED
plsql_debug boolean FALSE
plsql_optimize_level integer 2
plsql_v2_compatibility boolean FALSE
plsql_warnings string DISABLE:ALL
pre_page_sga boolean FALSE
processes integer 5000
query_rewrite_enabled string TRUE
query_rewrite_integrity string enforced
rdbms_server_dn string
read_only_open_delayed boolean FALSE
recovery_parallelism integer 0
recyclebin string on
redo_transport_user string
remote_dependencies_mode string TIMESTAMP
remote_listener string
remote_login_passwordfile string EXCLUSIVE
remote_os_authent boolean FALSE
remote_os_roles boolean FALSE
replication_dependency_tracking boolean TRUE
resource_limit boolean FALSE
resource_manager_cpu_allocation integer 8
resource_manager_plan string
result_cache_max_result integer 5
result_cache_max_size big integer 52448K
result_cache_mode string MANUAL
result_cache_remote_expiration integer 0
resumable_timeout integer 0
rollback_segments string
sec_case_sensitive_logon boolean TRUE
sec_max_failed_login_attempts integer 10
sec_protocol_error_further_action string CONTINUE
sec_protocol_error_trace_action string TRACE
sec_return_server_release_banner boolean FALSE
serial_reuse string disable
service_names string GOODUS
session_cached_cursors integer 500
session_max_open_files integer 10
sessions integer 7528
- 72 –
11gR2 RAC Installation Manual
sga_max_size big integer 16096M
sga_target big integer 0
shadow_core_dump string partial
shared_memory_address integer 0
shared_pool_reserved_size big integer 256M
shared_pool_size big integer 5G
shared_server_sessions integer
shared_servers integer 0
skip_unusable_indexes boolean TRUE
smtp_out_server string
sort_area_retained_size integer 0
sort_area_size integer 65536
spfile string /dev/rk2coredo38_lv
sql92_security boolean FALSE
sql_trace boolean FALSE
sqltune_category string DEFAULT
standby_archive_dest string ?/dbs/arch
standby_file_management string MANUAL
star_transformation_enabled string FALSE
statistics_level string TYPICAL
streams_pool_size big integer 0
tape_asynch_io boolean TRUE
thread integer 1
timed_os_statistics integer 0
timed_statistics boolean TRUE
trace_enabled boolean TRUE
tracefile_identifier string
transactions integer 8280
transactions_per_rollback_segment integer 5
undo_management string AUTO
undo_retention integer 21600
undo_tablespace string UNDOTBS1
use_indirect_data_buffers boolean FALSE
use_sigio boolean TRUE
user_dump_dest string /oracle/db/diag/rdbms/goodus/GOODUS1/trace
utl_file_dir string
workarea_size_policy string AUTO
xml_db_events string enable
10. 마치며
이상으로, 11gR2 RAC 를 RAWDEVICE 를 이용하여 구성하여 보았습니다.
11g R2 RAC 에 비록 GNS, GPnP, SCAN, ACFS 등 여러 개념이 새로 나왔지만, 특히나 이해가 어려운 부분이 네트
웍 환경 설정에 관한 부분 인 듯 합니다. SCAN 을 이용하여 모든 CLIENT 에서 접속을 단순화 하고 서버측에서
LOAD BALANCING 을 하겠다는 것은 일견 편리한 듯 보이지만, 국내에서 주로 사용하는 최대 4 노드 이하의 RAC
환경에서 크게 이점이 있는 부분인지는 고민이 필요할 듯 합니다.
이 문서상에서 최대한 기존에 사용 중이던, 10g RAC 환경과 유사하게 scan 과 gns 를 제거하고, rawdevice 로 환경
을 구성하였습니다.
이 문서에선 언급되지 않았지만, CFS 를 이용하여 OCR 과 VOTE 를 생성하고, 차후 RAWDEVICE 로 전환하는 것도
크게 다르지 않기 때문에 기존과 같이 RAWDEVICE 로 11gR2 RAC 를 구축하려는 분들께 조금이나마 도움이 되길
바랍니다.
- 73 –