Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
0: Grid Infrastructure
Alex Gorbachev 29 August, 2012
Agenda
Place of Grid Infrastructure in Oracle RAC Node membership and evictions Clusterware architecture & startup sequence Resources Management and troubleshooting
2009/2010 Pythian
Agenda
th Th
High
Need to memorize
le
ss
Low Shallow
Understanding
2009/2010 Pythian
In-depth
Query/DML/DDL
SERVER
INSTANCE Memory (SGA, PGA) Processes (PMON, SMON, LGWR and etc. + multiple shadow processes)
Read/write
Datales Controlles redo logs ashback logs, change tracking and etc...
Database
2012 Pythian
SERVER
INSTANCE
Database
2012 Pythian
SERVER 1
INSTANCE 1
Database
2012 Pythian
SERVER 1
SERVER 2
INSTANCE 1
INSTANCE 2
Database
2012 Pythian
SERVER 1
SERVER 2
SERVER 3
INSTANCE 1
INSTANCE 2
INSTANCE 3
Database
2012 Pythian
SERVER 1
SERVER 2
SERVER 3
INSTANCE 1
INSTANCE 2
INSTANCE 3
Database
2012 Pythian
SERVER 1
SERVER 2
SERVER 3
INSTANCE 1
INSTANCE 2
INSTANCE 3
Database
2012 Pythian
2012 Pythian
VIP Listener
Service
VIP Listener
Service
VIP Listener
Service
Instance ASM
Instance
Grid Infrastr.
Grid Infrastr.
Voting disk
2009/2010 Pythian
VIP Listener
Service
VIP Listener
Service
VIP Listener
Service
Instance ASM
Instance
Grid Infrastr.
Grid Infrastr.
Voting disk
2009/2010 Pythian
OS
OS
Clusterware
Clusterware
CSSD
OPROCD interconnect
CSSD
OPROCD
2009/2010 Pythian
OS
OS
Clusterware
Clusterware
CSSD
OPROCD interconnect
CSSD
OPROCD
2009/2010 Pythian
OS
OS
Clusterware
Clusterware
CSSD
OPROCD interconnect
CSSD
OPROCD
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD
OPROCD interconnect
CSSD
OPROCD
Voting disk
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD
OPROCD interconnect
CSSD
OPROCD
Voting disk
2009/2010 Pythian
OS
OS Clusterware
CSSD
interconnect
CSSD
OPROCD
OPROCD
Voting disk
2009/2010 Pythian
OS Clusterware
OS Clusterware
VIP
CSSD
OPROCD interconnect
CSSD
OPROCD
Voting disk
10
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD
OPROCD interconnect
CSSD
OPROCD
Voting disk
11
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD
OPROCD interconnect
CSSD
OPROCD
Voting disk
11
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD
OPROCD interconnect
CSSD
OPROCD
Voting disk
11
2009/2010 Pythian
OS Clusterware
CSSD
interconnect
CSSD
OPROCD
Voting disk
11
2009/2010 Pythian
OS Clusterware
CSSD
interconnect
CSSD
OPROCD
Voting disk
12
2009/2010 Pythian
OS Clusterware
OS Clusterware
CS SD
CSSD
interconnect OPROCD
OPROCD
Voting disk
13
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD Monitor/Agent
CS SD
CSSD
interconnect OPROCD
OPROCD
Voting disk
13
2009/2010 Pythian
OS Clusterware
CSSD Monitor/Agent
CSSD
interconnect OPROCD
Voting disk
13
2009/2010 Pythian
OS Clusterware
CSSD interconnect
CSSD
OPROCD
Voting disk
14
2009/2010 Pythian
OS Clusterware
CSSD interconnect
CSSD
OPROCD
Voting disk
14
2009/2010 Pythian
OS Clusterware
CSSD interconnect
CSSD
OPROCD
Voting disk
14
2009/2010 Pythian
OS Clusterware
CSSD interconnect
CSSD
OPROCD
OPROCD/CSSD Mon
Voting disk
14
2009/2010 Pythian
OS Clusterware
Voting disk
14
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD
interconnect OPROCD
CSSD
OPROCD
Voting disk
15
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD
interconnect OPROCD
CSSD
OPROCD
Voting disk
15
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD
interconnect OPROCD
CSSD
OPROCD
Voting disk
15
2009/2010 Pythian
OS Clusterware
CSSD
interconnect
CSSD
OPROCD
Voting disk
15
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD
interconnect OPROCD
CSSD
OPROCD
Voting disk
16
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD
interconnect OPROCD
CSSD
OPROCD
16
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD
interconnect OPROCD
CSSD
OPROCD
16
2009/2010 Pythian
CSSD
interconnect
CSSD
16
2009/2010 Pythian
17
2009/2010 Pythian
OS
OS
Clusterware Instance
CSSD
OPROCD interconnect
CSSD
OPROCD
18
2009/2010 Pythian
OS
OS
Clusterware Instance
member kill
CSSD
OPROCD interconnect
CSSD
OPROCD
18
2009/2010 Pythian
OS
OS
Clusterware Instance
member kill
CSSD
OPROCD interconnect
CSSD
OPROCD
18
2009/2010 Pythian
OS
OS
Clusterware Instance
member kill
CSSD
OPROCD interconnect
CSSD
OPROCD
18
2009/2010 Pythian
OS
OS
Clusterware Instance
member kill
CSSD
OPROCD interconnect
CSSD
OPROCD
18
2009/2010 Pythian
OS
OS
Clusterware Instance
member kill
CSSD
OPROCD interconnect
CSSD
OPROCD
18
2009/2010 Pythian
OS
member kill
CSSD
OPROCD interconnect
18
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD
OPROCD interconnect
CSSD
OPROCD
Voting disk
19
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD
OPROCD interconnect
CSSD
OPROCD
Voting disk
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD
OPROCD interconnect
CSSD
OPROCD
Voting disk
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD
OPROCD interconnect
CSSD
OPROCD
Voting disk
2009/2010 Pythian
OS Clusterware
OS Clusterware
CSSD
OPROCD interconnect
CSSD
OPROCD
Exadata Fencing
20
Voting disk
2009/2010 Pythian
10g / 11gR1
Linux
11gR2
Linux
init.ohasd run
init.crs
Linux
ohasd start
Windows
Windows
Services
Services
21
2009/2010 Pythian
root
4283
0 02:52 ?
[root@cheese2 ~]# ls -l /etc/rc3.d/*ohasd* lrwxrwxrwx 1 root root 17 Sep 15 02:02 /etc/rc3.d/K15ohasd -> /etc/init.d/ohasd lrwxrwxrwx 1 root root 17 Sep 15 02:02 /etc/rc3.d/S96ohasd -> /etc/init.d/ohasd
22
2009/2010 Pythian
/etc/oracle/scls_scr/{host}/root/cssrun
init.crs start
init.cssd autostart
init.cssd oprodc init.cssd oclsomon init.cssd oclsvmon init.cssd daemon oprocd oclsomon.bin oclsvmon.bin ocssd.bin
crsd.bin
t
23
2009/2010 Pythian
24
2009/2010 Pythian
init.ohasd run t
24
2009/2010 Pythian
init.ohasd run t
24
2009/2010 Pythian
init.ohasd run t
24
2009/2010 Pythian
/etc/init.d/ohasd
init.ohasd run t
24
2009/2010 Pythian
/etc/init.d/ohasd
init.ohasd run t
24
2009/2010 Pythian
/etc/init.d/ohasd
init.ohasd run t
24
2009/2010 Pythian
init.ohasd run t
24
2009/2010 Pythian
OHAS
init.ohasd run t
24
2009/2010 Pythian
Ora Agent
OHAS
init.ohasd run t
24
2009/2010 Pythian
Ora Agent
OHAS
Root Agent
init.ohasd run t
24
2009/2010 Pythian
Ora Agent
OHAS
init.ohasd run t
24
2009/2010 Pythian
Ora Agent
OHAS
init.ohasd run
CSSD Monitor t
24
2009/2010 Pythian
Ora Agent
OHAS
init.ohasd run
CSSD Monitor t
24
2009/2010 Pythian
Ora Agent
ACFS Drivers CTSS Disk Monitor OHAS Root Agent CSSD Agent init.ohasd run CSSD Monitor t CRS
CSS
24
2009/2010 Pythian
ACFS Drivers CTSS Disk Monitor OHAS Root Agent CSSD Agent init.ohasd run CSSD Monitor t CRS
CSS
24
2009/2010 Pythian
ACFS Drivers CTSS Disk Monitor OHAS Root Agent CSSD Agent init.ohasd run CSSD Monitor CRS
CSS
24
2009/2010 Pythian
ACFS Drivers CTSS Disk Monitor OHAS Root Agent CSSD Agent init.ohasd run CSSD Monitor CRS CRS Ora Agent
CSS
24
2009/2010 Pythian
ACFS Drivers CTSS Disk Monitor OHAS Root Agent CSSD Agent init.ohasd run CSSD Monitor CRS CRS Ora Agent
CSS
25
2009/2010 Pythian
26
2009/2010 Pythian
data
OCR
is accessed mostly read-only is written only by CRS - Oracle Local Registry (OLR)
Every component reads OCR only from a single OCR master node managed by ohasd
OCR
11gR2
27
2009/2010 Pythian
with ifconfig eth1 down on node 2 Both nodes notice the loss Racing to evict each other
from voting disk => 2 equal sub-clusters (cohorts) survives the one with the lowest leader #
Winner
11gR2
28
2009/2010 Pythian
CSSD
29
2009/2010 Pythian
using kill -STOP <cssd.bin pid> (and try kill -9) Another node observes NHB loss
CSSD
Commit suicide
30
2009/2010 Pythian
OPROCD
Doesnt
31
2009/2010 Pythian
before starting up
Check
processes using ps -fe | grep init Check syslog (/var/log/messages) boot sequence Clusterware log files
crsctl
resource type
APPLICATION was the only supported resource type in 11gR1 Resource types using resource agents
Cluster
33
2009/2010 Pythian
ora.cluster_interconnect.haip ASM
Ora Agent
MDNS
ACFS Drivers CTSS Disk Monitor OHAS Root Agent CSSD Agent init.ohasd run CSSD Monitor CRS
CRS Ora Agent
CSS
34
2009/2010 Pythian
up? OHASD Agents up? Internal resources up? CRSD Agents up? Managed resources up?
Have
MOS 1050908.1
MOS 1053970.1
MOS 1050693.1
MOS 942166.1
36
2009/2010 Pythian