Automatic Workload Management With RAC

Automatic \orkload Management
with Oracle Real Application

Clusters

.v Oracte 1ecbvicat !bite Paer

Automatic Workload Management with Oracle Real Application Clusters Page 2
Automatic \orkload Management with Oracle Real
Application Clusters
Introduction ....................................................................................................... 3
last Application Notiication ,lAN, ............................................................. 3
Serices........................................................................................................... 4
last Connection lailoer............................................................................. 4
Load Balancing Adisory............................................................................. 5
lAN Lents................................................................................................... 5
Database 1ier ..................................................................................................... 8
Serices........................................................................................................... 8
last Application Notiication ,lAN, Serer Side Callouts .................. 10
Oracle Net Serices Integration with lAN Lents............................... 12
Data Guard Integration with lAN Lents............................................. 12
Oracle Notiication Serice on the Database 1ier................................. 13
Using Oracle Streams Adance Queuing or lAN Lent Publication14
Application 1ier............................................................................................... 14
JDBC............................................................................................................. 14
JDBC last Connection lailoer ,lCl, .............................................. 15
Implicit Connection Cache................................................................... 15
JDBC 1CP Connect 1imeout Property.............................................. 16
Application Usage o last Connection lailoer................................ 16
Oracle Notiication Serice ,ONS, on the Application 1ier ........... 1
OC4J Data Sources..................................................................................... 1
Implicit Connection Cache................................................................... 1
last Connection lailoer ,lCl, .......................................................... 19
Using 1Al,lAN with JDBC 1hick Drier ........................................... 20
Oracle Data Proider or .NL1 ,ODP.NL1, ........................................ 21
Oracle Call Interace................................................................................... 22
Runtime Connection Load Balancing with OCI Session Pools...... 24
Oracle Notiication Serice Application Programming Interace ,ONS API,
....................................................................................................................... 24
Conclusion........................................................................................................ 25
Appendix A ONS Operation ....................................................................... 26
Appendix B SAMPLL JAVA PROGRAM USING ONS API.............. 28
Appendix C SAMPLL C Code USING ONS API .................................. 29
Appendix D 1urning on Logging with JDBC ............................................ 31

Automatic \orkload Management with Oracle Real
Application Clusters
INTRODUCTION
Applications using a clustered database generally want to load balance their workload across
the cluster. Starting with Oracle RAC 10g Release 2 the load balancing adisory proides real-
time inormation to the application tier on the serice leel being proided by the database.
Applications can utilize this inormation to proide the best possible throughput or
transaction response time to the application using the assigned resources in the cluster.
Oracle Real Application Clusters ,RAC, includes a highly aailable ,lA, application
ramework that proides the necessary serice and integration points between RAC and
custom enterprise applications. One o the main principles o a highly aailable application is
or it to be able to receie ast notiication when something happens to critical system
components ,both inside and outside the cluster,. 1his allows the application to execute
eent-handling programs. 1he timely execution o such programs minimizes the impact o
cluster component ailures, by aoiding costly connection time-outs, application timeouts, and
reacting to cluster resource reorganizations, in both planned and unplanned scenarios.
1his paper discusses the eatures o Oracle Real Application Clusters 11g that can be used by
applications to proide the best possible serice leels to the end user by maximizing
throughput or response time and minimizing the impact o cluster component ailures. 1his
paper also gies practical examples showing how you can integrate these eatures into your
enironment.
FAST APPLICATION NOTIFICATION (FAN)
last Application Notiication ,lAN,, is a eature o Oracle Real Application Clusters ,RAC,
that urther dierentiates it or high aailability and scalability. lAN enables the automated
recoery o applications when cluster components ail. lAN soles the ollowing problems:
Applications waiting or 1CP,IP time-outs when a node ails.
Applications attempting to connect when serices are down.
Applications not connecting when serices resume.
Applications processing the last result ater a node, instance, or serice has gone
down.
Applications not balancing across aailable systems when serices restart or expand.
Attempting to execute work on slow, hung, and dead nodes
lor cluster coniguration changes, the Oracle RAC lA ramework posts a lAN eent
immediately when a state change occurs in the cluster. Instead o waiting or the Application

1ier to poll the Database 1ier and ind a problem, using lAN your application tier will
receie these eents and react immediately to the eent. lor down eents, the disruption to
the application can be minimized as connections to the ailed instance or node can be
terminated. In-light transactions are terminated and the application user immediately
notiied. Application users requesting connections are directed to aailable instances only.
Serer side callouts can be used to log trouble tickets or page Administrators to alert them o
the ailure. lor Up eents, when serices and instances are started, new connections can be
created so the application can immediately take adantage o the extra resources.
\ith Oracle RAC 10g Release 2, lAN was extended to assist applications in directing work
requests to where the request can be best satisied based on the current application workload.
Applications can take adantage o the load balancing adisory lAN eents to direct work
requests to the instance in the cluster that is currently proiding the best serice.
\ou can take adantage o lAN in the ollowing 3 ways:
1. \our application can take adantage o lAN without any programmatic changes by
utilizing an integrated Oracle Client. 1he integrated clients or lAN eents include
Oracle Database 11g JDBC, Oracle Database 11g ODP.NL1, and Oracle Database
11g Oracle Call Interace ,OCI,.
2. Applications can take adantage o lAN programmatically by using the Oracle
Notiication Serice Application Programming Interace ,ONS API, to subscribe to
lAN eents and execute eent-handling actions upon receipt.
3. Implement lAN serer side callouts on your Database 1ier.
Services
1he use o lAN requires the use o Serices. Serices decouple any hardwired mapping
between a connection request and a RAC instance. Serices are an entity deined or a RAC
database that allows the workload or a RAC database to be managed. Serices diide the
entire workload executing in the Oracle Database into mutually disjoint classes. Lach serice
represents a workload with common attributes, serice leel thresholds, and priorities. 1he
grouping is based on attributes o the work that might include the application being inoked,
the application unction to be inoked, the priority o execution or the application unction,
the job class to be managed, or the data range used in the application unction or job class.
Do not use the deault database serice, create at least one serice as described in document.
Fast Connection Failover
last Connection lailoer is a eature o Oracle clients that hae integrated with lAN lA
Lents. 1his is the easiest way or an application to improe its aailability by taking
adantage o the ability to know a ailure has occurred at the RAC database and recoer rom
the ailure as ast as possible. Utilizing this eature, applications can mask ailures rom the
user. Oracle JDBC Implicit Connection Cache, Oracle Call Interace ,OCI,, and Oracle Data
Proider or .Net ,ODP.Net, include ast connection ailoer. \ith ast connection ailoer,
when a down eent is receied, cached connections aected by the down eent are
immediately marked inalid and cleaned up. In light transactions are aborted. 1he
application can either propagate the error to the end user or retry the transaction and get a

connection rom the cache to a suriing instance. I the error is propagated to the user, the
user can easily retry the transaction and get a connection to a suriing instance.
Load Balancing Advisory
1o make the best use o the cluster resources, applications need to spread the workload
across multiple serers in the cluster. Oracle RAC proides adanced capability to notiy the
application o the current serice leel proided by each node so that it can intelligently send
work requests to the instance that will proide the best serice at the time o the request. 1he
load balancing adisory continually analyzes the work being processed on each instance
proiding a serice and publishes an eent which includes a recommendation on the
percentage o work to send to this instance and a lag indicating the quality o the data
proided. 1he load balancing adisory adjusts distribution or nodes that hae dierent
power ,I.L. dierent number o CPUs or dierent CPU speeds,. It can react quickly to
changes in the cluster coniguration, changes in application workload, and hung or
oerworked nodes. 1he load balancing adisory is integrated with the Automatic \orkload
Repository built into Oracle Database 11g. 1he Automatic \orkload Repository measures
response time and CPU consumption or each serice. 1he iews V>SLRVICLML1RIC and
V>SLRVICLML1RIC_lIS1OR\ contain the serice time or eery serice. 1hese iews
are updated eery 60s and contain 1 hour o history.
1he load balancing adisory publishes its inormation as lAN eents. Oracle connection
pools hae been integrated with this load balancing adisory by subscribing to the lAN
eents to proide Runtime Connection Load Balancing or application. \hen an application
gets a connection rom the pool, it will get the best connection aailable to process the work
request instead o a random connection.
FAN Events
Beore using some o the lAN eatures such as Serer Side Callouts or the ONS API, you
should understand lAN eents. Oracle RAC lAN eents consist o header and detail
inormation deliered as a set o name-alue pairs accurately describing the name, type and
nature o the eent. Based on this inormation, the eent recipient can take concrete
management, notiication, or synchronization steps, such as shutting down the application
connection manager, rerouting existing database connection requests, rereshing stale
connection reerences, logging a trouble ticket, or sending a page to the database
administrator.
lAN eents are system eents, sent during periods when cluster serers may become
unreachable and network interaces slow or non-unctional. 1here is an inherent reliance on
minimal communication channel oerhead to send, queue and receie notiications quickly.
1he objectie is to delier lAN eents so that they precede regular connection timeouts or
typical polling interals.
1hree categories o eents are supported in Oracle RAC lAN:
Serice eents, which includes both application serices and database serices
Node eents, which includes cluster membership states and natie join,leae
operations
Load Balancing Lents sent rom the RAC Load Balancing Adisory

RAC standardizes the generation, presentation, and deliery o eents pertaining to managed
cluster resources. 1he ollowing examples show the structure o a lAN eent:
Parameter Description
Version Version o the eent payload. Used to identiy release changes.
Lent type
SLRVICL, SLRVICL_MLMBLR, DA1ABASL, INS1ANCL, NODL, ASM,
SRV_PRLCONNLC1
Note that Database and Instance types proide the database serice -
db_unique_name.db_domain
Serice name 1he serice name. Matches the serice in DBA_SLRVICLS.
Database unique
name
1he unique database supporting the serice. Matches the initialization parameter alue or
db_unique_name, which deaults to the alue o the initialization parameter DB_NAML.
Instance 1he name o the instance supporting the serice. Matches the ORACLL_SID
Node name
1he node name supporting the serice or the node that has gone down.
Always matches the node name known to CSS.
Status
Values are UP, DO\N, NO1_RLS1AR1ING, PRLCONN_UP, PRLCONN_DO\N,
UNKNO\N
Cardinality Number o serice members - included on all UP eents.
Reason lailure, Dependency, User, Autostart, Boot
Incarnation lor node down eents - the new cluster incarnation
1imestamp Date and time stamp ,local time zone, to order notiication eents
Iigure J HA IAN Lvent

Parameter Description
Version Version o the eent payload. Used to identiy release changes.
Lent type SLRVICL_ML1RICS
Serice Matches the serice in DBA_SLRVICLS.
Database
unique name
1he unique database supporting the serice. Matches the initialization parameter alue
or db_unique_name, which deaults to the alue o the initialization parameter
DB_NAML.
1imestamp Date and time stamp ,local time zone, to order eents
1he ollowing ields are repeated.
Instance 1he name o the instance supporting the serice. Matches the ORACLL_SID
Percent 1he percentage o work requests to send to this database and instance
llag Indication o the serice quality relatie to the serice goal - Values are GOOD ,metrics
are alid,, VIOLA1ING ,A\R LLAPSLD or CPU thresholds are iolated,, NO
DA1A ,instance did not send metrics,, UNKNO\N ,A\R has no data or serice,
Iigure 2 Load Balancing IAN Lvent
lor diagnostics, lAN lA eents related to serices and databases are reported to
>ORACLL_lOML,racg,dump. lAN eents related to Nodes are reported to
ORA_CRS_lOML,racg,dump. Set and export the parameter _USR_ORA_DEBUG=1 in
$ORACLE_HOME/bin/racgwrap


1o monitor the Load Balancing adisory eents, use the ollowing example:
set pages 60 space 2 lines 132 num 8 verify off feedback off
column user_data heading "AQ Service Metrics" format A60 wrap
break on SERVICE_NAME skip 1
select
to_char(ENQ_TIME, 'HH:MI:SS') Enq_time, user_data
from SYS.SYS$SERVICE_METRICS_TAB
order by 1 ;

Iigure 3 Lxample Application Setup
1o help understand what lAN eents look like, let us look at a sample coniguration that is
pictured in ligure 5. In this example, we hae a 3 node RAC database called PROD with 2
Serices. Serice LRP is deined as Primary~Node1,Node2 and Aailable~Node3. Serice
CRM is deined as Primary~Node2, Node3 and Aailable~Node1. \ith this coniguration,
the LRP serice will ail oer to Node3 i instance PROD1 or PROD2 ail and the CRM
serice will ail oer to Node1 i instance PROD2 or PROD3 ail. Normal operation is to
hae the LRP serice running on PROD1,PROD2 and the CRM serice to be running on
PROD2,PROD3.
I Node1 ails, the LRP serice will ailoer to Node3, serice PROD will stop on Node1.
1he ollowing down eents are sent:
Event 1: FAN event type: instance
Properties: version=1.0 service=PROD database=PROD instance=PROD1 host=node1
status=down
Event 2: FAN event type: service_member
Properties: version=1.0 service=ERP database=PROD instance=PROD1 host=node1
status=down


After service ERP fails over to instance PROD3, service member ERP is up on instance PROD3
and the event is sent as follows:
Event 3: FAN event type: service_member
Properties: version=1.0 service=ERP database=PROD instance=PROD3 host=node3
status=up
A load balancing advisory event for the above configuration would be:
Event 4: FAN-event type: service_metrics
Properties: version=2.0 service=ERP database=PROD instance=PROD1 percent=70
service_quality=GOOD instance=PROD2 percent=30 service_quality=GOOD
Event 5 :FAN-event type: service_metrics
Properties: version=2.0 service=CRM database=PROD instance=PROD2 percent=30
service_quality=GOOD instance=PROD3 percent=70 service_quality=GOOD
DATABASE TIER
Oracle Database 11g proides the inrastructure to make your application data highly aailable
through the Oracle Real Application Clusters ,RAC, option. Real Application Clusters
proides the ability to hae multiple instances running on multiple nodes accessing the same
database. In the eent o hardware or sotware ailure, access to your data is not lost and
only a subset o application sessions are aected by the ailure. 1he Oracle Clusterware will
automatically try to restart the ailed system or process. Oten the system is recoered beore
the Administrator is aware a problem occurred.
Services
1raditionally an Oracle database proided a single serice and all users connected to the same
serice. A database will always hae this deault database serice that is the database name.
1his serice cannot be modiied and will always allow you to connect to the database. Do not
use the deault database serice, create at least one serice as described in document. \ith
Oracle Database 11g, a database can hae many serices ,up to a maximum o 115 per
database,. 1he serices hide the complexity o the cluster rom the client by proiding a
single system image or managing work. Applications and mid-tier connection pools select a
serice by using the Serice Name in their connection data. 1he serice must match the
serice that has been created using add serice with Lnterprise Manager, SRVC1L, or
DBMS_SLRVICL PL,SQL package. It is recommended that you use Lnterprise Manager
to create and manage serices by selecting the Cluster Managed Database Serice screen in
Lnterprise Manager. \ou can check the actie serices by querying the
V>AC1IVL_SLRVICLS iew. \ou can check the sessions using a serice by querying the
V>SLSSION iew. \ou may ind eatures o Oracle database create their own serices such
as the 2 internal serices SYS$BACKGROUND which is used by the background processes
only and SYS$USERS which is the default service for user sessions that are not associated with
any application service. You will not be able to manage these internal services. As a rule of
thumb, if you did not create the service, you will not be able to manage it. Lnterprise Manager
supports creating, modiying, iewing, and operating serices as a whole, with drill down to
the instance-leel when needed. ligure 4 below shows the Lnterprise Manager 11g screen or

database serices, which is the recommended way to manage serices. Click on the
Aailability 1ab rom your clustered Database, and then click on the Clustered Managed
Serices link. A serice can span one or more instances o an Oracle database and a single
instance can support multiple serices. 1he number o instances oering the serice is
managed by the DBA independent o the application. All client connections should connect
to a serice.
Iigure 4 Lnterprise Manager JJg Cluster Managed Services Screen
Serices enable the automatic recoery o an application`s ability to connect to the database.
1his is achieed according to business rules. lollowing outages, the serice is recoered
quickly and automatically at the suriing instances. \hen instances are later repaired,
serices that are not running are restored quickly and automatically by RAC lA lramework.
Immediately when a serice changes state, either Up or Down, a ast application notiication
,lAN, eent is aailable or applications using the serice. Applications can use this
notiication to trigger recoery and load balancing actions.
1he use o Serices is required to take adantage o the load balancing adisory and runtime
connection load balancing eatures o Oracle RAC. \hen a serice is created, a serice leel
goal is deined. 1here are 3 options aailable
None - Deault setting, you are not taking adantage o this eature
1lROUGlPU1 - \ork requests are directed based on throughput. 1lROUGlPU1
should be used when the work in a serice completes at homogenous rates. An example is a
trading system where work requests are similar lengths.
SLRVICL_1IML - \ork requests are directed based on response time. SLRVICL_1IML
should be used when the work in a serice completes at arious rates. An example is as
internet shopping system where work requests are arious lengths
Oracle Net Serices proides the ability to load balance connections to the database at
connection time. \ou must conigure Oracle Net Serices or connection load balancing by
setting the local_listener and remote_listener parameters. 1his will be automatically done i

you used DBCA to create your database. Since Oracle Database 10g Release 2, a goal or
connection load balancing is set when a serice is created. 1here are two options or the
Connection Load Balancing goal:
CLB_GOAL_SlOR1 - used or application connections that are short in duration. 1his
should be used with connection pools integrated with the load balancing adisory.
CLB_GOAL_LONG - used or application connections that are connected or a long period
such as third party connection pools and SQLlorms applications. 1his is the deault alue.
Note: I you use the shared serices eature o Oracle Net Serices and set the serice
attribute o the dispatchers parameter in the init.ora, the serice name speciied in this
parameter cannot be managed. IL srctl stop serice will show the serice stopped in crs_stat
but the listener will still allow connections.
1ransparent Application lailoer ,1Al, can be deined at the serice. 1Al is a eature o
Oracle Net Serices that will ailoer a database session to a backup connection should the
session be disconnected. I a serice has a deined 1Al policy, this will oerride a 1Al
policy on the client connection. 1he minimum requirement to turn 1Al on is to set the
ailoer_type~session or select. 1Al restrictions hae not changed and 1Al is thereore
only alid or OCI connections ,do not use with JDBC lCl,. 1he 1Al ailoer method o
PRLCONNLC1 is not supported when deining a 1Al policy or a serice, you must use
BASIC. 1Al will use the lAN lA eents to proide ast ailoer or database sessions.
\ou must use DBMS_SLRVICL PL,SQL package to add the 1Al policy.
execute dbms_service.modify_service(
service_name => 'gl.us.oracle.com' -
, aq_ha_notifications => true -
, failover_method => dbms_service.failover_method_basic -
, failover_type => dbms_service.failover_type_select -
, failover_retries => 180 -
, failover_delay => 5 -
, clb_goal => dbms_service.clb_goal_long);
Iigure S Sample PL/SQL to add 1AI policy to Service
\ith Oracle RAC, there is a special serice option to support Distributed 1ransaction
Processing applications ,IL XA transactions
1
,. By defining the DTP property of a service, the
service is guaranteed to run on one instance at a time in a RAC database. All global distributed
transactions performed through the DTP service are ensured to hae their tightly-coupled
branches running on a single RAC instance. Define only one instance as the preferred instance.
You can have as many AVAILABLE instances as you want.
Fast Application Notification (FAN) Server Side Callouts
Serer-side callouts proide a simple, yet powerul integration mechanism with the ligh
Aailability lramework that is part o the Oracle Clusterware. Serer side callouts can be
deployed with minimal programmatic eort. A callout is essentially a shell script or pre-
compiled executable written in any programming language. Simply place an executable in the
directory ORA_CRS_lOML,racg,usrco on eery node that runs the Oracle Clusterware. I

1
Best Practices or XA 1ransactions with RAC can be ound
http:,,www.oracle.com,technology,products,database,clustering,pd,bestpracticesorxaandrac.pd

using scripts, remember to set the shell as the irst line o the executable. A callout script can
be as simple as callout.sh
2
:

#! /bin/ksh
FAN_LOGFILE= [your path name]/admin/log/`hostname`_uptime.log
echo $* "reported="`date` >> $FAN_LOGFILE &
\hen callout.sh is executed, it will produce output such as:
NODE VERSION=1.0 host=sun880-2 incarn=23 status=nodedown reason=
timestamp=08-Oct-2004 04:02:14 reported=Fri Oct 8 04:02:14 PDT 2004
All executables in the directory ORA_CRS_lOML,racg,usrco will execute immediately in
an asynchronous ashion when a condition occurs. A condition occurs when an lA eent is
receied in the cluster through the Oracle Notiication Serice ,ONS,. 1he Oracle RAC lA
ramework posts a lAN eent to ONS immediately when a state change occurs. A state
change could be the start or stop o a serice, instance, database or when a node leaes the
cluster. Oracle recommends eent-handling programs with similar semantics or whose
executions must be perormed in a particular order, be inoked rom a common callout. \ou
should careully test each new callout or execution perormance beore ormal deployment.
Note: As a security measure, it is recommended that the callout directory has write
permissions only to the system user who installed Oracle Clusterware, and that each callout
executable or script contained therein has execute permissions only to the same Oracle
Clusterware user.
\riting serer-side callouts inoles the ollowing steps:
1. Parse the eent argument list. A sample Bourne shell script that parses the eent
argument list is:
# Scan and parse HA event payload arguments:
#
NOTIFY_EVENTTYPE=$1 # Event type is handled differently

for ARGS in $*; do
PROPERTY=`echo $ARGS | $AWK -F"=" '{print $1}'`
VALUE=`echo $ARGS | $AWK -F"=" '{print $2}'`
case $PROPERTY in
VERSION|version) NOTIFY_VERSION=$VALUE ;;
SERVICE|service) NOTIFY_SERVICE=$VALUE ;;
DATABASE|database) NOTIFY_DATABASE=$VALUE ;;
INSTANCE|instance) NOTIFY_INSTANCE=$VALUE ;;
HOST|host) NOTIFY_HOST=$VALUE ;;
STATUS|status) NOTIFY_STATUS=$VALUE ;;
REASON|reason) NOTIFY_REASON=$VALUE ;;
CARD|card) NOTIFY_CARDINALITY=$VALUE ;;
TIMESTAMP|timestamp) NOTIFY_LOGDATE=$VALUE ;; # catch event date
??:??:??) NOTIFY_LOGTIME=$PROPERTY ;; # catch event time (hh24:mi:ss)
esac
done
2. lilter incoming lAN eents
Instead o propagating eery lAN eent to the local eent-handling program, you can ilter
the incoming eents based on payload inormation. In the ollowing Bourne shell script
example, a trouble ticket system ,using the second iltering example aboe, is inoked only

2
Sample Code can be ound
http:,,www.oracle.com,technology,sample_code,products,rac,index.html

when RAC lA ramework posts a SLRVICL, DA1ABASL or NODL eent type, with
status either down`, not_restarting`, restart_ailed` or nodedown`, and only or two
application serice names: lQPROD and lIN_APAC:
# Only FAN events with the following conditions will be inserted
# into the critical trouble ticket system:
# NOTIFY_EVENTTYPE => SERVICE | DATABASE | NODE
# NOTIFY_STATUS => down | not_restarting | restart_failed | nodedown
# NOTIFY_DATABASE => HQPROD | FIN_APAC
#
if ((( [ $NOTIFY_EVENTTYPE = "SERVICE" ] ||
[ $NOTIFY_EVENTTYPE = "DATABASE" ] || \
[ $NOTIFY_EVENTTYPE = "NODE" ] \
) && \
( [ $NOTIFY_STATUS = "down" ] || \
[ $NOTIFY_STATUS = "not_restarting" ] || \
[ $NOTIFY_STATUS = "restart_failed" ] || \
[ $NOTIFY_STATUS = "nodedown " ] \
)) && \
( [ $NOTIFY_DATABASE = "HQPROD" ] || \
[ $NOTIFY_DATABASE = "FIN_APAC" ] \
))
then
<< CALL TROUBLE TICKET LOGGING PROGRAM AND PASS RELEVANT NOTIFY_*
ARGUMENTS >>
fi
3. Lxecuting eent-handling programs
\ou must decide the actions that need to be taken once an eent is receied based on your
business requirements. 1hese can generally be it into three categories o eent handlers you
may want to implement:
Lent logging: Receies the lAN eent locally and copies a subset o the payload
data into some persistent repository, e.g. a local log ile, operating system logging
serice, or a remote database table.
Lent paging: Receies the lAN eent locally and issues a notiication message to a
remote deice, such as a pager, e-mail client or SNMP management console.
Lent start,stop action: Receies the lAN eent locally and issues a start or stop
command to a remote daemon or process. 1his requires that the local eent handler
be a trusted proxy application running on each RAC cluster node, capable o sel-
authentication and issuance o start,stop commands, through a secure network
connection and an API that the remote daemon or process exposes. lor example,
the local component execution program may be a small, custom compiled program,
or a perl script posting a URL through l11PS.
Oracle Net Services Integration with FAN Events
1he Connection Manager ,CMAN, and Oracle Net Serices Listeners take adantage o
lAN. 1his allows the Listener and CMAN to immediately de-register serices proided by
the ailed instance and aoid erroneously sending connection requests to ailed instances. 1he
Listener uses the load balancing adisory when load balancing connections where the serice
has CLB_GOAL_SlOR1, and GOAL~SLRVICL_1IML or 1lROUGlPU1. I
CLB_GOAL~LONG ,CLB_GOAL_LONG, then the listener will load balance based on
number o sessions and the GOAL setting will not be used.
Data Guard Integration with FAN Events
Oracle Database 11g Data Guard Broker is integrated with lAN. \hen the role changes or
a database that is part o a Data Guard physical standby coniguration, an eent is posted to
help administrators automate post role change actiities and notiy applications when the

database it is connected to is no longer operating in the primary role. 1he standby database
can be either a single instance or RAC database. 1he Data Guard Broker posts a database
down eent or the lost primary. 1he down eent is posted on successul completion o the
ailoer operation and is posted by the database that is operating in the new primary database
role on behal o the old primary. Oracle Call Interace ,OCI, has integrated with the Data
Guard database down eent. OCI clients that hae registered or lA eents, will receie this
eent. I the OCI connection is using 1ransparent Application lailoer ,1Al,, then the
client session can be ailed oer to the new primary database.
Oracle Notification Service on the Database Tier
Oracle Notiication Serice ,ONS, uses a simple publish,subscribe method to produce and
delier eent messages or both local and remote consumption. ONS daemons run locally
sending messages to and receiing messages rom a conigured list o nodes ,where other
ONS daemons are actie,. Oracle Clusterware and RAC utilize ONS to propagate lAN
messages both within the RAC cluster, and to client or mid-tier machines. ONS is installed
with Oracle Real Application Clusters ,RAC,, and the Oracle Clusterware resources to
manage the ONS daemon are created automatically during the RAC installation process.
Oracle Clusterware automatically starts the ONS daemon during a reboot. Lnsure your ONS
is conigured on each node running the Oracle Clusterware.
1he ONS daemon is running as a node application. 1o check node applications use the
command: srvctl status nodeapps. \our results should be similar to the ollowing:
>srvctl status nodeapps -n rca01
VIP is running on node: rca01
GSD is running on node: rca01
Listener is running on node: rca01
ONS daemon is running on node: rca01
Use onsctl ping to check that the ONS daemon is actie.
>onsctl ping
ONS is running
1he ONS coniguration ile, >ORACLL_lOML,opmn,con,ons.conig, contains the ONS
coniguration. 1his ile should always contain alues or localport, the port that ONS
binds to on the localhost interace to talk to local clients, remoteport, the port that ONS
binds to on all interaces or talking to other ONS daemons, and nodes, a list o other ONS
daemons to talk to speciied as either hostnames or IP addresses plus ports. 1he host
inormation can be speciied in the conig ile or be retrieed rom the OCR ,recommended,.
localport=6101 # This is the port ONS is writing to
remoteport=6201 # This is the port ONS is listening on
loglevel=3
useocr=on
Iigure 6 ONS Configuration using OCR with Oracle RAC
1he Oracle Uniersal Installer ,OUI, automatically conigures the inormation in the OCR
or ONS daemons. 1o update RAC nodes, use racgons in your >ORACLL_lOML,bin
1o add ONS daemons coniguration:
racgons.bin add_config hostname:port [hostname:port] ...
1o remoe ONS daemons coniguration:
racgons.bin remove_config hostname[:port] [hostname:port]...

Note: ONS can dynamically recognize a new ONS. I you add a node to the cluster, you do
not hae to restart the ONS on the existing nodes, as soon as the new ONS publishes an
eent to the existing nodes, they will dynamically add the new ONS to their coniguration.
Using Oracle Streams Advance Queuing for FAN Event Publication
Real Application Clusters publishes lAN eents to a system Alert queue in the database.
ODP.NL1 and OCI client integration use this method to subscribe to lAN eents.
1o hae lAN lA eents or a Serice posted to the alert queue, the notiication must be
turned on or the serice using dbms_serice PL,SQL package.
dbms_service.modify_service(service_name=>crm
,aq_ha_notifications=>true)
1o iew the lAN lA eents that are published, use the iew
DBA_OUTSTANDING_ALERTS or DBA_ALERT_HISTORY. \hen the GOAL is set on
a serice to either 1lROUGlPU1 or SLRVICL_1IML, load balancing adisory eents are
sent.
APPLICATION TIER
1he easiest way to receie the beneits o lAN with your application is to use an integrated
Oracle Client. Oracle Database 11g has integrated the connection pools or the JDBC
,supports both the JDBC thick and thin drier,, ODP.NL1 and OCI clients. I you are not
using one o the integrated clients, an API is aailable which allows your application to
subscribe directly to lAN eents. All client integration requires that your database serice be
deined with the appropriate lags set and that you hae Oracle Net Serices conigured or
connection load balancing.
JDBC
lor Jaa applications, you can use the Oracle Database 11g JDBC last Connection lailoer
eature o the Implicit Connection Cache. 1he steps required to implement lAN with
Oracle Database 10g JDBC are:
1. Lnable JDBC Implicit Connection Cache and JDBC last Connection lailoer in
your datasource. lor aster ailoer o the client connection set 1CP connect
timeout property.
2. Conigure the ONS on each RAC node to be aware o your application tiers.
3. Deine the JDBC data source parameters to point to the remote ONS in the RAC
cluster.
4. \hen starting the application, ensure that the ons.jar ile is located on the
application CLASSPA1l.
JDBC last Connection lailoer, JDBC Implicit Connection Cache and the ONS are
discussed in detail in the ollowing sections o this paper.
1urning on the last Connection lailoer eature with JDBC includes Runtime Connection
Load Balancing. 1he JDBC connection pool subscribes to the lAN Load Balancing eents
automatically with you conigure ast connection ailoer. Instead o randomly assigning a
ree connection, the connection pool will choose the connection that will gie the best serice
according to the latest inormation it has receied. I a node becomes hung, it will graitate
connections rom the hung node to other nodes in the cluster.

Oracle Application Serer 10g ,10.1.2, users with Jaa applications using serlets or beans,
will receie the beneits o lAN by utilizing the JDBC last Connection lailoer eature.
Oracle Application Serer 10g ,10.1.3, will ully integrate lAN eatures.
JDBC Fast Connection Failover (FCF)
1he JDBC connection pool when conigured to use last Connection lailoer, automatically
subscribes to lAN eents, thereore it can react to the up or down eents rom the database
cluster. 1he application will always be gien a alid connection to an actie instance
proiding the database serice requested. \hen a down eent is receied, all connections to
that instance using that serice are terminated. Any connections that were in use at the time
o ailure are cleaned up so the application will receie the ailure immediately. Database
recoery will take care o rolling back any in-light uncommitted transactions. 1he
connection pool will create additional connections only when it requires them ,i.e. a
getConnection comes in and there are no idle connections in the pool,. \hen that instance is
restarted, an up eent is sent and the connection pool will create new connections to the
database. 1he number o new connections is a portion o the connections it held to the
instance when it ailed. I you hae connection load balancing setup correctly, the new
connections should go to the instance that was restarted. \hen an up eent is receied or a
new instance, the connection pool tries to re-balance its connections by retiring a portion o
connections and then creating new connections. Assuming you are using load balancing, the
new connections will go to the instance that triggered the up eent and work is immediately
directed to the added instance with no application changes. 1he database connection string
used with last Connection lailoer cannot use 1ransparent Application lailoer ,1Al, and
the serice used, should not hae 1Al attributes deined.
1he pre-requisites or using last Connection lailoer are:
1he implicit connection cache is enabled. last Connection lailoer works in
conjunction with the JDBC connection caching mechanism. 1his helps applications
manage connections to ensure high aailability.
1he application uses serice names to connect to the database.
JDBC Datasource is conigured to or remote ONS subscription ,or a ONS is
running on the application tier as with Oracle AS 10g,
1he underlying database should be at a minimum Oracle Database 10g ,10.1.0.4,
Real Application Clusters ,RAC,. I ailoer eents are not propagated, connection
ailoer cannot occur.
Implicit Connection Cache
1o take adantage o lAN and lCl with Oracle Database 10g JDBC driers, you must be
using the Implicit Connection Cache. An application turns the implicit connection cache on
by inoking OracleDataSource.setConnectionCachingEnabled(true). Ater
implicit caching is turned on, the irst connection request to the OracleDataSource
transparently creates a connection cache.

OracleDataSource ods = new OracleDataSource();
// Set DataSource properties
ods.setUser("Scott");
ods.setConnectionCachingEnabled(true); // Turns on caching

ctx.bind("MyDS", ods);
// ...
1o take adantage o last Connection lailoer, a second Data source property
lastConnectionlailoerLnabled must be set to true. Note: ConnectionCacheName should
come ater the ConnectionCachingLnabled and lastConnectionlailoer properties.
OracleDataSource ods = new OracleDataSource()
...
ods.setUser(Scott);
ods.setPassword(tiger);
ods.setConnectionCachingEnabled(True);
ods.setFastConnectionFailoverEnabled(True);
ods.setConnectionCacheName(MyCache);
ods.setConnectionCacheProperties(cp);
ods.setONSConfiguration("nodes=racnode1:4200,racnode2.:4200");
ods.setURL("jdbc:oracle:thin:@(DESCRIPTION=
(LOAD_BALANCE=on)
(ADDRESS=(PROTOCOL=TCP)(HOST=VIP1)(PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=VIP2)(PORT=1521))
(CONNECT_DATA=(SERVICE_NAME=service_name)))");
JDBC TCP Connect Timeout Property
Due to the retry logic in 1CP, JDBC clients encounter delays between the irtual IP ailing
oer, and the client completing a successul connection. 1hese delays are aoided by setting
the oracle.net.ns.SQLnetDe.1CP_CONN1IMLOU1_S1R property. 1his time is in
milliseconds:
Properties prop = new Properties ();
prop.put (oracle.net.ns.SQLnetDef.TCP_CONNTIMEOUT_STR,
"" + (1 * 1000)); // 1 second
dbPools[ poolIndex ].setConnectionProperties ( prop );
Application Usage of Fast Connection Failover
\hen a connection is closed by the implicit connection cache, the application receies a
SQLLxception on each connection . 1he application can inokes
islatalConnectionLrror,SQLLxception e, to determine:
i each connection is still usable
i connection is not usable any more, retry connection request, and obtain a good
connection rom the pool
At this stage, the application may replay transaction.
try {
conn = getConnection();
// do some work
} catch (SQLException e) {
handleSQLException(e)
}
...
handleSQLException (SQLException e)
{
if
(OracleConnectionCacheManager.isFatalConnectionError(e))
ConnRetry = true; // Fatal Connection error detected,
}
Note: Oracle Application Serer handles connection retry transparently or Container
Managed Persistence

Oracle Notification Service (ONS) on the Application Tier
1he Oracle JDBC Implicit Connection Cache, when conigured or last Connection lailoer
,lCl,, requires an ONS daemon rom which to receie eent messages. 1he absence o ONS
will preent the connection cache rom receiing eents. I you are using Oracle Application
Serer, the ONS is installed as part o the Application Serer. I you are not using Oracle
Application Serer, you can install ONS on the application tier ,in this case the JVM in which
your JDBC instance is running must hae oracle.ons.oraclehome set to point to your
ORACLL_lOML where the ONS iles were installed, or use a remote ONS subscription by
setting the ONS subscription DataSource property ,recommended,. Remote ONS
subscription oers the ollowing adantages:
Support or an All Jaa mid-tier stack
No ONS daemon needed on the client machine. No need to manage this process
Simple coniguration ia DataSource property
\hen using remote ONS subscription or last Connection lailoer, an application inokes
setONSConfiguration(String remoteONSConfig) on an Oracle DataSource
instance as in the ollowing example:
ods.setONSConfiguration("nodes=racnode1:4200,racnode2.:4200");
1he ONS.JAR must be included in the CLASSPA1l on the client. 1he ONS.JAR can be
ound as part o the Oracle Client installation.
Inormation on managing the ONS is ound in Appendix A o this document. lor both
Oracle Application Serer and ONS installed rom Oracle Database 10g Client, you will need
to update the coniguration ile.
OC4J Data Sources
Oracle Containers or J2LL ,OC4J, data sources integrate the new Implicit Connection
Caching and last Connection lailoer eatures in Oracle Database 10g JDBC. lor Oracle AS
10g ersions 9.0.4.x and 10.1.2, Implicit Connection Caching support is limited to natie data
sources. Other types o data sources ,e.g., emulated and non-emulated data sources, do not
support this eature and use an older orm o caching. lor Oracle AS 10g ,9.0.4.x, ersions,
since the deault Oracle JDBC drier is older than Oracle 10g, users must upgrade the deault
drier to at least Oracle Database 10g JDBC in order to take adantage o this eature.
Implicit Connection Cache
Coniguring Implicit Connection Caching or these Oracle AS ersions is mostly done in a
declaratie way by modiying the data-sources.xml ile directly. 1he aected OC4J instance
has to be restarted or the changes to take eect.
1o conigure Implicit Connection Caching ,prior to AS 10g 10.1.3,, ollow these two steps:
1. speciy oracle.jdbc.pool.OracleDataSource` as the class` attribute in a data-
source element,
2. speciy the connectionCacheName` and connectionCachingLnabled` properties
within the same data-source element.

<data-source
class="oracle.jdbc.pool.OracleDataSource"
name="OracleDS"
location="jdbc/OracleCache"
connection-driver="oracle.jdbc.driver.OracleDriver"
username="scott"
password="tiger"
url="jdbc:oracle:thin:@localhost:1521:orcl">
<property name="connectionCacheName" value="ICC" />
<property name="connectionCachingEnabled" value="true" />
</data-source>

Oracle Application Serer 10g 10.1.3 proides improed integration with lCl. OC4J 10.1.3
oers two simple types o data sources: managed data source and natie data source. Implicit
Connection Caching is supported in both types o data sources. 1he primary tool or
coniguring Implicit Connection Caching in OC4J 10.1.3 data sources is the user-riendly
Oracle Lnterprise Manager ,LM, 10g Application Serer Control Console. Central
management o the data sources ia the Console signiicantly lowers administratie cost.
Standard JMX-based management supports the dynamic creation, deletion, and modiication
o both data source types, as well as the associated connection caches, without OC4J restart.
Alternatiely, Implicit Connection Caching can still be enabled or disabled declaratiely within
the data-sources.xml deployment descriptor ile or any OC4J data source. Because the
descriptor syntax has changed in 10.1.3, this would be dierent rom how it is done in earlier
Oracle Application Serer ersions. lor more details on OC4J Data Sources with Implicit
Connection Caching and last Connection lailoer, see the article on O1N at
http:,,www.oracle.com,technology,tech,jaa,newsletter,articles,oc4j_data_sources,oc4j_ds.htm

Iigure 7 Lnterprise Manager Application Server Control


Iigure 8 Sample data-sources.xml deployment descriptor
Fast Connection Failover (FCF)
\hen using oracle.jdbc.pool.OracleDataSource` as the natie data source, there is a
common way in all Oracle AS ersions to enable Implicit Connection Caching and last
Connection lailoer at the same time, with all the connection cache properties taking their
deault alues. 1hat is, setting the system property oracle.jdbc.lastConnectionlailoer` to
true when launching an OC4J instance. lor example,
jaa -Doracle.jdbc.lastConnectionlailoer~true -jar oc4j.jar
lor Oracle AS ersions ,9.0.4.x, and 10g ,10.1.2,, last Connection lailoer support is limited
to the natie data sources. Other types o data sources ,e.g., emulated and non-emulated data
sources, do not support this eature. lor Oracle AS 10g ,9.0.4.x, ersions, since the deault
Oracle JDBC drier is older than Oracle 10g, users must upgrade the deault drier to at least
Oracle Database 10g JDBC in order to take adantage o this eature.
Coniguring last Connection lailoer or these Oracle AS ersions is done in a declaratie
way by modiying the data-sources.xml ile directly. 1he aected OC4J instance has to be
restarted or the changes to take eect. 1he Implicit Connection Cache must be enabled in
order to use last Connection lailoer.
1o conigure last Connection lailoer, ollow these two steps:
1. Speciy oracle.jdbc.pool.OracleDataSource` as the class` attribute in a data-
source element,
2. Speciy the astConnectionlailoerLnabled` property with alue true` within the
same data-source element.
<data-source
class="oracle.jdbc.pool.OracleDataSource"
name="OracleDS"
location="jdbc/OracleCache"
connection-driver="oracle.jdbc.driver.OracleDriver"
username="scott"
password="tiger"
url="jdbc:oracle:thin:@localhost:1521:orcl">
<property name="connectionCacheName" value="ICC" />
<property name="connectionCachingEnabled" value="true" />
<property name="fastConnectionFailoverEnabled" value="true" />
</data-source>


lor OC4J 10.1.3 data sources, declaratie coniguring o last Connection lailoer is ery
similar to how it is done in earlier Oracle Application Serer ersions. Speciically, it can be
done in two steps, depending on the data source type.
lor managed data sources:
1. speciy oracle.jdbc.pool.OracleDataSource as the actory-class attribute in a
connection-actory element or a conigured connection-pool,
2. speciy the astConnectionlailoerLnabled property with alue true within the same
connection-actory element.
<managed-data-source
jndi-name="jdbc/ManagedDS"
description="Managed DataSource">
connection-pool-name="myConnectionPool"
name="ManagedDS"/>
<connection-pool
name="myConnectionPool"
min-connections="10"
max-connections="30"
inactivity-timeout="30">
<connection-factory
factory-class="oracle.jdbc.pool.OracleDataSource"
user="scott"
password="tiger"
url="jdbc:oracle:thin:@localhost:1521:oracle"/>
<property name="fastConnectionFailoverEnabled" value="true"/>
</connection-factory>
</connection-pool>
lor natie data sources:
1. speciy oracle.jdbc.pool.OracleDataSource as the data-source-class attribute in a
natie-data-source element,
2. speciy the astConnectionlailoerLnabled property with alue true within the same
natie-data-source element.
<native-data-source
name="nativeDataSource"
jndi-name="jdbc/nativeDS"
description="Native DataSource"
data-source-class="oracle.jdbc.pool.OracleDataSource"
user="scott"
password="tiger"
url="jdbc:oracle:thin:@localhost:1521:oracle">
<property name="connectionCacheName" value="ICC"/>
<property name="connectionCachingEnabled" value="true"/><property
name="fastConnectionFailoverEnabled" value="true"/>
</native-data-source>
Using TAF/FAN with JDBC Thick Driver
I you would like to use 1Al with the JDBC thick drier, then you should use the OCI
integration with lAN instead o lCl. 1o use 1Al, irst set the 1Al policy or the serice
with the DBMS_SLRVICL package and set AQ_lA_NO1IlICA1IONS~true. \ou can
also take adantage o the OCI callback unctionality with your application program. Do not
use both lCl and 1Al together.

execute dbms_service.modify_service (service_name =>
'gl.us.oracle.com' -
, aq_ha_notifications => true -
, failover_method => dbms_service.failover_method_basic -
, failover_type => dbms_service.failover_type_select -
, failover_retries => 180 -
, failover_delay => 5 -
, clb_goal => dbms_service.clb_goal_long);
Iigure 9 Using DBMS_SLRVICL to set 1AI policy
Oracle Data Provider for .NET (ODP.NET)
lor .NL1 applications, Oracle Database 10g Release 2 ODP.NL1 proides the ability to take
adantage o lAN eents or high aailability and connection pool load balancing. Oracle
Data Proider or .NL1 ,ODP.NL1, connection pools subscribe to lAN notiications rom
RAC that indicate when nodes are down and when serices are up or down. Based on these
notiications, ODP.NL1 connection pools make idle connections, connections that were
preiously connected to nodes that ailed, aailable again. It also creates new connections to
healthy nodes i possible. In the case o a DO\N eent, Oracle cleans up sessions in the
connection pool that is connected to the instance that stops. It will then create new
connections to an UP instance i it exists and the number o connections in the pool is below
min_pool_size. 1he steps required to implement lAN with Oracle Database 10g Release 2
are:
1. 1urn on AQ lA eent notiications
dbms_service.modify_service(service_name=>crm,aq_ha_notifications=>tru
e)

2. Grant permission to the application user,s, to de-queue the messages. 1his is the
user who is connecting rom the .NL1 application.
execute
dbms_aqadm.grant_queue_privilege('DEQUEUE','SYS.SYS$SERVICE_METRICS',
<your user>);
3. Lnable last Connection lailoer or ODP.NL1 connection pool by subscribing to
lAN lA eents. Set the HA Events string to true. 1his can be done at connect
time or in the data source deinition. Note this will only work with i you are using
connection pools, I.L. "pooling=true" attribute is set.
"user id=scott;password=tiger;data source=erp;HA events=true;"

// C#
using System;
using Oracle.DataAccess.Client;
class ConnectionPoolingSample
{
static void Main()
{
OracleConnection con = new OracleConnection();
//Open a connection using ConnectionString attributes
//related to connection pooling.
con.ConnectionString =
"User Id=scott;Password=tiger;Data Source=oracle;" +
"Min Pool Size=10;Connection Lifetime=120;Connection Timeout=60;" +
HA events=true, "Incr Pool Size=5; Decr Pool Size=2";
con.Open();
Console.WriteLine("Connection pool successfully created");

// Close and Dispose OracleConnection object
con.Close();
con.Dispose();
Console.WriteLine("Connection is placed back into the pool.");
}
}

lor aster ailoer o the client, set the SQLNet Connect 1imeout Property on the Client
,DO NO1 SL1 on RAC Serer,. 1he alue is seconds and should be set in the
SQLNL1.ORA.
sqlnet.outbound_connect_timeout = 3

ODP.NL1 proides runtime connection load balancing to proide enhanced load balancing
o the application workload. Instead o randomly selecting an aailable connection rom the
connection pool, it will choose the connection that will proide the best serice base on the
current workload inormation. 1he steps required to implement connection pool load
balancing, are:
1. 1urn on eent notiications and set a goal or your serice:
dbms_service.modify_service(service_name=>crm, goal=>THROUGPUT)

4. Lnable Runtime Connection Load Balancing by subscribing to lAN Load Balancing
eents. 1his can be done at connect time or in the data source deinition. Note this
will only work with i you are using connection pools, I.L. "pooling=true"
attribute is set.
"user id=scott;password=tiger;data source=erp;load balancing=true;"
Oracle Call Interface
1he Oracle Call Interace ,OCI, proides integration with lAN lA and Load Balancing
Adisory eents with Oracle RAC 11g. 1o take adantage o the Load Balancing Adisory,
you need to enable the OCI Session Pool. OCI clients can register to receie notiications
about RAC high aailability eents and respond when eents occur. 1his improes the
connection ailoer response time in OCI and also remoes terminated connections rom
connection and session pools. 1his eature works or all OCI client applications. OCI clients
with transparent application ailoer ,1Al, enabled are recommended to enable this eature
or ast ailoer. On receipt o a down eent or an instance or node, OCI will
1erminate aected connections at the client
Remoe connections rom the OCI connection pool and OCI session pool ,1he
session pool maps each session to a physical connection in the connection pool.
1here can be multiple sessions per connection,
I 1Al is conigured, the connection will ailoer, i not, the client receies an error
such as ORA-12543
I a 1Al callback has been registered, then the ailoer retries and ailoer delay are
ignored. I an error occurs, 1Al will continue to attempt to connect and
authenticate as long as the callback returns a alue o OCI_lO_RL1R\. Any delay
should be coded into the callback logic.
Client applications must connect to a RAC instance to enable eent notiication. Clients that
hae enabled high aailability eent notiication can optionally register client LVLN1
callbacks. 1his reduces the time that it takes to detect a connection ailure.

lor aster ailoer o the client, set the SQLNet Connect 1imeout Property on the Client
,DO NO1 SL1 on RAC Serer,. 1he alue is seconds and should be set in the
SQLNL1.ORA.
sqlnet.outbound_connect_timeout = 3
Perorm the ollowing three steps to conigure lAN notiications to an OCI client:
1. Conigure the serice at the serer to set the alue or the parameter
AQ_NOTIFICATIONS to TRUE. lor example:
dbms_service.modify_service(service_name=>crm,aq_ha_notifications=>tru
e)
2. Lnable OCI_EVENTS at enironment creation time, this tells Oracle you may be
interested in lA eents:
, Need to init w, OCI_LVLN1S to receie lA eents ,
if (checkerr(NULL, OCIInitialize((ub4) OCI_EVENTS, (dvoid *)0,
(dvoid * (*)(dvoid *, size_t)) 0,
(dvoid * (*)(dvoid *, dvoid *, size_t))0,
(void (*)(dvoid *, dvoid *)) 0)))
goto terminate;
3. In your program, you will need to check i an eent has occurred, this code gets
called when an lA eent occurs:
void evtcallback_fn(ha_ctx, eventhp)
dvoid *ha_ctx;
OCIEvent *eventhp;
{
OCIServer *srvhp;
OCIError *errhp;
sb4 retcode;
OraText *hostname;
OraText *dbname;
OraText *instname;
OraText *svcname;
OCIDate timestmp;
OCIEnv *envhp = (OCIEnv *)ha_ctx;

ub4 sizep;

printf("HA event received.\n");

if (OCIHandleAlloc( (dvoid *)envhp, (dvoid **)&errhp,
(ub4) OCI_HTYPE_ERROR,
(size_t) 0, (dvoid **) 0))
return;
if (retcode = OCIAttrGet(eventhp, OCI_HTYPE_EVENT, (dvoid *)&srvhp,
(ub4 *)0,
OCI_ATTR_HA_SRVFIRST, errhp))
checkerr (errhp, (sword)retcode);
else{
printf("found first server handle.\n");
/*get associated instance name, */
if (retcode = OCIAttrGet(srvhp, OCI_HTYPE_SERVER, (dvoid *)&instname,
(ub4 *)&sizep,
OCI_ATTR_INSTNAME, errhp))
checkerr(errhp, (sword)retcode);
else
printf("instance name is %s.\n", instname);
}
while(!retcode){
if (retcode = OCIAttrGet(eventhp, OCI_HTYPE_EVENT, (dvoid *)&srvhp,
(ub4 *)0,
OCI_ATTR_HA_SRVNEXT, errhp))
checkerr (errhp, (sword)retcode);
else{
printf("found another server handle.\n");
/*get associated instance name, */
if (retcode = OCIAttrGet(srvhp, OCI_HTYPE_SERVER, (dvoid *)&instname,

(ub4 *)&sizep,
OCI_ATTR_INSTNAME, errhp))
checkerr(errhp, (sword)retcode);
else
printf("instance name is %s.\n", instname);
}
}
OCIHandleFree((dvoid *)errhp, OCI_HTYPE_ERROR);
printf("Finished event callback function.\n");
}
4. Application must then decided what it wants to do when it receies an lA eent
/*Registering HA callback function. */
if (checkerr(errhp, OCIAttrSet(envhp, (ub4) OCI_HTYPE_ENV,
(dvoid *)evtcallback_fn, (ub4)0,
(ub4)OCI_ATTR_EVTCBK, errhp)))
{
printf("Failed to set register EVENT callback.\n");
return EX_FAILURE;
}
if (checkerr(errhp, OCIAttrSet(envhp, (ub4) OCI_HTYPE_ENV,
(dvoid *)envhp, (ub4)0,
(ub4)OCI_ATTR_EVTCTX, errhp)))
{
printf("Failed to set register EVENT callback context.\n");
return EX_FAILURE;
}
return EX_SUCCESS;
}
5. Client applications must link with the client thread or operating system thread library.
I.L. libthread or libpthread
Runtime Connection Load Balancing with OCI Session Pools
Runtime connection load balancing is basically routing work requests to sessions in a session
pool that best sere the work. It comes into eect when selecting a session rom an existing
session pool. Runtime connection load balancing is enabled by deault with Oracle Database
11g Release 1 or higher client talking to a serer o Oracle Database 10g Release 2 or higher.
Setting the mode parameter to OCI_SPC_NO_RLB when calling
OCISessionPoolCreate() disables runtime connection load balancing.
1o utilize runtime connection load balancing with your OCI Session Pool, the ollowing
steps required:
1he application must hae been linked with the threads library.
1he OCI enironment must be created in OCI_EVENTS and OCI_THREADED
mode.
Conigure the RAC serice with a GOAL, CLB_GOAL , and
AQ_NOTIFICATIONS to TRUE. lor example:
dbms_service.modify_service(service_name=>crm,
aq_ha_notifications=>true, goal=>service_time,
clb_goal=>short);
Oracle Notification Service Application Programming Interface (ONS API)
Oracle proides the ONS API or both C and Jaa clients. By using this API, your program
can process eents receied rom the local ONS daemon. Regardless o the programming
language o choice, an ONS daemon needs to be conigured and deployed at the client
application machine where database connections are being managed. 1o use the Oracle

Notiication Serice Application Programming Interace, your Oracle Client and Oracle
Database must be at least 10.1.0.3. lor security reasons, your application program must run
using the same user as the owner o the ONS daemon. A sample jaa program
3
that will
receie lAN eents and print them is included in Appendix B, sample C program in
Appendix C. Note: 1his method is not recommend as the lAN eents may changed with
any release o Oracle RAC and any programs that use the lAN eents directly may break.
CONCLUSION
Oracle Real Application Clusters proides many eatures that enterprise applications can take
adantage o or ery high aailability and scalability. 1he high aailability o your application
depends on your notiication and repair policies or the computing enironment running the
application. On the database serer, the RAC lA ramework proides notiications o any
change in the cluster coniguration. 1he application can subscribe to lAN eents and react
quickly so their users can immediately take adantage o additional resources and are
unaected ,or minimally aected, by a reduction in aailable resources. Applications are
proided with greater lexibility to manage the arious workloads that the database must
execute within gien serice leels. Using the lAN callouts and lAN eent system, repair
processes are inoked immediately when the ault is detected, and applications including
disaster recoery are integrated end to end, eliminating time-outs and wasted time spent
retrying.

3
A sample jaa program can be ound
http:,,www.oracle.com,technology,sample_code,products,rac,index.html

APPENDIX A ONS OPERATION
ONS is controlled through a utility script named onsctl located in ORACLE_HOME/bin.
1his command line utility accepts the ollowing commands:

ORACLE_HOME/bin/onsctl help

usage: ORACLE_HOME/bin/onsctl
start|stop|ping|reconfig|debug

start - Start opmn only.
stop - Stop ons daemon
ping - Test to see if ons daemon is running
debug - Display debug information for the ons
daemon
reconfig - Reload the ons configuration
help - Print a short syntax description
(this).
detailed - Print a verbose syntax description.
1o start ONS execute the ollowing command:
$ onsctl start
onsctl: ons started

1o test the ONS daemon use ping:
$ onsctl ping
ons is running ...

1o eriy the coniguration use the debug option:

$ onsctl debug
HTTP/1.1 200 OK
Content-Length: 1205
Content-Type: text/html
Response:
======== ONS ========
Listeners:
NAME BIND ADDRESS PORT FLAGS SOCKET
------- --------------- ----- -------- ------
Local 127.000.000.001 6100 00000142 7
Remote 139.185.140.063 6200 00000101 8
Request No listener

Server connections:

ID IP PORT FLAGS SENDQ WORKER BUSY SUBS
------- --------------- ----- -------- ---------- -------- ------ -----
1 130.035.176.193 6200 00010026 0 1 0

Client connections:

------- --------------- ----- -------- ---------- -------- ------ -----
Pending connections:
------- --------------- ----- -------- ---------- -------- ------ -----
0 127.000.000.001 6100 00020812 0 1 0
Worker Ticket: 0/0, Idle: 180

THREAD FLAGS
-------- --------
4 00000012
5 00000012
7 00000012

Resources:
Notifications:
Received: 0, in Receive Q: 0, Processed: 0, in Process Q: 0

Pools:
Message: 24/25 (1), Link: 25/25 (1), Subscription: 0/0 (0)

1he output rom the debug command is quite comprehensie. 1he dierent sections show
the ONS coniguration:
Listeners Shows the IP address and the port inormation or the local and
remote addresses.
Server connections show the serers and ports that this daemon is aware o.
Initially this will correspond to the nodes entry in ons.config, but as other
daemons are contacted any hosts they are in contact with will appear in this section.
1he other sections in the debug output gie inormation relating to current and
preious actiity. Messages sent and receied, threads currently actie and so orth.
\hen troubleshooting the ONS, there are some optional parameters in the ONS.conig ile
that may assist. 1hese include:
loglevel 1his speciies the leel o messages that should be logged by ONS. 1his
alue is an integer that ranges rom 1 ,least messages logged, to 9 ,most messages
logged, use only or debugging purposes,. 1he deault alue is 3.
logfile 1his speciies a log ile that ONS should use or logging messages. 1he
deault alue or logile is >ORACLL_lOML,opmn,logs,ons.log.
walletfile A wallet-ile is used by the Oracle Secure Sockets Layer ,SSL, to store SSL
certiicates. I a wallet ile is speciied to ONS, it will use SSL when communicating
with other ONS instances and require SSL certiicate authentication rom all ONS
instances that try to connect to it. 1his means that i you want to turn on SSL or
one ONS instance, you must turn it on or all instances that are connected. 1his
alue should point to the directory where your ewallet.p12 ile is located.

APPENDIX B SAMPLE JAVA PROGRAM USING ONS API
/*
* Copyright (c) 2001, 2004 by Oracle. All Rights Reserved
* ONC Subscription client. This client listens for all events ONS
* receives
* Based on the event type decisions are made on how and whether to
* print the event body.
* NOTE: The events are subject to change at any release of Oracle and any
* Program that subscribes directly to FAN events may break
*/

import oracle.ons.*;
import java.util.*;
import java.io.*;
import java.nio.*;

public class onc_subscriber
{

public static void main(String args[])
{
boolean debug = false;
// Set ONC-required System property for oracle.ons.oraclehome:
//System.setProperty("oracle.ons.oraclehome",
"/home/orauser/product/oracle/10gClient");
//Subscriber s = new Subscriber("\"oraDb/myProdCluster\"",
"database/event/*");
//Subscriber s = new Subscriber("", "database/event/*");
Subscriber s = new Subscriber("", ""); // subscribe to all events
Notification e;

System.out.println("ONC subscriber starting");

boolean shutdown = false;
while (!shutdown)
{
e = s.receive(true); // blocking wait for notification receive

System.out.println( "** HA event received -- Printing header:" );
e.print();
System.out.println( "** Body length = " + e.body().length);
System.out.println( "** Event type = " + e.type());

/* Test the event type to attempt to determine the event body format.
Database events generated by the racg code are "free-format" events - the
event body is a string. It consists of space delimited key=value pairs.
Events constructed using the ONC Java API have an event body that is a byte
array (byte []) which have a terminating character. The Java API call to read
the body waits on this character to terminate the read.
*/
if (e.type().startsWith("database")) {
if (debug) { System.out.println( "New print out"); }
evtPrint myEvtPrint = new evtPrint(e);
} else if (e.type().startsWith("javaAPI")){
oncPrint myPrint = new oncPrint(e);
} else {
System.out.println("Unknown event type. Not displaying body");
}

try
{
if (e.type().equals("onc/shutdown")) {
System.out.println("Shutdown event received.");
shutdown = true;
}
else {
java.lang.Thread.currentThread().sleep(100);
System.out.println("Sleep and retry.");
}
}
catch (Exception te)
{
te.printStackTrace();
}
}

s.close();
System.out.println(" ONC subscriber exiting!");
}
}


APPENDIX C SAMPLE C CODE USING ONS API

/* Create a subscriber that will listen to all events */
s = ons_subscriber_create("",database/event/service", &message );

if (!s)
{
fprintf(stderr, "Error: FAILED TO CREATE SUBSCRIBER,
message = %s\n",message ? message : "NULL");
fflush(stderr);
exit(-1);
}

recvthreadsready++;

/* Keep receiving notifications until shutdown event: */
while (!done)
{
body_len = 0;
/* Specify a blocking receive with an indefinite wait */
e = ons_subscriber_receive(s, 1, -1);

if (e)
{
/* Set shutdown flag if shutdown event received */
if (strcmp(ons_notification_type(e),
"ons/test/shutdown") == 0)
done=1;
numevents++;

/* If the event is generated by Oracle RAC system
perform additional processing */

if (!strcmp(ons_notification_type(e),
"database/event/service"))
{
event_type = "SERVICE";
}
else
{
event_type = "OTHER";
}
/* Retrieve body of event */
body_text = (char *) ons_notification_body(e,&body_len);

/* If the event has a payload (body), then parse this
free-format payload breaking each token up in to the appropriate
name value pairs and assigning to an array */

if (body_text) {
creationTime = (time_t) ons_notification_creationTime(e);
deliverTime = (time_t) ons_notification_deliveryTime(e);
body_copy = (char*) malloc (strlen(body_text));
strcpy(body_copy,body_text);
tok = strtok_r(body_copy," ",&name_value);
while ( tok != NULL) {
sub_tok = tok;
sub_tok = strtok(sub_tok,"=");
if (!strcmp(sub_tok,"VERSION")) {
sub_tok = strtok(NULL,"=");
body_array[0] = (char *) malloc(strlen(sub_tok)+1);
strcpy(body_array[0],sub_tok);
} else if (!strcmp(sub_tok,"service")) {
} else if (!strcmp(sub_tok,"database")) {

} else if (!strcmp(sub_tok,"host")) {
} else if (!strcmp(sub_tok,"status")) {
} else if (!strcmp(sub_tok,"reason")) {
} else if (!strcmp(sub_tok,"card")) {
} else if (!strcmp(sub_tok,"instance")) {
}
tok = strtok_r(NULL," ",&name_value);
}
}

if (printevents) {
/* Display the event body - payload */
ons_notification_print(e);
fprintf(stderr,"The notification body is:\n Length = %d
%s ! \n",body_len,body_text);
}
/* Perform action based on event contents */
if (body_text) eventAction(event_type,body_array,creationTime,deliverTime);

free(body_copy);

/* finished with event */
ons_subscriber_relinquish(s, e);

/* Re-initialise array of values and free memory*/
for (i=0; i< ARRAY_ELEMENTS;i++) {
free(body_array[i]);
body_array[i] = NULL;
}
}

}

APPENDIX D TURNING ON LOGGING WITH JDBC

I you are testing your JDBC enironment and want to see more detail o what is actually
happening, you can turn on logging rom your JDBC Data Source. lere are the steps with a
sample properties ile. 1he JDBC demo dir also has a sample logging properties ile --
OracleLog.properties.
1. Use JDK 1.4
2. Use debug jar ojdbc14_g.jar rom >ORACLL_lOML,jdbc,lib. I.L. rename the
ojdbc14.jar ,something like ojdbc14.jar_sae, and then rename ojdbc14_g.jar to
ojdbc.jar,. 1his needs to go in your CLASSPA1l
Include a properties ile, with contents ,be careul o the word line wrapping,:
============
handlers= java.util.logging.ConsoleHandler
# default file output is in user's home directory
java.util.logging.FileHandler.pattern = jdbc.log
java.util.logging.FileHandler.limit = 50000
java.util.logging.FileHandler.count = 1
java.util.logging.FileHandler.formatter=java.util.logging.SimpleFormatter
# Setting this to SEVERE avoids duplicate output from
# default logger
java.util.logging.ConsoleHandler.level = SEVERE
java.util.logging.ConsoleHandler.formatter=java.util.logging.SimpleFormatter
oracle.jdbc.level = FINEST
oracle.jdbc.pool.level = FINEST
==============
Some output goes to StandardOut so you may want to redirect to a ile.
3. Set the ollowing when starting the test:
...-Doracle.jdbc.trace=true
-Djava.util.logging.config.file=<properties file location> ...
\our JDBC log should show:
Calls to initlailoerParameters - this shows you that last Connection ailoer is
enabled
lCl eent1ype and eentBody ,showing the ull eent, - this shows you JDBC
got the eent when a ailure occured
Calls to abortConnection - this shows you that JDBC is cleaning up the
connections to the ailed instance.

Automatic Workload Management with Oracle Real Application Clusters
July 2007
Author: Barb Lundhild
Contributing Authors: Carol Colrain, Troy Anthony, Daniel Semler, Rajkumar Irudayaraj

Oracle Corporation
World Headquarters
500 Oracle Parkway
Redwood Shores, CA 94065
U.S.A.

Worldwide Inquiries:
Phone: +1.650.506.7000
Fax: +1.650.506.7200
oracle.com

Copyright 2007, Oracle. All rights reserved.
This document is provided for information purposes only and the
contents hereof are subject to change without notice.
This document is not warranted to be error-free, nor subject to any
other warranties or conditions, whether expressed orally or implied
in law, including implied warranties and conditions of merchantability
or fitness for a particular purpose. We specifically disclaim any
liability with respect to this document and no contractual obligations
are formed either directly or indirectly by this document. This document
may not be reproduced or transmitted in any form or by any means,
electronic or mechanical, for any purpose, without our prior written permission.
Oracle, JD Edwards, PeopleSoft, and Siebel are registered trademarks of Oracle
Corporation and/or its affiliates. Other names may be trademarks
of their respective owners.

Automatic Workload Management With RAC

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Automatic Workload Management With RAC

Caricato da

Copyright:

Formati disponibili

Automatic \orkload Management

with Oracle Real Application

Potrebbero piacerti anche