December 6, 2010Alert Log Monitoring script via ADRCI

Filed under: How To, Tips coskan @ 4:47 pm

20 iRate This
Before I start to write about the blog series in my mind which will be the base
to my first presentation again in my mind, I would like to share simple alert lo
g monitoring with adrci.
At first I was completely against Diagnostics Dest idea but after a bit of searc
hing and learning new tips tricks of adrci command like I think I am a big fan o
f diagnostics dest. Below is the script I wrote to monitor alert log hourly and
with a daily summary. Script is self explanatory, the only thing it does is gets
ADRCI homes put them in a bash array and grep the ORA-, TNS- errors for the las
t hour or last day. It is not rocket since and it is based on a simple command w
hich can also be added to your profiles as an alias.
--last day
adrci exec="set home ${adrci_home}; show alert -p \\\"message_text like '%ORA-%'
and originating_timestamp > systimestamp-1\\\""
--last hour
adrci exec="set home ${adrci_home}; show alert -p \\\"message_text like '%ORA-%'
and originating_timestamp > systimestamp-1/24\\\""
I am pretty sure this could be shortened but I did not bother to shorten it so I
leave it to your imagination:)
I did not add listener checks to hourly checks but to daily summary, because the
re can be many which will cause you to get mail every hour.
Good part of using ADRCI and hourly checks is you do not need to edit the file t
o get rid of the error because you will never see that alert again in the next h
our slot.
Script will send the output to the files and read those files back to the mail c
ontent and I am sure there may be better way to do this.
Complete script
#################################################
###### ALERT LOG CHECKING VIA ADRCI #############
#################################################
# Author : Coskan Gundogar
# Version Date: 02/12/2010
# Usage :
# To run for last 24 hours - ./check_alert.sh D
# To run for last hour - ./check_alert.sh
# Edit variables below for moving between servers
# Changes To the Script
#
export ORACLE_SID=+ASM1
export ORACLE_HOME=/u01/crs/oracle/product/11.2.0/grid/
export PATH=$PATH:$ORACLE_HOME/bin
DAILY_LOG=/home/oracle/bin/alert_log_check_daily.txt
HOURLY_LOG=/home/oracle/bin/alert_log_check_hourly.txt
MAIL_SUBJ="PRD:WARNING HOST: "
MAIL_RECIPIENT="your_dba_group@your_company.com"
HOST_NAME=`hostname -a`
if [ "$1" = "D" ]
then
############################################
###############DAILY CHECKS ################
############DBMS AND ASM CHECK##############
############################################
adrci_homes=( $(adrci exec="show homes" | grep -e rdbms -e asm))
echo '####################################################' > $D
AILY_LOG
echo '####### ALERT LOG OUTPUT FOR LAST 24 HOURS #########' >> $
DAILY_LOG
echo '####################################################' >> $
DAILY_LOG
echo '' >> $DAILY_LOG
echo '' >> $DAILY_LOG
echo '' >> $DAILY_LOG
for adrci_home in ${adrci_homes[@]}
do
echo $adrci_home' Alert Log' >> $DAILY_LOG
adrci exec="set home ${adrci_home}; show alert -p \\\"me
ssage_text like '%ORA-%' and originating_timestamp > systimestamp-1\\\"" -term >
> $DAILY_LOG
done
############################################
############## DAILY CHECKS ################
############# LISTENER CHECK###############
############################################
adrci_lsnr_homes=( $(adrci exec="show homes" | grep -e tnslsnr))
echo '' >> $DAILY_LOG
echo '' >> $DAILY_LOG
echo '' >> $DAILY_LOG
echo '####################################################' >> $
DAILY_LOG
echo '###### LISTENER LOG OUTPUT FOR LAST 24 Hours #######' >> $
DAILY_LOG
echo '####################################################' >> $
DAILY_LOG
echo '' >> $DAILY_LOG
echo '' >> $DAILY_LOG
echo '' >> $DAILY_LOG
for adrci_lsnr_home in ${adrci_lsnr_homes[@]}
do
echo $adrci_lsnr_home' Listener Log' >> $DAILY_LOG
adrci exec="set home ${adrci_lsnr_home}; show alert -p \
\\"message_text like '%TNS-%' and originating_timestamp > systimestamp-1\\\"" -
term >> $DAILY_LOG
done
num_errors=`grep -c -e 'TNS' -e 'ORA' $DAILY_LOG`
if [ $num_errors != 0 ]
then
MAIL_SUBJ=$MAIL_SUBJ$HOST_NAME" Errors Found in Daily Alert Summ
ary"
 mailx -s "$MAIL_SUBJ" $MAIL_RECIPIENT $HOURLY_LOG
echo '' >> $HOURLY_LOG
echo '' >> $HOURLY_LOG
echo '####################################################' >> $
HOURLY_LOG
echo '######### ALERT LOG OUTPUT FOR LAST HOUR ###########' >> $
HOURLY_LOG
echo '####################################################' >> $
HOURLY_LOG
echo '' >> $HOURLY_LOG
echo '' >> $HOURLY_LOG
echo '' >> $HOURLY_LOG
for adrci_home in ${adrci_homes[@]}
do
echo $adrci_home' Alert Log' >> $HOURLY_LOG
adrci exec="set home ${adrci_home}; show alert -p \\\"me
ssage_text like '%ORA-%' and originating_timestamp > systimestamp-1/24\\\"" -ter
m >> $HOURLY_LOG
done
num_errors=`grep -c -e 'TNS' -e 'ORA' $HOURLY_LOG`
if [ $num_errors != 0 ]
then
MAIL_SUBJ=$MAIL_SUBJ$HOST_NAME" Errors Found in Hourly Alert Sum
mary"
mailx -s "$MAIL_SUBJ" $MAIL_RECIPIENT < $HOURLY_LOG
fi