TheDBADepartureSurvivalGuide

YourOnlyDatabaseAdministratorIsWalkingOutTheDoor.
IsYourBusinessPrepared?

Copyright2010Ntirety,Inc.
www.ntirety.com

Page1of11

TableofContents

TableofContents....................................................................................................................................... 2
TheUnexpectedHappens ...................................................................................................................... 3
IamGoingtoMakeYouanOfferYouCantRefuse..................................................................... 4
Scenario#1:TheOrElse................................................................................................................ 4
Scenario#2:TheTwoWeekNotice.............................................................................................. 4
Scenario#3:JetBlueDjVu ........................................................................................................... 4
CommonReasonsDBAsSeekOtherEmployment ................................................................. 5
SecurityIsYourBestFriend(UnlessYouDontKnowthePassword)............................... 7
OperatingSystemLevelPasswords.............................................................................................. 7
ChangeThePasswords ...................................................................................................................... 8
HardCodedPasswordsAnyone ..................................................................................................... 8
GettheForwardingAddress................................................................................................................. 8
KeyNightlyProcessing ........................................................................................................................... 9
TheStateoftheDatabaseBackup ...................................................................................................... 9
ProactiveMonitoringinPlace............................................................................................................10
DBADepartureSurvivalGuideSummary.....................................................................................10
HowToContactNtirety ........................................................................................................................10
AbouttheAuthor .....................................................................................................................................10

Copyright2010Ntirety,Inc.
www.ntirety.com

Page2of11

TheUnexpectedHappens
Atruckhitsyouronlydatabaseadministrator.YouronlyDBAisrushedtothe
Hospitalbyambulance.YouronlyDBAisgoingonvacationfor4weeksinanother
country.YouronlyDBAjustgaveyouhis2weeknotice.Whateverthereasonis
yourbusinesspreparedforsuchanevent?
Fact:Ifyourdatabasegoesdown,theapplicationsyourbusinessreliesonto
conductbusinesswillcometoagrindinghalt.Ifyouarenotpreparedforthis
outcome,itcouldputyououtofbusiness.
Yet,weallknowpeoplegetsick,haveaccidents,arepulledoutoftheofficewith
littleornowarningandpeopletakenewjobs.Giventherisktoyourbusiness,itis
onlyprudentthatyouhaveaplantodealwiththeunexpectedlossofyouronly
databaseadministrator.Manytimesbusinessesdont.
Inthispaper,Illdiscuss:

Whatyoucandotoprotectyourbusinessfromsuchanevent

Thingstoconsiderbeforethishappenstoyou

SimplequestionsyoushouldaskyourDBAbeforehe/shewalksoutthedoor
andnevercomesback.

ThemostimportantadviceIcangiveis:dontpanic.Evenifyouronlydatabase
administratorhasanunexpectedaccident,walksoffthejobwithoutnotice,slides
downaplanesemergencyslide(likeJetBlueflightattendantStevenSlaterwitha
beerinhishand),oryourbusinessislocatedinSinclair,Wyoming,population406
(yes,thereisadatabaseinSinclairWyomingthatneedstobemanaged),atNtirety,
wehaveyoucovered.Wecanandhaveprovidedimmediatedatabase
administrationsupportwithlittleornowarning.
Thissortofthinghappensalotmoreoftenthanyouwouldexpect.AtNtirety,we
haveteamsofU.S.baseddatabaseadministratorswhoareequippedtodealwith
thisoranyotheremergencyyourOracle,SQLServerorMYSQLdatabase(s)could
everhave.
Helpisjustaphonecallaway:(781)4747700orour7x24Emergencyline:
(18776847389)

Copyright2010Ntirety,Inc.
www.ntirety.com

Page3of11

IamGoingtoMakeYouanOfferYouCantRefuse
ThelineIamgoingtomakeyouanofferyoucantrefuse,remindsmeofthefamous
sceneinthemovieTheGodfather.Yet,asamanagerofabusiness,Iamsureyoucan
relatetothatscene.Wehaveallexperiencedanemployeewalkingintoouroffice
andmakingveryunreasonabledemands.Toillustratethispoint,letsconsider
somereallifescenarios.

Scenario#1:TheOrElse

ItisFridayafternoonat3:00oclockandyouronlyDBAwalksintoyouroffice.She
informsyou,IwantmoremoneyorIquit.

Scenario#2:TheTwoWeekNotice

YouronlySQLServerdatabaseadministratorwalksintotheoffice.Ihavejust
acceptedanotherjoboffer.Iamofficiallygivingyoumytwoweeknoticeeffective
immediately.

Scenario#3:JetBlueDjVu

Onedayinthemiddleoftheweek,yourDBAwalksoffthejobwithoutwarning.
Couldbeworse,theycouldhaveopenedupacanofbeerandsliddowntheplanes
emergencychutelikeStevenSlaterfromJetBlueAirlines.

InthecaseofScenario#1,theOrElse,whenyourDBAwalksinanddemands
moremoney,youknowwhatyoushoulddodontletthedoorhitthepersoninthe
*_sonthewayout.

Copyright2010Ntirety,Inc.
www.ntirety.com

Page4of11

ManytimesatNtirety,wehavecomeintobusinesseswherewefindouttheymetan
employeesunreasonabledemandsnotonce,nottwice,butthreetimes.Itsafter
thethirdtimethatwetypicallygetaskedin,asthebusinessfinallyrealizedthehard
lessonthatenoughwasneverenoughtheemployeewhowalksinandmakessuch
ademandfeelsentitled,believestheyareindispensible,andthattheemployerhas
nochoicebuttomeettheirdemands.
Inmyexperiencetoevenconsidersuchademandalwaysleadstonogood.The
precedentyouaresettingforthebusinesswillbedisastrous.Bottomline:nooneis
indispensible.Anemployeewhomakessuchademandinthismannerneedstobe
shownthedoorandneverinvitedback.
NowtakeScenario#2HereismyTwoWeekNotice.Whenanemployeegives
youtheir2weeknotice,youhaveaproblem.IfthisisyouronlyDBAandyour
databasegoesdown,nooneistheretorespond.Thentheapplicationsyour
businessreliesontoconductbusinesswillcometoagrindinghalt.
Facedwiththis,manytimesbusinessesmakeacounteroffertogettheironlyDBA
tostayon.Yetbeforeyouconsidersuchastrategyasabusinessyouneedtobe
honestwithyourself.Whatistheunderlyingreasonthispersonisleaving?Ifyoudo
counterofferandyoudontaddresstheunderlyingreasonswhythispersonis
leavingmyexperiencehastaughtmeyouwillbewastingyourtimeandmoney.
Askyourselfifmoneytheonlyreasontheyareleaving?Mosttimestheanswerto
thisquestionisno.Ifyourcompanyispayingafairwage,thenthereareother
issuesatworkhere.

CommonReasonsDBASeekOtherEmployment

Moremoney
Seekingmorechallengingwork:
o TheywanttoworkonSQLServer2008andallofyourapplications
arestillrunningonSQLServer2000
o Youarerunningthestandardeditionandtheywanttousethe
featuresintheenterpriseedition
o Theywanttoworkinahighavailabilityenvironment(OracleRAC,
SQLServerClustering,replication)
Bettercommute
Tiredoftheendlessnights&weekendscalls
Wanttobepartofateam(tiredofbeingthelonewolf)
Accesstotraining
Careeradvancement

Letmesaythisagain,ifyoudocounterofferandyoudontaddresstheunderlying
reasonsforDBAattrition,youarewastingyourtimeandmoney.Itwillbejusta
matteroftimebeforethecorereasonsthatdrovethemtothepointofleavinginthe
Copyright2010Ntirety,Inc.
www.ntirety.com

Page5of11

firstplacetakeoveragainandtheyleave.Atthatpointintimenocounterofferyou
couldpossiblygivewillkeepthem.
Earlyinmycareer,Iworkedforamajorcomputercompany.Igavemynotice.To
keepme,theygavemealotmoremoneyandadoublepromotion.Istayed.Within
afewmonths,Imovedonanyway.Thecorereasonsthatdrovemetothepointof
lookingforanotherjobwereneveraddressed.
Finally,letslookatScenario#3:JetBlueDjVu.Wecomeacrossmany
companiesthathavethepotentialfortheirDBAtoonedayupandleaveina
dramaticfashionlikeStevenSlateratJetBlueAirlines.Weseemanycompaniesthat
arepushingpeopleveryclosetothebreakingpoint.
Toillustratethispoint,Iwilltalkaboutacompanywithwhichwearecurrently
working.Whenyoureadthisstory,beawarethatfrommyexperiencethishappens
farmorefrequentlythanyouwouldthink.Wearecurrentlyworkingwitha
customerwhokeepslosingtheironlytwodatabaseadministratorsevery6To9
months.WhenwecameinthedoortoprovidesupportasthelastDBAwasleaving
welearnedafewinterestingfacts.

TheyhavetwoDBAsbuttheyalwayshaveenoughworktokeepfourDBAs
busy.Intodaysworldofdoingmorewithless,thisisquiteacommon
situation.
TheDBAsaresobusydealingwithlastminuterequeststhattheyneverhave
timetodobasicmaintenanceonthedatabase.Maintainingthedatabase,
theyweretoldbymanagementwasnotapriority.IwonderhowBritish
Petroleumwouldfeelaboutthestatementmaintenanceisnotapriority
today.
Whenwereviewedthebackupsofthedatabases,theywerenotworking
correctly.Wefindthistobethecaseinabout1in5clientswestarttowork
with.
Therewasnoproactivemonitoringofthedatabaseinplacesoeverything
thatwasbeingdonewasconstantlyreactive.TheDBAteamswerealways
puttingoutfires.
Theneverendingnightsandweekendcallswereburningthedatabase
administratorsout.
Nomatterhowhardtheyworked,theDBAscouldnevercatchup.

Givenalltheissuesatthisclientsite,nomatterhowmuchmoneytheypaidtheir
DBAsitsjustamatteroftimebeforetheyleave.Thecompanyneedstobehonest
withitselfanddealwiththerootproblemwhytheykeeplosinggoodpeople.
InthecaseofScenario#3JetBlueDjvu,youhavetoaskyourselfiftheissueis
thepersonoristheissuetheworkenvironment.Haveyoucreatedawork
environmentthatputstheDBAundersomuchstressitsjustamatteroftimebefore
Copyright2010Ntirety,Inc.
www.ntirety.com

Page6of11

theybreak.BeingaDBAisaverystressfuljobtobeginwith.Itis7daysaweek,24
hoursaday.But,everyoneneedstobeabletogetabreak.
AtNtiretywehaveclientswhojustuseour7x24databaseservicedesktotakefirst
callsfortheirinternalDBAstaff.ByusingtheNtiretyDatabaseServiceDesktheyare
abletogivetheirinternalDBAstafftheirnightsandweekendsback.Thisprevents
theirDBAstafffromburningout.
Itshardenoughtolosegoodpeople.Itsimportantnottolosethemforthewrong
reasons.Remember:whentheywalkoutthedoor,alltheirinstitutionalknowledge
walksoutthedoorwiththem.

SecurityIsYourBestFriend(UnlessYouDontKnowthe
Password)

Everbeenlockedoutofyourownhomeandbeenforcedtowalkaroundyourhouse
lookingforawindowyouleftunlocked?Youronlydatabaseadministratoris
leavingintwoweeks.Makesureyougetthepasswordstothedatabaseaccounts
youwillneedbeforetheywalkoutthedoor.

Database
Oracle
MicrosoftSQLServer
MySQL

Account
SysAccount
SaAccount
SuperUserAccount

WhatifyouronlyDBAisrunoverbyatruck?Thinkthiscanthappen?Itcan.Think
ofhowmanytimesinyourownlifeyouhavehadtocallanambulanceorseen
someonetakenawayinanambulance.ImagineitisyouronlyDBAbeingcarried
away.Asabusinessyoumustsetupacontingencyplan.Itcouldbeassimpleas
puttingthepasswordsinasealedenvelopeinasafeonceamonth.Thissimplestep
couldpreventyoufromwhatcouldbecomeaverycostlymistake.

OperatingSystemLevelPasswords

Youwillalsoneedtheoperatingsystempasswordsforalltheaccountsthathouse
databasecodeorapplicationcode.Forexample,itisverycommontohavethe
OracleDatabaseinstalledinanaccountattheoperatingsystemnamedOracle.
Makesureyougetthepasswordtothataccount.InthecaseofSQLServertherewill
beanaccountattheWindowsoperatingsystemlevel.Thesamefollowsforthe
MySQLenvironment.
Whenindoubt,trytogetthesuperuseraccountfortheoperatingsystem.For
example,inaUnixenvironmentthatwouldbetherootaccount.
Copyright2010Ntirety,Inc.
www.ntirety.com

Page7of11

Dontforgetanynetworkpasswordsthatmaybeoutthere.Manyshopsputa
passwordontheOracleListener.Thelistenerishowaclientwouldtypically
connecttoanOracleserver.ItisbesttoasktheDBAforallthepasswordsthey
know.

ChangethePasswords

Whenadatabaseadministratorisleavingyourcompany,itisimportantthatyou
changethepasswords.Thisisaclearindustrybestpracticenoonewouldever
dispute.YoudothistonotonlyprotectyourcompanybutalsotoprotecttheDBA
whoisleaving.

HardCodedPasswordsAnyone?

Weallknowthathardcodedpasswordsarenotagoodideaforsomanyreasons.
Withthatsaid,asktheDBAthequestion,IfIchangethepassword,isthereanything
runningouttherethatmightbreak?Youwouldbesurprisedbyhowmanytimes
weseetheDBAstopintheirtracksandthensayyes,nowthatyouasked..
IndefenseoftheDBA,theyaretypicallyoverworkedandhavetocutcorners.So
theysetupjobstohelpthemmanagethedatabasequicklyandneverfindthetime
togobackandremovethehardcodedpasswords.

GettheForwardingAddress
ItisveryimportantthatyouasktheDBAwhoisleavinghowcanwereachyouif
wehaveanyquestions?Askthemiftheywouldbewillingtoansweranyquestion
youmighthaveaftertheyleave.Askfortheirphonenumberandanemailaddress
youmightuse.
Youhavetorespectthefactthattheyaremovingon.Onlyreachouttothemwhenit
isreallyneeded.Manytimescompaniesabusethisprivilegeoverandoverand
wonderwhythepastemployeewontreturnthecall.
Copyright2010Ntirety,Inc.
www.ntirety.com

Page8of11

KeyNightlyProcessing
Areyouaretailoperationwhereeachstorereportsineverynight?Perhapsyou
haveanumberofscheduledjobsthatruneachhourgatheringkeyinformationon
whatwassoldineachstorebysize,colorquantityandprice.Nomatterwhat
businessyouarein,thereistypicallykeyprocessingthathappensduringtheday
andnight.Whenitbreaks,itcouldmeanyourbusinessiswithoutkeyinformationit
needstobecompetitive.
Findingtheproblemcouldbelikelookingforaneedleinthehaystack.AsktheDBA
beforetheyleavetoshareanyinformationanddocumentationtheyhaveonkey
processingthathappens.

TheStateoftheDatabaseBackups
Therearelotsofwaystobackupadatabase.Youhavehotbackups,coldbackups,
fullbackups,incrementalbackups,andexportstonamejustafew.Somebackupsgo
todisksandthenarecopiedofftotape.OtherbackupsgodirectlyofftotheCloud.
AsktheDBAbeforetheyleaveaboutanydocumentationtheymayhaveonthe
databasebackups.Askthemwhatarethemostcommonproblemsthatoccurwith
thebackups?Itiscommontofindoutthatthereisnotenoughdiskspace,andthat
theDBAisspendingalotoftimetroubleshootingbackupissueseachnight.

Copyright2010Ntirety,Inc.
www.ntirety.com

Page9of11

ProactiveMonitoringinPlace
ManytimesDBAssetupsomebasicjobstomonitorthehealthofthedatabase.This
consistsofacollectionofhomemadescripts.Itisveryimporttohavesomelevelof
proactivemonitoringinplace.Bettertohavehomemadescriptsinplacethen
nothingatall.AsktheDBAifhehasanymonitoringinplace?Ifso,howdoesit
reachoutwhentheresaproblem?Manytimesitsassimpleasanemailthatgoes
out.Ifthatisthecase,makesurethatsomeoneisgettingthoseemails.

DBADepartureSurvivalGuideSummary
Insummary,

Dontpanic,youhaveoptions
UnderstandtheunderlyingreasonstheDBAisleaving
Getthekeypasswords
o DatabaseLevel
o O/SLevel
Changethepasswords
GetcontactinformationforthedepartingDBA
Obtainalldocumentationyouhaveonkeydaily/nightlyprocessing
Understandhowthedatabaseisbeingbackedup
Understandanyproactivemonitoringinplace

Nomatterwhat,rememberyouhaveoptions.

HowtoContactNtirety
NtiretyisthepioneerofDatabaseAdministrationasaService.AtNtirety,our
businessis100%focusedondatabaseadministration.Ourstaffis100%U.S.based.
Weareveryproudofourtenyeartrackrecordofnearperfectclientqualityscores.
Ifyouareinterestedinlearningmore,pleasecontactus:

Sales@Ntirety.com
17814747700

Ntirety,Inc
980WashingtonStreet,Suite124
Dedham,Massachusetts02026

AbouttheAuthor

MichaelCoreyisthefounderandChiefExecutiveOfficerofNtiretyInc.An
entrepreneur,authorandrecognizedexpertonRelationalDatabases,Database
AdministrationasaServiceandDataWarehousing,MichaelfoundedNtirety,Inc
Copyright2010Ntirety,Inc.
www.ntirety.com

Page10of11

inFebruary2001.AsChiefExecutiveOfficer,MichaelCoreyleverageshisextensive
experienceandbusinesssavvytomanageandexpandNtiretyintothepremier
DatabaseAdministrationasaServicefirmintheUnitedStates.

Michaelisafrequentspeakerattechnologyconferencesthroughouttheworldand
haswrittennumerousarticlesandbooks,publishedbyOsborneMcGrawHill,on
topicsrangingfromSQLServer7DataWarehousingtoOracleDatabase11g:A
BeginnersGuide.

MichaelisapastpresidentoftheIndependentOracleUsersGroupandafounderof
theProfessionalAssociationofSQLServer.Michaelistheoriginalrecipientofwhat
isnowknownastheChrisWooldridgeAwardfromtheIndependentOracleUsers
Group.MichaelreceivedagubernatorialappointmenttotheMassachusettsRobert
H.GoddardCouncilonScience,Technology,EngineeringandMathematics.

CheckoutMichaelsBLOGonallthingsrelatedtoremotedatabaseadministration:
http://michaelcorey.ntirety.com

CheckoutMichaelsTwitterAccount:
http://www.twitter.com/Michael_corey

Forquestionsorcommentsyoumayhave,contactMichaelviaemail:
Michael.corey@ntirety.com

Copyright2010Ntirety,Inc.
www.ntirety.com

Page11of11