BewareoftreatingNoncriticalresourceerrorsas

warningsMyLoadTest
ByStuartMoncrieffonApril2nd,2013
OneofthemostpoorlyunderstoodVuGenRuntimeSettingsistheonethatallowsaperformancetesterto
treatnoncriticalresourceerrorsaswarnings.ThemajorityofLoadRunnerusersIhavespokentodonot
realisethatenablingthissettingcanhideseriousloadrelateddefectsinthesystemundertest.

LoadRunnerautomaticallydetectsHTTP4xxor5xxresponsesbut,ifthisruntimesettingisenabled,
LoadRunnerwillignoretheHTTP4xxor5xxerrorcodewhenrequestingaresource(i.e.thecurrent
transactionwillnotfail).

Whatarenoncriticalresources?
Thewordingofthisruntimesettingisabitmisleading.Thereisnosuchthingasanoncriticalresource,
justresources,andnonresources.Examplesofcontenttypesthatareclassifiedasresourcesbydefault
are:images(PNG,GIF,JPEG),stylesheets(CSS),JavaScript,JSONand(sometimes)XML.Whetheran
HTTPrequestisforaresourceoranonresourceisdeterminedduringthescriptgenerationstep,
immediatelyafterrecording.
TheVuGenrecordingsettingsidentifynonresourcesbytheirMIMEtype.Bydefault,onlytext/htmland
text/xmlareidentifiedasnonresources.ThisomitssomeratherimportantMIMEtypesthatarecommonly
usedfordynamiccontent,suchasapplication/xmlandapplication/json.

WhenthesourcecodeoftheVuGenscriptisgenerated,requestsfornonresourcesareidentifiedbya
web_urlorweb_custom_requestfunctionwithaResource=0argument(theweb_submit_data,and
web_submit_formfunctionsarealwaysrequestsfornonresources,sotheydontneedthisfunction
argument).
web_url("portal",
"URL=http://www.example.com/portal",
"Resource=0",//<look!It's*not*aresource
"RecContentType=text/html",
"Referer=",
"Snapshot=t1.inf",
"Mode=HTML",//<HTMLmodemeansthattheHTMLsourceisparsedbythereplay
engine,andany"resources"thatarefoundareautomaticallydownloaded
EXTRARES,
"Url=/static/splashscreen.png",ENDITEM,//<anEXTRARES(OURCE)isa
"resource"thatwasdownloadedduringrecording,butwasnotfoundbytheHTMLparser
"Url=/static/background.png",ENDITEM,
"Url=/static/formvalidation.js",ENDITEM,
LAST);

web_url("loginform.js",//Iftherecordingenginecannotfindaparentrequestfora
resourceitsawduringrecording,itwillgenerateaseparateweb_urlstatementwith
"Resource=1"
"URL=http://www.example.com/static/loginform.js",
"Resource=1",//<notefunctionargumentmakingthisa"resource"
"RecContentType=application/javascript",
"Referer=http://www.example.com",
"Snapshot=t2.inf",
LAST);
Duringreplay,LoadRunnerusestheResource=functionargumenttodetermineifarequestiffora
resourceoranonresource.

Noncriticalresourcesareactuallyreallyimportant

VuGenhasnowayofdeterminingwhatcontenttypesaredynamicallygeneratedontheserver,exceptby
lookingattheMIMEtypereturnedduringrecording.Sometimesdynamicallygeneratedcontentistreatedas
anoncriticalresource,causingtheperformancetestertoaccidentallyignorecriticalerrors.
MyfavouriteexampleofsomeonegettingthemselvesintodifficultiesisfromthetestingofanewHRself
serviceportal.UserscouldlogonandviewtheirpayslipsinPDFformat.EachPDFwasdynamically
generatedthefirsttimeitwasviewed,andcachedafterwards.Duringtheloadtesttherequestsforthe
PDFswerereturninganHTTP500error,buttheperformancetesterdidntnotice,asitwasanEXTRARES
inthescript,andhehadnoncriticalresourceerrorsaswarningsenabled.Theonlyreasonanyonefound
outaboutthisproblemwasbecausetheserveradminswerewonderingwhythereweresomanyexceptions
intheirappserverlogsafterthetest.IfyourscriptdownloadsaPDF,pleasemakesurethatyoucheckthe
PDFcontenttoverifythateverythingisworkingcorrectly.
Evenifyouhavesuccessfullyidentifiedallthedynamiccontenttypes,andLoadRunneristreatingthemas
nonresources,itisstillimportantthatallofthestaticcontentisdownloadedsuccessfully.Bywayof
example,hereisascreenshotofAmazon.com:

Hereisthesamepagewithmostoftheresourcesremoved.ALoadRunnerscriptwiththenoncritical
resourceerrorsaswarningsruntimesettingenableddoesnotdetectthisproblem,andgivesthe
transactionastatusofpass.Ifrealuserswereseeingawebsitethatlookedlikethiswhenthesystemwas
underload,itwouldbeaveryhighseveritydefect,notsomethingthatshouldbeignored.

IcanimaginethattheBusinessownersofthenewITsystemwouldbeveryunhappyiftheiruserswere
seeingabrokenwebpagelikethiswhentheirsystemwasunderload,andtheirperformancetesterhadnt
toldthemaboutit.

Howresourceerrorscanoccurunderload
Themainreasonforloadtestingisthatanapplicationcanbehaveverydifferentlyunderloadtohowit
behaveswhenthereisonlyasmallnumberofusers.Itmightrunslowly,orrequeststhatnormallyworkfine
mightsuddenlystartreturningerrors.HereisagreatexamplefromanASP.NETforum:
IhaveanASP.NETapplicationthatusesbothanHTTPModuleandseveralHTTPHandlers.Under
somewhatheavyload(~25req/sec)theapplicationbeginstoreturnHTTP404errorsforrequestsofa
specifichandler.
Evenpurelystaticcontentcanbehavedifferentlywhenthesystemisunderload.Thesequencediagram
belowshowsaproductpagebeingloadedsuccessfullyfromwww.example.com,buttheproductimagesare
notloadedsuccessfully,astheyarehostedonanotherserverwhichisoverloaded.PerhapsLoadRunner
getsaTCPrelatederrorduetoaconnectionlimit(failedtoconnecttoserverorserverhasshutdownthe
connectionprematurely)oranHTTPerrorcodeisreturned.

RecommendedRuntimeSettings
Webbasedscriptsshouldalwaysdisable(untick)theNoncriticalresourceerrorsaswarningscheckboxin
thescriptsruntimesettings.Thisistheonlywaytobesurethatthissettingisnothidingerrorsduringyour
test.

Ifyourapplicationseemstoworkfinemanually,butyourscriptfailsduringreplayduetoanHTTP404error
foraresource,thecorrectworkaroundistotemporarilyaddadownloadfilterfortheURLthatisfailing,
ratherthanhidingallerrorsofthistype.
//RequestsforthisresourceareconsistentlyreceivinganHTTP404response.This
//hasbeenraisedasdefect#2096.Whenthedefectisreadyforretesting,this
//filtercanberemoved.
web_add_auto_filter("Action=Exclude","URL=https://www.example.com/resources/form
validation.js",LAST);

Abaddefaultsetting
Ifyoureadthisfar,IamsurethatyouarewonderingwhyHPleavesthenoncriticalerrorsaswarnings
settingenabledbydefault.Weallknowthaterrorsarebad,andthattheaimoftestingunderloadisto
discovererrors,right?
Itiseasytoforgetwhatitwaslikewhenyouwerejuststartingasaperformancetester.Spareathoughtfor
ajuniorperformancetesterwhoisattemptingtorecordtheirfirstVuGenscript.Iftheirnewscriptfailsfora

confusingreason(likeaninvisible11pixelspacerGIFbeingmissing),thentheymightbecome
discouraged.Thereisanobvioustradeoffbetweenthis,andtheriskofhidingseriousapplicationerrorsthat
occurunderload.
Personally,IreallywanttohelpperformancetesterswritebetterscriptsthisiswhyIhavedevelopedthe
VuGenValidator,whichhighlightscommonscriptingmistakeslikeenablingthenoncriticalerrorsas
warningsruntimesetting.
Relatedposts: