1

Performance audit: contributing to accountability in democratic government1 TomLing

Summary State Introduction Audit Institutions have become a key part of the architecture of accountabilityindemocraticsocietiesandtheyhavedevelopedtheirskills,theircontacts, and the range of work they take on during recent decades. This has created the risk of missioncreepandthispaperidentifieswherethecorestrengthsofSAIslieandsuggests how best to exploit these strengths in order better to contribute to accountability in democraticgovernment.

Introduction StateAuditInstitutions(SAIs)had,bytheendofthe1990s,becomeaknownquantity,a familiarpresenceintheprocessofgovernance(Pollittetal1999,219).Inthispaperitis arguedthat,asSAIsbecomeamorefamiliarpartofthelandscape,theymaybetempted intostrayingfromtheircoremission.Thispaperarguesagainstsuchdevelopment.

1

This paper draws upon the final chapter of Lonsdale J, Wilkins P, and Ling T

(editors) (forthcoming). The author gratefully acknowledges the key contributions made by both his fellow editors both in the development of this paper and in their contributions to discussions and debates enjoyed over many years. The views expressed here do not necessarily reflect the views of Jeremy Lonsdale nor his employer the UK National Audit Office, nor the views of Peter Wilkins and his employer, the Ombudsmans Office of Western Autralia.

2 However,iftheyaretooccupytheirmorefocussedroleeffectivelytherearechanges theyneedtomakeandbuttressesrequiredinthewiderarchitectureofaccountability. Pollittaddedthat: Indifferentwaysandatdifferentspeeds,eachSAIhas,throughamixtureof pressureandchoice,beguntoemergeintothewiderworld.Thisemergencehas involveddialogueswithexecutivesandlegislatures,theadoptionofmore proactivepoliciestowardsthemedia,andgraduallyincreasingcooperationwith otherSAIs.Itisatrajectorywhich,soonerorlater,leadstonewcomparisons beingmadeanddemandsbeingput(Pollittetal1999,219). Overall,performanceaudithascontinuedtodevelopoverthelastdecade.Itisnowa stronger(oratleastmorevisible)disciplineinmanycountries,abletodrawonwide statutorypowers,andwithatrackrecordofimpact.Performanceauditsexamineawide rangeofimportant,complexandpressingsubjectsinareasasdiverseas,forexample, health,education,defence,infrastructure,regeneration,transport,andtheenvironment. Theyhaveadaptedtheirworktofocusonnewandemergingsubjectareasprivatisation, theuseofprivatefinance,thecreationofpublicprivatepartnershipsandtheuseof regulatorymechanisms.Mostrecently,theyhavestartedtoreportontheresponses madebygovernmentstotheeconomicdownturn,suchastherescuemeasuresforthe bankingsector,fiscalstimuliandthesupportfordifferentindustries. Performanceaudithasdrawnonotherfieldsofenquirytobecomeamorerigorous activityandhasdevelopedqualityassurancearrangementswhichareatleasttheequalof

3 thosefoundinmanyotherformsofevaluativeactivity(LonsdaleandMayne,2005).We canlookatthisinalittlemoredepth. Firstly,therehasbeenagrowthinthescaleandintensityofperformanceauditwork.Itis nowundertakenbymoreSAIsthaneverandonalargerscale.TheGAO,forexample, undertakesaround1,000performanceauditsayear,whilstduringthemid2000sthe NAOincreaseditsoutputfrom45to60majorreportsayear.TheotherSAIscontinueto pursuesignificantprogrammesofwork.InAustralia,auditbodiescarryoutperformance auditsineachofthestateandterritories,aswellasnationally,andinCanada,ineachof theprovinces.Thesearesubstantialprojects;manyoftheauditsmakeuseofmultiple datasourcesandmoreintrusiveandcomplexevidencecollectionmethodsthaninthe past. OneAmericanoverview(BurkeandHaynes,2008)suggestedthatwhilstperformance auditwaslessubiquitous[thanfinancialaudit,itis]abroadandgrowingprofessional activity,asmanygovernmentsfeeltheneedtosubmittheirphysicaloperationstoclose scrutinybasedonthevaluesofefficiencyandeffectiveness.Elsewhere,wehaveseenthe creationofauditbodiesinScotlandandWales,underdevolutionarrangements,which haveincreasedtheamountofperformanceauditundertakenintheUKandtakenitin differentdirections.ManySAIshavestartedtocarryoutperformanceauditwherethere wasnonebefore;forexample,inmanydevelopingnations.Someofthegrowthhasbeen stimulatedbyprojectsfunded,forexample,bytheWorldBank,EuropeanCommission,

4 theAsianDevelopmentBank,orledbySAIswithwelldevelopedperformanceaudit regimes.Therearemanypowerfuladvocatesforperformanceaudit. SuchdevelopmentsmightbeseenaspartofMichaelPowersauditexplosion,withthe associatednegativeconnotations,butthereisevidencethatthereisanappetiteforthe workfromparliaments,governments,themediaandcommunitygroups.Onereview identifiedthatGovernmentshadfoundnewrolesforaudittoplayinanumberof countriesforexample,inAustralia,NewZealandandtheUnitedKingdom(Wilkinsand Lonsdale,2007). Itisunclear,however,howmuchperformanceauditisoptimal,orwhatfactorsshould limititsfurthergrowth.SAIscannottakeattentionforgranted.Nichol(2007)found limitedinterestintheAustralianHouseofRepresentativescommitteesinANAO performanceaudits,andCrerar(2007)reportedgreaterfamiliaritywithinspectionand complaintshandlingthanregulationandauditamongstthepublicinScotland. Secondly,wehaveseenthecontinuingdevelopmentofverydistinctiveapproachesin differentcountries,driveninpartbytheneedsandrequestsoflegislaturesand governments,theskillsofSAIstaffandexternalexpertsandthedifferingphilosophiesof theauditbodies.Thereisnoreasontoexpectthattherewillbeasingle,converging,role forSAIsinaccountability.Put,quotedinBoukaertandHalligan,2008hasarguedthat, forexample,theDutchandtheUKapproachestoperformanceauditarealmostexactly

5 oppositestrategies.IntheUK,managementissuesarepriorities,whereaspolicy evaluationissecondary.IntheNetherlands,hearguestheoppositeisthecase. Thirdly,wehaveseenabroadeningofmethodsandapproaches.BurkeandHaynes(2008) suggestthatUltimately,theperformanceauditingprofessionalisasmuchanartistasshe orheisascientist,workingfromabroadmandatewithanabundanttoolboxof techniquesandbroaddiscretioninapproachandfocus.Differentapproacheshavebeen fuelledbyseparaterecruitmentpractices,differentviewsonwhatconstitutes appropriateskills,andvariedtrainingregimes.Lonsdale(2000)commentedthat performanceauditorsinfourSAIsintheNetherlands,Sweden,FinlandandtheUKhad beenatpainstohighlightthattheyhadgivengreaterattentiontohowtheyapplied methodsinthelate1990sthaninthepast.AstillandPage(2006)alsoconcludedthat therewasbothpressurefromthetopoftheUKandFrenchSAIstodeveloptheir methodologies,aswellaseffectsfrombelowduetoauditorspushingoutthe boundaries.Lonsdalegoesfurtherinthisvolumetoidentifyaseriesoffactorsthat shapethechoicesandconstraintsonauditors,andalsohighlightssomeofthe innovationsthathavetakenplacewithintheworkoftheUKNAO.Factorsinfluencingthe choiceofmethodsincludethepracticalconstraintsoftime,costandevidence expectations,butalsomultipleinfluencesatteam,project,organisationaland environmentallevelswhichhavestimulatedtheuseofnewmethods. However,therehavebeenlongstandinganxietiesaboutthedegreeofsophisticationin themethodologicalqualityofsomeperformanceauditwork.(RobertsandPollitt,1994,

6 Schwartz,1999).TheseconcernsarealignedwiththoseraisedbyDubnick(2005)that thereisnothingintheliteraturetoprovidealogicallinkbetweenaccountgivingand performance.Thesamemightwellbesaidforthelinkbetweenaccountgivingand accountability. Afourththemehasbeenthepursuitofhigherstandards.Inmanywayswecansee parallelswiththedebateintheworldofevaluation,withSAIsseekingtocodify, standardiseandcommunicatetheexpectationstowhichtheywork.INTOSAIhas developedperformanceauditstandardsandindividualauditofficeshaveeither developedtheirownorchosentoadheretoestablishedones.Anunresolvedissueis whetheragreaterconsistencyofstandardsandpracticeswillemergeincomingyearsor whetherthediversityofapproacheswillcontinue.IntheUnitedStates,BurkeandHaynes (2008)haveseenamovementinrecentdecadestoinsertacommonframeworkthrough legislativeandbureaucraticaction,withtheGAOsgenerallyacceptedgovernment auditingstandards(GAGAS),firstreleasedin1972,becomingthenormfornational auditing,andstateandlocalgovernments,aswellashighereducationinstitutionsand nonprofitagencies,thatreceivefederalassistancerequiredtouseGAGAS.Thereis certainlyadesireforconsistencyandaneed,giventheincreasedexpectationsof transparencyunderfreedomofinformationlegislation(andevencourtproceedingsasin Canada(Fraser,2007)),tobeabletoshowthestandardsthathavebeenapplied. However,another(possiblymorepromising)responsecouldbeforSAIstobeless concernedwithbeingtheauthoritativesourceofevidenceaboutperformance,andmore

7 concernedwithestablishingthemselvesasthesourceofauthoritativeandindependent judgementbasedonthisevidence;theirrolewouldbetomakesenseoftheburgeoning bodyofperformancedata. SomethinghappenedintheWestwhenSocratesbegantoconfrontthepoliticians ofhisday,themenwhospokeintheassembliesandledthearmiesandnaviesof Athens,stoppingtheminthemarketplaceandaskingthemtogiveanaccountof whattheyweredoingandwhytheyweredoingit.Itwasnotexactlyanewform ofinquirythatwasbeinginvented,butanewstyleofinvestigation.(Bourkeand Geuss,Introduction,2009) SAIs,then,mightcontinueSocrateseffortsbycontributingjudgements.Itisnot judgementsabouttheoverallpackagesofgoodsofferedbydifferentpoliticalparties.This istheproperplaceofelectoralpoliticsandpartycompetition(oftenplayedoutthrough themediaandcontributedtobyorganisedcampaignsandlobbygroups).Norisitthe dramaticandcompetitivedramatisingofindividualpoliticaldecisionsplayedoutin electedassemblies.Itisjudgementsabouthowwellfoundeddecisionswereinasetof evidenceorvalues,howwellconsideredweretheinevitabletradeoffsmade,andwhose voicesandinterestswereprivilegedinthecomplexprocessesofimplementingpolicies.In asense,therefore,itismakingjudgementsaboutjudgements(orthelackofthem) includingmakingthevaluesandevidenceunderpinningthesejudgementsexplicit.The easierissuesarewhereprivateinterestsareprivilegedthroughcorruptionor concealment,orwhereimplementationactivitiesarepursueddespiteclearevidencethat theyarenotworking(ietheimportantquestionsofcorruptionandinertiaindemocratic government).However,moresubtleissuesoftradeoffsandcompromisesbetween

8 interestsandovertime(includingissuesrelatingtosustainabilityandtheinterestsof futuregenerations)wouldbenefitfromadispassionateassessment. Fifthly,wehaveseenincreasedinteractionbetweenSAIsandtheoutsideworldin developingtheirperformanceaudit.Thisisarolebuildingtrustandunderstandingacross theinstitutionsofgovernmentandinparticularbysharingmethodsanddataina transparentmanner.Wearenotawarethatsufficientattentionhasbeengiventothis importantfunctioninwiderresearchontheroleofSAIs.Lonsdale(2008)hasarguedthat thehistoryofVFMworkattheNAOcantoagreatextentbeseenasthedevelopmentof arangeofexternalcontacts.Indeed,performanceauditorshaveincreasinglysoughtto explaintheconductoftheirworktoothers.In2008,theOAGinCanadaprepareda detaileddocumentforthestaffofauditeddepartmentsentitledWhattoExpect:An AuditeesGuidetothePerformanceAuditProcess,designedto: encourageproductiveandrespectfulrelationsbetweenentitiesandauditstaff.The toneofthedesiredrelationshipisoneofmutualrespectandthedocumentsetsout therolesandresponsibilitiesonbothsides. Asixthandfinalthemehasbeentheneedtotackleincreasedcomplexityinthesubject matteraddressed.VanderKnaap(forthcoming,inLonsdale,WilkinsandLing)comments thatifperformanceauditistocontributetoeffectivegovernment,itmustpayclose attentiontothecomplexanddynamicnatureofpublicpolicy.Suchcomplexityincludes differentcentresofpower,variedpopulationswithdifferentculturalheritages, technologicaldevelopmentsandconstantlychangingadministrativearrangements.

9 Performanceaudithasalsoextendedtopartsofgovernmentactivityoftenspecialist andcomplexwhereitwaspreviouslynotseenforexample,theroleofprivatesector providersandtheircontractswiththestate.Wealsoseesomeauditofficesnowtackling theworkofgovernmentstodealwiththeeconomiccrisis.InMay2009,forexample,the AlgemeneRekenkamerpublisheditsfirstreport(AlgemeneRekenkamer,2009)onthe maininterventionsandarrangementsmadebytheMinistryofFinanceinresponsetothe creditcrisis.InMay2010,itreportedonthefurtherdevelopmentoftheseinterventions andarrangements(AlgemeneRekenkamer,2010).TheNationalAuditOfficeintheUKhas reportedonthenationalisationofthebank,NorthernRock,widersupportforthe bankingsectorandtheeffectofthecreditcrisisoninfrastructuredevelopment(National AuditOffice,2009,2010).AndtheSwedishNAOreportedin2009onStateGuaranteesin theFinancialCrisis,concludingthattheGovernmenthadnotprovidedcomprehensive andtransparentstatusreportstoParliamentsettingouttherisksforthestateand taxpayers(Riksrevisionen,2009). Ithasnotalwaysbeeneasy.Thedifficultiesandrisksbothmethodologicalandpolitical withsuchworkcanbeseen,forexample,inthecriticismoftheworkoftheauditoffices inAustraliainscrutinisingpublicprivatepartnerships,wheretherewereperceptionsthat theymayhavecompromisedtheirindependencethroughtheirattemptstofostergood managementpracticeinthepublicsector(English,2007).Thechallengesofthisgreater complexityandSAIsresponsestoithaveledsometoquestionwhetherperformance auditandtheexistinginstitutionalarrangementsarebeingstretchedtoofar(Ling,2007).

10 Performanceauditandaccountability WecanbreakthecontributionofSAIsandperformanceaudittoaccountabilitydown furtherintermsoftherelevanceto:trust,theavailabilityofindependentinformation, andtherelationshipbetweengovernmentandcitizens Arelatedaspectofperformanceauditsrelationshipwithdemocraticgovernmentrelates totheissueoftrustinwhatgovernmentssay.Variouscommentatorshavesuggestedthat trustinsocietyisindecline;OnoraONeillhasspokenofourpresentcultureofsuspicion (ONeill,2002),andpoliticians,professionalsofmanykindsandotherauthorityfigures haveseentheirstandingfall(Seldon,2009).Inthesecircumstances,thoseresponsiblefor reportingontheperformanceofgovernmentandtaskedwithprovidingassuranceon whetherpublicfundshavebeenspentasanticipatedhaveanambiguousrole.AsDe LancerJulnes(2006)hasarguedThecurrentemphasisonresultsforbothaccountability andperformancemeasurementhasbeenassociatedwiththeincreasedskepticismand discontentoftheAmericanpublicwithhowtheirtaxdollarsarebeingspent. Therelationshipbetweenperformanceauditandtrustiscontentious.Fromone perspective,performanceauditreportswhichregularlyidentifydeficienciesandfailures weakentrustinourpublicinstitutions.Alternatively,itcanbearguedthatperformance auditreports,inplacinginformationinthepublicdomain,havehelpedtoincreasetrustin democraticgovernment(orshowwhereitismisplaced).Publishedauditreportsprovide informationandanalysisindependentofgovernmentandtheexistenceofthefunction

11 givesgeneralassurancesomeoneiskeepinganeyeonhowpublicmoneyisspent.Our viewontrustdependsonourviewofpublicservantsandwhetherweregardthemas saints,whoarebestimprovedthroughguidance,orrascals,whoneedtobecontrolled. Intruth,itprobablytakesanoddviewoftheworldtothinkwecantrusteveryone withoutevidence.Andwemightarguethattrustisnotgoodinitself,butonlyhas significancewhenithasbeenauthentically(i.e.honestly)won. AfurtherdimensionconcernstheindependenceofinformationthatSAIsissueintheir performanceauditreports.Dahl(2000)hascommentedthatLikefreedomofexpression, theavailabilityofalternativeandrelativelyindependentsourcesofinformationis requiredbyseveralofthebasicdemocraticcriteria.He,arguesthatcitizensmusthave accesstoalternativesourcesofinformationthatarenotunderthecontrolofthe governmentordominatedbyanyothergroupofpointofview. Finally,performanceauditplaysaroleinopeningupgovernmenttoexaminationby citizens.Providinginformationaboutpublicsectorperformancehasgrownintheformof increaseddatainannualreports,leaguetables,andsummariesofinformationinlocal papersandtheinternet.AsoneobserverhasputitGovernmentsnowattempttoshow theirconstituentswhattheyaregettingfortheirtaxdollars,howefficientlyand effectivelytheirtaxdollarsarespent,andhowexpendituresbenefitconstituentslives. (CallahanandHolzer,1999indeLancerJulnes,2006).Mulgantoo(Mulgan,2006)has arguedthat:

12 thebestmoderngovernmentsopenthemselvesuptodoubtandlearning...Thisdoes notimplylimitlesstolerance.....Insteaditsdefiningfeatureisthatittoleratesmany pathstotruth,andtriestointernalizewithinitselfsufficientchallengeandargument toprotectitfromitsownbeliefs,withofficialsandadviserswhoarewillingtospeak truthtopower. AnumberofSAIshavearguedthattheytakeacitizensperspective.TheSwedishNAO, forexample,statesOneimportantrole...istoensuredemocratictransparency,i.e providecitizenswiththeopportunitytoseehowdemocraticdecisionsaremadeand implemented,howtheirtaxmoneyisused,andwhetherpublicadministrationfollows directives,rulesandregulationsandachievestheobjectivessetforit.(Riksrevisionen, 2010). Overall,performanceauditprovidesauniqueflowofacertainkindofevidenceand argument.Throughitseffortstoprovidedispassionateaccountsbasedonevidenceand argumentationitlendsitselftoaparticular,moredeliberative,viewofdemocraticlife, whichrecognisesthepracticalimportanceofmajoritarianismbutprivilegesthe importanceofauthenticdeliberationasthebasisfordecisionmaking.Arguably,whereit doesitbestiswhereitreportscomplexmaterialinwayswhichnonspecialistreaderscan understand. Agendaforthefuture Weconcludethisbriefpaperwithsomediscussionpoints.

13 1. Definewhatperformanceauditisandsticktoit.WethinkitisessentialthatSAIs focusontheircoreactivitiesandbeanSAI.Otheragenciesarebetterplacedto playtheroleofmanagementconsultancies,generatorsofnewideas,orchange agents. 2. Improvethetheoryandpracticeofperformanceaudit.Performanceauditisa practicaldiscipline,designedtoassistinimprovingtheoperationofgovernment. 3. Linkedtothis,developtheabilitytoassessandmeasuretheimpactand influenceofperformanceauditwork.Everypublicbodyneedstobeableto explainandevidencethebenefitssecuredfromitswork. 4. Improvetheroleofperformanceauditinrespectoflearning.Ourdefinitionof performanceauditplaceslearningfromtheworkasasecondaryoccupationbut neverthelesshighlightsitsimportance.Whileavoidingmissioncreep,weare clearthatSAIsareuniquelyplacedtoassistwithlearningbecauseoftheirstatus andstanding,theircrossgovernmentposition,thedepthoftheirknowledgeof governmentbodiesandthestrengthoftheiraccesstodataandtheiranalytical skillsandjudgement. 5. Continuetoincreasetheirresponsiveness.Thismayrequirefindingnewwaysof workingtoreducethetimetaken,newmeansofreportinginlinewithdeveloping knowledgeofwhatkindofinformationworksbestwithdecisionmakers(Pollitt, 2006,Boyle,2009)andnewwaysofapproachingtheircoreroles. 6. Exploittheirpositionintheinterestsofdeliberationanddemocraticdecision makingbystrikingabalancebetweensupportingaccountabilityprocessesand generatinglearning.Thiscanbedoneby:

14 Focusingonproblemsandissuesstretchingbeyondindividualorganisationsand programmes; Contributingtowellinformeddebate; Supportingdemocraticcitizenship; Askinginconvenientquestionstokeepgovernmentresponsivetocitizens; Championingtransparency; Contributingtothecreationofanenvironmentforproperdebateabout performance.

15

References AlgemeneRekenkamer(2006),Performanceauditmanual,TheHague:Algemene Rekenkamer. AlgemeneRekenkamer(2009),Thefinancialcrisis20082009,TheHague:Algemene Rekenkamer. AlgemeneRekenkamer(2010),Financialcrisis:interventionsandfollowup;fourth quarterof2009andfirstquarterof2010,TheHague:AlgemeneRekenkamer. Astill,StuartandEdPage(2006),Aims,InstrumentsandInteractioninVFMAudit:A CrossNationalStudy,PaperfortheNationalAuditOffice,London. BemelmansVidec,MarieLouise,JeremyLonsdaleandBurtPerrin(eds)(2007), MakingAccountabilityWork:DilemmasforEvaluationandforAudit,NewBrunswick, NJ:TransactionPublishers. Bouckaert,GertandJohnHalligan,(2008),ManagingPerformance:International Comparisons,Routledge,London. Burke,BrendanandWendyHaynes(2008),PerformanceAuditsinJackRabinandT AaronWachhaus(eds),EncyclopaediaofPublicAdministrationandPublicPolicy,Second edition. Callahan,KandHolzer,M(1999),ResultsOrientedGovernment:CitizenInvolvementin PerformanceMeasurementinAHalachmi(ed.)PerformanceandQualityMeasurement inGovernment.IssuesandExperiences,Burke,VA:ChatelainePress,pp.5164. Crerar,Lorne(2007),TheReportoftheIndependentReviewofRegulation,Audit, InspectionandComplaintsHandlingofPublicServicesinScotland,Edinburgh:Scottish

16 Government,availableatwww.scotland.gov.uk/Publications/2007/September(accessed 13May2010). Dahl,Robert(2000),OnDemocracy,Yale:YaleUniversityPress. Dubnick,Melvin(2005),AccountabilityandthePromiseofPerformance:InSearchof Mechanisms,PublicPerformanceandManagementReview,28(3),376417. Dunleavy, Patrick, Helen Margetts, Simon Bastow and Jane Tinkler (2006), Digital era goveranance: IT corporations, the state and egovernment, Oxford: Oxford University Press. Dunleavy, Patrick, Christopher Gilson, Simon Bastow and Jane Tinkler, (2009), The NationalAuditOffice,thePublicAccountsCommitteeandtherisklandscapeinUKpublic policy,TheRiskandRegulationAdvisoryCouncil,availableat: www.berr.gov.uk/deliverypartners/list/rrac/index.html English, L (2007), Performance audit of Australian Public Private Partnerships: Legitimising Government Policies or Providing Independent Oversight?, Financial AccountabilityandManagement,23(3),313336. Fraser, S (2007), Public Scrutiny Puts SAI Audit Practices to the Test in International JournalofGovernmentAuditing,January,15 INTOSAI.(2004),ImplementationGuidelinesforPerformanceAuditing,International OrganisationofSupremeAuditInstitutions. De Lancer Julnes, P (2006), Performance Measurement: An Effective Tool for GovernmentAccountability?TheDebateGoesOn,Evaluation,12(2),219235

17 Ling,Tom(2007),NewWineinoldBottles?WhenAudit,Accountabilityand EvaluationMeetinBemelmansVidec,MarieLouise,JeremyLonsdaleandBurt PerrinMakingAccountabilityWork.NewBrunswick,NJ:TransactionPublishers. Lonsdale,Jeremy(2000),Advancingbeyondregularity:developmentsinVFMmethodsat theNationalAuditOffice19841999,UnpublishedPhDthesis,BrunelUniversity. Lonsdale,J(2007),Helpinggovernmenttoberesponsive,responsibleandrespected:a reaction to Jocelyne Bourgon, International Review of Adminstrative Sciences, March, 73,3136. Lonsdale,J(2008),BalancingIndependenceandResponsiveness,Evaluation14(2), 227248. Lonsdale,JandMayne,J(2005),NeatandTidyand100%correct:assuringthequality ofsupremeauditinstitutionperformanceauditwork,inJohnMayneandRobSchwartz (eds), Quality Matters: seeking confidence in evaluating, auditing and performance reporting,NewJersey,Transaction. NationalAuditOffice(2008),ValueforMoneyHandbook,London:NationalAuditOffice. NationalAuditOffice(2010),ProgresswithVFMsavingsandlessonsforcostreduction programmes,London:NationalAuditOffice. Nichol, Paul (2007), Audit in a Democracy: the Australian Model of Public Sector AuditanditsApplicationtoEmergingMarkets,Ashgate,Aldershot. Office of the Auditor General (2008), What to Expect An Auditees Guide to the PerformanceAuditProcessat: www.oagbvg.gc.ca/internet/English/meth_lp_e_30860.html(accessed24April2010)

18 ONeill, Onora (2002), Reith Lectures 2002, Lecture 1 at

www.bbc.co.uk/radio4/reith2002/lecture1.shtml Pollitt,Christopher,XavierGirre,JeremyLonsdale,RobertMul,HilkkaSummaand MaritWaerness(1999),PerformanceAuditandPublicManagementinFive Countries.OxfordandNewYork:OxfordUniversityPress. Riksrevisionen (2009), Annual Report of the Auditors General 2009, Stockholm: Riksrevisionen. Riksrevisionen (2010), The role of the Swedish NAO at

www.riksrevisionen.se/templib/pages/NormalPage_1768.aspx(accessed25April2010) Roberts, Simon and Christopher Pollitt (1994) Audit or evaluation ? A National Audit OfficeVFMstudy,PublicAdministration72(4),527549. Schwartz,R(1999),CopingwiththeEffectivenessDilemma:StrategiesAdoptedbyState Auditors,InternationalReviewofAdministrativeSciences,65,511526. Seldon,A(2009),Trust:HowWeLostItandHowtoGetinBack,Biteback. Wilkins,PeterandJeremyLonsdale(2007),PublicSectorAuditingforAccountability: NewDirections,NewTricks?.inMarieLouiseBemelmansVidec,JeremyLonsdale andBurtPerrin(eds).MakingAccountabilityWork.DilemmasforEvaluationandfor Audit.NewBrunswick:TransactionPublishers. BourkeRandGeussR(2009)PoliticalJudgementEssaysforJohnDunnCambridge: CambridgeUniversityPress