JOURNAL OF CLINICAL MICROBIOLOGY, Sept. 2011, p. S11S14 0095-1137/11/$12.00 doi:10.1128/JCM.00580-11 Copyright 2011, American Society for Microbiology.

. All Rights Reserved.

VOL. 49, NO. 9 SUPPL.

The Clinical Predictive Value (or Lack Thereof) of the Results of In Vitro Antimicrobial Susceptibility Tests
Gary V. Doern1* and Stephen M. Brecher2
Clinical Microbiology Laboratories, University of Iowa Carver College of Medicine, Iowa City, Iowa 52242,1 and Department of Clinical Microbiology, VA Boston Healthcare System, Boston, Massachusetts 2 Estimates of antibacterial activity can be determined using a wide variety of different in vitro methods. These include manual methods such as disk diffusion, broth microdilution, and Etests or instrument-based methods such as Vitek (bioMerieux, Durham, NC), Phoenix (BD, Franklin Lakes, NJ), MicroScan (Siemens, Sacramento, CA), and TREK (Trek Diagnostics Systems, Cleveland, OH). Depending on the method employed, results are conveyed in the form of either susceptibility categories (susceptible [S], intermediate [I], or resistant [R]) or quantitative estimates of antibacterial activity, MICs. The measurement units for MICs are micrograms per milliliter. MICs represent the most rened estimate of in vitro antibacterial effect. With category methods, the results S, I, or R are predicated on either interpretation of MICs or correlations between MICs and zone diameters obtained with the disk diffusion procedure. The rst paper in this supplement explores the manner in which the criteria for interpreting MICs, i.e., breakpoints, are developed. In this discussion, we address a second issue. How well do the results of antimicrobial susceptibility tests (ASTs) predict therapeutic outcome in patients with infections? Stated another way, what is the clinical predictive value of in vitro susceptibility tests? In general, resistance as determined by use of in vitro susceptibility tests is nearly always an independent risk factor for therapeutic failure in patients with infection who are treated with antimicrobial agents (5, 6, 13, 14). That is to say, use of antimicrobials that have been determined in the laboratory to be more active is invariably associated with higher rates of response to therapy than use of agents with diminished activity. The important question is, Does resistance always predict failure; does susceptible always denote favorable response to therapy? Table 1 provides the results of a prospective study that assessed therapeutic response in a large number of patients with a variety of different monomicrobic Gram-negative infections who were treated with cefotaxime, usually at a dose of 2 g intravenously (i.v.) every 8 h (q8h), in the context of the cefotaxime MICs of the bacteria thought to be responsible for infection (16). These data were derived from phase III clinical trials with cefotaxime, so patient inclusion criteria and assessments of response to therapy were both objective and rigorous. Generally, there was trending toward higher response rates among patients infected with organisms with lower cefotaxime MICs, but there were certainly no absolutes. Note that 64% of
* Corresponding author. Mailing address: Clinical Microbiology Laboratories, University of Iowa Carver College of Medicine, C606GH, Iowa City, IA 52242. Phone: (319) 356-8615. Fax: (319) 356-4916. E-mail: gdoern@nc.rr.com. S11

patients infected with organisms whose cefotaxime MICs were 64 g/ml were judged to have responded to therapy. Further, applying current CLSI breakpoints for cefotaxime (4) as a means for dening organisms as being resistant or susceptible, 93% of patients infected with bacteria categorized as being susceptible responded to therapy while 64% of those infected with resistance organisms were judged to have experienced a favorable response. Essentially the same observations were made in two other studies, one which examined the clinical predictive value of meropenem MICs (8) and another which characterized response to therapy based on cefoperazone MICs (11). These and other investigations (10, 17, 21, 22) prompted Rex and Pfaller (18) to coin the phrase, the 9060 rule when considering the expected correlation between outcome and the results of in vitro susceptibility tests. In general, a susceptible result is associated with a favorable therapeutic response in 90 to 95% of patients. When the infecting bacterium has been determined to be resistant, notwithstanding this result, nearly two-thirds of patients can be expected to respond to therapy. These observations apply to immunocompetent patients with monomicrobic bacterial infections who are treated with a single antimicrobial agent which is administered parenterally in circumstances in which the penetration of drug to the site of infection is predictable. In one survey at the University of Iowa, this represented ca. 25% of instances in which in vitro susceptibility tests were performed in the laboratory (Erik Munson, personal communication, 2005). Who were the other 75%? These were patients with polymicrobic infections, patients whose infections were treated with multiple antibiotics, patients receiving orally administered agents, immunocompromised patients, or patients with infections at sites in which antimicrobial agents were either diluted or concentrated. What is the clinical predictive value of antimicrobial susceptibility test results in these situations? Even lower. Indeed, in many of these situations there is no predictive value whatsoever for the results generated by in vitro susceptibility tests (7, 9, 12, 19, 20). In other words, notwithstanding the extraordinary reproducibility of conventional laboratory susceptibility test methods, they often fall far short of proving accurate predictions of outcome. Why is that? Drugs are tested in the laboratory as single agents against pure cultures of putative pathogens. Is it reasonable to think that an in vitro assessment of activity determined under those conditions is likely to reliably predict outcome in patients receiving combination therapy or patients with polymicrobial infections? As outlined in the rst paper in this supplement, the MIC breakpoints that are used to dene test organisms as being resistant or susceptible are often pred-

S12

CAMP CLIN MICRO

J. CLIN. MICROBIOL.

TABLE 1. Correlation of disease outcome with the results of MIC determinations in patients with infection who were treated with cefotaximea (16)
Cefotaxime MIC ( g/ml) Categoryb Number of patients % Cured or improved % Eradication

in vitro tests to reliable predict failure is much less, i.e., 35%. As Christine Sanders opined 13 years ago, what we really need are antimicrobial resistance tests, and currently we do not have them (C. Sanders, ICAAC Meeting, 1998). DISCUSSION Two practical matters were considered related to the information presented above. Should laboratories that perform MIC tests report actual MIC values or rather provide only category interpretations? Our consensus view is that in all but selected situations, only the category interpretation should be reported routinely. This view is predicated on the relatively obscure relationship that exists between individual MIC values and dened predictions of therapeutic response together with the inherent variability of MIC determinations. We know absolutely that the wiggle room in an MIC test, even one done under the most scrupulous of conditions is 1 2-fold concentration increment. That is, when you say the organism has an MIC of 2 g/ml, what you really mean is that it has an MIC of 1 to 4 g/ml. In this regard, it makes at least intuitive sense to report a more broadly representative estimate of antimicrobial effect, the category interpretation, rather than an MIC value. A second question is when to perform the test in the rst place. If, as outlined above, the results of in vitro susceptibility tests have little to no clinical predictive value in infections such as those occurring in the urinary tract, polymicrobial infections, outpatient infections treated with oral antimicrobials, or infections treated with multiple agents, why bother to do susceptibility tests in these situations? The possibility exists that the results of such tests could be harmful, encouraging the use of agents that are more toxic, agents that are needlessly expensive, or agents characterized by greater drug-drug interactions. One implication of diminished testing would be the impact such an approach would have on the generation of cumulative antimicrobial susceptibility information. If all of the bugs have not been tested, how can the laboratory generate the handydandy annual cumulative antibiograms that t so nicely into the breast pockets of the lab coats worn by health care providers? It cannot. But is that necessarily a bad thing? Construction of cumulative antibiograms is problematic in a number of important respects (2). Moreover, the approach that is most often used, that is, to tabulate AST results in the laboratory without regard to the true clinical signicance of the isolates, has recently been shown to yield erroneous information (1). And nally, what is the real proven value of cumulative antibiograms in the care of patients? That said, the reality is that many care providers have come to rely on antibiograms in guiding empirical therapeutic decisions at the bedside and the expectation is that the laboratory will provide this information. Further, there are other, more general potential uses for antibiograms. These include tracking of institutional resistance trends longitudinally over time as a resource for use in antibiotic stewardship programs and as a means for complying with regulatory dictates. So again, as stated above, if a decision is made to restrict the performance of ASTs to selected clinical situations, where is the information upon which antibiograms are constructed going to come from? One approach would be to continue to perform susceptibil-

4 8 16 32 64

S S I I R

1003 273 151 70 19

94 90 77 84 64

91 86 75 71 50

a All patients had dened monomicrobic infections and were treated with intravenous cefotaxime alone, typically at a dosage of 2 g q8h. b Susceptibility categories: S, susceptible; I, intermediate; R, resistant.

icated on or at least inuenced by pharmacokinetic and pharmacodynamic constructs. In other words, drug levels over time are an important part of the equation. What happens to the predictive value of pharmacokinetic-pharmacodynamic (PKPD)-derived breakpoints when patients receive nonstandardized dosage amounts of drug or have an infection in sites where drug concentrations are either lower or higher than what would be predicted based on plasma pharmacokinetic determinants? Monte Carlo analysis as a tool in developing pharmacodynamic MIC breakpoints has eliminated some of this variability but certainly not all of it. Another important determinant in the outcome of infection is the presence or absence of bacterial virulence factors. More virulent organisms are often associated with more severe disease and are more difcult to treat, and patients may not respond to antimicrobial therapy irrespective of the activity prole of the agent(s) chosen for management. Organisms that lack virulence determinants cause less severe infections that are easier to treat. The presence or absence and level of expression of virulence determinants are factors which are not taken into consideration with in vitro susceptibility tests. Arguably the single most important factor in understanding the relatively poor clinical predictive value of in vitro susceptibility tests is the fact that in the test tube or microdilution tray, on the disk diffusion or Etest plate, or in the AST instrument, there are no host factors that mitigate for or against improvement or disease progression in patients with infections, unrelated to antimicrobial effect. We do not take into account the effect of white blood cells (WBCs), complement, cytokines, or antibody in our in vitro estimates of response to therapy. All of these factors, of course, tend to assist the patient in ghting off infection. Perhaps the high frequency with which patients infected with resistant bacteria seem nonetheless to clear their infections despite what should be an ineffective antimicrobial is not surprising. Indeed, the clinical predictive value of in vitro susceptibility tests with chemotherapeutic agents used in the management of hematologic malignancies is much higher than that usually observed with antibacterial agents (3). This is likely explained by the circumstances that exist in patients with leukemias and lymphomas, circumstances that result in drug effect emerging as the primary determinant of outcome. Host factors are relatively less important. In the end, it is probably appropriate that we refer to our in vitro methods as antimicrobial susceptibility tests. The clinical predictive value of a susceptible result is excellent, i.e., 90%. Unfortunately, the ability of

VOL. 49, NO. 9 SUPPL., 2011

ANTIMICROBIAL SUSCEPTIBILITY TESTING

S13

ity tests but with the expressed purpose of generating antibiograms, not for reporting in cases of individual infections. Many laboratories use this approach for testing anaerobic bacteria. Isolates are saved and batch tested, and in this manner, cumulative resistance data are generated. A second approach to this dilemma would be to use data obtained from large national surveillance programs that track antimicrobial resistance to generate antibiograms. The strength of these data is that they are usually derived from reference standard testing performed in experienced laboratories. Their weakness lies in the fact that the results of such surveys often represent a very broad picture of resistance rates. What one really needs is information that is more locally representative. Then, the question becomes where and how those data are going to be developed and promulgated. What can be done about enhancing the clinical predictive value of in vitro susceptibility tests? Several issues were discussed, all with the aim of increasing the clinical predictive value of ASTs. As noted previously, one approach that is used to establish MIC breakpoints is correlation of MICs with outcome in patients with infection. It is understood that these data are very difcult to obtain insofar as they must be derived from carefully structured clinical studies with both rigorous inclusion criteria and explicitly dened and tracked outcome measures. Further, clinical correlation as a means for establishing MIC breakpoints would optimally require large numbers of patients infected with organisms with high MICs at or near the putative breakpoint for resistance who were nonetheless treated with an antibiotic that might not work. Ethical considerations make this type of study very challenging. Although less than perfect, an alternative approach might be to perform retrospective observational surveys, in which patients found in the laboratory to be infected with bacteria with MICs of a given antibiotic close to the resistance breakpoint would be queried retrospectively for which antibiotic(s) they had received or were receiving for management. In cases where a patient had for some meaningful period of time been found to have received monotherapy with the antimicrobial agent with a high MIC, an assessment of outcome could be made. Large numbers of patients would have to be surveyed, it would be essential that the clinical assessment was performed by someone with serious medical expertise, preferably an infectious disease physician, and it would be hugely important to track only outcome measures that were clearly attributable to infection. Accepting the nonrandomized, relatively uncontrolled, and observational nature of such a survey, some insight permitting the establishment of breakpoints that would more reliably equate to therapeutic failure might be obtained. Another notion that was discussed was the use of assays for detecting bacterial resistance determinants directly as a surrogate for, or replacement of, in vitro tests for antibacterial activity. In other words, skip go. The reason an organism escapes antimicrobial inhibition and/or killing in a patient with infection and as a result is at higher risk for failing therapy is that the organism rst harbors and then expresses some kind of resistance factor. It might be an enzyme that inactivates the antimicrobial agent, it might be an altered antibiotic binding target, it might be the ability of an organism to actively extrude an antibiotic with an intracellular base of action from inside to outside the cell (i.e., efux), or it might be a circumstance in

which antibiotic penetration to its site of action is somehow compromised. In all of these instances, detection of the resistance factor either genotypically or phenotypically might serve as a reliable predictor of failure. There are several examples of where this seems to be the case. The presence or absence of mecA with isolates of Staphylococcus aureus is a much better predictor of failure in patients with S. aureus infections who are treated with betalactam antibiotics than is any in vitro assay which measures the activity prole of beta-lactam antimicrobials (4). Another example pertains to extended-spectrum -lactamases (ESBLs) when produced by Escherichia coli, Klebsiella pneumoniae, and selected other Enterobacteriaceae. Such organisms are often found in the laboratory to have very low MICs to many betalactam antimicrobial agents, and yet patients fail therapy when treated with these agents; this is so even for patients with urinary tract infections, a circumstance where the antimicrobial agent is concentrated (23). In other words, the presence of the dened resistance factor is a better predictor of failure than is the MIC. Further, this approach offers the possibility of detecting resistance determinants directly in clinical material, thus obviating the need to recover the pathogen in culture prior to testing for the resistance factor, in turn considerably shortening the time to provision of a test result. One major caveat with this approach is the fact that what is important is the level of expression of the resistance determinant. Simply nding a gene or a protein that indicates that an organism has the ability to produce a resistance factor is not sufcient. It must be expressed in order to exert its effect. As we learn more about the regulatory pathways that control the expression of resistance factors, a new avenue for measuring resistance mechanisms in an even more clinically relevant manner is on the horizon. (Additional aspects of this exciting new diagnostic modality are found in the fourth paper in this supplement.) Another matter that was discussed is the simple reality which denes the clinical situation in which ASTs are performed. ASTs represent a single time point in the entire continuum that constitutes the care of a patient with infection. A specimen is obtained, it is transported to and then processed in the clinical microbiology laboratory, an organism is recovered, susceptibility tests are performed, and the information is provided back to care providers generally using a notoriously inefcient passive reporting system, the laboratory information system (LIS). In the meantime, dozens, if not hundreds, of other things are happening to the patient with infection that may directly inuence their outcome beyond the effect of the antibiotic. Examples include nutritional management, the treatment of underlying diseases which in some cases may have predisposed to infection in the rst place, surgical procedures, physiological reconstitution, physical therapy, and visits from the chaplain and the long-lost uncle, the uncle who had not been included in the patients will. AST results and their effect or lack of effect on antimicrobial prescribing practice is but one small piece of a very large and complicated clinical puzzle. Furthermore, in the hospital environment, generally speaking, there is a lag of between 2 and 4 days after a specimen is obtained until the results of susceptibility tests become available. In the setting of true community-based ambulatory practice, this lag period may be even longer. In the interim, patients

S14

CAMP CLIN MICRO

J. CLIN. MICROBIOL.
biograms: potential implications for selection of empirical antimicrobial therapy. Infect. Control Hosp. Epidemiol. 27:682687. 3. Bosanquet, E. G. 1991. Correlations between therapeutic response of leukemias and in vitro drug sensitivity assay. Lancet 337:711714. 4. Clinical and Laboratory Standards Institute. 2010. Performance standards for antimicrobial susceptibility testing (twentieth informational supplement), M100S20. Clinical and Laboratory Standards Institute, Wayne, PA. 5. Cosgrove, S., et al. 2003. Mortality related to methicillin-resistant Staphylococcus aureus compared to methicillin-susceptibile S. aureus: a meta-analysis. Clin. Infect. Dis. 36:5359. 6. Defe, R., M. H. Scheetz, J. M. Feinglass, M. J. Psotelnick, and K. Scarsi. 2009. Effect of differences in MIC values on clinical outcomes in patients with blood stream infection caused by Gram-negative organisms treated with levooxacin. Antimicrob. Agents Chemother. 53:10741079. 7. Doern, G. V. 1992. In vitro activity of loracarbef and effects of susceptibility test methods. Am. J. Med. 92:7S15S. 8. Doern, G. V. 1995. Interpretive criteria for in vitro antimicrobial susceptibility tests. Rev. Med. Microbiol. 6:126136. 9. Evans, M. R., et al. 2009. Short-term and medium-term outcomes of quinolone-resistant Campylobacter infection. Clin. Infect. Dis. 48:15001506. 10. Forrest, A., et al. 1993. Pharmacodynamics of intravenous ciprooxacin in seriously ill patients. Antimicrob. Agents Chemother. 37:10731081. 11. Gerber, A. U., and W. A. Craig. 1981. Worldwide clinical experience with cefoperazone. Drugs 22:108118. 12. Hilf, M., et al. 1989. Antibiotic therapy for Pseudomonas aeruginosa bacteremia: outcome correlations in a prospective study of 200 patients. Am. J. Med. 87:541546. 13. Jones, R. N., and M. N. Dudley. 1997. Microbiological and pharmacodynamic principals applied to the antimicrobial susceptibility testing of ampicillin/sulbactam: analysis of the correlations between in vitro test results and clinical response. Diagn. Microbiol. Infect. Dis. 28:518. 13a.Ledeboer, N. A., and R. L. Hodinka. 2011. Molecular detection of resistance determinants. J. Clin. Microbiol. 49(Suppl.):S20S24. 14. Maragakis, L. L., E. N. Perencevich, and S. E. Cosgrove. 2008. Clinical and economic burden of antimicrobial resistance. Expert Rev. Anti Infect. Ther. 5:751763. 15. Munson, E. L., D. J. Diekema, S. E. Beekmann, K. C. Chapin, and G. V. Doern. 2003. Detection and treatment of bloodstream infection: laboratory reporting and antimicrobial management. J. Clin. Microbiol. 41:495497. 16. Murray, P. R., R. Jones, and W. Novick. 1983. Analysis of the clinical predictive value of quantitative and qualitative susceptibility tests with cefotaxime. Abstr. 23rd Intersci. Conf. Antimicrob. Agents Chemother. abstr. 545, p. 182. 17. Nguyen, M. H., V. L. Yu, and A. J. Morris. 2000. Antimicrobial resistance and clinical outcome of Bacteroides bacteremia: ndings of a multicenter prospective observational trial. Clin. Infect. Dis. 30:870876. 18. Rex, J. H., and M. A. Pfaller. 2002. Has antifungal susceptibility testing come of age? Clin. Infect. Dis. 35:982989. 19. Schultz, H. J., L. A. McCaffrey, T. F. Keys, and F. T. Nobrega. 1984. Acute cystitis: a prospective study of laboratory tests and duration of therapy. Mayo Clin. Proc. 59:391397. 20. Snydman, D. R., G. J. Cuchural, L. McDermott, and M. Gill. 1992. Correlation of various in vitro testing methods with clinical outcomes in patients with Bacteroides fragilis group infections treated with cefoxitin: a retrospective analysis. Antimicrob. Agents Chemother. 36:540544. 21. Washington, J. A. 1983. Discrepancies between in vitro activity and in vivo response to antimicrobial agents. Diagn. Microbiol. Infect. Dis. 1:2531. 22. Weinstein, M. P., J. R. Murray, L. B. Reller, and K. A. Lichtenstein. 1983. The clinical signicance of positive blood cultures: a comprehensive analysis of 500 episodes of bacteremia and fungemia in adults. Rev. Infect. Dis. 5:5470. 23. Yang, Y. S., et al. 2010. Impact of extended-spectrum -lactamase-producing Escherichia coli and Klebsiella pneumoniae on the outcome of communityonset bacteremic urinary tract infections. J. Microbiol. Immunol. Infect. 43:194199.

will invariably have been started on empirical therapy and have had a chance to declare themselves clinically. What is the likely effect of a therapeutic intervention at that point? Often, the effect is very little. Perhaps this explains the observations of Munson et al. (15), in which even in patients with proven bacteremia, only a fraction of the time did the results of ASTs seem to inuence therapeutic decisions when they were nally made available to care providers. Each of these points seems to underscore the importance of laboratorians extending their scope of activities to include extensive interaction with care providers in trying to optimize the use of the AST results. Susceptibility testing is not done in a vacuum. It is simply not OK for the laboratorian to blindly generate susceptibility test results, load them onto the LIS, hit the button that sends those results scurrying off into cyberspace, and then pat themselves on the back thinking theyve fullled their responsibilities as a member of the health care team. Stated another way, The distance between the clinical microbiology laboratory and the ill patients bed is only as long as you, Dr. Laboratory Director, choose to make it (Silas G. Farmer, 1977, personal communication). In summary, the clinical predictive value of in vitro susceptibility tests is currently often limited. For this reason, care should be exercised in deciding when to perform susceptibility tests on bacteria recovered from patients with infection. Potential future approaches to resolving this problem include increased reliance on clinical correlation as a means for establishing MIC breakpoints and the detection of resistance determinants directly, using either phenotypic or genotypic methods. The detection of resistance factors could potentially serve as a replacement for in vitro ASTs and could be applied either to organisms recovered in culture or, even better, directly on clinical specimens. Finally, until we have better in vitro predictors of outcome, it is more important than ever that the clinical microbiology laboratory director step outside of the laboratory and become an integral player in the larger health care enterprise.
Session discussants: Lynn Boyer, Eileen Burd, Kimberle Chapin, Daniel Diekema, David Durack, Larry Hambleton, Markus Kostrzewa, Michael Loefeholz, Susan Novak-Weekly, Ribhi Shawar, Mary Ann Silvius, Jane Smith, Ben Turng, Markita Weaver, and Donna Wolk.
REFERENCES 1. Bantar, C., G. Alcazar, D. Franco, F. Salamone, and M. E. Oliva. 2007. Are laboratory-based antibiograms reliable to guide the selection of empirical antimicrobial treatment in patients with hospital-acquired infections? J. Antimicrob. Chemother. 59:140143. 2. Binkley, S., et al. 2006. Comparison of unit-specic and hospital-wide anti-