Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Laboratory Management
School of Industrial Engineering, University of Oklahoma, 202 West Boyd, Suite 124, Norman, OK 73019. * Author for correspondence. Fax 405-325-7555; e-mail dechert@mailhost. ecn.ou.edu. Received January 21, 1998; revision accepted May 22, 1998.
1959
1960
collected data from clinical chemistry and drug monitoring assays and evaluated correlation in these data sets. The data used in the evaluation included actual laboratory monitoring data along with some data generated according to NCCLS EP-5 protocol for evaluating precision. Estimated correlation coefficients between concentrations across four assays were 0.05, 0.40, 0.67, and 0.70, respectively. From this small number of assays, it is clear that the presence of correlation in testing data is real; however, the degree of correlation may vary from situation to situation. In addition to using the aforementioned data for evaluating correlation within an analytical run, the data were evaluated for the presence of correlation from run to run within a control concentration. Autocorrelation plots were analyzed for these data, and these plots indicate no significant correlation from run to run within one concentration of control material. Because there are many possible sources of variability leading to correlation, it is incumbent on laboratories to check quality-control data for the presence of correlation. By calculating the sample correlation coefficient r, a laboratory can readily quantify the degree of correlation between concentrations. If substantial correlation is evident, then the laboratory should consider the use of quality-control approaches that will properly account for correlation in the data. Given the possibility of correlation in control data, the next step is to determine its impact on the performance of quality-control methods. If one considers the instance of two control materials, then the difference between assuming independent and correlated control concentrations can be demonstrated graphically, as shown in Fig. 1. In the case where independence applies, the in-control region determined by UCL (mean k SD) and LCL (mean k SD) presents itself as a square. In the case where correlation is present, the true in-control region presents itself as an ellipse (7). This means that assuming independence of control concentrations and applying k SD limits as such when the control concentrations are truly correlated will produce areas of over-control and
under-control, as shown in Fig. 1. An over-control area is the region in which independent control limits would indicate a shift in the control materials when no true shift has occurred (i.e., commit an alpha error). The undercontrol area is the region in which independent control limits would fail to indicate a true shift in the control materials (i.e., commit a beta error). As stated previously, the purpose here is not to definitively characterize the nature of correlation between control concentrations within a given analytical run. The purpose is to acknowledge the real existence of this correlation in many instances and to provide a means for dealing with this correlation. A method for dealing with correlation, the 2 chart, is detailed in the next section.
The 2 Chart
Although a number of multivariate approaches are available for quality-control application, the 2 chart (8, 9) was used for this research. This selection was based on the fact that the 2 chart assumes a known covariance matrix, the same assumption made for evaluating traditional qualitycontrol approaches. In practice, a T2 chart (10, 11) can be used in the case of an unknown covariance structure with results similar to the 2 chart. Assuming a known covariance matrix for the control concentrations being monitored, application of the 2 chart is very straightforward. There is only an upper limit for the 2 chart, and it is based on the 2 distribution. The 2 upper limit for the chart is ,p, where p is the number of parameters being monitored and is the probability of a type I error (i.e., the probability of the 2 chart signaling a shift in the control materials when none has occurred). In the clinical application, p would be the number of control concentrations being monitored (typically two or three). Given the upper limit for the chart, the statistic plotted is simply n(x 0)/1(x 0), where n is the sample size (typically n 1 for the clinical application), x is the observed sample mean vector (observed values from the quality-control batch), 0 is the original mean vector (control concentration targets or historical means), and 1 is the inverse of the covariance matrix of the control concentrations. A simple algebraic formula for calculating this statistic for two concentrations of control material is as follows: n
12 22 1 22 122
2
x 1 1 2 x 2 2 2 2 12 x 1 1 x 2 2 12 22 12 22
where x1 and x2 are observations for control concentrations I and II, 1 and 2 represent historical means for control concentrations I and II, and 1 and 2 represent the SDs of control concentrations I and II, respectively, with 12 the covariance between concentrations I and II. By plotting this calculated statistic against the upper limit
1961
of the 2 chart, one can determine if the measurement system has shifted.
Method Comparison
To understand the performance of a multivariate approach, it is necessary to compare such an approach to traditional methods of clinical quality control. For the purposes of this research, traditional methods of quality control include the strict application of a single limit set at 2.75 SD limits (12.75s), the use of an immediate retest for limits set at 1.91 SD (21.91s), and Westgards Multirule Procedure as originally published. Although typical approaches would be the strict application of 2 SD limits (12s) and 2 SD limits with an immediate retest (22s), these rules have been replaced with 12.75s and 21.91s to match the in-control ARL of the Westgard Multirule Procedure. The multivariate approach chosen for application is the 2 chart with alpha 0.0119 to also match the in-control ARL of the Westgard Multirule Procedure. The comparison of the three traditional methods and the 2 chart is made for the typical clinical application of N 2 (i.e., n 1, p 2 from the multivariate perspective). However, different shifts in the control materials are considered in this research from those that generally appear in the literature. The comparisons are made for the usual instance when both concentrations of control material shift together (i.e., concentration I and concentration II both shift the same direction at the same time), denoted as N 2/2. Additionally, comparisons are made for the case when a single concentration of control shifts while the other concentration of control remains on target (i.e., concentration I experiences a shift while concentration II stays centered or vice versa), denoted as N 2/1, and the case where the two control concentrations diverge (i.e., concentration I shifts upward while concentration II shifts downward), denoted as N 2/2. Fig. 2 graphically portrays the shifts investigated in this research. Although all these shifts are not typically evaluated in the literature, they may easily occur in practice and should be considered.
In addition, the comparisons of the quality-control methods are made based on ARLs, where the ARL is simply the number of quality-control batches tested (on average) before the quality-control method will signal. In the case where no change has occurred in the measurement process, the ARL corresponds to the average number of quality-control batches until a false rejection. In the case where a true shift has occurred in the measurement process, the ARL corresponds to the average number of quality-control batches until a true detection. The reason for applying ARLs for comparison purposes as opposed to probabilities is that the Westgard Multirule Procedure involves information over multiple quality-control batches. Adding a run rule that requires four qualitycontrol batches (i.e., 41s) for evaluation in no way improves the probability of detecting a true shift with the first sample evaluated after the shift occurs. It does, however, reduce the average number of quality-control batches required to detect the shift in the long run. Therefore, the ARL is used for comparison purposes in this research. To generate the ARL information, the calculations for 12.75s and 21.91s are straightforward probability calculations. The ARLs for the Westgard Multirule Procedure are determined using computer simulation. For the 2 chart, ARLs are generated by integrating the non-central 2 distribution (12). Additionally, a correlation structure must be specified to evaluate the 2 chart, and the correlation between the concentrations is assumed to be 0.60 and held constant for all analysis in this research. Here, we consider only two concentrations of control material; however, the modeling can be extended to the instance of three concentrations of control with similar results. The results from the statistical modeling are shown in Figs. 35. Fig. 3 shows the ARLs for the case when two concentrations of control are both shifted together (N
1962
2/2). Note that both means are shifted in terms of SDs (i.e., both control materials experience a 1.0 SD shift in their means). Figs. 4 and 5 show the ARLs when the control concentrations are shifted independently (i.e., N 2/1 and N 2/2). Examination of the graphs yields some valuable insights. The first is that the Westgard Multirule Procedure is the most sensitive to changes in both concentrations of control materials in the same direction (N 2/2). The other traditional approaches show performance close to the Westgard Multirule Procedure, with the 2 chart being the least sensitive to this type of shift. However, the 2 is more sensitive to the other two types of shifts considered in the research, N 2/1 and N 2/2. In fact, the 2 chart substantially outperforms the other approaches for the case of diverging control concentrations. Although the traditional methods have reduced sensitivity to a shift in a single concentration of control or two diverging control concentrations, the performance of the multivariate approach actually improves. This is because of the nature of the in-control ellipse for the 2 chart. For
a shift common to two concentrations of control, the direction of the shift moves along the axis of the in-control ellipse. However, for a shift in a single concentration of control, the direction of the shift moves away from the axis of the ellipse, making the approach more sensitive to these kinds of shifts. For two diverging concentrations of control material, the 2 chart is even more sensitive to the change because of the direction of the mean vector shift vs the in-control ellipse. These results are also important from the standpoint of the perceived error protection for the traditional qualitycontrol approaches. When the ARLs are examined for the different types of mean shifts, it is clear that the error protection capabilities for the traditional methods decrease. For the multivariate case, the error protection for a shift in a single concentration or diverging concentrations actually improves. This means that a 2 chart developed for the case of a shift in both materials will provide that minimum degree of error protection regardless of the type of shift encountered. This is not the case for traditional quality-control approaches. To further evaluate the performance of the 2 chart, the sensitivities to changes in precision for all the qualitycontrol approaches are evaluated in Fig. 6. The chart shows the ARLs for detection of changes in the control concentration SDs by multiples of the SD. Only the case where both SDs are inflated is considered. From an examination of the chart, it is clear that all the methods considered have essentially the same sensitivities to changes in random error or precision. Therefore, none of the approaches has a particular advantage over the other methods in terms of detecting changes in random error.
Conclusion
The existence of correlation between control concentrations in quality-control data is very real. Although this correlation may not always be large and certainly varies from assay to assay, its presence can play a large role in the performance of a selected quality-control approach. To deal with correlated control concentrations in clinical quality control, a multivariate approach has been proposed. This approach utilizes the correlation structure of multiple concentrations of control materials to determine
1963
the appropriate in-control region for the monitoring application. There are a number of advantages to the use of the multivariate approach for monitoring clinical quality control. The first is that the approach provides the appropriate control region for the application. If the assumption of independence does not hold, then the assumed performance of traditional approaches can be misleading. The multivariate approach, however, can guarantee error protection for a variety of different types of shifts in the control materials. Another advantage of the 2 chart is that it moves away from the application of runs rules. There is an inherent uneasiness with detecting shifts that occurred four or five quality-control batches ago and shutting down the system. What about all the patient samples in between that were subjected to the same shift in the measurement process? With the 2 chart, each quality-control batch would be either acceptable or unacceptable on its own merit. There is still a risk that a shift could go undetected; however, that risk does not change from quality-control batch to quality-control batch, and that risk is considered explicitly in the design of the 2 chart. Given the potential for correlation in clinical qualitycontrol data, it would be advisable for laboratories to check for correlation in their data. If the assumption of independence holds, then traditional methods should be continued. If substantial correlation is evident, then a
multivariate approach, such as the 2 chart, should be applied to the quality-control application.
References
1. Levey S, Jennings ER. The use of control charts in the clinical laboratory. Am J Clin Pathol 1950;20:1059 66. 2. Westgard JO, Groth T, Aronsson T, de Verdier C.-H. Combined Shewhart-Cusum control chart for improved quality control in clinical chemistry. Clin Chem 1977;23:18817. 3. Parvin CA, Gronowski AM. Effect of analytical run length on quality-control (QC) performance and the QC planning process. Clin Chem 1997;43:2149 54. 4. Westgard JO, Barry PL, Hunt MR. A multi-rule Shewhart chart for quality control in clinical chemistry. Clin Chem 1981;27:493501. 5. Linnet K. Choosing quality-control systems to detect maximum clinically allowable analytical errors. Clin Chem 1989;35:284 8. 6. Petersen PH, Fraser CG. Setting quality standards in clinical chemistry: can competing models based on analytical, biological, and clinical outcomes be harmonized? Clin Chem 1994;40: 1865 8. 7. Jackson JE. Quality control methods for two related variables. Ind Qual Control 1956;1:1 4. 8. Alt FB. Encyclopedia of statistical sciences. New York: John Wiley & Sons, 1985:110 22. 9. Tracy ND, Young JC, Mason RL. Multivariate control charts for individual observations. J Qual Technol 1992;24:88 95. 10. Alt FB. Multivariate quality control: state of the art. Am Soc Qual Control Qual Congr Trans 1982;886 93. 11. Jackson JE. Multivariate quality control. Commun Statistics 1985; 14:2657 88. 12. Anderson TW. An introduction to multivariate statistical analysis. New York: John Wiley & Sons, 1958:727.