Trends in Analytical Chemistry

Use of recovery and bias information in analytical chemistry and estimation of its uncertainty contribution
Thomas P.J.Linsinger

Arianne G. Letada MS-Chemistry

BACKGROUND-LAB METHOD FLOW

Method Development

Method Validation

Method Transfer

Approved

Definitions

Validation is the process of demonstrating or confirming the performance characteristics of a method of analysis. A process of evaluating method performance and demonstrating that it meets a particular requirement. Validation applies to a specific operator, laboratory, and equipment utilizing the method over a reasonable concentration range and period of time.

Why Method Validation?

To minimize analytical and instrumental errors To give reliable and reproducible results in accordance with the given specifications of the test method To ensure the quality of the test results To meet accreditation requirement Objective evidence for defense against challenges To be assured of the correctness of results

Method bias is generally recognized as intrinsic part of method validation There are different points of view on how to estimate potential bias

whether to correct for it how to accommodate it its uncertainty in the uncertainty budget for a particular measurement.

Introduction
It is a truism in the analytical community that not every method that should work in principle will also work in practice and deliver accurate results. Analytical methods therefore have to be validated (either in-house or by laboratory intercomparison) to demonstrate the reliability of the results obtained by these methods.

Parameters for Method Validation

Accuracy Precision (repeatability, reproducibility) Specificity Limit of detection Limit of quantitation Linearity and range

Definition
IUPAC Trueness: Closeness of the agreement between the average value obtained from a large series of test results and an accepted reference value Bias: The difference between the limiting mean and the true value ISO Guide 99 Trueness: Closeness of agreement between the average of an infinite number of replicate measured quantity values and a reference quantity value. Bias: Systematic measurement error or its estimate, with respect to a reference quantity value.

This review will focus on the estimation of bias and its uncertainty, essential for assessment of the significance of an eventual bias, and the use of uncertainty in the estimation of measurement uncertainties. Particular emphasis is given not only to peerreviewed literature but also to guidelines issued by international organizations and accreditation bodies.

Estimation of bias
b = xmeas xref eq. 1 This bias may be a function of the analyte content. b = xmeas / xref eq.2 The methods of bias estimation differ in the source of the reference value.

Requirements in selecting the materials for bias estimation

Materials shall resemble real-life samples as closely as possible Materials with different analyte levels shall be available The reference values shall be reliable and their uncertainties as small as possible.

Certified reference materials (CRMs): well-designed intercomparisons or measurements with another method of demonstrated accuracy
Advantage high reliability of the reference values with low uncertainties. Disadvantages only in few cases are materials at different concentration levels available for a certain matrix. due to the longer storage of CRMs, trade-offs between stability and realistic presentation must frequently be made

Therefore recommended for bias estimation (e.g., IUPAC, ISO 17025 and Eurachem)

Harmonised guidelines for the use of recovery information in analytical measurement: use of surrogates (spikes) as options for estimating recoveries
Advantages

Disadvatages

materials with different analyte concentrations are readily available the uncertainties in the analyte contents are generally low

spikes (isotopically labeled or not) generally do not reach an equilibrium in the spiked samples and hence result in heterogeneous samples
The recovery of the surrogate is therefore likely to be greater than that of the native analyte, so a bias in an estimated recovery may arise

Meija and Mester Isotope-dilution massspectrometry: their review showed that equal behavior cannot be expected for trace metals or organometallic substances. Analytical Methods Committee: which recognized that all methods for estimating recovery are unsatisfactory in some circumstances

ISO TS 21748: Guide to the use of repeatability, reproducibility and trueness estimates in measurement uncertainty estimation
Advantages only possible for a single laboratory to demonstrate bias control with respect to other laboratories, as it usually has to assume that the results of the other laboratories are correct. Disadvantages the lower reliability of the assigned values

Testing for significance

If the bias obtained is smaller than the expanded uncertainty of this bias, there is in reality no evidence of a bias. However, even if found statistically significant, a bias may still be deemed practically insignificant if it is small compared to the measurement uncertainty of the measurement in question.

The Eurachem Guide on measurement uncertainty defines this small as not larger than one-third of the largest uncertainty component. This general recommendation has been confirmed by Monte Carlo simulations showing that uncertainties are significantly underestimated if a bias larger than one third of the other uncertainty contributions remains uncorrected.

Uncertainty of bias
ucorr = u2meas + u2b eq.3 ub= u2meas,b + u2ref eq.4 These two equations illustrate the basic principle of estimating the uncertainty of bias namely that it can never be smaller than the uncertainty of the reference value. It also breaks down the estimation of bias uncertainty into two sub-problems, namely determination of the uncertainty of the reference value and estimation of the measurement variation.

Disadvantages

First, it treats uncertainties of recovery as combined uncertainty, thus forgoing any deeper insight into source of the potential bias; and, second, it implicitly assumes that the results obtained from the samples during the study are valid for future, yet unknown, samples.

Bettencourt et al., showed that deeper insight of the source of bias can also be gained from validation data.

They applied an intricate analytical design that aimed at separating uncertainties from re-extraction of the extracted sample, sample-processing recovery and extraction recovery. In this way, they managed to identify the main sources of bias, which subsequently could be used for a targeted method optimization. Their approach is close to the bottom-up approach of uncertainty estimation

Maroto et al., who constructed Youden plots for

samples of various weights used to obtain an estimate of the constant bias, which later on were combined with the uncertainty from the proportional bias.

The second problem is not specific to bias estimation but extends to all aspects of method validations; samples always have to be chosen to reflect samples encountered in daily use. The issue of bias is in this respect no different from determination of repeatability, intermediate precision or limit of quantification.

Barwick and Ellison discussed how to accommodate variation between concentrations, matrices and spike vs. incurred samples in the general uncertainty model.

They gave guidance on how to include additional uncertainty sources corresponding to differences between recovery of the spiked and the incurred analyte and changes of recovery with concentration and matrix. This is done by estimating uncertainty contributions for all of these variations and including them in the overall uncertainty model

Eurachem Guide on measurement uncertainty recommended inclusion of these uncertainties

Bot et al. used a nested design to evaluate mean uncertainty of recovery, variation of recovery due to difference in matrices and variation of recovery depending on the content of endocrine disruptors in a sample (sediments and waters). Their uncertainties are estimated from ANOVA. They generally found that variation of recovery depending on the analyte content was negligible. The problem of matrix mismatch can sometimes be overcome by using materials from proficiency tests (PTs) for assessing method trueness. PT samples are frequently closer to real-life samples than spikes or CRMs, and the problem of matrix mismatch is hence less prominent.

Desenfant and Priel explained the use of PT results for the estimation of uncertainty of bias. The uncertainty of bias is the standard deviation of the biases in the various rounds of the PT scheme and must be included in the overall uncertainty. If the average bias is not zero, an additional uncertainty contribution relating to this bias must be added. However, Desenfant and Priel discouraged this latter approach. While the use of PT data certainly has its advantages, one main disadvantage is that assigned values are often less reliable.

Correction versus non-correction of bias

There is evidence that recovery correction leads in most cases to better comparable results, but legislation is sometimes unclear or even explicitly states that results should not be corrected for recoveries. Recoveries in organic analysis are often regarded as acceptable if they are 70110%

It is understood that any correction of bias should be a method of last resort only. Removal of a bias should always be preferred over accommodation of a bias. Magnusson and Ellison gave four criteria that need to be fulfilled to warrant bias correction: (1) evidence of a significant effect; (2) a causal relationship; (3) the estimation of the bias must be sufficiently accurate; and, (4) correction must reduce the uncertainty.

Guide to the Expression of Uncertainty in Measurement(GUM) explicitly states that known biases must be corrected for, and only in exceptional circumstances is it acceptable to increase the uncertainty to allow for the bias. Eurachems translation of the GUM for analytical chemistry also states that known biases must be corrected for. IUPAC also follows this policy. Its guide on the use of recovery information distinguishes between rational and empirical methods

The Asia Pacific Laboratory Accreditation Cooperation (APLAC) states that results must be corrected for recovery. Results not corrected for recovery are only traceable to the specific working instruction. The AMC states even more drastically that results obtained without correction for recovery are necessarily empirical The same conclusions are drawn by the Australian National Pathology Accreditation Advisory Council (NPAAC). Eurolab, Nordtest, the Nordic Committee on Food Analysis (NMKL), European Accreditation (EA) and the American Association for Laboratory Accreditation (A2LA) agree that bias must be corrected for. This unanimous agreement between the institutions in Europe, Asia and the Americas that results must be corrected for recoveries is encouraging with respect to the ultimate goal of achieving comparability of measurement results world-wide.

Use of the uncertainty of bias

According to the law of error propagation, uncertainties in the basic analytical procedure and the uncertainty of bias/recovery must be combined to obtain the full measurement uncertainty. There is agreement among international bodies that the uncertainty of the bias correction is a part of the measurement uncertainty.

The EA Guidelines for uncertainty estimation state explicitly that in general, the uncertainty associated with the determination of the bias is an important component of overall uncertainty. Lyn et al. showed that ignoring bias, in this case from sample preparation, underestimated uncertainty. Feinberg and Laurentie found that inclusion of uncertainties of the recovery factors increased uncertainties.

Guidance exists on the issue of how uncorrected bias should be included in an uncertainty statement. Magnusson and Ellison devoted an entire review article to the treatment of uncorrected bias. Also, Phillips and Eberhardt, ODonnell and Hibbert and Synek investigated several options for allowing for uncorrected bias. A recovery of 100% does not eliminate the recovery factor from the basic measurement equation. The problem basically is that, if the variation in results of a method is high enough or if the uncertainties of the certified values of the CRMs used are large enough, any bias will remain undetected.

Maroto et al. found that ignoring non-significant bias results in underestimations of the uncertainty, if the uncertainty of the bias is large and if the bias contributes significantly to the overall uncertainty. Ellison and Barwick also explicitly stated that a recovery of 100% has an uncertainty, but they also warned of double counting so they advised planning validation experiments carefully.

IUPAC, Eurachem and Nordtest follow this line by stating explicitly that an uncertainty of recovery needs to be included in the overall uncertainty even for 100% recovery or a bias of zero. The NPAAC guide qualifies this point of view by recommending inclusion of the uncertainty of bias only when significant. The ISO explicitly assumes that the uncertainty of the bias check is negligible compared to the other uncertainty sources, so does not have to be included. If this is not the case, an additional uncertainty contribution must be added.

Hasselbarth the uncertainty of bias is not a part of the overall measurement uncertainty. The condition is that a full uncertainty budget is available and bias is tested as final step in an exhaustive evaluation of measurement uncertainty. In this case, all uncertainty sources are already included and no additional uncertainty needs to be added when no significant bias is found. He agrees that, whenever bias is tested in a withinlaboratory or a between-laboratory validation procedure, the model still includes the recovery factor and its uncertainty must be included even if recovery is found to be 100%.

Conclusions and suggested procedure

1.

2.
3.

4.

5.

Materials that are sufficiently close to real-life samples with sufficiently accurately assigned reference values shall be selected for the assessment of method bias. Method bias shall be estimated. The uncertainty of the bias shall be estimated. This estimation includes in all cases the uncertainty of the assigned value and a contribution of the variation of the measurement results used for the bias estimation. The significance of the bias is tested by comparing the bias determined with its expanded uncertainty estimated. If the bias is found to be significant and no further method optimizations are performed, results, in general shall be corrected for this bias. The uncertainty of the bias is included in the uncertainty budget for measurements from this method, regardless of whether or not the bias was found to be significant. If no bias correction is applied for insignificant bias, an additional uncertainty contribution accounting for this uncorrected bias has to be added.