Institute of Hygiene And Environment City of Hamburg, Germany Dr. Georg Giese (author), Dr.

Manfred Kutzke (translation)

03.12.2004

Uncertainty of measurement 1 Basics

Testing laboratories need to define the amount of uncertainty thats originating from the different steps in the analytical process. The rigor needed in an estimation of uncertainty of measurement depends on the requirements of the method and the customer (fitness for purpose). With environmental analysis, uncertainty of measurement should comprise sample preparation, sample measurement, and also sampling, where appropriate and feasible. These provisions do not apply to qualitative and semi-quantitative work. Standardized methods dont need additional definition of uncertainty of measurement, provided the lab has demonstrated its ability reach the level of performance stated in the standard. These provisions favor the concept of expanded measurement uncertainty. For estimating levels of uncertainty, you may use quality data derived from routine work. Additionally, the main components of uncertainty should be identified and discussed as well. Its not really necessary to do a proper mathematical calculation. Attached to this document youll find some examples which should help clarify the concept.

2 Description and definition of standard uncertainties

The method concerned should be named and its error-afflicted steps (individual standard uncertainty compounds) should be described in a way that the relevant contributions to method uncertainty can be identified. A cause-and-effect diagram may be useful.

Calibration Preparation of standard Measurement of standard

Result
Dilution as needed Matrix effect Filtration Measurement of sample

There are two types of information used for estimating uncertainty of measurement: Components of uncertainty that can be measured by experiments regarding precision and bias1 (Type A). May be derived from standard deviations from quality control charts and from calibrations. Additional Components that may be estimated from analytical experience, from the literature, from instrument manuals2 (Type B)

Particularly useful is using data from measurement of certified reference materials, yielding the complete method uncertainty in a single step (excluding sampling, of course). It is reasonable to combine single steps to uncertainty modules (uy) integrating numerous uncertainties, which dont have to be estimated separately any more. Frequently, we will calculate uncertainty modules from quality control charts, resulting from routine work, as Type A error (standard deviation). A X-bar chart from an independent check standard reflects the error contributions from preparation of standards, calibration and measurement. By using an R chart, you may include systematic error from instrument fluctuations as well. Using R charts with real samples will also include matrix effects. By using (certified) reference materials as quality check samples, the complete method may be monitored in one step and the total measurement uncertainty may be estimated.

Uncertainty u derived from standard deviation s from n repeated measurements: u = Tolerances from manufacturers manuals typically follow a rectangular distribution:

s n

u=

tolerance 3

3 Combined standard measurement uncertainties

We may combine standard uncertainties of i steps to uncertainty modules uy by applying the law of error propagation:

f = u xi
2 N y i =1

x= u

2 i

f f + 2 x x= x x= u ij i< j i i

with

y = f (x1, x2, x3, x4xi)

and

= mean value

For simple, linear relationships (y=x1+x2++xi), variances may simply be added

u i2

If the partial result is the product of the terms, relative variances must be added

y , rel

u i2, rel

4 Dependence of uncertainty on concentration

In environmental analysis, data quite often may span several orders of magnitude. Hence, variances may vary with concentration. It is advisable, therefore, to perform several estimations in ranges of concentrations relevant to decision-making. At least we should estimate using data from concentration-adapted quality control charts. A procedure for interpolation of measuring uncertainties to concentrations not investigated is given by EURACHEM3 Alternatively, we may estimate an averaged uncertainty by using data from R charts from real samples, as long as the concentrations are representative for the working range.

EURACHEM/CITAC Guide: Quantifying Uncertainty in Analytical Measurement, Final Draft, April 2000

5 Expanded uncertainty
Quite often, it will not be possible to include all the components of uncertainty into the estimation. Therefore it is advisable to include a safety factor. According to GUM4, we should use a factor of 2 to assure a 95% confidence interval (C)

U = k utotal with

k=2

6 Documentation and expression of results

Using this expanded uncertainty U, results Y should be reported as

Y U (C = 95%)

or Y

Urel

including a reference to the uncertaintys quality, i.e. the confidence interval used. All the steps included in estimating measurement uncertainty should be documented transparently and kept with the validation data. The most important factors of influence should be named. Make sure the estimated uncertainties are matching the requirements of the assignment. Documentation of the estimation process is part of the validation documentation. It should be made available to customers on request. Measurement uncertainties should be reported on customers request

7 Examples
7.1 Estimation of measurement uncertainties in the preparation of an ammonia-N standard solution
Method: Photometric measurement of ammonia, using a continuous-flow analyzer. Calibration is done by using 5 standards. Prepare stock solution 1 by weighing of ammonia chloride; prepare calibration solutions by dilution from stock solution 1 or from intermediate solution (stock solution 2). Preparation of standard solutions: For stock solution 1 (100mg/l N), weigh ammonium chloride into 1000ml volumetric flask und add water up to the mark. We need to look at purity of ammonia chloride, the balances tolerance, errors in weighing and filling the flask, and tolerance of the flask. Tolerances given by the manufacturers are uncertainties of type B, which follow a rectangular distribution. Therefore, we calculate uncertainties by dividing by 3 .

Guide to the Expression of Uncertainty in Measurement (GUM), chapter 6

We can investigate errors in filling the volumetric flask, pipetting and weighing by repeating these activities several times (n) and calculating the standard deviation (s). These are type A errors that we may calculate by u= s n

We normally do not repeat these activities for any given step, so n=1 and u=s Weighing of ammonia chloride: Purity is 99.8%, so there may be up to 0.2% impurities. Weighed portion is 100mg N (or 381.9mg ammonia chloride), impurities are 0.2mg The balances tolerance is 0.1mg u (impurity ) = u (balance) = tolerance 3 tolerance 3 = = 0.2 3 0.1 3 = 0.115mg = 0.058mg

Weighing-in with 10 repeats, we found a standard deviation of 0.08mg

u ( weighing in) = s = 0.08mg

Combining these uncertainties gives

u (1) = u (impurity ) 2 + u (balance) 2 + u ( weighing ) 2 = 0.115 2 + 0.058 2 + 0.08 2 = 0.151mg

We get the relative uncertainty by dividing by the weighed portion

u (1) rel = 0.151 u (1) = = 0.00151 100 weighed portion

Solving ammonia chloride in water and filling the 1000ml volumetric flask Volumetric flask tolerance is 0.6ml u ( flask ) = tolerance 3 = 0.6 3 = 0.346ml

Filling the flask ten times, we found a standard deviation of 0.096ml

u ( filling ) = s = 0,096ml

Combining these uncertainties, we get

u (2) = u ( flask ) 2 + u ( filling ) 2 = 0.346 2 + 0.096 2 = 0.36ml

We get the relative uncertainty by dividing by the set volume

u (2) rel = u ( 2) 0.36 = = 0.00036 volume 1000

Uncertainties of stock solution 1 and dilutions By adding the relative uncertainties using the law of error propagation, we get the uncertainty of the stock solution 1

u ( stock solution 1) rel = u (1) 2 + u (2) 2 = 0.00155 rel rel

u ( stock solution 1) = 100mg / l 0.00155 = 0.155mg / l or 0.16%

We prepare stock solution 2 and the calibration solutions 1 to 5 by diluting stock solution 1 or stock solution 2. Here, we have to include the tolerances of pipettes and volumetric flasks and uncertainties in pipetting and filling the flasks. Resulting uncertainties are between 0.6 and 0.8%.

7.2 Estimation of measurement uncertainties of photometrical determination of ammonia-N in water samples, using main uncertainty contributions
Method: Photometric measurement of ammonia, using a continuous-flow analyzer. Calibration is done by using 5 standards. Level of quantitation is 0.04 mg/l.

Sample preparation: membrane filtration 0.45m Regular quality assurance: X-bar chart, using an independent check standard. Certified reference materials not available. Two separate R charts: S1 Triple analysis of real samples, once at each series S2 Single analysis of real sample, filtrated three times, once a week. We use this quality control chart for estimating uncertainties of real samples.
Main uncertainty contributions: Possibly resulting from calibration (preparation and measurement of standards) and from measuring real samples (sample preparation, sample matrix).

In this example, we calculate the uncertainty component arising from calibration from results of an arbitrarily chosen calibration. With real samples, the actual data may be used if feasible. In case routine calibration uses only few standards, or a prior calibration is just checked, we may also use data from validation studies or 10-point-calibrations. For uncertainty contribution real samples we use the results from control chart S2.

calibration standard preparation standard measurement result

dilution as needed

matrix influence

filtration sample measurement

Calibration:

Preparation of standards Relative uncertainties between 0.6 and 0.8% (refer to previous chapter), which is not very relevant (< 1%). This contribution will partly be included in the uncertainty of measuring standards anyway. Therefore, we will not regard standard preparation a main uncertainty contribution and will not include it in further computations.

Measuring of standards We will analyze 5 standard solutions within each series of samples. Compute residual standard deviation sxy and standard deviations for slope sb and intercept sa from

(y y )
i i

s yx =
) y=

n2
) calculated results , y = bxi + a measured value

yi =

sb =

2 yx

(x
i =1

x)2

sa =

2 s yx xi2 i =1

n ( xi x ) 2
i =1

For any given sample, the standard deviation is

s xy ( Analysis ) 1 1 sb = + + * b m n b s xy
2

ya s yx

n = number of calibration points m = number of repeats We use an Excel sheet for computation. Inserting measuring values between 0.04 and 1 mg/l, we get uncertainties between 0.005 and 0.007 mg/l.

Determination of ammonia-N in real water samples

By evaluating R chart S2 we do not find a significant increase of standard deviations within the range of 0.04 to 1 mg/l. Mean standard deviation is 0.009 mg/l, which is, not surprisingly, a bit higher than what we find from calibration data. We will use this value for further computation.

Combined and extended measuring uncertainties

We may compute the combines uncertainty from the uncertainty contributions of the calibration and the measuring of real samples. Calculations are done by an Excel sheet, using this formula:
R R R sc sb + sa + uc = a b c
2 2 2

where and a: intercept (calibration) b: slope (calibration) c: measured value (sample)

R=

c sample a b

sa: standard deviation of intercept sb: standard deviation of slope sc: standard deviation of measured value

If we substitute measured values within the calibration range, well get the following combined measurement uncertainties uc. We get expanded measurement uncertainties U by multiplying uc with the factor of 2.

measured value [mg/l]

uc [mg/l]

uc [%]

Uext [mg/l]

Uext [%]

0.04 0.5 1

0.0095 0.01 0.012

23.7 2 1.2

0.019 0.02 0.024

47.4 4 2.4

7.3 Estimation of measurement uncertainties of photometrical determination of ammonia-N in water samples, using a span control chart
Basics: Procedure has been taken from German Standard Methods for Water, Wastewater and Sludge Analysis5, which describes the use of an R chart for estimating measuring uncertainties. It neglects any influence of sample concentration, yielding relative measurement uncertainty for the methods whole range. Prerequisite is an R chart from real samples with representative sample matrix, comprising the complete method including sample preparation. If the R chart does not include sample preparation or sample matrix, these contributions to measuring uncertainty must be estimated separately. From the span control chart, we estimate the mean relative standard deviation srel by s rel = R rel d2

R rel : mean relative span d2=1.128 (duplicate) or d2=1.693 (triplicate analysis) In our example (determination of ammonia-N), the complete set of R charts yields a mean relative span of 9.8% in the calibration range of 0.04 to 1 mg/l. The chart includes filtration, which is done in triple and the filtrates measured separately. This gives a mean relative standard deviation of 5.8%. This may be regarded as the relative measuring uncertainty in the calibration range of 0.04 to 1 mg/l. We get the extended measuring uncertainty of 11.6% by multiplying with the factor of 2.

7.4 Measurement uncertainty for a continuous automated instrument in a monitoring network and effect on the annual mean value
Prerequisites for operating a monitoring network are the current EU directives. Measurement uncertainty is assessed according to: EN ISO 14956, ISO 11222 (Draft), EN14211 (Draft)
5

Strategien fr die Wasseranalytik: Verfahrensentwicklung, Validierung und Qualittssicherung in der Routine; published in:Deutsche Einheitsverfahren zur Wasser-, Abwasser- und Schlammuntersuchung, 39. Lieferung 1997, Beuth-Verlag

Additional requirements: Use of air conditioned monitoring stations with maximum temperature change of 5 K. Biannual maintenance and cleaning and quarterly checks of the sampling train. Continuous monitoring of sample gas flow. Automated recalibration of instruments every 25 hours. Use of recalibration results for correction of data. Daily plausibility checks on recalibrations, triggering maintenance within 4 days if necessary. Removal of data found to be invalid. Monthly maintenance of instruments and checks using test gases. n case instrument zero or span are exceeding predefined limits, additional maintenance is triggered, the instrument is repaired and recalibrated using a transfer standard. Biannual recalibration using a transfer standard.
Estimation of measurement uncertainty for annual mean value of SO2 for the year 2000 EU limit value for annual mean is 20 g/m3 . The 1 h mean limit value is 350 g/m. All uncertainties are given in g/m3. The basis for estimating components of error is the s 1 calculation of standard deviation s from n repeated measurements u = . A factor of is 3 n added whenever measurements where conducted in a field environment. Systematic deviations are recognized and corrected by biannual control calibrations. The uncertainty is given by control calibration standard deviation. Control calibration also monitors any instrument drift. The uncertainty of the calibration test gas is given with 2.3% at 1200 g/m3 . The uncertainty of the annual mean limit value is given by

ucalib =

0,023 20 = 0,2656 3

for the annual mean limit value

Random deviations: Data sources are a) 25 h automated calibration (function check), b) biannual control calibration, c) time of passage of sample gas inside the sampling train. Only the 25 h calibration is evaluated statistically. a) Function check: The set value of the test gas source is 800 g/m3. There had been 360 checks in the year 2000. The standard deviation taken from the function checks is sFC = 17,92 g/m3 = 2,24%. This results in an uncertainty of the annual mean limit value

ufunctioncheck =

20 0,0224 = 0,0027 360 25 3

b) Biannual control calibration Uses a three-point dilution system. Set concentration depends on concentration of test gas and may vary from calibration to calibration. We use the standard deviation of the absolute difference of set value and actual value. Each calibration is valid for 4380 1 hour mean values. scalibration =10,37 g/m at 539 g/m = 1,92%

Number of checks: 2; number of values: 4380. This results in an uncertainty of the annual mean limit value

u calibration =

20 * 0,0192 2 * 4380 * 3

= 0,0024

c) Time of passage of sample gas inside the sampling train Not evaluated yet, and not included in the calculation.

Non-random deviation Established during instrument evaluation. a) Cross-sensitivity for humidity had been found to be -5% (-1 g/m3 for the annual mean limit value) at 90% relative humidity. t has been found, however, that no relevant influence has to be expected below 60% relative humidity. Average relative humidity in the sampling train has been found at 30% - 65%, so the cross-sensitivity will have no real impact. The maximum uncertainty of the 1 hour mean limit value computes to

H 2O =

17.5 * 90 + 90 * 10 + 10 = 10.71 90 3

For the annual mean limit value, influence of humidity is 1 g/m(5% of 20 g/m) Given the fact that relative humidity is variable over time, the error for the annual mean limit value is reduced by dividing by the square root of the number of days.

H 2O =

1 90

90 + 90 * 10 + 10 1 = 0.032 for the annual mean limit value 3 365

b) Cross-sensitivity for nitrogen monoxide Cross-sensitivity for nitrogen monoxide had been found to be +2.2% (+7.7 g/m3 for 1 hour mean limit value) at a reading of 200 g/m3 SO2 and 0 670 g/m3 NO.

u = +7.7 670
NO

670 + 670 * 0 + 0 = 4.45 for the 1 h mean limit value 3

The NO load at the station is rather low, but there may be higher loads for a couple of hours due to a nearby power plant. Therefore, uncertainty is divided by the square root of the number of hours during a sampling period. The uncertainty of the annual mean limit value computes to (2.2% of 20 g/m is 0.44)

u = +0.44 670
NO

670 + 670 0 + 0 1 = 0.0027 for the annual mean limit value 3 8760

c) nfluence of container temperature The influence had been found to be 2% for each 10K. Due to air conditioning there is a variation in temperature of 3K, giving raise to 0.6% deviation (2.1 g/m3 in respect to the 1 h mean limit value). The instrument has temperature correction. Thus,

u temp = 2.1

3 = 3.64 for the 1 h limit value 3

Container temperature depends on daily temperature fluctuations Thus, uncertainty for the annual mean limit value is divided by the square root of the number of days in the sampling period. (0.6 % of 20 g/m is 0.12 g/m)

u temp = 0.12

3 3 365

= 0.0109 for the annual limit value

d) Influence of ambient air pressure Not investigated during method validation. Experience shows the instrument is not air pressure dependant. However, the internal calibration source used for the 25 h function test is, so data is influenced by uncertainty of the calibration system. Average air pressure deviation is around 5 kPa. The deviation of the calibration sources set value is 0.99% per kPa change in air pressure. The 1 h mean limit value is 350 g/m.

uPu = 0.0099 * 350

5 = 10.003 3

for the 1 h mean limit value

Assuming a change of ambient air pressure in northern Germany twice a week, the uncertainty will be reduced by the square root of 104 changes per year.

u = 0.0099 20
Pu

5 3 104

= 0.056 for the annual mean limit value

e) Dependency on fluctuations of voltage Fluctuation of voltage have not been recorded yet. f) Uncertainty from the instruments within-run standard deviation Method validation gave a within-run standard deviation of 0.363% under laboratory conditions. The 1 h mean limit value is 350 g/m.

u =s
r

u =
r

350 0.363 = 1.27 100

for the 1 h mean limit value

The uncertainty for the yearly mean limit value is reduced by the square root of the number of 1 h mean values. (0.363% of 20 g/m is 0.0726 g/m)

u =
r

0.0726 = 0.000776 for the annual mean limit value 8760

g) Uncertainty because of non-linearities Linearity of the monitor had not been investigated during method validation. The manufacturers specification is 1% for the range 100 ppm, which had been verified by the laboratory. For uncertainty estimation we use the maximum error allowed

u =
lin

0.01 * 20 = 0.1155 for the annual mean limit value 3

Overall measuring uncertainty The overall result utotal is calculated by

utot =

u
x =1

in the following table. Results printed in italics are neglected because they are less than 10% of the biggest uncertainty, or because they are not applicable in this context.
Component Period Instrument Function check Correction Temperature deviation Air pressure deviation Yearly mean limit value SO2 01.01.2001 - 31.12.2001 TE 43 C corrected for temperature and air pressure IZS Permeation tube Zero ; Span ON 3K 5 kPa 20 g/m u per year Data Components of uncertainty nonrandom g/m 0.2656 0.0003 0.0027 0.0003 0.0024 x x x x x x x 0.032 0.0027 0.0109 0.0568 0.000776 0.1155 0.295 1.48 @20 g/m

Sources of error

systematic

random

Test gas Function check Zero Function check Span Biannual calibr. Zero Biannual calibr. Span Cross sensitivity (H2O) Cross sensitivity (NO) Dep. Temp. Dep. air pressure Dep. voltage Within-run standard dev. Linearity utotal [g/m] utotal [%]

QC autom. autom. QC QC Validation Validation Validation Validation Validation Validation Validation

x x x x x