Accuracy, Precision, & Calibration

EE 483 - Spring 1998

Accuracy: The difference between the expected value of a sensor's output and the actual value of the quantity being measured. The mean of a set of data is related to the accuracy of the data. Precision: The variations in a measured signal due to noise or other phenomena. The standard deviation of a set of data is a measure of the precision. Theoretical Calibration: Define a mapping based upon how an ideal system should behave: End-Point Calibration: Define a linear map such that the data passes through the endpoints: y = ax + b such that y(x1)=x1, y(xn)=xn.

For example, if the temperature of a room from 0C to 10C is measured, the following data shows measurements which are neither accurate nor precise, accurate, precise, and both.

Measured T vs Actual T
Zero-Based Calibration: Define a linear map which passes through the origin and minimizes the mean squared error: This is useful in that it forces the output to zero when the measurement is zero. y = ax
Accurate

Precise

Accurate and Precise

Calibration:
Calibration improves the accuracy and precision of a sensor by defining a mathematical relationship between what you measured and what the reading should be. For example, the third plot above could be improved by subtracting a constant from the measured temperature. This would be one type of calibration. In general, calibration attempts to map the measurement to its ideal value by some function: y = f (x ) where 'x' is the measured quantity and 'y' is an improved estimate of what you're trying to measure. Different functions are as follows:

Linear Calibration: Try to fit a straight line to the data: y = ax + b a,b free

1-1

Polynomial Calibration: Try to fit a polynomial to the data: y = ax 2 + bx + c

Y=T; X=R; % Zero-Based Calibration X=R; A=inv(X'*X)*X'*Y A = 6.1341e-004

So the best you can do is T 0.0006143R Checking this for accuracy and precision...
Yest=A*X; [Y,Yest] ans =

Nonlinear Calibration: Try to fit some other function to the data.

0 10.0000 20.0000 30.0000 40.0000

18.0895 11.5260 7.5204 5.0263 3.4302

Examples: Regression using MATLAB:

Problem: The resistance vs. temperature for a thermister is given below. Find a curve which passes through the following points and minimizes the squared error (Least Squares solution). T R 0 10 20 30 8,194 40 5,592

mean(Y-Yest) ans = 10.8815 std(Y-Yest) ans = 21.5308

29,490 18,790 12,260

Not surprisingly, zero-based calibration is fairly poor. Solution (Least Squares): Express the problem as Y = XA where Y is the output, X is a known function matrix, and A is a constant but unknown matrix. The least squares solution for A will then be A (X T X ) X T Y The estimated output is then Y = XA and the error in this estimate is E = YY So T 0.0016R + 43.6790 Checking this calibration scheme for accuracy and precision...
Yest=X*A; [Y,Yest]
1

Linear Calibration
Next try T aR + b
% Linear X=[R,R.^0]; A=inv(X'*X)*X'*Y A = -0.0016 43.6790

Zero-Based Calibration:
First, set this up as Y = AX
R=[29490,18790,12260,8194,5592]'; T=[0,10,20,30,40]';

ans = 0 10.0000 20.0000 30.0000 40.0000 -3.2961 13.7481 24.1499 30.6267 34.7714

1-2

 mean(Y-Yest) ans = -5.6843e-015 std(Y-Yest) ans = 4.1793

red = Y blue = polynomial green = linear

8.25C from 0 to 40C.

Polynomial Calibration:
The data looks like a quadratic function. relationship like T aR 2 + bR + c may work better. In MATLAB
% polynomial X=[R.^2,R,R.^0]; A2=inv(X'*X)*X'*Y Warning: Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND = 2.743875e-019 A2 = 0.0000 -0.0038 58.0711

Hence, a

Noise and Higher-Order Calibration Schemes:

Given enough terms, you can curve fit any set of data. The approximation may actually get worse with more terms, however, in that eventually you start fitting noise rather than data. One way to tell if you should include more terms is to look at the residual (Y-Yest). If this looks like random noise, there is no reason to include more terms. If it is a smooth curve, however, then you might want to include a term to fit this residual. For example, if you approximate temperture as a constant Ta and find 'a', you will get a constant line at about 20C (the midpoint of the data). The residual will be the actual data, shifted down so that the mean error is zero. This curve looks somewhat a line , however, so another term may be added.

so T 0.0000R 2 0.0038R + 58.0711 Checking the accuracy and precision...

Y2est=X*A2; [Y,Y2est] ans = 0 10.0000 20.0000 30.0000 40.0000 0.2871 8.7736 20.9364 31.1840 38.8189

T aR + b This results in a residual which looks like

mean(Y-Y2est) ans = -4.9605e-014 std(Y-Y2est)

Again, the accuracy of higher-order polynomials is still zero. The precision is better, however. Plotting R vs. Y and its estimates gives the following 1-3

This looks somewhat a polynomial. another term may make sense: T = aR 2 + bR + c This residual then is

Hence, adding

At this point, the residual is starting to look like noise. Including a third term is probably not warranted.

1-4