Testing and evaluation of thermal cameras for absolute temperature measurement

Krzysztof Chrzanowski, MEMBER Military University of Technology Inst. of Optoelectronics 00-908 Warsaw, Poland E-mail: kchrza@wat.waw.pl
SPIE

Joachim Fischer Physikalisch-Technische Bundesanstalt Temperature Radiation Section Abbestr. 2-12 D-10587 Berlin, Germany Robert Matyszkiel Military Institute of Communication 05-130 Zegrze Pld. Poland

Abstract. The accuracy of temperature measurement is the most important criterion for the evaluation of thermal cameras used in applications requiring absolute temperature measurement. All the main international metrological organizations currently propose a parameter called uncertainty as a measure of measurement accuracy. We propose a set of parameters for the characterization of thermal measurement cameras. It is shown that if these parameters are known, then it is possible to determine the uncertainty of temperature measurement due to only the internal errors of these cameras. Values of this uncertainty can be used as an objective criterion for comparisons of different thermal measurement cameras. 2000 Society of Photo-Optical Instrumentation Engineers.
[S0091-3286(00)03709-0]

Subject terms: infrared; forward-looking infrared; temperature; instrumentation; measurement; metrology. Paper 200031 received Jan. 24, 2000; revised manuscript received Apr. 12, 2000; accepted for publication Apr. 13, 2000.

Introduction

Thermal cameras can be generally divided into two basic groups: observation cameras and measurement cameras. Observation cameras are mostly used in military applications to enable observation of a battleeld in darkness and in difcult atmospheric conditions by creating the relative temperature distribution of the terrestrial scenery being observed. Measurement cameras are used for civilian applications in industry and science, mostly for noncontact measurement of temperature distributions on the surface of the tested objects. Currently the border between these two groups is becoming more blurred as some cameras can be used for both observation and measurement applications. However, this situation is still the exception to the rule, as most observation cameras do not have the capability to measure the temperature of the observed objects, and the image quality of the measurement systems is inferior to the image quality of the thermal observation cameras. Applications of thermal measurement cameras can be divided into two general groups: applications that require only relative temperature measurement and applications that require absolute temperature measurement. Although the same cameras can typically be used in both applications, there are different camera suitability assessment criteria for these two groups of applications. If the camera is used in applications where only relative temperature measurement is required, such as in nondestructive thermal testing NDTT , then the quality of the thermal image of the tested object is usually the most important criterion. If the camera is used in applications where absolute temperature measurement is required, then the accuracy of measurement results is the most important criterion. Therefore, different sets of parameters are required to evaluate the suitability of any thermal camera for these two groups of applications.
Opt. Eng. 39(9) 25352544 (September 2000) 0091-3286/2000/$15.00

Numerous papers, books, and reports have been published on the subject of the evaluation and testing of observation thermal cameras,15 and a few standards precisely dene methods for testing these cameras.69 Three standards1012 also exist on the subject of testing measurement thermal cameras used in NDTT that describe methods for testing three parameters: noise equivalent temperature difference NETD , minimum resolvable temperature difference MRTD and minimum detectable temperature difference MDTD . These parameters, however, enable the evaluation of thermal image quality, not the measurement accuracy, and are useful for the evaluation of only measurement cameras used in applications that require relative temperature measurement. In contrast to the observation thermal cameras and the measurement thermal cameras for the NDTT, the situation in the area of the testing and evaluation of measurement thermal cameras used for absolute temperature measurement is confusing. The accuracy of temperature measurement is the most important criterion for the evaluation of cameras used in such applications. Therefore the parameters of the thermal camera delivered by the manufacturer should enable determination of measurement accuracy when inuence of external factors is negligible or when inuence of these external factors is known. According to the International Vocabulary of Basic and General Terms in Metrology13 VIM , published jointly by seven leading international metrological organizations the International Bureau of Weights and Measures BIPM , the International Electrotechnical Commission IEC , the International Organization for Standardization ISO , the International Organization of Legal Metrology OIML , the International Federation of Clinical Chemistry IFCC , the International Union of Pure and Applied Chemistry IUPAC , and the International Union of Pure and Applied Physics IUPAP , accuracy is dened as the closeness of
2000 Society of Photo-Optical Instrumentation Engineers 2535

Downloaded from SPIE Digital Library on 18 Jan 2012 to 130.89.66.195. Terms of Use: http://spiedl.org/terms

Chrzanowski, Fischer, and Matyszkiel: Testing and evaluation . . .

the agreement between the result of a measurement and the true value of the measured quantity. This means that the accuracy is only a qualitative concept that should not be associated with numbers. According to the VIM we are allowed to say only that accuracy is good, bad etc. As a measure of measurement accuracy the VIM proposes a parameter called uncertainty of measurement, which is dened as a parameter associated with the result of a measurement that characterizes the dispersion of the values that could be reasonably attributed to the measured quantity. Rules of evaluation of measurement uncertainty are presented in the well known Guide to the expression of uncertainty in measurement, commonly abbreviated as GUM, published in 1993 in the name of the seven already mentioned main international metrological organizations.14,15 Note that these rules from Ref. 14 are now fully accepted by other organizations such as the National Institute of Standards and Technology in the United States, the European Cooperation for Accreditation, the American National Standard Institute, the National Conference of Standards Laboratories, the American Association for Laboratory Accreditation, the International Laboratory Accreditation Cooperation and many national standards institutes all over the world. Currently, there is a general trend in industry to accept the quality systems according to the international ISO 9001-9004 norms and in cooperating accreditation laboratories, according to the EN 45001-45003 norms. According to these standards, centers that implement quality systems are required to evaluate an uncertainty of measurements and present it in their certicates. A few examples of the calculation of uncertainty of different measurement results are presented in Ref. 14. More examples are presented in publications of the European Cooperation for Accreditation. However, no method of determining the uncertainty of measurement results with thermal cameras has been published in available literature. Difculties in determining measurement uncertainty is an important limitation of thermal cameras as a veried technique of temperature measurement for the industrial plants and the cooperating accreditation laboratories that have implemented quality systems according to the international norms ISO 9001-9004 and EN 45001-45003. The uncertainty of the measurement result depends on both the intrinsic uncertainty of the thermal camera and external uncertainties. The external sources already mentioned are errors in the determination of the effective emissivity of the tested object, of the effective background temperature, and of the effective atmospheric transmittance by the user of the thermal camera, who is required by the measurement procedure to estimate values for these parameters. We limit our analysis in this paper to the problem of determining the intrinsic uncertainty of thermal cameras. Manufacturers of thermal measurement cameras often state a parameter called accuracy. However, this parameter is not useful for estimating the real measurement accuracy of a thermal camera for two basic reasons. First, there are problems with the interpretation of this parameter. As it was already presented according to international metrological organizations, accuracy is only a qualitative concept that should not be associated with numbers. Second, the so called accuracy is often measured as a range around the
2536 Optical Engineering, Vol. 39 No. 9, September 2000

true object temperature T ob in which the output temperature T out is located when the measurements are carried out under conditions identical to the conditions of calibration of the thermal camera. Errors of temperature measurement generated within the camera under real conditions can be many times higher than the errors suggested by accuracy value. Apart from the accuracy parameter, the manufacturers of the thermal measurement cameras present in product specications other parameters such as NETD, MRT, spatial resolution, instantaneous eld of view, and image resolution. These parameters provide some information about the inuence of specic sources of errors on nal measurement accuracy. However, the manufacturers do not provide any set of parameters that would enable determination of the measurement uncertainty when the inuence of external sources of errors is negligible or when inuence of these external sources is known. Additionally, catalogs often present parameters without a description of measurement methods. The external sources mentioned are errors in the determination of the effective emissivity of the tested object, of the effective background temperature, and the effective atmospheric transmittance by the user of the thermal camera, who is required by the measurement procedure to estimate values for these parameters. The situation is different in the eld of the radiation thermometers, which are used similarly to the thermal measurement cameras for noncontact temperature measurement; there, several manufacturers have recently agreed to overcome this situation and recommend mandatory specications with boundary conditions for radiation thermometers.16 For thermal measurement cameras, published papers present models of measurement errors.17,18 These models enable determination of the true temperature measurement errors only when detail design parameters of the camera and the true temperature of the tested object are known. Because of these requirements the models can be really useful only for thermal camera designers. They cannot be directly used to evaluate measurement uncertainty when only the parameters of a thermal camera, treated as a black box, are known. To summarize the situation, we can say that two unanswered questions remain. First, which parameters of thermal measurement cameras should be measured? Second, how should the uncertainty of the measurement with thermal cameras due to internal errors of these systems be calculated to evaluate their suitability for applications requiring absolute temperature measurement. Section 2 presents a review of the confused situation in area of specications for measurement thermal cameras. A set of seven parameters that fully characterize the capabilities of measurement thermal cameras for absolute temperature measurement are dened in Sec. 3, where methods for measuring these parameters are also proposed. A method to calculate measurement uncertainty with thermal cameras due to internal systems errors is presented in Sec. 4 and the measurement uncertainty calculation of results for an example thermal camera are shown Sec. 5. Finally, basic conclusions of the paper are presented in Sec. 6.

Downloaded from SPIE Digital Library on 18 Jan 2012 to 130.89.66.195. Terms of Use: http://spiedl.org/terms

Chrzanowski, Fischer, and Matyszkiel: Testing and evaluation . . . Table 1 Thermal camera parameters related to temperature measurement accuracy. No. 1 Parameter Accuracy Exemplary Value 1% of the measured object temperature but not less than 1C Comments This typically ranges around the true object temperature in which the output temperature is located when the measurements are carried out in conditions identical with the conditions of calibration of the thermal camera. All these terms are used as equivalent of the NETD.

Thermal sensitivity Thermal resolution Temperature resolution NETD MDTD

0.1C at 30C

0.05C at 30C

MRTD

1.2C at 0.5 cycles/mrad

Spatial resolution Geometrical resolution Instantaneous eld of view (IFOV) Image resolution (IR lines/frame)

1.8 mrad

2 mrad

The thermal resolution of the system as perceived by the human eye for large targets is called MDT. The MRTD is a function of both the spatial frequency of the bar pattern and the ambient temperature of the background. It also takes into account the visual acuity of the observer and the quality and settings of the display device. It is usually measured as angular dimension of the slit for which the slit response function (SRF) equals 0.5. Angular dimension of a single detector or angular dimension of an element of a matrix of detectors

640 horizontal, 480 vertical 300

2 Review of Present-Day Situation Information about thermal measurement cameras can be presented in the form of large volume texts with descriptions of principle of system work, diagrams, photos, system parameters, features, etc. However, the technical specications understood as parameters and features represent the most consistent form of this information and are typically used in literature to describe capabilities of thermal cameras. Manufacturers of thermal measurement cameras use sets of numerous parameters and features in the descriptions of their products. These sets differ from one case to another and technical specications of even quite similar systems manufactured by different manufacturers can differ signicantly. From many parameters used by manufacturers of thermal measurement cameras, we present and discuss only the parameters of a thermal camera treated as a black box not design parameters of blocks of the imager that give some information about the temperature measurement accuracy achieved with the camera. Table 1 lists these parameters with exemplary values and additional comments. The rst parameter in this list, called accuracy, is usually measured as maximal error of temperature measurement of a blackbody under calibration conditions if the temperature of the blackbody varies within the measurement range of the camera. The accuracy provides information about the best measurement capability of the camera that can be achieved under calibration conditions. However, for the reasons presented in the following, real measurement conditions can differ signicantly from the

calibration conditions and real uncertainty can differ quite signicantly from the accuracy suggested by the manufacturers. First, modern thermal cameras equipped with internal digital processors are able to average individual picture elements from several elds or to utilize area weight averaging. Averaging reduces the random noise content of the thermal image by a factor equal to the square root of the number of elds averaged. This effect is easily noticeable as a reduction in snow and smoothing of color patterns. This means that using averaging, it is possible to improve camera sensitivity by reducing the inuence of noise generated in the electrical analog channel of the camera. The averaging effect is typically used during calibration of the camera and also during measurement of the accuracy parameter. However, this effect also reduces measurement speed and cannot be used for dynamic images. Second, the results of measurements with modern thermal cameras are always digitized; however, the resolution of the digital channel of the thermal camera is always limited. This means that a certain difference exists between two temperature levels that cannot be distinguished because of the limited resolution of camera digital channel. The value of this difference depends not only on bit number of the analog to digital A/D converter used by the camera but also on temperature span of the thermal camera used during measurements. The accuracy parameter is usually measured for small temperature spans such as 1 or 2C. However, it is sometimes necessary to carry out measurements for temperature spans as high as 1000C. Therefore
Optical Engineering, Vol. 39 No. 9, September 2000 2537

Downloaded from SPIE Digital Library on 18 Jan 2012 to 130.89.66.195. Terms of Use: http://spiedl.org/terms

Chrzanowski, Fischer, and Matyszkiel: Testing and evaluation . . .

the value of the accuracy does not provide information about possible temperature measurement errors due to the limited resolution of camera digital channel. Third, the accuracy parameter is typically measured under laboratory conditions when environment temperature is close to 20C. During real measurements, however, the temperature of the environment can vary signicantly within wide limits determined by the manufacturer like from 10 to 40C. Changes of environmental temperature can have a signicant effect on measurement results for several reasons. First, radiation emitted by the optical elements of the camera depends directly on temperature of these elements and indirectly on the environmental temperature. Second, variation of the environmental temperature can cause variation of the temperature of the detector and change detector sensitivity. Third, changes of the environmental temperature directly cause changes of temperature of the electronic blocks, but they also indirectly cause changes of the gain and offset of these blocks. The inuence of the environmental temperature on measurement results can be corrected. Modern thermal cameras are equipped with software and hardware mechanisms that should automatically correct this inuence. However, only a partial correction of the inuence of the environmental temperature on measurement results with thermal cameras is practically possible and changes of ambient temperature always cause changes of measurement results. Therefore measurement uncertainties carried out at temperatures that differ signicantly from laboratory temperatures can differ signicantly from values suggested by the accuracy. Fourth, it is commonly known that readings of any measuring instrument vary if measurements are repeated even under identical conditions. Thermal cameras are not an exception to this rule. Variations of readings of new thermal cameras are usually negligible. However, these variations can become quite signicant during measurements carried out with older thermal cameras. As shown, the value of the accuracy parameter does not give information about errors caused by noise in the analog channel, limited resolution of the A/D converter, variations of the environmental temperature and limited repeatability of the camera measurements. Because the accuracy is measured using a large blackbody as a tested object it also provides no information about possible additional errors when objects of small size are being measured. However, other parameters presented in thermal camera catalogs give some indications concerning these factors. First, a group of parameters called thermal sensitivity, thermal resolution, temperature resolution or NETD provide information concerning the inuence of noise in electrical channel on measurement errors. The parameters have different names but they are usually all measured as NETD, dened as the blackbody temperature difference between a target and its background required to produce a peak-signal-to-rms root mean square -noise ratio of unity at a suitable point in the output electrical channel.6 It was shown in Ref. 19 that NETD equals the standard deviation of the output temperature dispersion caused by the system noise. Therefore, the NETD can be treated as a good estimation of uncertainties due to system noise. We must, however, remember that the NETD depends on object temperature. It is typically measured only
2538 Optical Engineering, Vol. 39 No. 9, September 2000

for one xed value of this temperature usually close to 30C and it is necessary to correct the NETD value when the temperature of the tested object differs from the value used during NETD measurement. Thermal camera manufacturers also present in the specications of their products two other parameters related to the NETD. The MRTD is a function of a minimum temperature difference between bars of the standard four-bar target and the background required to resolve the thermal image of the bars by an observer versus spatial frequency of the target. The MDTD is a function of a minimum temperature difference between a single circular target and the background required by an observer to detect the thermal image of the target versus inverse spatial dimension of the target. Although both MDTD and MRTD are functions, they are often presented as single value parameters. MDTD is typically measured for targets of large size, and it equals 50 to 70% of NETD. For MRTD, it is difcult to formulate a similar rule as it is measured for targets of different spatial frequency. Generally, the MRTD is the most important measure of the ability of a thermal camera to detect and identify a target and is an excellent tool for evaluating observation thermal cameras. However, its use to evaluate measurement thermal cameras for absolute temperature measurement is problematic. It gives some indications about system temperature resolution and about system ability to measure small size objects. However, it is impossible to connect the MRTD and the uncertainty of the thermal camera. The inuence of the size of the tested object on measurement results should apparently be easily determined based on other parameters in the list, such as IFOV, spatial resolution or image resolution. In reality, however, it is typically difcult or impossible for the following reasons. The IFOV is dened as the angular dimension of a single detector or the angular dimension of an element of a matrix of detectors. It is related to the minimum angular size of the tested object for which the inuence of the size of the tested object on measurement results is still negligible. However, this minimum size also depends on the parameters of other blocks of the thermal camera such as aberration of the optical block, diffraction effects, and frequency bandwidth of the electrical channel and it is not possible to determine this minimum size based on the IFOV. The spatial resolution or the geometrical resolution is usually measured as angular slit dimension for which the SRF of the tested thermal camera is equal to 0.5. The SRF is dened as a function of signal generated by a slit versus the width of the slit normalized to the signal generated by a very wide slit. Values of the spatial resolution dened in this way can usually be found in scanning thermal camera catalogs. The spatial resolution dened in the way already presented is a good measure of the cameras ability to create a thermal image of the tested object and is sometimes called the observation spatial resolution. However, this parameter does not provide information as to whether the size of the tested object is high enough to ensure negligible inuence of this size on the measurement results. This information is provided by the measurement spatial resolution dened as angular slit dimension for which the SRF of the tested thermal camera is equal to 0.99. When the angu-

Downloaded from SPIE Digital Library on 18 Jan 2012 to 130.89.66.195. Terms of Use: http://spiedl.org/terms

Chrzanowski, Fischer, and Matyszkiel: Testing and evaluation . . .

lar size of the tested object is higher than the measurement spatial resolution, we can assume that the inuence of the size of this object on temperature measurement results is negligible. The measurement spatial resolution, however, is usually a few times higher than the observation spatial resolution, and manufacturers prefer to present only the values of the rst parameter. Image resolution, presented as number of pixels or number of lines per frame, is a good measure of the quality of thermal image of the tested object. It is related to both the previously dened observation and measurement spatial resolution. However, it is impossible to determine the exact value of the measurement spatial resolution based only on the known image resolution. To summarize our discussion we can say that based on the typical parameters of thermal cameras provided by their manufacturers, it is impossible to determine the temperature measurement uncertainty due to errors generated within the camera. 3 Testing of Thermal Cameras for Absolute Temperature Measurement To evaluate thermal cameras in applications requiring absolute temperature measurements it is necessary to determine the uncertainty of temperature measurement due only to internal errors generated within the camera. Such a situation, where the external errors are negligible, exists during the temperature measurement of a blackbody for a short distance between the object and the camera. In contrast to the presented conditions for measurement of the so called accuracy, however, this is a case of dynamic measurement carried out for large temperature spans, for different temperatures of the environment, repeated in long time intervals and for objects of different size. The value of the uncertainty of measurement due to the internal errors of the camera would be a good objective measure for comparisons of different thermal cameras. Let us now nd a set of parameters of a thermal camera treated as a black box that would enable us to determine this uncertainty. When we analyze the measurement process inside a thermal camera, we nd that there are many possible sources of internal errors. These errors can be caused by changes of radiation emitted by optical elements or changes of their transmission, limited accuracy of internal blackbodies, detector noise and nonlinearity, limited stability of the cooling system, variation of the preamplier gain and offset, limited resolution and limited linearity of the A/D converters, limited accuracy of calibration chart etc. Generally these errors cannot be modeled theoretically; they can be determined experimentally. Due to the sophisticated equipment required and the detailed knowledge about design of the camera needed, these errors can be determined only by manufacturers; they cannot be determined by camera users. However, instead of analyzing separately all blocks of camera let us treat the camera as a unity. Then we can propose parameters that could provide information about all of these internal errors and at the same time the parameters could be easily measured. There is no simple experimental way to determine the errors of thermal cameras due to internal sources by measurement of a single parameter. However, it seems that these errors can be reasonably well represented by a set of

seven parameters: minimum error ME , noise generated error NGE , digital temperature resolution DTR , temperature stability TS , repeatability RE , measurement uniformity MU , and measurement spatial resolution MSR . The ME is dened as a range around the output temperature T out in which the true temperature T ob is located when the measurements are carried out in conditions identical with the conditions during calibration of the thermal camera. The calibration conditions are dened as 1. The tested object is a sufciently large blackbody. 2. The distance between the tested object and the thermal camera is short so that the inuence of limited transmittance of the atmosphere is negligible. 3. The temperature of the environment is in the typical laboratory range 20 to 30C. 4. The object is located in the center of the system eld of view. 5. Measurements are carried out for the shortest temperature span of the thermal camera. 6. The effect of averaging a dozen or more measurement results is used. If we analyze these conditions, we nd that they are the same as the conditions if the earlier discussed accuracy parameter is typically measured. Therefore the ME can be treated as equivalent of the accuracy parameter typically presented in thermal measurement camera catalogs. The NGE is dened as the standard deviation of the output temperature dispersion caused by noise of the system. The denition of the NGE presented here is almost the same as the original denition of this parameter shown in Ref. 19. The difference is that the original denition of the NGE refers only to the noise caused by the detector; here the NGE refers to analog noise caused by all blocks of the camera. It was shown in the same Ref. 19 that in the case of typical singleband thermal cameras, the NGE equals the NETD. Therefore the NETD values typically shown in thermal camera catalogs can be used to determine the NGE. The NGE can be also measured using any of the measurement methods described in Refs. 19 and 20 if the rst ve of the preceding six measurement conditions of the ME are fullled. The NETD is generally a function of object temperature, but it is typically presented by manufacturers as a single value parameter measured for object temperature located within the range from 20 to 30C. The NGE is also a function of the object temperature T ob . It can be calculated based on this single value NETD measured for the object temperature T m using the following formula NGE T ob NETD T ob T m L L ,T m / T sys ,T ob / T sys d , d 1

where sys is the relative spectral sensitivity of the thermal camera and L( ,T) is Plancks function. The DTR is dened as the smallest difference between two temperature levels that can be distinguished because of
Optical Engineering, Vol. 39 No. 9, September 2000 2539

Downloaded from SPIE Digital Library on 18 Jan 2012 to 130.89.66.195. Terms of Use: http://spiedl.org/terms

Chrzanowski, Fischer, and Matyszkiel: Testing and evaluation . . .

the limited resolution of the digital channel of the thermal camera. This difference depends on the temperature span being used during measurement and the bit number of the A/D converter. Because of nonlinear dependence of the output temperature T out on the output electrical signal S out , however, the DTR is also a function of temperature of the tested object T ob . It is possible to calculate the DTR as a function of object temperature when detailed data concerning design of electronic blocks is available. However, as these data are usually available only for thermal camera designers, let us assume a linear dependence of output temperature on the output electrical signal to enable estimation of the DRT using the formula DTR T span , 2k 2

low. Therefore, to shorten the TS measurement time and avoid the mentioned problems, it would be desirable to determine the temperature stability TS for any object temperature T ob based on the TS measured for only one value of T ob . This can be done using the formula TS T ob TS T ob m T obRDRF T ob m , T ob m RDRF T ob 3

where TS(T ob) is the temperature stability for the object temperature T ob , TS T ob(m) is the temperature stability measured for object temperature T ob(m) , and the RDRF is the relative disturbance resistance function of the thermal camera. The latter function is dened as the ratio of the relative error of signal measurement and the relative error of temperature measurement and can be calculated as21
RDRF dS bb T ob / dT ob T ob S bb T ob T ob
2 1

where T span is the temperature span of the camera used during measurements, and k is the bit number of the thermal camera A/D converter. The DTR is usually many times smaller than the NGE for small temperature spans of a few degrees Celsius. However, for an 8-bit converter of the older cameras and large temperature spans of hundreds of degrees, we can have a drastically inverse situation. If an accuracy of the DTR determination higher than that achieved using Eq. 2 is required, then this parameter can be measured. The measurements must be carried out under conditions where the inuence of other sources of errors, particularly the inuence of the noise of the system, on the measurement results is negligible. It is usually possible to fulll this condition using the effect of averaging measurement results and by carrying measurements for wide temperature spans of the camera such as hundreds of degrees or more. The TS is dened as a range in which the results of the measurements carried out in different environmental temperatures, within limits determined by the camera manufacturer, are located. Changes of temperature of the environment in comparison to the temperature when the camera calibration was carried out cause changes of the output electrical signal. The changes in this signal can be treated as errors of the signal measurement. Due to the nonlinear dependence of output temperature on electrical signal output, the same signal measurement error can cause different temperature measurement errors. This means that the temperature stability of the thermal camera depends on the temperature of the tested object. It is possible to measure the TS for different object temperatures T ob . To eliminate the inuence of other sources of measurement error during determination of the TS, the presented conditions 1, 2, and 4 to 6 of the ME should be fullled. Measurements of the TS are difcult and time consuming for the following reasons. First, a special large environmental chamber is required to enable measurement and it often takes many hours to signicantly change temperature within this chamber. Second, the measurements of the TS for any single temperature in the chamber should be made for many temperatures of the blackbody. Third, it is sometimes impossible to achieve a high temperature of the tested object close to 1000C if the temperature in the chamber is
2540 Optical Engineering, Vol. 39 No. 9, September 2000

sys
2 1

exp c 2 / T ob / sys /
5

T 2 exp c 2 / T ob ob 1 d

d ,

exp c 2 / T ob

where S bb(T ob) is the output electrical signal during calibration process of the thermal camera generated by the is relative specblackbody of temperature T ob , and sys tral sensitivity of the camera. Note that the unit of object temperatures in Eqs. 3 and 4 is Kelvin and not degree Celsius. The RE is dened as a range in which the results of the measurements are located when measurements are repeated under identical measurement conditions. Measurements of the repeatability should be carried out in conditions identical with the conditions during measurement of the ME. Similar to the TS, repeatability is a function of the object temperature T ob . It can be calculated for any object temperature based on the value of this parameter measured for a single temperature of the object using a modied Eq. 3 by exchanging TS with RE. In contrast to TS, however, it can also be easily measured for different temperatures of the tested object. The MU is dened as a range in which the results of the measurements are located when the tested object is located at different places within the camera eld of view. The measurements of the MU should be carried out under conditions where the rst ve conditions for the measurement of the ME are fullled. Here we have the same situation as with RE, and we can calculate the measurement uniformity for any object temperature based on the value of this parameter measured for a single temperature of the object using a modied Eq. 3 . We can also measure the MU for a few different values of the temperature T ob . The MSR is dened as the minimum angular dimension of the tested object when there is still no inuence of limited size of this object on temperature measurement results. We can say, in other words, that if the angular size of the tested object varies but is always higher than MSR, then the output temperature will be the same. The MSR is related to the IFOV of the single detector or the IFOV of the element of the linear or staring matrix, to the blur of the imaging optics, to the spacing between scanning lines or spacing

Downloaded from SPIE Digital Library on 18 Jan 2012 to 130.89.66.195. Terms of Use: http://spiedl.org/terms

Chrzanowski, Fischer, and Matyszkiel: Testing and evaluation . . . Table 2 Thermal camera parameters related to the uncertainty of temperature measurement. Parameter ME NGE 1% T out or Value 1C

0.2C at 30C for f R 25 Hz 0.1C at 30C for f R 6.25 Hz 0.05C at 30C for f R 1.56 Hz 0.0078C for temperature span 2C 0.019C for temperature span 5C 0.039C for temperature span 10C 0.078C for temperature span 20C 0.39C for temperature span 50C 1.95C for temperature span 100C 3.9C for temperature span 500C 19.5C for temperature span 1000C Bit number of the A/D converter 8

DTR

Fig. 1 Images of a rectangular object on a 2D matrix of detectors for two different locations of this object (in case of an ideal optics).

between elements of the linear matrix. In the case of single detector cameras or linear matrix cameras, the MSR does not depend on the location of the tested object in the horizontal direction, but it depends on it in the vertical direction. The MSR depends on the location of the tested object in both horizontal and vertical directions in the case of cameras built using area matrix detectors. The reasons for the dependence of the MSR on location of the tested object are presented graphically in Fig. 1. It is proposed to consider the value measured for the worst case as the MSR of the tested thermal camera. 4 Evaluation of Thermal Cameras for Absolute Temperature Measurement The presented parameters of the thermal camera treated as a black box should enable determination of measurement uncertainty if the inuence of external factors is negligible or if the inuence of these external factors is known. External factors can be considered as negligible, for example, during short distance measurements of a blackbody. The inuence of the same factors on the output electrical signal can be considered as known and fully corrected if the effective emissivity of the tested object, the effective temperature of the background of the tested object, and the effective transmittance of the atmosphere are known. Uncertainty as a measure of accuracy of this measurement could be an objective criterion for comparisons of different thermal cameras. Uncertainty is a quantiable attribute of the quality of a result of a measurement. It is a parameter, associated with the result of a measurement, that characterizes the dispersion of values that could be reasonably attributed to the measurand.1315 When reporting the result of a measurement of a physical quantity, it is obligatory that some indications of the quality of the results be given so that those who use it can assess its reliability. Without such an indication, measurement results cannot be compared. The total uncertainty of the result is determined as an estimated standard deviation of the dispersion of values that could reasonably be attributed to the measurand, which, in our case, is object temperature. Let us now determine the combined standard temperature measurement uncertainty with thermal cameras due only to internal sources of errors based on the presented set

TS

0.3C for T E (20 to 30C) 0.9C for T E (10 to 35C) 1.95C for T E (0 to 40C) Measurements for T out 100C

RE MU MSR

0.2% T out or

0.2C

0.1% T out or 0.05C 3 mrad in the horizontal direction 3.2 mrad in the vertical direction Values for 7 deg eld of view

of parameters: ME, NGE, DTR, TS, RE, MU and MSR. These parameters of thermal cameras provide direct information about output temperature uncertainties due to different internal sources of uncertainties. Let us analyze the case of a thermal camera with the parameters shown in Table 2. Note that the results shown in Table 2 are not the results of measurement of a real thermal camera. The parameters presented in this table should be treated as an example chosen to enable analysis of a case close to real systems. The parameters can be considered as typical for older measurement thermal cameras built using a single IR detector and scanning technology. For modern cameras built using a 2D matrix of detectors and A/D converters of higher bit numbers, we can expect better values for the NGE, the DTR, and the MSR, but at the same time much worse data for the ME, the TS, the RE, and the MU. Before we begin to calculate the standard uncertainty of the measurement results with the thermal camera of parameters shown in Table 2 we discuss the way the ME, RE and MU are presented in this table. These parameters provide information concerning the limits within which the T errors difference between temperature of the blackbody and the output temperature are located. The T error parameters depend on the output temperature T out , and this dependency can be quite sophisticated. If we measure the T errors for any thermal cameras, we nd that in spite of some variations there is a general trend that the values of T rise with the output temperature or temperature of the tested blackbody . This effect is described mathematically in the form of the RDRF function Eq. 4 . It can be also
Optical Engineering, Vol. 39 No. 9, September 2000 2541

Downloaded from SPIE Digital Library on 18 Jan 2012 to 130.89.66.195. Terms of Use: http://spiedl.org/terms

Chrzanowski, Fischer, and Matyszkiel: Testing and evaluation . . .

in which the true temperature T ob is located. This means that we know the bounds of probability distribution of the quantity T ob but there is no specic knowledge about posT out sible values of T ob within the range 0.5ME, T out 0.5ME . To enable us to calculate the standard uncertainty of the output temperature T out , we must make an assumption about the type of distribution of the quantity T ob . Let us assume the uniform distribution of the quantity T ob within the mentioned limits. This assumption is commonly made in situations where there is no specic knowledge about possible values of the measurand within a certain range.14,15 Then the standard uncertainty u ME of output temperature T out during measurements carried out in calibration conditions can be calculated as
Fig. 2 Results of measurements of the ME for an example thermal camera.

u ME described physically as presented in the following. The sources of errors within the thermal camera before the amplier inuence the output electrical signal S and generate an error of signal measurement S, which later generates errors of temperature measurement T. The S error is proportional to the value of the signal S. However, the signal S is not proportional to the temperature of the measured blackbody. It rises more quickly almost T 4 , as suggested by the Boltzmann law than that temperature, and that causes a signicant increase of the T errors with the temperature of the tested object. Measurement of all the mentioned parameters, and particularly of the TS, requires sophisticated, accurate laboratory equipment. We plan to carry out measurement of all proposed parameters of a few thermal cameras of different type in near future at the Physikalisch-Technische Bundesanstalt in Berlin, where the necessary setup will be prepared. At present, however, we cannot provide the results of thorough testing. Therefore the exemplary values presented in Fig. 2 and Table 2 can be considered as values of a ctitious measurement that should be reasonably close to the values of real measurement in our opinion based on some experience with testing of measurement thermal cameras. The results of ctitious measurements of the ME of an example thermal camera are shown in Fig. 2, where T is the difference between output temperature T out and true temperature T ob of the blackbody used during measurements. As we can see, it is possible to determine different approximation curves that could dene a range around the output temperature in which the true object temperature is located. These curves determine values of the ME for different temperatures T ob . The approximation curve in the analyzed case that equals 1% of the output temperature T out or 1C was chosen among many other functions because the manufactures typically use this way of presenting the accuracy and repeatability parameters. We must, however, remember that the ME is dened as a range, and even if it is presented as, for example, 1C, the total span is actually 2C and ME equals 2C. The ME in the form shown in Table 2 provides information about the range around the output temperature T out
2542 Optical Engineering, Vol. 39 No. 9, September 2000

ME 12

The standard deviation of output temperature dispersion due to detector noise can be calculated when the camera NGE and the type of distribution of the output temperature are known. The NGE can be measured or calculated using any of the presented methods. The type of the noise distribution can usually be assumed with reasonably accuracy. The central limit theorem of statistics as applied to thermal imaging system states that the noise distribution generated by the system tends toward a Gaussian shape independently of type of distribution of the noise generated by components of the system. Therefore, the normal distribution of the output temperature dispersion due to the noise in the electrical channel can usually be assumed and the standard uncertainty u NGE of output temperature equals the NGE u NGE NGE. 6

The standard uncertainty of the output temperature distribution due to limited DTR can be easily determined when the latter parameter is known by making an assumption about uniform distribution of the true temperature T ob within limits determined by camera digital temperature resolution T out 0.5DTR, T out 0.5DTR . The assumption about the uniform distribution is here fully physically realistic in contrast to the earlier analyzed case. Then the standards uncertainty u DTR of the output temperature caused by limited digital temperature resolutions equals DTR 12

u DTR

Values of the TS, the RE and the MU also provide information concerning the bounds of probability distribution of the quantity T ob but no information about the type of probability distribution of this quantity. Therefore, similarly to the case of the ME, let us assume the uniform distribution and then the standard uncertainty of the output temperature due to limited temperature stability, limited repeatability and limited measurement uniformity can be calculated as

Downloaded from SPIE Digital Library on 18 Jan 2012 to 130.89.66.195. Terms of Use: http://spiedl.org/terms

Chrzanowski, Fischer, and Matyszkiel: Testing and evaluation . . .

u TS

TS 12 RE 12

and different measurement speed. Therefore let us assume for the purpose of the calculations the following measurement conditions: 1. angular size of the object: 15 mrad 2. spectral band of the thermal camera: 8 to 12 m 3. temperature of the environment within the range 10 to 35C 4. measurement frequency: 25 Hz 5. temperature span of the camera: 50C Now let us calculate the combined standard uncertainty u int during these measurements for three different output temperatures: 0, 500, and 1000C. The calculations are carried out in a few stages. First, the values of ME, NGE, DTR, TS, RE, MU and MSR for these three output temperatures are calculated based on the values shown in Table 2. DTR and MSR do not depend on the output temperature and suitable values can be taken directly from Table 2. ME, RE and MU depend on the output temperature according to very simple functions shown in Table 2. The calculations are more complicated in case of NGE and TS, which depend on the output temperature in a more complicated way. The values of NGE for output temperatures 0, 500, and 1000C can be calculated using Eq. 1 . One component of this formula is the relative spectral sensitivity sys of the thermal camera. There can be some differences between sys of different cameras of the 8 to 12 m spectral band. To simplify the analysis, however, the case of sys being 1 within 8 to 12 m and 0 outside this range was assumed during the calculations. The values of TS for the output temperatures 0, 500, and 1000C can be calculated using Eqs. 3 and 4 . Next, the partial uncertainties u ME , u NGE , u DTR , u TS , u RE , and u MU can be calculated using Eqs. 5 10 . Finally, the combined standard uncertainty u int for the mentioned values of the output temperatures can be calculated using Eq. 11 . The calculation results are presented in Table 3. The value of the combined standard uncertainties u int could be a good objective criterion for comparisons between different thermal cameras. 6 Conclusions We proposed a set of parameters for characterization of measurement thermal cameras. It was shown that if these parameters are known, then it is possible to determine the uncertainty of temperature measurement due to internal errors of these cameras. Values of this uncertainty can be used as an objective criterion for comparisons of different thermal cameras used in applications requiring absolute temperature measurement. International metrological organizations currently recommend characterization of measuring systems by their uncertainty. The proposed set of parameters fullls these recommendations as it enables characterization of the thermal camera by the uncertainty of temperature measurement due to internal errors of the system. However, further research in this area is required to precisely dene measurement methods for the proposed parameters. Next, to fully evaluate the proposed method of characterization of thermal
Optical Engineering, Vol. 39 No. 9, September 2000 2543

u RE

u MU

MU 12

10

Equations 5 10 enable us to calculate the partial standard uncertainties u ME , u NGE , u DTR , u TS , u RE and u MU of the output temperature T out due to different sources of internal errors. The combined standard uncertainty u int of the output temperature T out due to all internal errors of the camera depends on values of these partial uncertainties but also on the degree of correlation between probability distributions represented by these partial uncertainties. It is theoretically possible to experimentally determine correlation coefcients between the mentioned probability distributions. However, the experiments would be very costly and time consuming due to the required resources and the complicated measurement procedure. At the same time, it seems that only an insignicant correlation exists between these probability distributions. Therefore let us assume here a case of uncorrelated distributions and then the combined standard uncertainty u int can be calculated as a positive square root of the sum of the squares of partial standard uncertainties u ME , u NGE , u DTR , u TS , u RE , and u MU u init
2 2 2 2 2 2 u ME u NGE u DTR u TS u RE u MU 1/2

11

If we analyze Eq. 11 , we easily nd that the uncertainty of output temperature T out due to the limited MSR of the thermal camera is not represented in this formula. This means that it was assumed that this partial uncertainty equals zero. This seems surprising as the MSR is one of the most important parameters of thermal measurement cameras and it is clear that the output temperature depends on the size of the tested object. The assumption about negligible uncertainty of the output temperature T out due to the limited MSR of the thermal camera was made due to two reasons. First, the angular size of the tested objects is almost always higher than the MSR, and then this uncertainty equals zero. Second, it is possible to develop a mathematical formula to calculate the uncertainty of the output temperature T out due to the limited MSR of the thermal camera if the angular size of the object is smaller than the MSR. It would be quite difcult, however, to develop such a formula without knowledge of the design parameters of the camera, particularly in the case of thermal cameras built using a 2D matrix of detectors. 5 Calculations A thermal camera with the parameters shown in Table 2 can measure objects of different temperatures and of different sizes, can work in different ranges of environment temperature T E , and can be set for different temperature spans

Downloaded from SPIE Digital Library on 18 Jan 2012 to 130.89.66.195. Terms of Use: http://spiedl.org/terms

Chrzanowski, Fischer, and Matyszkiel: Testing and evaluation . . . Table 3 Calculation results.

T out 0C
ME 2C NGE 0.29C DTR 0.39C TS 1C RE 0.4C MU 0.1C MSR 3 mrad

T out 500C u ME 0.58C u NGE 0.29C u DTR 0.11C u TS 0.29C u RE 0.11C u MU 0.03C u MSR 0
ME 10C NGE 0.047C DTR 0.39C TS 6.6C RE 2C MU 0.5C MSR 3 mrad

T out 1000C
ME 20C NGE 0.038C DTR 0.39C TS 14.5C RE 4C MU 1C MSR 3 mrad

u ME 2.9C u NGE 0.047C u DTR 0.11C u TS 1.9C u RE 0.58C u MU 0.14C u MSR 0 u int 3.51C

u ME 5.8C u NGE 0.038C u DTR 0.11C u TS 4.2C u RE 1.16C u MU 0.29C u MSR 0 u int 7.26C

u int 0.73C

measurement cameras, it is necessary to carry out measurements of the proposed parameters for quite a few different systems.

Acknowledgments
Funding in support of this research was provided by the State Committee for Scientic Research of Republic of Poland Program No. 8 T10C 018 16.

19. K. Chrzanowski and M. Szulim, A measure of inuence of detector noise on temperature measurement accuracy with IR systems, Appl. Opt. 37, 50515057 1998 . 20. K. Chrzanowski, Z. Bielecki, and M. Szulim, Comparison of temperature resolution of singleband, dualband and multiband infrared systems, Appl. Opt. 38 13 , 28202823 1999 . 21. K. Chrzanowski, Non-contact temperature measurement measurement errors, Research and Development Treaties, Vol. 7, SPIE Poland chapter, Warsaw 2000 . Krzysztof Chrzanowski received his MS degree in 1986, his PhD degree in 1991 and his DSc in 1997 from the Military Academy of Technology in Warsaw, where he is currently an assistant professor with the Institute of Optoelectronics. He has contributed to more than 50 publications and has taken part in over 10 scientic projects. His research interests include design, simulation, applications, testing of optoelectronic systems and uncertainty analysis of measurement processes. He is a member of SPIE. Joachim Fischer attended the University of Stuttgart, Germany, and received his MS degree in physics in 1981 and his PhD degree in physics in 1985 form the Technical University of Berlin, Germany. He has since been with the PhysikalischTechnische Bundesanstalt (PTB) in Berlin. He was rst with the laboratory of synchrotron radiation at the electron storage ring BESSY and in 1986 he joined the PTB temperature radiation section. He is responsible for realization and dissemination of the international temperature scale using optical methods and for the PTB scale of spectral radiance. His work has been devoted to radiation thermometry through the development of absolute cryogenic radiometry. He was one of the rst to apply these systems for the determination of thermodynamic temperatures using advanced blackbody sources. His other major projects include remote sensing and thermography. Robert Matyszkiel received his MSc in electronics from the Military University of Technology in Warsaw, Poland, in 1995, and has since been with the Military Communication Institute in Zegrze, Department of Management of Communication Networks. His main interests are thermovision, military IR systems, metrology and the management of communication networks.

References
1. M. C. Dudzik, Ed., Infrared imaging system testing, Chap. 4 in The Infrared & Electro-Optical Systems Handbook, Vol. 4, ElectroOptical Systems Design, Analysis and Testing, SPIE Optical Engineering Press, Bellingham, WA 1993 . 2. G. C. Holst, Testing and Evaluation of Infrared Imaging Systems, JCD Publishing 1993 . 3. J. M. Lloyd, Thermal Imaging System, Plenum Press, New York 1975 . 4. J. A. Ratches, Static performance model for thermal imaging systems, Opt. Eng. 15, 525530 1976 . 5. J. A. Ratches, W. R. Lawson, L. P. Obert, and J. M. Swenson, Night Vision Laboratory static performance model for thermal imaging systems, USA Electronics Command Report 7043, AD/A 011212 Apr. 1975 . 6. Thermal imaging devices, performance parameters of, MIL-STD1859 1989 . 7. Infrared thermal imaging survey procedure for electrical equipment, MIL-STB-2194 1992 . 8. Thermal sight TS-3681/VSG, Military standard MIL-T-49381 Test Set, USAERADCOM 1980 . 9. Infrared thermal imaging systems, Military standard MIL-I24698 SH , Department of Defense 1988 . 10. Standard test method for minimum resolvable temperature difference for thermal imaging systems, ASTM standard E 1213-92 1992 . 11. Standard test method for noise equivalent temperature difference of thermal imaging systems, ASTM standard E 154313-94 1994 . 12. Standard test method for minimum detectable temperature difference for thermal imaging systems, ASTM standard E 1311-89 1989 . 13. International Vocabulary of Basic and General Terms in Metrology, International Organisation for Standardisation 1993 . 14. Guide to the Expression of Uncertainty in Measurement, International Organisation for Standardisation-International Electrotechnical Commission-International Organisation of Legal MetrologyInternational Bureau of Weights and Measures, TAG 4/WG 3 1993 . 15. Guideline for evaluating and expressing the uncertainty of NIST measurement results, NIST Technical Note 1297 1994 . 16. Temperature measurement in industry, Part 4.1 Specication for radiation thermometers draft printing , VDI/VDE Guideline 3511, Beuth Verlag, Berlin 1999 . 17. K. Chrzanowski, Comparison of shortwave and longwave measuring thermal imaging systems, Appl. Opt. 34, 28882897 1995 . 18. K. Chrzanowski, Experimental verication of theory of inuence from measurement conditions and system parameters on temperature measurement accuracy with IR systems, Appl. Opt. 35, 35403547 1996 .

2544 Optical Engineering, Vol. 39 No. 9, September 2000

Downloaded from SPIE Digital Library on 18 Jan 2012 to 130.89.66.195. Terms of Use: http://spiedl.org/terms