INFRARED THERMOGRAPHY BY MODERN SAMPLING METHODS N. Ratner Dept. Electrical Engineering Technion - Israel Inst.

Technology Haifa 32000, ISRAEL

ABSTRACT Infrared (IR) thermography is an imaging modality that is used to measure the temperature distribution of a body from its IR radiance. As any other imaging modality, IR imaging is prone to measurement errors. We argue that an effective way of overcoming the imaging noise in the temperature prole reconstruction is to constrain the reconstructed temperature prole to lie in the sub-space of feasible solutions of the heatconduction equation of the imaged scene. In this work, we present a novel scheme for the robust reconstruction of a temperature prole of a body from its IR radiation samples. The robustness of the algorithm stems from constraining the satisfaction of the heat equation. Our algorithm is strongly based on recent developments in sampling theory that concern the treatment of non-linear processes that the input signal undergoes before sampling. Index Terms infrared, thermography, generalized sampling 1. INTRODUCTION Infrared (IR) thermography is an imaging modality that is used to measure the temperature distribution of a body from its IR radiance [1, 2, 3]. It is useful for a variety of applications e. g. integrity checks of semiconductor devices [4, 5, 6, 7, 8, 9, 10], medical diagnostics [11, 12, 13], mechanical and aerodynamics design [14] etc. As any other imaging modality, IR imaging is prone to measurement errors. These errors are the result of noise in the sensors electronics, irradiance from the imaging device itself and the stochastic nature of electromagnetic radiation [15]. Various methods of image post-processing were developed to reduce the noise. Among these methods are general methods for treatment of noise in images. Namely, linear and non-linear space invariant ltering (e. g. gaussian smoothing or median ltering) [16], as well as more advanced context dependent ltering (e. g. anisotropic diffusion [17] ,principal component analysis [18] and trend removal [19]). Another class of denoising methods is more adapted to IR images and is commonly used in thermography. Ref. [19] reviews the main methods. Thermal tomography [20] produces an image of thermal contrast which has the same dimensionality as the source IR sequence. In Pulsed phase thermography [21] the inspected image sequence is produced by measuring the response of the imaged specimen to a varying sinusoidal heat pulses. Our approach further adapts the denoising process to the imaged scene by constraining the denoised information to be feasible. Namely, IR radiation, which is related to heat, should be interpreted as a result of a feasible solution of the heat-conduction equation of the imaged scene. Hence, the primary task of our work will be to reconstruct this solution. In this work, we propose a novel method to reconstruct the continuous temperature prole of a given body. This body is imaged by an IR sensor, thus, as an input, we are given its discrete pixeled radiance samples. For our method, we make use of a recent advance in sampling theory [22]. Ref. [22] presented a new reconstruction scheme for a continuous function from the generalized samples of a non-linear transformation of this function. Ref. [22] achieved perfect reconstruction using the assumption that the continuous function is embedded in some subspace. 2. THEORY OF NONLINEAR AND NON-IDEAL SAMPLING For this paper to be self contained, we briey repeat the analysis and derivations presented in [22]. 2.1. Problem formulation The problem faced by [22] is to reconstruct a continuoustime signal y(t) from samples of x(t), which is obtained by a nonlinear, memoryless and invertible mapping M of y, i. e. , x = M (y). The measurements are the generalized samples of x, with the nth sample given by [n] = sn , xL2 = sn (t)x(t)dt , (1)

where, sn is the nth sampling function. Here, S L2 is the sampling space, which is the closure of span{sn } and S is the set transformation corresponding to {sn }. With this notation,

the generalized samples [n] can be written as = Sx . (2)

2.2.2. Iterating Oblique Projections The results for the linear case lead to an iterative recovery algorithm in the presence of nonlinearities. The true input y is a function consistent with the measured samples , i. e. , it satises = S M (y) (7) To recover y eq. (7) is rst linearized. By starting with some initial guess y0 A and approximating the memoryless nonlinear operator M using its Frechet derivative at y0
M (y) = M (y0 ) + M0 (y y0 )

If there are no constraints on the structure of y, the problem of reconstructing this input from the samples is illposed. Namely, there are innitely many functions in L2 that can yield the known samples, as any signal of the form M (xs + v) is a possible candidate. Here, v is any vector in S and xs = PS x = S(S S) (3) is the orthogonal projection of x onto the sampling space. The analysis in [22] treats the problem of reconstructing y which is known to lie in an arbitrary closed subspace A L2 . This analysis states that to recover y from its samples, one needs to determine a bijection between this function and the sample sequence. Unfortunately, the restriction to a closed linear subspace A of L2 is not a sufcient condition for the existence of such a bijection. Ref. [22] shows that if M (A) S = L2 (4)

(8)

Here, for brevity, the notation Mn = Myn is used. Rewriting eq. (7) using (8) yields S M0 y = S [M (y0 ) + M0 y0 ] .

(9)

Eq. (9) is a linear problem, hence, its solution is of the form given in eq. (6).
y = (M0 )1 PM0 (A),S (xs [M (y0 ) M0 y0 ]) = y0 (M0 )1 PM0 (A),S (M (y0 ) xs )

is satised, and under proper assumptions on the nonlinear distortion M , there is a unique function y within A, which is consistent with the measured samples. These results lead to a theory and a concrete iterative method for obtaining the perfect reconstruction of a signal in a subspace, despite the fact that it is measured through a nonlinear and non-ideal acquisition device. 2.2. Reconstruction algorithm As mentioned in Sec. 2.1, one wishes to nd a function y A which is consistent with the measured samples , i. e. {y A| = S M (y)}. This is a set of non-linear equations. Ref. [22] relies on local linearization to solve it. 2.2.1. Perfect Reconstruction for Linear Schemes Assume a linear relation between y and x, namely x = M (y) = Ry , (5)

(10)

The process can now be repeated, using y1 = y as the new approximation point. Hence, an iterative scheme of the reconstruction algorithm can be written as yl+1 = yl (Ml )1 PMl (A),S (M (yl ) xs ) = yl+1 = yl (Ml )1 PMl (A),S (Ps M (yl ) xs ) (11)

Note that PMl (A),S = PMl (A),S Ps is used for the last equality. The general reconstruction algorithm, presented by [22] is: Using the current approximation yl , calculate es = PS M (yl ) xs , the error within the sampling space. Solve a linear problem of nding a function within the reconstruction space A, consistent with es when M = Ml . The solution is (Ml )1 PMl (A),S es . Update the current estimate yl using the obtained correction term. Finally, note that in practice, one needs only to update the representation coefcients of yl within A. Thus, we ref. [22] write yl = Aal , where the set transformation A corresponds to a Riesz basis for A, and al l2 are the coefcients. This results in a discrete version of the algorithm: al+1 = al (S Ml A) (S M (Aal ) ) (12)

where R is taken to be a linear, bounded and bijective mapping. Assuming eq. (4) is satised, it is known that x R(A) can be perfectly reconstructed from its samples, by oblique projecting xs of eq. (3) along S onto the reconstruction space. Since R is a bijection, obtaining y is straightforward, once x is reconstructed. y = R1 PR(A);S xs = A(S RA) c . (6)

Here, A is a set transformation corresponding to a Riesz basis for A and PR(A);S is the oblique projection operator.

where the equalities (Ml )1 PMl (A),S = (S Ml A) S , and S xs = are used.

3. BASIC THERMOGRAPHY
Radiance [Js1m2sr1 m1]

2.5

x 10

10

The law that governs the behavior of the temperature prole of a body is the differential equation of heat conduction: T = 2 T , t (13)

1.5

where is the thermal diffusivity coefcient. Further assuming bodies with a rotational symmetry, we may consider only the radial component of the heat prole. Hence, the heat equation is one dimensional, T 2T = 2 t z . (14)

0.5

0 0

500

1000 Temperature [oK]

1500

2000

Separation of variables yields the general solution to this differential equation [23]: [ (z) ( z )] t T (z, t) = e 2 C sin + D cos , (15) where C and D are constants to be found under the boundary conditions of the specic problem. Here is the separation coefcient. It stems directly from the solution method of separation of variables. It is well known, that all bodies emit electro-magnetic radiation. The intensity of this radiation depends on the temperature of the body and its emissivity. This intensity varies with the considered part of the spectrum. Assuming a black body, the relation between the temperature of this body and the intensity of the electromagnetic radiance emitted by this body is given by Plancks law [24]: I(, T ) = 2hc2 5 (e kT 1)
hc

Fig. 1. Plancks law applied to a wavelength of 4m. Temperature range is 0 to 20000 K. hit the detector. Those photons are originated by the radiating considered black body. The number of emitted photons in a given spectrum per time frame is merely the intensity of the radiation as expressed by eq. (16). Hence, the role of the non-linear measurement system M (y) is played by I(0 , T ) of eq. (16). Here, 0 is the considered wavelength. The detector measures the number of hitting photons in a narrow band about this wavelength. The IR radiation emitted by the black body is sampled by a detector. This radiation function is typically convolved with the point spread function (PSF) of the optics and integrated over the active area of the detector. We thus have the sampling space S, spanned by shifted versions of the PSF. 5. SIMULATIONS To examine the performance of the offered temperature prole reconstruction procedure, we conducted some MATLAB simulations. Our simulation setting assumed a onedimensional metal bar of length L = 9.9[m]. The temperature prole of the bar in t = 10[Sec] was T (z, t = 10) = 10
4 4995 l=0

(16)

where h is plancks constant, c is the speed of light, k is Boltzmans constant, T stands for the temperature and stands for the wavelength of the radiation. Here, I(, T ) is the spectral radiance, or energy per unit time per unit surface area per unit solid angle per unit wavelength. Plancks law is a bijection between the temperature of a black body and its radiance for a given wavelength (see g. 1). Thus, given radiation measurements of some black body in some pre-dened spectral band, one can extract the temperature of this black body. 4. TEMPERATURE PROFILE RECONSTRUCTION We now implement the signal reconstruction algorithm described in Sec. 2.2 in our thermography problem. The continuous function y, that we wish to reconstruct is the onedimensional temperature prole of a body in an arbitrary time instant T (z, t0 ). The actual quantities that we measure are the voltage values in the pixels of a detector. These values are assumed to linearly relate to the number of photons that

10 (0.01+0.002l)2

[ ( ) ( )] 100z 100z (l + 1) sin + (4996 l) cos . (17) 1 + 0.2l 1 + 0.2l Sampling the radiance was simulated by weighted spatial summation of several dense values of the radiance. The densely sampled radiance was obtained trough non-linear transformation of the temperature prole given in eq. (17), by eq. (16). The weighted summation models the process of photons owing through the PSF of the optics and integrated by pixels of nite dimensions.

2000 1800 1600 0 Tmperature [ K]

o

850 800
21 17 13 9 5 1

750 700 Tmperature [oK] 650

33 29

25

1400 1200 1000 800 600 1 2 3 4 5 6 400 0

600 550 500 450 400

37

4 6 Length [m]

10

350 0

4 6 Length [m]

10

Fig. 2. Results of the temperature prole reconstruction. The solid blue line is the ground truth, while the dotted red lines are the results of the updated reconstruction for each iteration (numbered in black). Sampling rate is 1 sample per pixel. No blur occurs in the sampling process.
Tmperature [oK]

(a) no noise
850 800 750 700
29 25 21 17 13 9 5 1

In order to examine practical aspects of using the proposed algorithm, our simulations included variations of parameters that are involved with the samplings process. The inuence of those variations on the quality of the reconstruction was studied. Varied parameters included: The resolution of the sensor. Effective width of the PSF. In our simulation, the PSF is taken to be gaussian and its width corresponds to the standard deviation. Additive sensor noise. Generally, there is a linkage between the PSF and the pixel size. Typically, the optic system is designed so that a point blurred by the PSF will approximately match the size of one pixel. Hence, throughout the simulation, we varied the size of the pixel and the PSF simultaneously. In the ideal case the size of a pixel is innitesimal and there is also no blur. In our simulation this corresponds to sampling the radiance in its nest given resolution with the PSF taken to be a function. The results of the reconstruction algorithm in this case are depicted in Fig. 2. Non-ideally. the pixel is of nite dimensions and the PSF blurs the imaged radiance. Another effect that should be considered is the additive noise. Some of this noise results from the sensors readout electronics and part of it stems directly from the stochastic nature of electromagnetic radiation [15]. Fig. 3 shows the reconstructed temperature prole for varying sizes of pixels and PSF. The results relate to both the noiseless case, and to a case when noise was added to the samples. The distribution of the noise was Poisson with a mean parameter that is 5% of the mean radiance level.

650 600 550 500 450 400 350 0

37

33

4 6 Length [m]

10

(b) poisson noise of 5% Fig. 3. Results of the temperature prole reconstruction with and without added noise. The solid blue line is the ground truth, while the dotted red lines are the results of the reconstruction process with increasing pixel and PSF width. The width is given by the black numeric labels.

6. DISCUSSION We now explore the the practical aspects of using the reconduction algorithm in light of the simulation results. We concentrate on two issues. The sensitivity of the reconstruction to sampling noise and the inuence of the sampling functions on the results. 6.1. Sensitivity to noise An inherent feature of the electro-magnetic radiation is its stochastic nature. Hence, even for an ideal (noiseless) imaging device, uctuations in the measured radiance are bound to happen. In our terminology this means that does not accu-

rately follow eq. (2). Rather, the non-ideal equation is = S [M (y) + ] , (18)

where is the random noise. In general is not constrained to be spanned by M (A). Similarly, the samples of M (y) + are not constrained to lay in S M (A). Nevertheless, might have components in M (A) and S [M (y) + ] might have components in S M (A). These components will be interpreted by the reconstruction algorithm as a legitimate component of the sampled signal. On the other hand, components of that are orthogonal to M (A) will not affect the convergence of the reconstruction algorithm and will only be expressed by a non zero (S M (Aal ) ), which is nulled by (S Ml A) . This can be seen in the temperature prole depicted in Fig. 3(b) , where in the presence of noise, the obtained prole is a legitimate solution of the heat equation (spanned by eq. (15) yet it is not the correct solution. 6.2. PSF of the Imaging Device The inuence of the PSF and sensor resolution on the reconstruction results concerns the requirement of direct sum expressed in eq. (4). As the resolution decreases the effective dimension of the sampling space S is reduced and respectively, the dimension of S increases. This effect can be clearly seen in the simulations by considering the rank of S for the decreasing sensor resolution. Further decrease is caused by the PSF. The PSF further degenerates S making it more unstable to invert. The result is the deterioration in the quality of the reconstruction that can clearly be seen in Fig. 3(a). 7. CONCLUSIONS In this work, we presented a novel scheme for the reconstruction of a continuous temperature prole of a body from its IR images. Our algorithm utilizes recent developments in sampling theory, concerning the treatment of non-linear systems that the input signal undergoes before sampling. Namely, we nd the optimal temperature prole that is constrained to be a legitimate solution of the heat conduction equation as well as consistent with its samples. We examined our approach with respect to the sampling resolution and noise levels and found it to be relatively robust. 8. REFERENCES [1] A. Armitage, An introduction to infrared thermography, Measurement + Control, vol. 31, pp. 264267, 1998. [2] T. Astarita, G. Cardone, G. M. Carlomagno, and C. Meola, A survey of infrared thermography for convective heat transfer measurements, Optics and Laser Technology, vol. 32, pp. 593610, 2000.

[3] T. Hamrelius, Accurate temperature measurement in thermography. an overview of relevant features, parameters and denitions, proc. SPIE Thermosense XIII, pp. 448457, 1467. [4] O. Breitenstein and M. Langenkamp, Lock-in contact thermography investigation of lateral electronic inhomogeneities in semiconductor devices, Sensors and Actuators A: Physical, vol. 71, pp. 4650, 1998. [5] O. Breitenstein, M. Langenkamp, F. Altmann, D. Katzer, A. Lindner, and H. Eggers, Microscopic lock-in thermography investigation of leakage sites in integrated circuits, Review of Scientic Instruments, vol. 71, pp. 41554160, 2000. [6] O. Breitenstein, J.P. Rakotoniaina, F. Altmann, J. Schulz, and G. Linse, Fault localization and functional testing of ICs by lock-in thermography, . [7] S. Huth, O. Breitenstein, A. Huber, and U. Lambert, Localization of gate oxide integrity defects in siliconmetal-oxide semiconductor structures with lock-in ir thermography, J. Appl. Phys., vol. 88, pp. 40004003, 2000. [8] S. Huth, O. Breitenstein1, U. Lambert, and A. Huber, Imaging of the lateral goi-defect distribution in silicon mos wafers with lock-in ir-thermography, Materials Science in Semiconductor Proc., vol. 4, pp. 3942, 2001. [9] J. P. Rakotoniaina, O. Breitenstein, and M. Langenkamp, Localization of weak heat sources in electronic devices using highly sensitive lock-in thermography, Materials Science and Engineering B, vol. 91, pp. 481485, 2002. [10] D. Teyssieux, D. Briand, J. Charnay, N. F. De Rooij, and B. Cretin, Dynamic and static thermal study of micromachined heaters: the advantages of visible and nearinfrared thermography compared to classical methods, J. Micromechanics and Microengineering, vol. 18, pp. 065005065014, 2008. [11] R. Oliver and D. J. Watmough, Infrared thermography and breast disease, British Medical J., vol. 1, pp. 51, 1971. [12] L. M. Van hoogmoed and J. R. Snyder, Use of infrared thermography to detect injections and palmar digital neurectomy in horses, Veterinary Journal, vol. 164, pp. 129141, 2002. [13] D. Zhang, Y. Zhu, S. Wang, H. Ma, Y. Ye, W. Fu, and W. Hu, Infrared thermoimages display of body surface temperature reaction in experimental cholecystitis, World J. Gastroenterol, vol. 8, pp. 323327, 2002.

[14] Y. Le Sant, M. Marchand, P. Millan, and J. Fontaine, An overview of infrared thermography techniques used in large wind tunnels, Aero. science and tech., vol. 6, pp. 355366, 2006. [15] A. Rogalski, infrared detectors, CRC Press, 2000. [16] A. K. Jain, Fundamentals of Digital Image Processing, Prentice-Hall, 1989. [17] G. Gilboa, N. Sochen, and Y. Y. Zeevi, Image enhancement and denoising by complex diffusion processes, IEEE PAMI, vol. 26, pp. 10201036, 2004. [18] S. Marinetti, E. Grinzato, P. G. Bison, E. Bozzi, M. Chimenti, G. Pieri, and O. Solvetti, Statistical analysis of ir thermographic sequences by PCA, Infrared Physics and Technology, vol. 46, 2004. [19] G. M. Revel and S. Rocchi, Defect detection in ceramic materials by quantitative infrared thermography, 2006. [20] X. P. V. Maldague, Theory and practice of infrared technology for nondestructive testing, 2001. [21] X. P. V. Maldague and S. Marinetti, Pulse phase infrared thermography, Journal of Applied Physics,, vol. 79, 1996. [22] T. G. Dvorkind, Y. C. Eldar, and E. Matusiak, Nonlinear and non-ideal sampling: Theory and methods, to appear in IEEE Trans. on Signal Processing. [23] S. J. Farlow, Partial Differential Equations for Scientists and Engineers, Dover Publications, 1993. [24] S. T. Thornton and A. Rex, Modern Physics for Scientists and Engineers, Thomson Learning, 2006.