leas A New Approach to Linear Filtering and Prediction Problems ASME Transactions, Volune 82, Part D (Journal of BasicRearited Jrom the Journal of Bavic Boginering for March 1960, with permission af Copyright Owner) A New Approach to Linear Filtering and Prediction Problems’ RE. KALMAN otlimore, Mi The classical filtering and prediction problem is re-examined using the Bode-Shannon representation of random processes and the "state iranstion” method of analysis of dynamic systems New results ore (d) The formulation and methods of solution of the problem apply without modifica ASME ‘Transactions, Yolune 82, Part D (Journal of Basic Engineering), 1960, pp. 35-45. ‘memory filers tion to staiionary and nonstationary statistics and to growing-memory and infntle (Q) A nonlinear diference (or diferential) equation is derived for the covariance matric of the optimal estimation error, From the solution of this equation the co (Gicients of the difference (or differential) equation of the optimal linear filler are ob: tained withou further calculations. () The filtering Problens is shown lo be the dual of she noise-free regulator problem. The new method developed here is applied to two well-Anown problems, confirming ‘ond extending earlier results The discussion is largely self-contained and proceeds from frst principles; basic concept ofthe theory of rondome processes are reviewed inthe Appendix, Introduction Aa nrorsane clase of thoretial and practical problems in ommmunieation and contel fv tatiana och probleme are) Predton of random signa (i para tion of random egnal rom nor noi (i deter of inal ‘known for (plars, noid) in Ue presence of random noe. In his ponering work, Wener (1) snowed that probleme (i) ani) lea to the wealed Wiener Hop ntgral equation; he fico gave a method (pects factorization) forthe molto of {hi ntgrl equation inthe prateeliy important special ene of ationary tiie and retool spect Mang tatensions and gepeatizatons flloned Wiener asic work, Zach and Teyuarin solved the Stemomory coe 2) Cncarently and independently of Boden Shannon (3), they also gave a simplified method [2] of solution. Booton dis fussed the nonstationary Wiener-Hopf equation [4]. These ells are now in wandard teste [5-6]. A somewhat diferent Snproach along these sin lines hax been given recently hy Darlington [7 For extensions ta sampled signal, se, 6, Franklin [8], Lees [0 Another approach based on the eigen functions of the Wiener-Hopf equation (shih applies ako to nonstationary probleme whereas the preceding methods in general {don't}, has been pinneered by Devis [10] and applied hy many ‘others, eg, Shintrot (11), Blum [12], Pogachew [13], Solodov- niko [11] Inall these works, the objective isto obtain the sperifieation of ‘linear dynamic system (Wiener filter) which aecomplises the on, separation, oF detection of random signal rnbera i brackets desiznnte References at end of paper {Of courae, fn general these tasks nay be done better hy nonlinear fitora. At preset, however. Hite or nothing i known alot how to ractienl) these nonlinear ter ‘Contrivated by the Inctrunients and. Regulators Diviso and resented atthe fstruments and Hepulatars Conference, Nareh 29- Rill. 1980 of Tae Auieas Soctrny or Stovawent ENA INE Re (act Statements und ‘opinions ndvanewd fr ppers are tn De tnderasesd ne invndun expressone of their asthore a not those Of tie Siniety. Mamgurrpt recived at ASNIIE Headguartea, Febru 24, Abit.” Paper No, 89 TAD Journal of Basit Engineering Present methods for solving the Wiener problem are subject to ‘4 number of limitations whieh seriously curtail their practical tefl: (1) The optimal filter is specified by ita impulse response, Tt isnt a simple task to synthesize the filter from such data (2) Numeriesl determination of the optimal impulse response is often quite involved and poorly muted £0 machine computa- tion, The situation gete rapidly worse with increasing compl ity of the problem (3) Important generalizations (e.g, growing-memory filters, nonstationary prediction) require new derivations, frequently of considerable difieulty to the aonspecalis. (4) The mathematics of the derivations are not transparent Fundamental assumptions and their consequences tend to be obser, "This paper intraduces «new look at this whole assemblage of| problems, sidestepping the diffeulties just mentioned. ‘The Following are the highlighte ofthe paper: (5) Optimal Euimatee and Orthogonal Projections. The Wiener problem is approached from the point of view of condi tional distributions and expectations, In this way, bate facts of ‘the Wiener theury are quickly obtained; the seope of the results land the fundamental aumptions appear clearly. Ttieeeen that All statistical caleulations and eerulte are aged on fist and second fonder averages; no other statistical data are needed. Thus df= fieulty (4) eliminated. Tie method je well known in probabi- ity theory (see pp. 75-78 and 148-135 of Doob (18) and pp. 455~ 464 of Latve [16}) but has not yet been used extensively in ene sneering. > (G) Models for Random Proceies. Following, in partiulas, Rode and Shannon {3}, arbitrary random siznals are represented {up to second order average state properties) as the output ff a linear dynamic system excited by independent or unvorre- Jated random signals (white noi”). ‘This ia standard trick in the engineering applications of the Wiener theory [2-7]. "The ppronch taken here difer fom the conventional one only in the ‘way’ in sehich linear dynamic systems are deseribed. We shall emphasize the coneepla of slate and sate transition; in other words, linear aystems will be specified by systems of first-order difference (oF diferential) equations. This point of view i march 1960 / 35tural soil who necemury in order take advantage of the implications mentioned under (3) (2) Solution of the Iiener Probe, With the stae-transition method, « single derivation cuvers a large variely of problems: sowing and infinite menvory filters, stationary and nonstationary atistirs, ele; aifienlty (3) disappears. Having guesed the “state af the estimation (i, filtering oF prediction} problem corectly, one is lod to 2 nonlinear differenie (or differential) vation for the covariance mattis of the optimal estimation error. This is vaguely analogous to the Wienet-Hopf equation Soliton of the equation for thr eowarianre matrix star at Lhe time fg when the fist observation is taken at carl later time & Ue solution of the eqtation represents thr eovarianee of the op ‘inal predietinn evtar given olmervations in the interval (t, ¢ From the eowarsanec matrix atime ¢ we obtain al oner, it further esteulations, the eveifirients (in general, time-varying) charseterising the opti lina Bite (8) The Dual Prodem..“The nose formation of the Wiener robles brings it inte euntart with the growing new theory of eontrol systems based om the "state point of vies [I7-24). Tt tens. surprisingly that te Wiener prem ie Use dual af the noise-fre optimal mgulator problem, which has ben salve Drevinasly by the author, using the state-transition ethod 10 srrat ulwantage (18, 25,24], ‘The mothematial backgrotnd of te te prabians i ientieal—this haa been speed all along, oat until now the anges have never bron made exp HP Applications. "The power nf the new method i most sp: parent in tienaetra! investigations and in numerical answers Lo ‘omplex peartval probleme. Inthe lttor ese, iis best to resort to mmachine computation. Example of this pe will be dies Inter. To provide some feel for applications, two standard ‘examples from nonstationary prediction are inehuded; in these ‘ease the sation of the nonlinear ference equation mentioned tinder (7) above ean Iw tained even in lose frm, Forvaxy reference, the main rests ate displayed in the form of theorems ‘Daly Theorems 3 au 4 are original. The next es tion and Uke Appenclx were mainly to review well-known mi teri in form suitable fur the present prpnies Notation Conventions ‘Throughout the papes, we shall deal mainly with iserete cor sampled) dynamic systems; in other words, signals will be ob served at equally spaced pointe in time (sampling instants). By fitable chojee of Uhe time keae, the constant intervale between ‘ueceasive sampling instante (aampling perfads) may be chosen at Tunity. Thus variables referring to Lime, auch ae ty 7, 7 i rlways be integers. ‘The restrieion to diserele dynamic systems isnot at all essential (at last from the engineering point of view) boy ssing the disreteness, hoverver, we ean Keep the mathe- matics rigorous and yet wlementary. Vectors will be denoted by revall boldface letters: a, By 4 8 8, ys A tector or more precisely an m-edor isa set of w numbers 4,» q; the x, are the ‘nordinates or components of the veeor Matrires will be denoted hy expital boldface letter: A, B, Q, Fy they arom weareays of element? yur ur ‘The: transpose (interchanging rows and columns) of w matrix will tye denote by the prime, In manipulating formulas, it will be convenient to regard a vevtor as matric wath a single column, Using the conventional deGinition of matrix multiplication, we ite the solar product of eo n-veetore x, ¥ ae rane “The salar product is elearly a sealar, Le, not a veetor, quantity BJ march 1960 Similurly, the quadratic form associated with then 1 matsix We define the expression xy! where x isan mevector and y is an revert tobe the mn mattis with elersents We write Hix} = Bx for the expeeted value of the random veo- turx(eoe Appenilix). Hisusually eonvoaient to omit the brackets after E. This doce aot result confusion in simple eases sinee fomtants and the operator # commute. This Bey” with slements Ezy); BxBy’ = matrs with elements B798(y,) For ease of reference, list of the prineipal aymble sed ia riven below ls pest in fe time at which olbservstions start 40-240) Vasie random variable, al) obwerved ranlom vari nUhl0) optimal rstnnnte of 24) pivew les 1s funetion (noneandunn fone tion fie arnt) estimation erroe (sndom variable) Yee) fineae 9 tt Gt) orthogonal peojetion of hon Ye) 2G]0) component of (4) orthogonal to Yl. fold generated Wy the ran 1m variables At), Medals for Rondom Procenes C+: transition matric Qi) covarianre of random oxitation Selatan of he Winn Prblom (0) basie random varia. 0) obeerved rand variable Ye) linear manifold generated by vie, WO > Kina manifold generated iy tlt ~ 2(G|0) optimal etimae of a) given YOO, HUG|0)_ error in optimal etinate uf ace) given ‘YO. Optimal Estimates, ‘To have a conerete description of the type of problems to be studied, consider the following situation, "We are given signal {n(¢) anid noise x40). Only the sum (0) = 2(0) + xK0) can be ob ferved. Suppose we have observed and know exactly the values of X),---s 0. What ean we infer from this knostledge in regard tothe (unobservable) value ofthe signa at €= &, whore temay be loss than, equal to, or gevater than (7 AF < this ia a ‘ata-amoething (interpolation) problem. If, = this it ealled filering. If, > f,w8e have a prediction problem. Sinee our teat- ‘nest will be general enotigh to ilu these and similar problems, twechall use hereafter the colertive term estimation. "As seas pointed out hy Wiener [1], Uhe antural setting of the estimation problem belongs to the realm of probability theory land statisticn. ‘Thus sign, noise, and (heir sim wil be random viriables, and consequently they may be regarded se random processes. From the probubiltie deseription of the random processes we ean determine the probability with which a par- ticular sample of the signal and noive sil cour. For any given set of measured valurs ml, (0) ofthe random variable y(®) fone ean then sls determine n prneiple, the probability of sims taneous occurrence of various values (tof the random variable 2G). ‘This isthe conditional probability dsteibution function ‘Transactions of the ASME