CHAPTER 7 © Exgodicity Statistics, Estimation, and Simulation The method estimates an unknown, but fixed, parameter @ of px(xl0) by determining the value of that maximizes py(x|6), where the observed value of xis substituted int palal8). The parameter 0 is unknown and is to be estimated. The objective ofthe MLE pehod is to determine the most probable value of 6 after the data have been measured ‘ince after esimation the probability density function is also a function of 6 itis referes to asa likelihood function. The idea is that since the observed data x are assumed to Be taqualy likely foreach measurement, then the value of 0 is,the one that wil give the Targest probability for the observed data, And this value of 6 has a high probability o being close tothe tre value of 8. “h maximumn likelihood estimate will be unbiased as the number of samples, d= noted as, N, increases. Its also consistent and will become Gaussian as N increases ‘The estimate has other properties as well, which are discussed in more advanced texts Several disadvantages of the MLE are that the probability density function must Be nown (or assumed) and thatthe estimate may be difficult to compute analytically Definition 7.14. The maximum likelihood estimate (MLE) Oya. of the fixed, bse unknown parameter 0 isthe value of © a, that maximizes the probability densi function p(x 0yq,). The particular value of © x, that does this for the given dat is the most likely value of © yn, which is 6a We typically have more than one value of data that is observed, for instance, ase quence of valves x(0) x(D, .. .x(N ~ 1, which we assume are independent and ies ‘Rally distributed. In this case the probability density function likelihood function becomes L = L(x(0)x(1)...x0N — 1)18) Paco)... xtw-1) (0), x(1), +. (N ~ 118) 3x40) €(0) | 8) px) (X(1) [8) «- - Pxos-n (CN ~ 1) 18) L= [exo (x10) ‘The log likelihood function is toefL] = > tosfexas(xCi) 16)1 vs “The MLE is computed by differentiating L or log{L] with respect to @ and si the derivative equal to zero, namely aL /30 = 0 or dlog{L]/a8 = 0 vs “Example 7.22 Suppose we measure a signal plus noise, such that X()=s+V@) i=0, 1. «WN ~ Do wes VG) are independent, identically distributed Gaussian white noise samples with 270 meme Yarlance o?. Since the measured data is XG) and V{)= X()~s, the MLE for sas a functions data is Found as follows: sich isthe som likeli 76 £ eo majo applicatio pediction Defi