Slides for Introduction to Stochastic Search

and Optimization (ISSO) by J. C. Spall

CHAPTER 3

RECURSIVE ESTIMATION FOR

LINEAR MODELS
Organization of chapter in ISSO
Linear models
Relationship between least-squares and mean-square

LMS and RLS estimation

Applications in adaptive control

LMS, RLS, and Kalman filter for time-varying solution

Case study: Oboe reed data

Basic Linear Model

Consider estimation of vector in model that is linear in
Model has classical linear form

zk hkT v k ,

where zk is kth measurement, hk is corresponding design vector,

and vk is unknown noise value
Model used extensively in control, statistics, signal processing, etc.
Many estimation/optimization criteria based on squared-errortype loss functions
Leads to criteria that are quadratic in
Unique (global) estimate

32

Least-Squares Estimation
Most common method for estimating in linear model is by method
of least squares
Criterion (loss function) has form

1 n
1
T 2
T
(
z

(
Z

)
(Z n Hn )

k
k
n
n
2n k 1
2n
T

where Zn = [z1, z2 ,, zn] and Hn is n p concatenated matrix of hkT

row vectors
Classical batch least-squares estimate is
Popular recursive estimates
T
1 Kalman
T
RLS,
filter) may be
( n ) (H(LMS,
H
)
H
Z
n n
n n
derived from batch estimate

33

Geometric Interpretation of Least-Squares

Estimate when p = 2 and n = 3

34

Recursive Estimation
Batch form not convenient in many applications
E.g., data arrive over time and want easy way to update
estimate at time k to estimate at time k+1

Least-mean-squares (LMS) method is very popular recursive

method
Stochastic analogue of steepest descent algorithm

LMS recursion:

k 1 k ahk 1(hkT1 k zk 1), a 0

Convergence theory based on stochastic approximation (e.g.,

Ljung, et al., 1992; Gerencsr, 1995)
Less rigorous theory based on connections to steepest descent
(ignores noise) (Widrow and Stearns, 1985; Haykin, 1996)

35

LMS in Closed-Loop Control

Suppose process is modeled according to autoregressive (AR)
form:

xk 1 0 xk 1xk 1 K m xk m uk w k ,

where xk represents state, and i are unknown parameters, uk is

control, and wk is noise
Let target (desired) value for xk be dk
Optimal control law known (minimizes mean-square tracking error):

dk 1 0principle
xk 1xkjustifies
m xk m of parameter
1 K substitution
Certaintyuequivalence
k
estimates for unknown true parameters

LMS used to estimate and i in closed-loop mode

36

LMS in Closed-Loop Control for

First-Order AR Model

37

Recursive Least Squares (RLS)

Alternative to LMS is RLS
Recall LMS is stochastic analogue of steepest descent (first
order method)
RLS is stochastic analogue of Newton-Raphson (second order
method) faster convergence than LMS in practice

RLS algorithm (2 recursions):

Pk 1 Pk

Pk hk 1hkT1Pk

1 hkT1Pk hk 1

k 1 k Pk 1hk 1(hkT1 k zk 1)
Need P0 and

to initialize RLS recursions

38

Recursive Methods for Estimation of TimeVarying Parameters

It is common to have the underlying true evolve in time
(e.g., target tracking, adaptive control, sequential experimental
design, etc.)
Time-varying parameters implies replaced with k

Consider modified linear model

zk hkT k v k
Prototype recursive form for estimating k is

k 1 Ak k k 1(hkT1Ak k zk 1),
where choice of Ak and k depends on specific algorithm
39

Three Important Algorithms for Estimation

of Time-Varying Parameters
LMS
Goal is to minimize instantaneous squared-error criteria across
iterations
General form for evolution of true parameters k

RLS
Goal is to minimize weighted sum of squared errors
Sum criterion creates inertia not present in LMS
General form for evolution of k

Kalman filter
Minimizes instantaneous squared-error criteria
Requires precise statistical description of evolution of k via
state-space model

Details for above algorithms in terms of prototype algorithm

(previous slide) are in Section 3.3 of ISSO
310

Case Study: LMS and RLS with Oboe Reed Data

an ill wind that nobody blows good.
Comedian Danny Kaye in speaking of the oboe in the The Secret
Life of Walter Mitty (1947)

Section 3.4 of ISSO reports on linear and curvilinear models for

predicting quality of oboe reeds
Linear model has 7 parameters; curvilinear has 4 parameters

This study compares LMS and RLS with batch least-squares

estimates
160 data points for fitting models ( reeddatafit ); 80
(independent) data points for testing models ( reeddatatest)
reeddatafit and reeddatatest data sets available from
ISSO Web site

311

Oboe with
Attached Reed

Comparison of Fitting Results for

reeddatafit and reeddata-test
To test similarity of fit and test data sets, performed
model fitting using test data set
This comparison is for checking consistency of the two
data sets; not for checking accuracy of LMS or RLS
estimates
Compared model fits for parameters in
Basic linear model (eqn. (3.25) in ISSO) (p = 7)
Curvilinear model (eqn. (3.26) in ISSO) (p = 4)

Results on next slide for basic linear model

313

Comparison of Batch Parameter Estimates for

Basic Linear Model. Approximate 95%
Confidence Intervals Shown in [, ]

314

Comparison of Batch and RLS with

Oboe Reed Data
Compared batch and RLS using 160 data points in
reeddatafitand 80 data points for testing models
in reeddatatest
Two slides to follow present results
First slide compares parameter estimates in pure linear
model
Second slide compares prediction errors for linear and
curvilinear models

315

Batch and RLS Parameter Estimates for Basic

Linear Model (Data from reeddatafit )
Constant,
const
Top close, T
Appearance,
A
Ease of
Gouge, E
Vascular, V
Shininess,
S
First blow, F

Batch
Estimates

RLS
Estimates

0.156

0.079

0.102

0.101

0.055

0.046

0.175

0.171

0.044

0.043

0.056

0.056

0.579

0.540
316

Mean and Median Absolute Prediction

Errors for the Linear and Curvilinear Models
(Model fits from reeddatafit; Prediction
Errors from reeddatatest)
Batch linear
model
Mean
Median

0.242
0.243

RLS
linear
model
0.242
0.250

Batch
curvilinear
model
0.235
0.227

RLS
curvilinear
model
0.235
0.224

Ran matched-pairs t-test on linear versus curvilinear

models. Used one-sided test.
P-value for Batch/linear versus Batch/curvilinear is
0.077
P-value for RLS/linear vs. RLS/curvilinear is 0.10
Modest evidence for superiority of curvilinear model
317