Contributions to the Theory
of Optimal Control
R. E. KALMAN
T HIS is one of the two ground-breaking papers by Kalman
that appeared in 1960—with the other one (discussed next) be-
ing the filtering and prediction paper. This first paper, which
deals with linear-quadratic feedback control, set the stage for
what came to be known as LQR (Linear-Quadratic-Regulator)
control, while the combination of the two papers formed the
basis for LQG (Linear-Quadratic-Gaussian) control. Both LQR
and LQG control had major influence on researchers, teachers,
and practitioners of control in the decades that followed.
The idea of designing a feedback controller such that the in-
tegral of the square of tracking error is minimized was first pro-
posed by Wiener [17] and Hall [8], and further developed in the
influential book by Newton, Gould and Kaiser [12]. However,
the problem formulation in this book remained unsatisfactory
from a mathematical point of view, but, more importantly, the
algorithms obtained allowed application only to rather low order
systems and were thus of limited value. This is not surprising
since it basically took until the H2 -interpretation in the 1980s of
LQG control before a satisfactory formulation of least squares
feedback control design was obtained. Kalman’s formulation in
terms of finding the least squares control that evolves from an
arbitrary initial state is a precise formulation of the optimal least
squares transient control problem.
The paper introduced the very important notion of controlla-
bility, as the possibility of transfering any initial state to zero by
a suitable control action. It includes the necessary and sufficient
condition for controllability in terms of the positive definiteness
of the Controllability Grammian, and the fact that the linear
time-invariant system with n states,
d was included in influential textbooks [2], [5], [11], [1], and
x = F x + Gu
dt
is controllable if and only if the matrix [G, FG, . . . , F n−1 G]
has rank n. As is well known, this concept of controllability, its
implications (e.g., in pole placement and stabilization), and gen-
eralizations to nonlinear or infinite-dimensional systems became
one of the main leitmotivs in control research. Controllability is
indeed one of the compelling notions that is truly endogenous
to the field of control.
147
The paper also introduced the notion of observability, but as a
mere “dual” of controllability. Contemporaneously Kalman pro-
vided an alternative, more satisfactory definition in [10], where
observability is defined in a more intrinsic way in terms of the
possibility of deducing the state trajectory from input/output
measurements.
Kalman actually states (p. 102, fourth paragraph) that he views
the introduction of the notions of controllability and observ-
ability, and their exploitation in the regulator problem, as the
principal contribution of the present paper. Controllability and
observability are shown in the paper to be of central impor-
tance in the analysis of the least squares control problem over
an infinite horizon. They are also used to obtain the asymptotic
properties of the Riccati differential equation [Equation (6.3)
of the paper—henceforth referred to as RDE], and the stability
properties of its limiting solution. This paper was in fact the first
to introduce the RDE as an algorithm for computing the state
feedback gain of the optimal controller for a general linear sys-
tem with a quadratic performance criterion. RDE had emerged
earlier in the study of the second variations in the calculus of
variations, but its use in general linear systems, where the opti-
mal trajectory needs to be generated by a control input, was new.
The analysis throughout the paper concentrates on time-
varying systems, and uses the Hamilton-Jacobi theory to arrive
at RDE and to deduce optimality of the LQ control gain. We
now know, however, that an alternative way to prove optimality
in least squares is by showing how RDE allows one to “complete
the square” (see, e.g., [5], [18]).
Almost immediately after its appearance, the LQ-problem
extended in a number of directions. For example, the case of
indefinite cost is treated in [18], which requires a more del-
icate analysis, and later had many applications, in particular,
in H∞ -control; and extensions to zero-sum and nonzero-sum
differential games are discussed in [9] and [16], where again
RDE-based feedback policies arise, albeit with more general
structures [3]. Kalman’s paper actually also motivated and led
the way to a great deal of research on RDE, particularly on its
algebraic version, and algorithms for solving it appeared very
soon [13]. Most computer packages that aim at linear systems
and control implement today a Riccati equation solver. A wealth
of information and references on the work on the LQ problem
can be found in [15] and [4].
Kalman’s paper deals only with state feedback. The formula-
tion of the output feedback version of the LQ problem requires
either introducing, as in LQG, stochastic disturbances, or a for-
mulation in terms of the H2 -norm. It is worth noting, however,
that Kalman’s paper contains (p. 104) an informal, unproven but
quite precise statement of the separation theorem and the cer-
tainty equivalence principle, which are further discussed in this
volume in the preamble to Feldbaum’s papers.
The LQ problem and its sister, LQG, received enormous atten-
tion, and were sometimes presented as a panacea for solving lin-
ear control problems. Critics [14], [16] pointed to the excessive
bandwidth and poor robustness properties of the LQG controller.
In fact, Doyle’s paper [6] about guaranteed margin of LQG-
controllers had the abstract “There are none”—fortuitously mak-
ing reading of the article superfluous. This lack of robustness led
to the H∞ control problem which was formulated by Zames [19]
in part as a reaction to the LQG problems (see the comments in
the preamble to [19], the last paper in this volume). However,
it turns out that also in the H∞ -problem it was the Riccati equa-
tion that finally prevailed [7]! The introduction of the Riccati
equation, along with the notions of controllability and observ-
ability, are among the many gems in the crown of original ideas
in this seminal paper by Kalman.
REFERENCES
[1] B.D.O. ANDERSON AND J.B. MOORE, Linear Optimal Control, Prentice
Hall (Englewood Cliffs, NJ), 1971.
[2] M. ATHANS AND P.L. FALB, Optimal Control, McGraw-Hill (New York),
1966.
148
[3] T. BAŞAR AND G.J. OLSDER, Dynamic Noncooperative Game Theory, Clas-
sics in Applied Mathematics, SIAM (Philadelphia), 1999.
[4] S. BITTANTI, A.J. LAUB, AND J.C. WILLEMS, edts, The Riccati Equation,
Springer Verlag (Berlin), 1991.
[5] R.W. BROCKETT, Finite Dimensional Linear Systems, Wiley (New York),
1970.
[6] J.C. DOYLE, “Guaranteed margins for LQG regulators,” IEEE Trans. Au-
tomat. Contr., AC-23:756–757, 1978.
[7] J.C. DOYLE, K. GLOVER, P.P. KHARGONEKAR AND B.A. FRANCIS, “State-
space solutions to standard H2 and H∞ control problems,” IEEE Trans.
Automat. Contr., AC-34:831–847, 1989.
[8] A.C. HALL, The Analysis and Synthesis of Linear Servomechanisms, The
Technology Press, M.I.T. (Cambridge, MA), 1943.
[9] Y.C. HO, A.E. BRYSON, JR. AND S. BARON, “Differential games and optimal
pursuit-evasion strategies,” IEEE Trans. Automat. Contr., AC-10(4):385–
389, 1965.
[10] R.E. KALMAN, “On the general theory of control systems,” in Proc.
First Internat. Congress Automat. Contr., pp. 481–491, Moscow,
1960.
[11] H. KWAKERNAAK AND R. SIVAN, Linear Optimal Control Systems, Wiley
(New York), 1972.
[12] G.C. NEWTON, JR., L.A. GOULD AND J.F. KAISER, Analytical Design of
Linear Feedback Controls, Wiley (New York), 1957.
[13] J.E. POTTER, “Matrix quadratic solutions,” SIAM J. Appl. Math., 14:496–
501, 1964.
[14] H.H. ROSENBROCK AND P.D. MCMORRAN, “Good, bad, or optimal?,” IEEE
Trans. Automat. Contr., AC-16(6):552–554, 1971.
[15] Special Issue on the Linear-Quadratic-Gaussian Estimation and Control
Problem, IEEE Trans. Automat. Contr., AC-16(6), 1971.
[16] A.W. STARR AND Y.C. HO, “Nonzero-sum differential games,” J. Optimiz.
Theory Appl., 3(3):184–206, 1969.
[17] N. WIENER, Extrapolation, Interpolation, and Smoothing of Stationary
Time Series, MIT Press (Cambridge, MA), 1949.
[18] J.C. WILLEMS, “Least squares stationary optimal control and the alge-
braic Riccati equation,” IEEE Trans. Automat. Contr., AC-16:621–634,
1971.
[19] G. ZAMES, “Feedback and optimal sensitivity: Model reference transforma-
tions, multiplicative seminorms, and approximate inverses,” IEEE Trans.
Automat. Contr., AC-26:301–320, 1981.
W.S.L. & J.C.W.
Reprinted with permission from Boletin de la Sociedad Matematica Mexicana, R. E. Kalman,
“Contributions to the Theory of Optimal Control,” Vol. 5, 1960, pp. 102–119.

149