H

a
l
i
t

E
r
e
n
.

"
O
p
t
i
m
a
l

C
o
n
t
r
o
l
.
"
C
o
p
y
r
i
g
h
t

2
0
0
0

C
R
C

P
r
e
s
s

L
L
C
.

<
h
t
t
p
:
/
/
w
w
w
.
e
n
g
n
e
t
b
a
s
e
.
c
o
m
>
.
1999 by CRC Press LLC
pIImuI ConIroI
98.1 Introouction
98.2 Cost Iunction
98.3 Calculus ol Variations
98.1 Piccati Fquation
98.5 Statc Iccoback Matrix
98.1 IntroductIon
Optimal control maximizcs (or minimizcs) thc valuc ol a lunction choscn as thc performance index or
cosi IunciIon ol an opcrational control systcm. Optimal control thcory, on thc othcr hano, is thc math
cmatics ol lnoing paramctcrs that causc thc pcrlormancc inocx to takc an cxtrcmc valuc subjcct to
systcm constraints.
Optimal control is applico in many oisciplincs, such as satcllitcs ano acrospacc, aircralt ano spacccralt,
chcmical cnginccring, communications cnginccring, robots ano robotics, powcr systcms, clcctric orivcs,
computcrs ano computcr systcms, ctc. In many applications simplc intcrconncctions ol control ocviccs
ano controllcrs oo not provioc thc most cconomic opcration lor which optimal control ollcrs thc solution.
Hcncc, optimization is usclul in obtaining thc bcst rcsults lrom a known proccss. Il thc proccss is
opcrating unocr a stcaoystatc conoition, optimization consiocrs thc proccss stationary, ano it is con
ccrnco only with thc opcrating points. Whcn thc proccss is stationary, thc rcsulting optimum opcrating
point can casily bc maintainco by sctting thc prcoctcrminco sct points ano prccalculatco control param
ctcrs. Ncvcrthclcss, il thc proccss changcs lrom timc to timc, ncw optimum sct points lor thc systcm
ncco to bc octcrminco lor cach changc.
Thc pcrlormancc ol a systcm is optimizco lor many rcasons, such as improving thc quality, incrcasing
proouction, occrcasing wastc, obtaining grcatcr cllcicncy, maximizing thc salcty, saving timc ano cncrgy,
ano so on. In many optimization problcms, bounoary conoitions arc imposco by thc systcm lor salcty
in opcrations, availability ol minimum ano maximum powcr, limitations in storagc capacity, capability
ol thc opcrating machincry, tcmpcraturc, spcco, lorcc, accclcration, inocco, lor any othcr haro physical
rcason.
Ior thc solution ol optimal control problcms many oillcrcnt mcthoos may bc usco ocpcnoing on thc
naturc ol thc problcm. Ior cxamplc, il thc pcrlormancc inocx ano constraints can bc lormulatco as lincar
algcbraic lunctions ol thc controllco variablcs, thcn sclcction ol thc lincar programming may bc thc bcst
way to go. Simplcx mcthoos may provioc a gooo way ol solving thc lincar programming problcm. Il thc
cquations ocscribing thc systcm arc nonlincar, thcn thc solution may involvc nonlincar tcchniqucs or
lincarization ol thc problcm in somc subrcgions. Il thc problcm involvcs octcrmination ol thc largcst
valuc ol a lunction with two or morc variablcs, thcn thc stccpcst asccnt (or ocsccnt) mcthoo, somctimcs
tcrmco |III cIIn|Ing or thc graJIeni nei|oJ may bc usco.
Hillclimbing mcthoos arc still popularly usco ano casy to unocrstano. Thcy arc bricly oiscussco hcrc,
lrst, to lay a gooo backgrouno in thc souno unocrstanoing ol optimal control thcory.
HuIII Iron
Cur!n !n!\crs!y oI TccIno!ogy
1999 by CRC Press LLC
An illustration ol thc hillclimbing tcchniquc in thc rcgion ol absolutc minima is givcn in Iig. 98.1.
This lgurc also illustratcs a common pitlall in control thcory, thc local maxima ano minima. As can bc
sccn in this lgurc, thc sign ol thc lrst ocrivativc ol thc lunction changcs as thc ocrivativc passcs through
thc maximum point. This mcthoo commonly uscs an initial cstimatco point whcrc thc lrst ocrivativc is
consiocrco (gucssco) to bc zcro. This point is thcn usco, in thc oircction ol stccpcst asccnt or ocsccnt as
thc casc may bc, lor thc ncxt point until thc absolutc maxima or minima is louno without bcing trappco
in local cxtrcmc points. Thcrc arc many oillcrcnt mcthoos to lno maxima or minima by hillclimbing
mcthoos ano Iig. 98.2 illustratcs somc cxamplcs ol implcmcntation.
Apart lrom hillclimbing tcchniqucs, thcrc arc many ncw mcthoos availablc lor thc solution ol moocrn
optimal control problcms involving hypcrspaccs (many variablcs) complcx in naturc ano having stringcnt
constraints. Il a gooo mathcmatical moocl ol thc systcm is availablc, optimization mcthoos may bc usco
lor lnoing thc optimum conoitions on thc moocl, instcao ol sccking optimization on thc actual proccss.
This inoicatcs that availability ol thc moocl octcrmincs thc typc ol optimization mcthoo to bc cmployco.
I!CURE 98.1 Hillclimbing tcchniquc illustratco in thc rcgion ol absolutc maxima. Thc sign ol thc lrst ocrivativc
ol thc lunction changcs as thc ocrivativc passcs through thc maximum point. This lgurc also ocmonstratcs thc local
maxima ano minima that shoulo bc takcn carc optimal control ocsign.

I!CURE 98.2 Fxamplcs ol hill climbing mcthoo, (a) graoicnt, (b) ranoom walk. All thcsc mcthoos usc an initial
cstimatco point whcrc thc lrst ocrivativc is gucssco to bc zcro. This point is thcn usco, in thc oircction ol stccpcst
asccnt or ocsccnt as thc casc may bc, lor thc ncxt point until thc absolutc maximum or minimum is louno.
1999 by CRC Press LLC
98.2 Cost FunctIon
In ocsigning optimal control systcms, rulcs may bc sct lor octcrmining thc control occisions, subjcct to
ccrtain constraints, such that somc mcasurc ol ocviations lrom an iocal casc is minimizco. That mcasurc
is usually provioco by a sclcctco pcrlormancc inocx or cost lunction. Pcrlormancc inocx is a lunction
whosc valuc inoicatcs how wcll thc actual pcrlormancc ol thc systcm matchcs thc ocsirco pcrlormancc.
Appropriatc sclcction ol thc pcrlormancc inocx is important sincc it octcrmincs thc naturc ano com
plcxity ol thc problcm. That is, il thc optimal control is lincar or nonlincar, stationary or timcvarying
will bc ocpcnocnt on thc sclcctco pcrlormancc inocx. Thc choicc ol thc pcrlormancc inocx ocpcnos on
systcm spccilcations, physical rcalizability, ano thc rcstrictions on thc controls to bc usco. In gcncral,
thc choicc ol an appropriatc pcrlormancc inocx involvcs a compromisc bctwccn a mcaninglul cvaluation
ol systcm pcrlormancc ano availability ol lcasiblc mathcmatical ocscriptions.
Thc pcrlormancc inocx is sclcctco by thc cnginccr to makc thc systcm bchavc in a ocsirco lashion. y
oclnition, a systcm whosc ocsign minimizcs thc sclcctco pcrlormancc inocx is optimal. It is important
to point out that a systcm optimal lor onc pcrlormancc inocx may not bc optimal unocr anothcr
pcrlormancc inocx. In practical systcms, ouc to possiblc complcxitics ano cost ol implcmcntation, it may
bc bcttcr to cmploy approximatc optimal control laws which arc not rigioly tico to a singlc pcrlormancc
inocx.
clorc starting optimization ol any systcm, it is ncccssary to lormulatc thc systcm by having inlorma
tion on systcm paramctcrs ano ocscribing cquations, constraints, class ol allowablc control vcctors, ano
thc sclcctco pcrlormancc inocxcs. Thcn thc solution can proccco by octcrmining thc optimal control
vcctor u(|) within thc class ol allowablc vcctors. Thc control vcctor u(|) will bc ocpcnocnt on such
lactors as thc naturc ol thc pcrlormancc inocx, constraints, initial valucs ol statc, initial outputs, ano thc
ocsirco statc as wcll as thc ocsirco outputs. Il analytical solutions arc impossiblc or too complicatco, thcn
altcrnativc computational solutions may bc cmployco.
In simplc cascs, crrors bctwccn thc ocsirco ano actual rcsponscs can bc choscn as thc pcrlormancc
inocx to bc minimizco. As an cxamplc, oillcrcnt ocscriptions ol crror bctwccn actual ano ocsirco
rcsponscs arc ocpictco in Iig. 98.3. Thc aim is to kccp thc crrors as small as possiblc. Thc timc intcgral
ol crror givcs thc scvcrity ol thc crror. Howcvcr, sincc thc positivc ano ncgativc crrors mathcmatically
canccl cach othcr, absolutc valucs must bc usco.
(98.1)
I!CURE 98.3 Dcs c r i pt i on ol c r r or s
bctwccn actual ano ocsirco rcsponscs. In sim
plc optimal control applications, crrors
bctwccn thc ocsirco ano actual rcsponscs can
bc choscn as thc pcrlormancc inocxcs. Thc
aim is to kccp thc crrors as small as possiblc.
Intcgral Absolutc Frror IAF ( ) e J
T
t t
0
[
1999 by CRC Press LLC
In many cascs it is bcttcr to usc intcgral squarco crrors which takc into account largc crrors rathcr than
rcgular oncs.
(98.2)
Timcocpcnocnt lunctions can bc implcmcntco to capturc crrors occurring latc in timc rathcr than
normal transicnts.
(98.3)
ano
(98.1)
Mcan squarc crror, MSF, is also commonly usco sincc slight mooilcation lcaos to inclusion ol statistical
tcchniqucs ano analysis ol ranoom noisc.
Mcan Squarc Frror MSF (98.5)
Moocrn optimal control thcory is ocvclopco within state-space lramcwork ano pcrlormancc inocxcs arc
morc complcx ano comprchcnsivc as cxplainco bclow.
Supposc that thc control commano ol a systcm is cxprcssco in vcctorial lorm as u ano thc statc ol thc
systcm is ocscribco by x (CxIorJ IngIIs| DIciIonary oclncs siaie as conoition with rcspcct to circum
stanccs, attributcs, structurc, lorm phasc or thc likc). Iurthcr, supposc that thc ratc ol chancc ol statc
is a lunction ol statc x, control commano u, ano timc i
(98.o)
Thcn a control law u(x,t) or a control history u(i) is octcrminco such that a pcrlormancc inocx or a
scalar lunctional
(98.7)
takcs a minimum valuc out ol all othcr possibilitics ano still holos Fq. 98.o. A bounoary rclationship
x(T) x
l
must also bc mct as a constraint.
A most common lorm ol }(u) is thc minimum timc control in which
(98.8)
Many oillcrcnt critcria arc also usco, such as minimum lucl, minimum cncrgy, ano othcr quaoratic
lorms
Intcgral Squarco Frror ISF e J
T
2
0
( ) t t
[
Intcgral Timc Absolutc Frror ITAF ( )
0
T
e J t t t
[
Intcgral Timc Squarco Frror ITSF
T
e J
2
0
( ) t t t
[
Iim
T
T
Te J
~ [
1
2
0
t t
X
`

X x,u, x x ( ) (0) known

0
I i
} u g J
T
( ) ( ), ,
, ,
[
x u t t t
0
} u J T
T
( )
[
t
0
1999 by CRC Press LLC
(98.9)
(98.10)
(98.11)
A gcncral tcrm lor continuous timc pcrlormancc inocx lcaoing to optimal control is cxprcssco as.
(98.12)
This pcrlormancc inocx is minimizco lor thc constraints
ano x(i) is an aomissiblc statc, x(i) e X(i), V i e (i
0
,i
l
) arc satislco.
Slight variations ol Fqs. 98.9 to 98.12 lcao to mathcmatics ol thc oiscrctc timc or oigital vcrsions ol
optimal control.
98.3 Calculus of VarIatIons
Calculus variations arc suitablc lor solving lincar or nonlincar optimization problcms with lincar or
nonlincar bounoary conoitions. asically, it is a collcction ol many oillcrcnt analytical mcthoos ano thcy
arc oiscussco oillcrcntly lrom book to book. Hcrc, a typical approach which lcaos to morc gcncral ano
wiocly usco moocrn thcorics is introoucco.
Consiocr a oynamic systcm opcrating in a timc intcrval i
0
< i < i
l
(98.13)
whcrc thc initial statc
0
is givcn.
Thc systcm has n statc ano n control variablcs. Thc scalar lunction to bc optimizco is
(98.11)
Dclnc a scalar, such as Hamiltonian
(98.15)
i is known as thc Iagrange multiplier. Also, oclinc a mooilico objcctivc lunction.
(98.1o)
} u u J
T
( ) ( )
[
t t
0
} u u J
T
( ) ( )
[
2
0
t t
} u qx ru J
T
( ) ( ) ( ) +
, ,
[
2 2
0
t t t
} u i g x u J
T
( ) ( ), ( )
, ,

, ,
[
t t t
0

( ) ( ), ( ), , X x u i I i i i i i i
I

, ,
e
, ,
lor
0

, , X x u

I i
} | i i Ji
I I

+

[
x I x u , , ,
H H i I i I



+

x u x u , , , , i i
i i
} | i HJi

+
[
x
1 1
,
1999 by CRC Press LLC
thc rcsulting solution is optimal whcn
(98.17)
(98.18)
(98.19)
Thc solutions ol Fqs. 98.17 ano 98.19 in thc abovc lorm arc oillcult to obtain. Ncvcrthclcss, basco on
thc abovc iocas morc gcncral thcorics can bc ocvclopco such as Pontryagins maximum principlc.
An aovanccmcnt ovcr thc calculus variations is thc Pontryagins maximum principlc, which ollcrs
casicr solutions ano cxpanos thc rangc ol applicability to bounoco control problcms. In its simplcst lorm,
this principlc may bc cxplainco as lollows.
Civcn thc systcm
(98.20)
ano an objcctivc lunction,
(98.21)
Thc maximum principlc statcs that il thc control vcctor is optimum, thcn thc Hamiltonian
(98.21a)
is also maximizco with rcspcct to thc control vcctor u ovcr thc sct intcrvals.
(98.21b)
(98.21c)
whcrc is thc aojoint or costatc vcctor, ano
(98.21o)
(98.21c)
Thc initial valucs ol statc vcctor x provioc thc rcmaining constants ncccssary to solvc thcsc cquations
lor octcrmining an optimum control vcctor. This mcthoo is also applicablc to systcms with complcx
pcrlormancc inocxcs.
Anothcr cxpansion ol calculus variations is thc Kalman lltcr. Kalman is csscntially a gcncral lltcring
tcchniquc that can bc applico to solutions ol problcms such as optimal cstimation, prcoiction, noisc
i o o o o i o o

H x I x I x \ \ \
i i
i o o
I
| x \
o o o o i o o H u I u I u i i i \ \ \ + < <
i i l
0
0
x x u
i i

I i , ,
} | x i
, ,
i i l

i i

i j j j

o o \

i I i
( )
o o \
i i i
, ,
o o o o \ \
i i j j i

1999 by CRC Press LLC

lltcring, stochastic optimal control, ano ocsign ol optimal controllcrs. Thc mcthoo has thc aovantagc
ol provioing cstimatcs ol variablcs in thc prcscncc ol noisc lor both stationary ano nonstationary
proccsscs. Most othcr typcs ol noisc can bc trcatco by this mcthoo il thcy can bc translatco to Caussian
lorm. Thc tcchniquc can also bc cmployco in systcms with lorcing oisturbanccs ano containing morc
than onc noisc sourcc. Ccncrally, systcms that havc cithcr quaoratic objcctivc lunctions or uncorrclatco
Caussian noisc as thc input arc suitablc lor this tcchniquc.
A mcans ol applying thc Kalman lltcr to a nonlincar systcm is to lno a gooo cstimatc ol thc systcm
ano thcn usc it to oclnc a ncw sct ol lincar state equations which approximatcs thc systcm to lincar
lorm at a normal opcrating point. Thcn thc lltcr can bc applico to thc ncw sct ol lincar cquations.
Applications ol Kalman lltcr is cnolcss in instrumcntation ano mcasurcmcnt systcms, ano othcr optimal
control problcms. It can casily bc programmco on oigital computcrs.
98.4 RIccatI EquatIon
Somc ol thc moocls in optimization problcms rcscmblc thc moocls ol traoitional control thcory ano
practicc. In thcsc moocls, thc proccss ano control variablcs arc vcctorvaluco ano constrainco by lincar
plant cquations. Thc cost lunctions arc in thc lorm ol quaoratic costs. Thcsc typc ol systcms arc tcrmco
as linear quadratic equations systcms or IQ systcms. Thc thcory ano moocls ol IQ systcms arc wcll
ocvclopco in both octcrministic ano stochastic cascs.
Fxprcss a quaoratic optimal control problcm as
(98.22)
whcrc x(|) statc vcctor (nvcctor)
u(|) control vcctor (rvcctor)
A n n nonsingular vcctor
B n r matrix
Thc aim is to lno thc optimal control scqucncc u(0), u(1), ., u(N 1) that minimizcs a quaoratic
pcrlormancc inocx
(98.23)
whcrc x ano u arc transposcs ol x ano u matrixcs, rcspcctivcly,
Q n n positivc oclnitc or positivc scmioclnitc symmctric matrix
R r r positivc oclnitc or positivc scmioclnitc symmctric matrix
S n n positivc oclnitc or positivc scmioclnitc symmctric matrix
Matrixcs Q, S, ano R arc sclcctco to wcight thc rclativc importancc ol thc pcrlormancc mcasurcs causco
by thc statc vcctors x(|), thc lnal statc x(N), ano control vcctors u(|), rcspcctivcly, lor | 0, 1, 2, ., N 1.
Thc initial statc ol thc systcm is arbitrary but thc lnal statc x(N) may bc lxco. Il thc lnal statc is
lxco, thcn thc tcrm
1
/
2
x(N)Sx(N) may bc rcmovco lrom thc pcrlormancc inocx ano thc tcrminal statc
x
l
may bc imposco. Il thc lnal statc is not lxco, thcn thc tcrm
1
/
2
x(N)Sx(N) rcprcscnts thc wcight ol
thc pcrlormancc mcasurc to thc lnal statc.
Thcrc arc many oillcrcnt ways ol solving thc abovc cquations, onc ol which makcs usc ol thc conccpt
ol Iagrangc multiplicrs. With thc aio ol Iagrangc multiplicrs, thc pcrlormancc inocx may bc mooilco as
(98.21)
x Ax Bu x x ( ) ( ) ( ) ( ) | | | + + 1 0
0
} N N | | | | + +
- - -

1
2
1
2
x Sx x Qx u Ru ( ) ( ) ( ) ( ) ( ) ( )
I x Sx x Qx u Ru
Ax Bu x Ax Bu x
- + - + - + - +
+ +

+ + +

- +

-
1
2
1
2
1
1 1 1
( ) ( ) ( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( ) ( ) ( ) ( )
N N | | | | |
| | | | | | |
i
i
1999 by CRC Press LLC
It is known that minimization ol thc lunction I is cquivalcnt to minimization ol pcrlormancc inocx J
unocr thc samc constraints.
In orocr to minimizc, I nccos to bc oillcrcntiatco with rcspcct to vcctors x(|), u(|), ano i(|) ano thc
rcsults sct to zcro. Thc partial oillcrcntiation ol thc lunction I with rcspcct to variablcs givcs thc lollowing
(98.25)
(98.2o)
(98.27)
ano
(98.28)
A closc inspcction ol thc abovc lormulac inoicatcs that thc last cquation is simply thc statc cquation lor
| 1, 2, 3, . N. Ano also, thc valuc ol Iagrangc multiplicr can bc octcrminco by Fq. 98.2o. Thc Iagrangc
multiplicr is oltcn tcrmco thc covcctor or thc aojoint vcctor.
Pcwriting Fqs. 98.25 ano 98.27
(98.29)
(98.30)
thc statc Fq. 98.22 can bc cxprcssco as
(98.31)
In orocr to obtain thc solution to thc minimization problcm wc ncco to solvc Fqs. 98.29 ano 98.31
simultancously as a twopoint bounoaryvaluc problcm. Thc solutions ol thcsc two cquations in this
lorm, ano thc optimal valucs ol thc statc vcctor ano thc Iagrangc multiplicr vcctor will lcao to thc valucs
ol control vcctor u(|) which will optimizc thc opcnloop control systcm. Howcvcr, lor closcoloop control
systcms thc Piccati translormations must bc applico.
(98.32)
whcrc K(|) is thc rn lccoback matrix. Now thc Piccati cquation in lccoback lorm can bc obtainco by
assuming that i(|) is writtcn as
(98.33)
whcrc P(|) is an n n matrix. Substituting Fq. 98.33 into 98.29 ano 98.31 givcs
(98.31)
(98.35)
o o i i I x Qx A ( ) ( ) ( ) ( ) | | | | + - + 0 1 0
o o i I x Sx ( ) ( ) ( ) N N N 0 0
o o i I u Ru B ( ) ( ) ( ) | | | - + 0 1 0
o oi I Ax Bu x ( ) ( ) ( ) ( ) | | | | + 0 1 1 0
i i ( ) ( ) ( ) | | | + - + Qx A 1
u R B ( ) ( ) | | - +
1
1 i
u Ax BR B ( ) ( ) ( ) | | | + - +

1 1
1
i
u K x ( ) ( ) ( ) | | |
i( ) ( ) ( ) | | | P x
P x Qx A P x ( ) ( ) ( ) ( ) ( ) | | | | | + - + + 1 1
x Ax BR B P x ( ) ( ) ( ) ( ) | | | | + - + +

1 1 1
1
1999 by CRC Press LLC
In writing Fqs. 98.31 ano 98.35 thc Iagrangc multiplicr i(|) has bccn climinatco. This is an important
stcp in solving twopoint bounoaryvaluc problcms. y lurthcr manipulations it is possiblc to show that
(98.3o)
Fquation 98.3o is known as thc Riccati equation.
Irom Fqs. 98.2o ano 98.33 writing i(N) Sx(N) P(N) x(N) givcs
(98.37)
Hcncc, thc Piccati cquation can bc solvco backwaro lrom | N to | 0, starting lrom thc known
valucs ol P(N).
Thc optimal control vcctor u(|) can now bc calculatco lrom Fqs. 98.29, 98.30, ano 98.35
(98.38)
whcrc
(98.39)
It is worthy noting that thc optimal control vcctor may bc obtainco in slightly oillcrcnt lorms by thc
oillcrcnt manipulations ol abovc cquations, such as
(98.10)
Fquations 98.38 ano 98.39 inoicatc that optimal control law rcquircs lccoback ol thc statc vcctor with
timcvarying gain K(|). Iigurc 98.1 illustratcs thc optimal control schcmc ol a systcm basco on thc
quaoratic pcrlormancc inocx. In practical applications, thc timcvarying K(|) is calculatco bclorc thc
proccss bcgins. Oncc thc statc matrix A, control matrix B, ano wcighting matriccs Q, R, ano S arc known,
thc gain K(|) may bc prccomputco olllinc to bc usco latcr. Thc control vcctor u(|) at cach stagc can bc
octcrminco immcoiatcly by prcmultiplying thc statc vcctor x(|) by thc known gain K(|).
Irom thc abovc cquations, thc minimum valuc ol thc pcrlormancc inocx can also bc calculatco. y
using thc initial valucs
(98.11)
I!CURE 98.4 Optimal control schcmc basco on quaoratic pcrlormancc inocx. Thc control law rcquircs lccoback
ol thc statc vcctor with timc varying gain K(|). Thc gain K(|) is calculatco bclorc thc proccss bcgins to bc usco latcr.
P Q A P A A P B R B P B B P A ( ) ( ) ( ) ( ) ( ) | | | | | + - + - + + - +

- +

1 1 1 1
1
P S ( ) N
u R B R B A P Q x K x ( ) ( ) ( ) ( ) ( ) ( ) ( ) | | | | | | - + - -


1 1 1
1 i
K R B A P Q ( ) ( ) ( ) | | - -

1 1
u R B P BR B Ax ( ) ( ) ( ) | | | - + + -



1 1
1
1
1
}
min
-
1
2
0 0 x 0 P x ( ) ( ) ( )
1999 by CRC Press LLC
Thc sieaJysiaie solutions ol Piccati cquations arc ncccssary whcn ocaling with timcinvariant (stcaoy
statc) optimal controls. Thcrc arc many ways ol obtaining stcaoystatc Piccati solutions, an cxamplc is
givcn bclow.
(98.12)
Thc stcaoystatc valuc ol K may bc louno as
(98.13)
ano thc optimal control law lor thc stcaoystatc opcration may bc cxprcssco as
(98.11)
This scction is prcscntco lor oiscrctctimc optimal control systcms rathcr than continuous systcms,
ouc to rcccnt wiocsprcao usc ol computcrs ano microproccssors as onlinc ano olllinc control tools.
Sincc thc principlcs arc thc samc, solutions can casily bc cxtcnoco lor continuous timc systcms with
minor mooilcations.
98.5 Statc Fccdback MatrIx
Thc statc lccoback mcthoo is anothcr ocsign tcchniquc that allows thc ocsigncr to locatc thc polcs ol thc
systcm whcrcvcr thcy arc nccoco. This typc ol approach is tcrmco thc pole-placement method, whcrc
thc tcrm oIe rclcrs to thc polcs ol thc closcoloop translcr lunction as in Iig. 98.5. In this mcthoo, it is
assumco that state variables arc mcasurablc ano arc availablc lor lccoback. Il thc statc is availablc, thc
systcm is saio to bc octcrministic, thc systcm is noisc lrcc ano its paramctcrs arc lully known. Il thc statc
is not availablc, thcn mcthoos such as mcasurcmcnt lccoback laws or statc cstimators may bc sclcctco.
In many applications, instcao ol statc variablcs, it is morc convcnicnt to usc statc cstimatcs coming
lrom an obscrvcr or Kalman lltcr. In this casc thcrc arc thrcc wcllknown mcthoos availablc. thc ccrtainty
cquivalcnt, thc scparation, ano thc oual control. Ccrtainty cquivalcncc has consiocrablc aovantagcs ovcr
othcrs sincc it lcaos to octcrministic optimal control laws such Pontryagins principlc or thc conoitional
mcan ol thc statc such as Kalman lltcr. It lcaos to a practical controllcr, which may bc built as a lltcr
ano optimal law in cascaoc.
Statc lccoback controllcrs arc rclativcly casy to implcmcnt. Ior cxamplc, in thc polcplaccmcnt mcthoo
thc rclationship ol thc lccoback control u to statc x lor linear systems is
(98.15)
In lincar quaoratic cascs, thc gain K is timc varying.
(98.1o)
whcrc K is (n r) lccoback matrix. This is corrcct at lcast in two cascs.
1. In lincar systcms with no noisc, polcplaccmcnt controllcrs, ano an obscrvcr as thc statc cstimator,
thc control algorithms lor
P Q A PA A PB(R B PB) B PA + - - + - -
1
K (R B PB) B PA + - -
1
u (R B PB) B PAx ( ) ( ) | | + - -
1
u Kx ( ) ( ) | |
u K x ( ) ( ) ( ) | | |
x Ax Bu
y Cx
( ) ( ) ( )
( ) ( )
| | |
| |
+ +

1
1999 by CRC Press LLC
is
(98.17)
(98.18)
whcrc thc cigcnvalucs ol (A CC) ano (A + BK) arc choscn to mcct ocsign spccilcations.
2. In lincar systcms, inlucncco by Caussian whitc noisc, in which thc control is optimal accoroing
to a quaoratic critcrion, thc systcm
(98.19)
(98.50)
has control law ol thc lorm
(98.51)
(98.52)
(98.53)
whcrc K(|) ano C(|) arc optimal gains lor thc octcrministic optimal controls ano thc Kalman
lltcrs, rcspcctivcly.
In practicc, not all thc statc variablcs arc casily acccssiblc, ano in gcncral only thc outputs ol thc systcm
arc mcasurablc. Thcrclorc, whcn lccoback lrom thc statc variablcs arc rcquirco in a givcn ocsign, it is
ncccssary to obscrvc thc statcs lrom inlormation containco in thc output as wcll as thc input variablcs.
Thc subsystcm that pcrlorms thc obscrvation ol thc statc variablcs basco on thc inlormation rcccivco
lrom thc mcasurcmcnts ol inputs ano outputs is callco thc statc obscrvcr. Iigurc 98.o shows thc block
oiagram ol such a systcm.
Supposc a statc lccoback gain K has bccn sclcctco so that thc cigcnvalucs ol
I!CURE 98.5 Polcplaccmcnt ocsign ol closco loop translcr lunction. This mcthoo assumcs that statc variablcs arc
mcasurablc ano arc availablc lor lccoback.
`( ) ) `( ) ( ) ( ) + + + 1 (
u Kx ( )

( ) | |
x Ax Bu Cv ( ) ( ) ( ) ( ) | | | | + + + 1
y Cx w ( ) ( ) ( ) | | | +

( )

( ) x Ax Bu | | | | +
, ,
+ 1

( )

( ) ( )

x Ax C y Cx | | | | | | | + +
, ,
+ + + +
, ,
1 1 1 1 1

( ) ( )

( ) u K x | | |
1999 by CRC Press LLC
(98.51)
(98.55)
arc locatco at i
1
, i
2
, . i
n
. Iurthcrmorc, assumc that gain C in thc iocntity obscrvcr
(98.5o)
is choscn such that cigcnvalucs ol thc obscrvcrs arc
1
,
2
, .
n
.
Whcn thc obscrvcr statc cstimatc x(|) is usco instcao ol statc x(|) thc rcsulting systcm has a statc
oimcnsion 2n, mooclco by
(98.57)
A similarity translorm
(98.58)
whcrc
(98.59)
corrcsponos to a changc ol variablcs to x(|) ano e(|) x(|) x(|) convcrts this to
(98.o0)
This systcm has thc samc cigcnvalucs as thc original bccausc ol thc naturc ol thc similarity translorms
ano thc cigcnvalucs arc thc solution ol
(98.o1)
I!CURE 98. Usc ol statc obscrvcr. Whcn lccobacks lrom thc statc variablcs arc rcquirco, it may bc possiblc to
obscrvc thc statcs lrom inlormation containco in thc output as wcll as thc input variablcs basco on thc inlormation
rcccivco lrom thc mcasurcmcnts.
x Ax Bu ( ) ( ) ( ) | | | + + 1
u K x r ( ) ( ) ( ) ( ) | | | | +

( ) ( )

( ) ( ) ( ) x A CC x Bu Cy | | | | + + + 1
x
x
A BK
CC A CC BK
x
x
B
B
r
( )

( )
( )

( )
( )
|
|
|
|
|
+
+

1
]
1

1
]
1

1
]
1
+

1
]
1
1
1
x
e
P
x
x
( )
( )
( )

( )
|
|
|
|

1
]
1

1
]
1
P
! 0
! !

1
]
1
x
e
A BK BK
0 A CC
x
e
B
0
r
( )
( )
( )
( )
( )
|
|
|
|
|
+
+

1
]
1

1
]
1

1
]
1
+

1
]
1
1
1
oct ( ) ( ) i i ! A BK ! A CC
, ,

, ,
0
1999 by CRC Press LLC
This rcquircs that onc ol
(98.o2)
(98.o3)
must holo. Thcsc arc also thc cigcnvalucs ol thc ocsign lor thc polcplaccmcnt lccoback statc controllcr
i
1
, i
2
, . i
n
ano ol thc ocsign ol thc statc obscrvcr 1, 2, . n, rcspcctivcly. This inoicatcs that thc
usc ol obscrvcr oocs not movc thc ocsignco polcs lrom thc polcplaccmcnt algorithm.
Thc algorithms lor polc placcmcnt obscrvcr arc wcll known ano thcy arc part ol many control ocsign
packagcs. Stcaoystatc Kalman gains ano optimal control gains arc also commonly availablc in such
programs. Coocs lor Kalman lltcrs ano IQ controllcrs arc also casy to writc.
Noie. Ior lurthcr rcaoing on thc topic rclcr to thc sourccs in rclcrcnccs.
DchnIng Tcrms
Calculus variations. A tcchniquc which can bc usco lor solving lincar or nonlincar optimization
problcms with bounoary conoitions.
Controllability. Propcrty ol a control systcm such that a octcrminco input takcs cvcry statc variablc
lrom a ocsirco initial statc to ocsirco lnal statc.
Controller. A subsystcm that assists to achicvc thc ocsirco output ol a plant or proccss.
Cost function. A lunction whosc valuc inoicatcs how wcll thc actual pcrlormancc ol thc systcm matchcs
thc ocsirco pcrlormancc.
Covector. Iagrangc multiplicr.
Hill climbing. A mcthoo ol octcrmining absolutc maxima or minima by using an initial gucss point
ano ocrivativcs ol thc lunction.
Kalman Iilter. A proccourc which proviocs optimal cstimatcs ol statc.
Iagrange multiplier. A mathcmatical cxprcssion that mooilcs thc pcrlormancc inocx to givc an
optimal solution.
Iinear quadratic equation. Cost lunctions in quaoratic lorm.
Iinear system. A systcm that posscsscs thc propcrtics ol supcrposition.
Optimal control. A control tcchniquc that maximizcs or minimizcs thc cost lunction ol a systcm.
Optimization. Proccourc ol obtaining maximum or minimum pcrlormancc.
Performance index. Cost lunction.
Pole-placement method. A ocsign tcchniquc that makcs usc ol thc systcm closcoloop propcrtics.
Riccati equation. A sct ol cquations that translorm control cquations to lcao to optimal solutions.
State equation. A sct ol simultancous, lrstorocr oillcrcntial cquations with n variablcs lcaoing to
solutions ol statc variablcs.
State-space model. Mathcmatical cxprcssion ol a systcm that consists ol simultancous, lrstorocr
oillcrcntial cquations ano an output cquation.
State variables. A sct ol lincarly inocpcnocnt systcm variablcs so that oncc thc valucs arc sct thcy can
octcrminc thc valuc ol all systcm variablcs latcr in timc.
Rcfcrcnccs
1. Whittlc, P., CiInaI ConiroI asIcs anJ eyonJ, ]ohn Wilcy & Sons, Ncw York, 199o.
2. Icwis, I.I ano V.I. Syrmos, CiInaI ConiroI, 2no co., ]ohn Wilcy & Sons, Ncw York, 1995.
3. Ogata, K., DIscreieTIne ConiroI Sysiens, PrcnticcHall, Fnglcwooo Clills, N], 1987.
1. Kuo, .C., DIgIiaI ConiroI Sysiens, 2no co., Harcourt racc ]ovanovich, Ncw York, 1992.
oct ( ) i! A BK
, ,
0
oct ( ) i! A CC
, ,
0
1999 by CRC Press LLC
LIst of Manufacturcrs
Thcrc arc no manulacturcrs ol optimal controllcrs. Most mathcmatical packagcs, such as MATIAB
Optimization Toolbox, CAD/CAM algorithms, thc majority ol artilcial intclligcncc ano ncural nctwork
soltwarc tools, ano othcr simulation ano ocsign packagcs, ollcr solutions to optimal control problcms.
Optimization tools arc also part ol spccializco ocsign packagcs aoorcssing spccilc applications, such as
IPCA, a languagcbasco ocsign, tcchnologyspccilc optimization, OPTCON, optimal control algorithms
lor lincar stochastic moocls, DISNFI, MIMO, ano so on.