ECE 551 LECTURE 1

Preface

This lecture reviews the optimal control problem. The concepts of an admissible control, an admissible
trajectory, and the general form of the performance measure are presented. We also review the basic
forms of performance measures corresponding to certain classes (types) of optimal control problem.
During the course we will repeatedly use this material as we learn methods of solving the optimal control
problem.

Optimal Control Problem

We begin with the statement of the optimal control problem as follows:

Find an admissible control u which causes the system

x (t ) a( x(t ), u(t ), t )

to follow an admissible trajectory x that minimizes the performance measure

tf

J h( x(t f ), t f ) g ( x(t ), u(t ), t )dt

t0

u is called an optimal control and x an optimal trajectory.

An admissible control is a control history (i.e. control input values during the interval [ t 0 , t f ] ) which

satisfies the control constraints during the entire interval [ t 0 , t f ] . Similarly, an admissible trajectory is

a state trajectory which satisfies the state variable constraints during the entire interval [ t 0 , t f ] .

Let us examine a simple optimal control problem, i.e. a minimum time problem, involving an automobile.
The engine of the automobile limits its acceleration to some maximum value M 1 , while the automobiles
brakes limit the deceleration to some maximum value M 2 . Thus, the controls must satisfy the following
control input constraints:

0 u1 (t ) M 1
(2.1)
M 2 u 2 (t ) 0

1
ECE 551 LECTURE 1

Assume that the automobile starts from rest and accelerates in a straight-line path stopping a distance R
later, so that the state boundary conditions are as follows:

0
x(t 0 )
0
(2.2)
R
x(t f )
0

If the automobile never backs up, then we also have the following additional state constraints:

0 x1 (t ) R
(2.3)
0 x 2 (t )

We must also consider an additional constraint on the control input, i.e.

tf

k u (t ) k u
t0
1 1 2 2 (t )dt G (2.4)

Where, G represents the quantity of fuel in the gas tank. Taken collectively, the constraints determine
admissibility. We can solve this problem using the techniques of this course to obtain the following
solution:

u1*( t )

M1

t0 (t0+tf) tf t

u2*( t )
(t0+tf) tf

t0
M2 t

2
ECE 551 LECTURE 1

x1*( t )

t0 (t0+tf) tf t

x2*( t )

(t0+tf) tf t
t0

Performance Measures

The performance measure defines optimality for a particular situation or problem. The most general form
of the performance measure is given by the following expression:

tf

J h( x(t f ), t f ) g ( x(t ), u(t ), t )dt (2.5)

t0

The first term on the right of equation (2.5) represents the cost of reaching the final state. It is often
referred to as the terminal cost. The integral term in (2.5) represents costs associated with the control
input and the resulting state trajectory. The integrand is referred to as the Lagrangian after Lagrange who
studied optimization problems involving integral cost functions. Performance measures that only involve
terminal costs are referred to as Mayer cost functions after Christian Mayer. Performance measures that
only involve an integral cost function are referred to as Lagrange cost functions after Lagrange. The
general form which includes both terminal and integral cost functions is referred to as Bolza cost functions
after Oscar Bolza. Let us now consider some specific examples of performance measure types.

3
ECE 551 LECTURE 1

Minimum-time Problem

Transfer a system from an arbitrary initial state x(t 0 ) x 0 to a specified target set S in minimum time.
Hence, we have that

tf

J dt t f t 0 (2.6)
t0

Recall that our automobile example above was a minimum-time problem.

Terminal Control Problem

Minimize the deviation of the final state of a system from its desired value r (t f ) . As one possible
measure, we have that

N 2

J xi (t f ) ri (t f ) (2.7)
i 1

We can also express equation (2.7) in matrix form, i.e.

J x(t f ) r (t f ) x(t
T
f ) r (t f )
2
(2.8)
x(t f ) r (t f )

Where, a is the norm of the vector a . For greater generality, we can insert a real symmetric positive
semi-definite n X n weighting matrix H , i.e. we have the following quadratic form

J x(t f ) r (t f ) H x(t
T
f ) r (t f )
2 (2.9)
x(t f ) r (t f )
H

4
ECE 551 LECTURE 1

Note that if H I (identity matrix), then equations (2.8) and (2.9) are equal. Also, if H is a diagonal
matrix, then each of its diagonal elements must be non-negative, since H is positive semi-definite. By
adjusting or choosing the values of the elements of H , we can quantify the importance of deviations of
each final state of the system xi (t f ) from its desired value ri (t f ) . Thus, relatively larger values of select
elements of H assign greater significance to those deviations, while if an element is set equal to zero,
then no significance is attributed to that deviation.

Minimum Control Effort Problem

Transfer a system from an arbitrary initial state x(t 0 ) x 0 to a specified target set S with minimum
expenditure of control effort. One possible measure is as follows:

tf

J
t0
u (t ) dt (2.10)

Equation (2.9) could correspond to a spacecraft whose rocket engines thrust is proportional to fuel
consumption. If there are multiple controls, then we have that

tf
M

J i u i (t ) dt (2.11)
t0 i 1

Where, i is a non-negative weighting factor. Another possible form of the performance measure might
be as follows:

tf

J u 2 (t ) dt (2.12)
t0

In this case, the situation might be a source driving a passive resistor network. If there are multiple
controls, then we have that

5
ECE 551 LECTURE 1

tf

J u
T
(t ) R u(t ) dt
t0
(2.13)
tf

2
u(t ) R
dt
t0

Where, R is a real symmetric positive definite weighting matrix. In general, the elements of R can be
functions of time.

Tracking Problem

Maintain the system state x(t ) as close as possible to the desired state r (t ) in the interval [ t 0 , t f ] . Thus,
we have that

tf

J x(t ) r (t )
2
Q (t )
dt (2.14)
t0

Where, Q (t ) is a real symmetric n X n matrix that is positive semi-definite for all t t 0 , t f . Note that
equation (2.14) assumes that the controls (admissible set) are bounded. If that is not the case, then we
have that

dt
tf

J x(t ) r (t ) u(t )
2 2
Q (t ) R (t )
(2.15)
t0

Where, R (t ) is a real symmetric m X m matrix that is positive definite for all t t 0 , t f . If it is also
important that the states be close to their desired values at t t f , then we have

dt
tf

2
J x(t f ) r (t f ) x(t ) r (t ) u(t )
2 2
Q (t ) R (t )
(2.16)
H
t0

Where, H is a real symmetric positive semi-definite n X n matrix.

6
ECE 551 LECTURE 1

Regulator Problem

This is a special case of the tracking problem where the desired state r (t ) is equal to zero for all

t t0 , t f .

Performance Measure Example

As an example of constructing a performance measure, let us consider an automatic landing of a jet

aircraft. This is a tracking problem. We shall assume the following:

1. Only motion in the vertical (x-y) plane is considered

2. All random disturbances are ignored
3. The glide path angle is small so that cos 1 and sin
4. Aircraft velocity is maintained constant at 160 mph (235 ft/sec) relative to the landing point
5. Aircraft longitudinal motion is controlled by the elevator deflection angle e
6. All dynamics are linearized to form a set of ordinary differential equations (ODE)

Let us make the following state variable and control input assignments:

x1 (t ) h Aircraft altitude

x 2 (t ) h Altitude rate

x3 (t ) Aircraft pitch angle

x 4 (t ) Pitch angle rate

u (t ) e Elevator deflection angle

Thus, the linearized state equations are as follows:

0 1 0 0 0
0 a a 23 0
x (t ) 22 x(t ) 0 u (t ) (2.17)
0 0 0 1 0

0 a 42 a 43 a 44 b4

7
ECE 551 LECTURE 1

The desired trajectory is given by the following:

hd ( t )

450
= 0.064 rad = 3.650

0 30 Time (sec)

7050 0 Range (ft)

d/dt ( hd ( t ) )
30 Time (sec)
0
7050
0 Range (ft)

15

d ( t ) )

0.0873 radian = 50

30 Time (sec)
0
7050
0 Range (ft)
d/dt ( d ( t ) )

30 Time (sec)
0
7050
0 Range (ft)

Thus, a possible performance measure given this is a tracking problem is as follows:

8
ECE 551 LECTURE 1

2
2

J k h h(30) hd (30) k h h(30) hd (30) k (30) d (30)
2

30
q h ( ) h( ) hd ( ) q h ( ) h( ) hd ( ) d
2 2

(2.18)
30
q ( ) ( ) d ( ) q ( ) ( ) d ( ) d
2 2

30
r e ( ) e ( ) ed ( ) d
2

Where, is a dummy variable of integration. Recalling the desired state trajectory, we have that

hd (t ) 450 15t Feet

hd (30) 0 Feet

hd (t ) 15 Feet/sec

hd (30) 15 Feet/sec

d (t ) 0.0873 Radians

d (30) 0.0873 Radians

d (t ) 0 Radians

d (30) 0 Radians

Hence, we have that

9
ECE 551 LECTURE 1

2
2

J k h h(30) k h h(30) 15 k (30) 0.0873
2

30
q h ( ) h( ) 450 15 q h ( ) h( ) 15 d
2 2

(2.19)
30
q ( ) ( ) 0.0873 q ( ) ( ) d
2 2

30
r e ( ) e ( ) d
2

Or, in matrix form

J x(30) r (30) H x(30) r (30)

T

x( ) r( )
30 (2.20)
Q( ) x( ) r ( ) r e ( ) u 2 ( ) d
T

Where,

k h 0 0 0
0 k h 0 0
H
0 0 k 0

0 0 0 0
And,

q h ( ) 0 0 0
0 q h ( ) 0 0
Q( ) (2.21)
0 0 q ( ) 0

0 0 0 q ( )

10