Professor Dr.

Sebastian Engell
Process Dynamics and Operations Group

4.3 Problems with Inequality Constraints

General form: min f (x )
x

h (x )
1
subject to: h(x ) = = 0
h ( x )
m

and

g1( x )

g (x ) = 0

g (x )
p

Idea: Extend the Lagrangian by multipliers for the inequalities

General Lagrange function: L( x , ) = f (x ) + T h(x ) + T g (x )

1

with the Kuhn-Tucker multipliers: = , i 0


p

Process Optimization

20114.3.1

Professor Dr. Sebastian Engell

Process Dynamics and Operations Group

Necessary Conditions for a Local Minimum

Kuhn-Tucker conditions:

(
(
g (x* ) 0

(*, * ) exist such that:

)
( ) ( )
) ( )
and g i (x* ) i* = 0, i* 0

( )

h
g *

x L x* , * , * = x f x* +
x* * +
x * = 0
x

L x* , * , * = h x* = 0

If an inequality gi(x) is:

(a) inactive:
(b) just active:
(c) strictly active:

( )
g i (x* ) = 0
g i x* < 0

for i = 1, , p
x2

(a)
f

(b)

i = 0

(c)

i 0
i > 0

x*
gi(x) 0
x1

Process Optimization

20114.3.2

Professor Dr. Sebastian Engell

Process Dynamics and Operations Group

Sufficient Conditions
In addition to the necessary conditions:
For a variation v = x - x*: v T 2xx L(x*, *, * ) v > 0
T

g a *

*
where:
x v = 0,
x v = 0,
x

h *
strictly active
x

x
must have full rank.

J
=
and
a
g

*
x

x2
x

( )
( )

( )

( )

balance of the forces:

xf , xh ,

xga

xga

g j *

x v 0
x

( )

just active

xf
xh

g2(x)

h(x)

where , are weights

g1(x)

Process Optimization

x1
20114.3.3

Professor Dr. Sebastian Engell

Process Dynamics and Operations Group

Nonlinear Programming
The analytical solution is often intractable!
Then alternatively: numerical solution by Nonlinear Programming
Several methods exist:

...

quadratic programming
penalty function methods
methods based on augmented Lagrangians
generalized reduced gradient method
sequential quadratic programming
global minimization methods (e.g., simulated annealing)
random search methods
evolutionary / genetic algorithms

Process Optimization

20114.3.4

Professor Dr. Sebastian Engell

Process Dynamics and Operations Group

Generalized Reduced Gradient Method

Idea: Treat active inequality constraints as equality constraints
and ignore inactive inequality constraints (in each step)
Scheme:
(1) Choose a feasible x(0), k = 0
(2) Add active inequalities to the equality constraints

Feasible path
approach

(3) Determine a search direction by using the negative

reduced gradient (considering the equalities)
(4) Line search along the direction; check
whether inactive constraints become active

~ (k +1)
> go to (2) with k = k+1

(5) If x 2 f x

x2

(6) Check whether f(x) can be reduced by moving

away from the inequality constraints: determine
* from the KT-conditions
remove constraints with negative i*

x~
f
f

x(0)

g1(x)

Terminate if all i* 0, else go to (2) with k = k+1

Process Optimization

g2(x)

20114.3.5

x1

Professor Dr. Sebastian Engell

Process Dynamics and Operations Group

Sequential Quadratic Programming

Idea: Extend the Newton method to constrained problems by
using the KT conditions
Scheme:
(1) Choose x(0); k = 0
(2) Linearize the KT-conditions for the active constraints in x(0)
quadratic program: solution is the optimal direction d
(3) Line search in direction d (constrained violation is penalized)
determines the step size x(k+1)
(4) Check termination criterion:
Terminate if x L x (k ), (k ) <

else go to (2) with k = k+1

Characteristics: not every x(k) is feasible

infeasible path approach

good convergence for suitable x(0)

Process Optimization

20114.3.6