Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
1
Quadratic forms
ak 1 ak 2 ... akk
2
Quadratic forms
4
Local maximum/Local minimum
6
T
Gradient vector f , f , , f
x x xn
1 2
2 f 2 f 2 f
. .
x1 x1x2 x1xn
2
2 f 2 f 2 f
. .
x x x22 x2xn
Hessian matrix H ( x) 1 2 .
.
.
2
f 2 f f
2
. . .
xn x1 xn x2 xn2
7
Stationary point
8
Inflection and Saddle Points
9
Theorem
10
Theorem (necessary and Sufficient condition)
12
f is said to be strictly convex if for each pair of points
X,Y on the graph,
f ((1-) X + Y) < (1-) f(X)+ f(Y)
for all such that 0 < < 1.
13
Convex/Concave function
d2 f
convex if 2
0
dx
d2 f
concave if 2
0
dx
15
Constrained optimization
KarushKuhnTucker (KKT) conditions:
17
KKT conditions
L
f ( X ) g ( X ) 0
X
L
2 s 0
s
L
( g ( X ) s 2 ) 0
18
KKT conditions
19
KKT conditions
i gi ( X ) 0
gi ( X ) 0 i=1,2,.m
20
Sufficiency of the KKT conditions:
Required conditions
Sense of optimization Objective function Solution space
maximization concave Convex set
minimization convex Convex set
21
Problem 1: Use the KKT conditions to derive an
optimal solution for the following problem:
maximize f(x1, x2) = x1+ 2x2 x23
subject to x1 + x2 1
x1 0
x2 0
Solution: Here there are three constraints namely,
g1(x1,x2) = x1+x2 - 1 0
g2(x1,x2) = -x1 0
g3(x1,x2) = -x2 0
22
Hence the KKT conditions become
f g1 g 2 g 3
1 2 3 0
x1 x1 x1 x1
f g1 g 2 g 3
1 2 3 0
x 2 x 2 x2 x 2
1g1(x1,x2) = 0
2g2(x1,x2) = 0 (note: f is concave, gi are
3g3(x1,x2) = 0 convex, maximization problem
g1(x1,x2) 0 these KKT conditions are
g2(x1,x2) 0 sufficient at the optimum point)
g3(x1,x2) 0 and 10, 20, 30
23
i.e. 1 1 + 2 = 0 (1)
2 3x22 1 + 3 = 0 (2)
1(x1 + x2 1) = 0 (3)
2 x1 =0 (4)
3 x2 =0 (5)
x1 + x2 1 0 (6)
x1 0 (7)
x2 0 (8) 1 0 (9)
2 0 (10) and 3 0 (11)
24
(1) gives 1 = 1 + 2 1 >0 (using 10)
Hence (3) gives x1 + x2 = 1 (12)
Thus both x1, x2 cannot be zero, therefore let x1>0, then (4)
gives 2 = 0. therefore 1 = 1
if now x2 = 0, then (2) gives 2 0 1 + 3 = 0
=> 3 < 0 not possible
Therefore x2 > 0, hence (5) gives 3 = 0 and then (2) gives
x22 = 1/3 => x2 =1/3.
And so x1 = 1- 1/3. Therefore
Max f = 1 - 1/3 + 2/3 1/33 = 1 + 2/33.
25
Problem 2:
Maximize f(x) = 20x1 + 10 x2 (0,1)
26
4 x2 =0
x12 + x22 1
x1 + 2x2 2
x1 0
x2 0
1 0
2 0
3 0
4 0
27
From the figure it is clear that max f occurs at (x1, x2) where
x1, x2 >0. Therefore, we have 3 = 0, 4 = 0.
Suppose x1 + 2x2 2 0
2 = 0 , therefore we get 20 - 21 x1=0
10 - 21 x2=0
1x1=10 and 1x2=5, squaring and adding we get
12 = 125 1 = 55
therefore x1 = 2/5, x2 = 1/5, f = 50/5 > 22
28
2 0 x1 +2 x2 2 = 0
Therefore x1=0, x2=1, f =10
Or x1= 4/5, x2= 3/5, f=22
Therefore max f occurs at x1 = 2/5, x2 = 1/5
29
Problem 3: Use the KKT conditions to derive an optimal
solution for the following problem:
minimize f(x1, x2) = x12+ x2
subject to x12 + x22 9
x1 + x2 1
Solution: Here there are two constraints namely,
g1(x1,x2) = x12+x22 - 9 0
g2(x1,x2) = x1 + x2 -1 0
30
Thus the KKT conditions are:
1 : 1 0, 2 0 as it is a minimization problem
2: 2x1 - 21x1 - 2 = 0
1 - 21x2 - 2 = 0
3 : 1 ( x12 x22 9) 0
2 ( x1 x2 1) 0
4 : x12 x22 9
x1 x2 1
31
Now 1 0 (from 2) gives 2 1 Not possible.
Hence 1 0 and so x x 9
2
1
2
2
(5)