1 Fin500J Topic 4 Fall 2010 Olin Business School

Fin500J: Mathematical Foundations in Finance

Topic 4: Unconstrained and Constrained
Optimization
Philip H. Dybvig
Reference: Mathematics for Economists, Carl Simon and Lawrence Blume,
Chapter 17, 18, 19
Slides designed by Yajun Wang
Outline
Unconstrained Optimization
Functions of One Variable
o General Ideas of Optimization
o First and Second Order Conditions
o Local v.s. Global Extremum
Functions of Several Variables
o First and Second Order Conditions
o Local v.s. Global Extremum
Constrained Optimization
Kuhn-Tucker Conditions
Sensitivity Analysis
Second Order Conditions

2 Fin500J Topic 4 Fall 2010 Olin Business School
Unconstrained Optimization
An unconstrained optimization problem is one where
you only have to be concerned with the objective
function you are trying to optimize.
An objective function is a function that you are trying to
optimize.
None of the variables in the objective function are
constrained.
3 Fin500J Topic 4 Fall 2010 Olin Business School
General Ideas of Optimization
There are two ways of examining optimization.
Maximization (example: maximize profit)
In this case you are looking for the highest point on the
function.
Minimization (example: minimize cost)
In this case you are looking for the lowest point on the
function.
Maximization f(x) is equivalent to minimization f(x)






4 Fin500J Topic 4 Fall 2010 Olin Business School
Graphical Representation of a Maximum
5
y
x
y = f(x) = -x
2
+ 8x

8
16
4
Fin500J Topic 4 Fall 2010 Olin Business School
Questions Regarding the Maximum
What is the sign of f '(x) when x < x*?
Note: x* denotes the point where the function is at a
maximum.
What is the sign of f '(x) when x > x*?
What is f '(x) when x = x*?
Definition:
A point x* on a function is said to be a critical point if
f ' (x*) = 0.
This is the first order condition for x* to be a
maximum/minimum.


6 Fin500J Topic 4 Fall 2010 Olin Business School
Second Order Conditions

If x* is a critical point of function f(x), can we decide whether
it is a max, a min or neither?
Yes! Examine the second derivative of f(x) at x*, f ' '(x*);
x* is a maximum of f(x) if f ' '(x*) < 0;
x* is a minimum of f(x) if f ' '(x*) > 0;
x* can be a maximum, a minimum or neither if f ' '(x*) = 0;


7 Fin500J Topic 4 Fall 2010 Olin Business School
An Example of f''(x*)=0
Suppose y = f(x) = x
3
, then f '(x) = 3x
2
and f ''(x) =6x,
This implies that x* = 0 and f ''(x*=0) = 0.
8
y=f(x)=x
3

x
y
Fin500J Topic 4 Fall 2010 Olin Business School
x*=0 is a saddle point where the point is neither a maximum nor a
minimum
Example of Using First and Second
Order Conditions
Suppose you have the following function:
f(x) = x
3
6x
2
+ 9x
Then the first order condition to find the critical
points is:
f(x) = 3x
2
- 12x + 9 = 0
This implies that the critical points are at x = 1 and x = 3.
9 Fin500J Topic 4 Fall 2010 Olin Business School
-0.5 0 0.5 1 1.5 2 2.5 3 3.5 4
-8
-6
-4
-2
0
2
4
Example of Using First and Second
Order Conditions (Cont.)
The next step is to determine whether the critical points
are maximums or minimums.
These can be found by using the second order condition.
f ' '(x) = 6x 12 = 6(x-2)

Testing x = 1 implies:
f ' '(1) = 6(1-2) = -6 < 0.
Hence at x =1, we have a maximum.
Testing x = 3 implies:
f ' '(3) = 6(3-2) = 6 > 0.
Hence at x =3, we have a minimum.

Are these the ultimate maximum and minimum of the
function f(x)?
10 Fin500J Topic 4 Fall 2010 Olin Business School
Local Vs. Global Maxima/Minima
A local maximum is a point that f(x*) f(x) for all x in some
open interval containing x* and a local minimum is a point
that f(x*) f(x) for all x in some open interval containing x*;

A global maximum is a point that f(x*) f(x) for all x in the
domain of f and a global minimum is a point that f(x*) f(x)
for all x in the domain of f.

For the previous example, f(x) as x and f(x) -
as x -. Neither critical point is a global max or min of f(x).


11 Fin500J Topic 4 Fall 2010 Olin Business School
Local Vs. Global Maxima/Minima (Cont.)
When f ''(x)0 for all x, i.e., f(x) is a convex function, then
the local minimum x* is the global minimum of f(x)

When f ''(x)0 for all x, i.e., f(x) is a concave function, then
the local maximum x* is the global maximum of f(x)

12 Fin500J Topic 4 Fall 2010 Olin Business School
Conditions for a Minimum or a Maximum
Value of a Function of Several Variables
Correspondingly, for a function f(x) of several
independent variables x
Calculate and set it to zero.
Solve the equation set to get a solution vector x*.
Calculate .
Evaluate it at x*.
Inspect the Hessian matrix at point x*.
( ) x f V
( ) x f
2
V
( ) ( ) x x H f
2
V =
Fin500J Topic 4 13 Fall 2010 Olin Business School
Hessian Matrix of f(x)
( )
( ) ( )
( ) ( )
( ) ( )
matrix. symmetric a is
H(x) function, C a for equal are partials - cross Since
.
s, n variable of function C a is
2
2
2
1
2
1
2
2
1
2
2
2
(
(
(
(
(
(

c
c
c c
c
c c
c
c
c
= V
n n
n
x
f
x x
f
x x
f
x
f
f
f
x x
x x
x x H
x

Fin500J Topic 4 14 Fall 2010 Olin Business School

Conditions for a Minimum or a Maximum Value of a
Function of Several Variables (cont.)
Let f(x) be a C
2
function in R
n
. Suppose that x* is a critical
point of f(x), i.e., .

1. If the Hessian is a positive definite matrix, then x*
is a local minimum of f(x);
2. If the Hessian is a negative definite matrix, then
x* is a local maximum of f(x).
3. If the Hessian is an indefinite matrix, then x* is
neither a local maximum nor a local minimum of f(x).

( ) * x H
Fin500J Topic 4 15 Fall 2010 Olin Business School
( ) 0 * = V x f
( ) * x H
( ) * x H
Example
xy y x y x f 9 ) , (
3 3
+ =
Fin500J Topic 4 16 Fall 2010 Olin Business School
Find the local maxs and mins of f(x,y)
Firstly, computing the first order partial derivatives (i.e.,
gradient of f(x,y)) and setting them to zero
( ) . 3 3 and (0,0) is * *, points critical
0
9 3
9 3
) , (
2
2
) , - ( y x
x y
y x
y
f
x
f
y x f

=
|
|
.
|

\
|
+
+
=
|
|
|
|
.
|

\
|
c
c
c
c
= V
Example (Cont.)
.
6 9
9 6
) , (
2
2 2
2
2
2
2
|
|
.
|

\
|

=
|
|
|
|
|
.
|

\
|
c
c
c c
c
c c
c
c
c
= V
y
x
y
f
x y
f
y x
f
x
f
y x f
Fin500J Topic 4 17 Fall 2010 Olin Business School
We now compute the Hessian of f(x,y)
The first order leading principal minor is 6x and the second order
principal minor is -36xy-81.
At (0,0), these two minors are 0 and -81, respectively. Since the
second order leading principal minor is negative, (0,0) is a saddle
of f(x,y), i.e., neither a max nor a min.
At (3, -3), these two minors are 18 and 243. So, the Hessian is
positive definite and (3,-3) is a local min of f(x,y).
Is (3, -3) a global min?
Global Maxima and Minima of a Function
of Several Variables
Let f(x) be a C
2
function in R
n
, then

When f(x) is a concave function, i.e., is negative
semidefinite for all x and , then x* is a global
max of f(x);

When f(x) is a convex function, i.e., is positive
semidefinite for all x and , then x* is a global
min of f(x);

18 Fin500J Topic 4 Fall 2010 Olin Business School
( ) x f
2
V
( ) 0 * = V x f
( ) x f
2
V
( ) 0 * = V x f
Example (Discriminating Monopolist)
Fin500J Topic 4 19 Fall 2010 Olin Business School
A monopolist producing a single output has two types of customers. If it
produces q
1
units for type 1, then these customers are willing to pay a price
of 50-5q
1
per unit. If it produces q
2
units for type 2, then these customers
are willing to pay a price of 100-10q
2
per unit.
The monopolists cost of manufacturing q units of output is 90+20q.
In order to maximize profits, how much should the monopolist produce
for each market?
Profit is:
plan. supply maximizing - profit the is (3,4) definite negative is
. 0 , 20 , 10
. 4 0 20 20 100
q
f
, 3 0 20 10 50
q
f
are points critical The
)). ( 20 90 ( ) 10 100 ( ) 5 50 ( ) , (
2
1 2
2
2 1
2
2
2
2
2
1
2
2 2
2
1 1
1
2 1 2 2 1 1 2 1
V
=
c c
c
=
c c
c
=
c
c
=
c
c
= = =
c
c
= = =
c
c
+ + + =
f
q q
f
q q
f
q
f
q
f
q q q q
q q q q q q q q f
Constrained Optimization
Examples:
Individuals maximizing utility will be subject to a
budget constraint
Firms maximising output will be subject to a cost
constraint

The function we want to maximize/minimize is called
the objective function
The restriction is called the constraint
Fin500J Topic 4 Fall 2010 Olin Business School 20
Constrained Optimization (General Form)
A general mixed constrained multi-dimensional
maximization problem is
=
s s s
= = =
1
1 1 1 2 1 2 1 ,
1 1 1 2 1 2 1
max ( ) ( ,..., )
subject to
( ,..., ) , ( ,..., ) , , ( ,..., )
( ,..., ) , ( ,..., ) , , ( ,..., ) .
n
n n k n k
n n m n m
f x f x x
g x x b g x x b g x x b
h x x c h x x c h x x c
Fin500J Topic 4 21 Fall 2010 Olin Business School
Constrained Optimization (Lagrangian Form)
The Lagrangian approach is to associate a Lagrange
multiplier
i
with the i
th
inequality constraint and
i
with
the i
th
equality constraint.

We then form the Lagrangian
( )
( )

=
=
=

1 1 1
1 1
1
1
1
( ,..., , ,..., , ,..., )
( ,..., ) ( ,..., )
( ,..., ) .
n k m
k
n i i n i
i
m
i i n i
i
L x x
f x x g x x b
h x x c
Fin500J Topic 4 22 Fall 2010 Olin Business School
Constrained Optimization (Kuhn-Tucker
Conditions)
If is a local maximum of f on the constraint set
defined by the k inequalities and m equalities, then,
there exists multipliers satisfying
*
x

* * * *
1 1
, , , , ,
k m
Fin500J Topic 4 23 Fall 2008 23 Fall 2010 Olin Business School
( )
* * * * * *
* *
1 1
*
* *
*
( ) ( ) ( , , ) ( )
0, 1, , n
( ) , 1, ,
( ) 0, 1, ,
( ) , 1, ,

k m
i i
i i
i i
j j j j
i i
i i i
i i
g x h x L x f x
j
x x x x
h x c i m
g x b i k
g x b i k

= =
c c c c
= = =
c c c c
= =
= =
s =

*
0, 1, ,
i
i k > =
Fin500J Topic 4
Constrained Optimization (Kuhn-Tucker
Conditions)
The first set of KT conditions generalizes the
unconstrained critical point condition
The second set of KT conditions says that x needs to
satisfy the equality constraints
The third set of KT conditions is


That is to say
> =
* *
if 0 then ( )
i i i
g x b
< =
* *
i
if ( ) then 0
i i
g x b
Fin500J Topic 4 24 Fall 2008
( )
* *
( ) 0, 1, ,
i i i
g x b i k = =
24 Fall 2010 Olin Business School Fin500J Topic 4
Constrained Optimization (Kuhn-Tucker
Conditions)
This can be interpreted as follows:
Additional units of the resource b
i
only have value if the
available units are used fully in the optimal solution, i.e., if
the constraint is not binding thus it does not
make difference in the optimal solution and
i
*=0.

Finally, note that increasing b
i
enlarges the feasible region,
and therefore increases the objective value
Therefore,
i
>0 for all i
Fin500J Topic 4 25 Fall 2008
<
*
( )
i i
g x b
25 Fall 2010 Olin Business School Fin500J Topic 4
Example

+ =
> >
2
2 2
max
4
0, 0.
x y
subject to
x y
x y
Fin500J Topic 4 26 Fall 2008





+ + +
c
= + =
c
c
= + =
c
+ =
= =
> >
> >
2 2 2
1 2
1
2
2 2
1 2
1 2
Form the Lagrangian
L=x-y ( 4) .
The first order conditions become:
(1) 1 2 0,
(2) 2 2 0,
(3) 4 0
(4) 0, (5) 0,
(6) 0, (7) 0,
(8) 0, (8) 0.
x y x y
L
x
x
L
y y
y
x y
x y
x y
26 Fall 2010 Olin Business School Fin500J Topic 4
Example (cont.)
Fin500J Topic 4 27 Fall 2008

+ = > + >
> > =
= + + >

= =
=
1 1 1
1
2
2
2
1
By (1), 1 2 , since 0, 1 0
0 and 0, by (4), 0.
by (2), 2 (1 ), since 1 0
either both and are zero, or both
are positive, from (5) 0, 0.
By (3) and (8) x=2, from (4), 0,
(
x
x
y
y
y
by = =
1 2
1 1
1), . So, ( , , , , ) (2, 0, , 0, 0).
4 4
x y
27 Fall 2010 Olin Business School Fin500J Topic 4
Sensitivity Analysis
We notice that


What happens to the optimal solution value if the
right-hand side of constraint i is changed by a small
amount, say Ab
i ,
i=1

k
.
or Ac
i ,
i=1
.
m.
It changes by approximately or
is the shadow price of i
th
inequality constraint and
( )

'
= = ( *, *, *) ( *) * ( *) *'( ( *) ) ( *) L x f x g x b h x c f x
*
i i
b A
*
i

Fin500J Topic 4 28 Fall 2010 Olin Business School

A
*
i i
c

*
i
is the shadow price of i
th
equality constraint
Sensitivity Analysis (Example)
In the previous example, if we change the first
constraint to x
2
+y
2
=3.9, then we predict that the new
optimal value would be 2+1/4(-0.1)=1.975.
If we compute that problem with this new constraint,
then x-y
2
=
If, instead, we change the second constraint from x0
to x0.1, we do not change the solution or the
optimum value since

Fin500J Topic 4 29 Fall 2010 Olin Business School
=
*
1
0.
= 3.9 1.9748.
Utility Maximization Example
( )
( )
0 4
0
0 2
: Conditions Order - First
4 100 )
is function Lagrangian Our
exercise. and movies watch to available are day per hours Four
100
function by the described is
(M) movies watching and (X) exercise from derived utility The
2
2
2
= + =
= =
= =
+ =
=

M X L
e L
e L
M X e e L(X,M,
e e X,M U

M
M
X
X
M X
M X

Fin500J Topic 4 30 Fall 2010 Olin Business School

Utility Max Example Continued
3
) 2 ln( 8
3
4 ) 2 ln(
X
4 ln(2) 3X or X -4 2X - ln(2) 0 2e
get we ng, substituti and for (3) Solving 0 2
get we (1), into ng Substituti . get that we (2) From
) 3 ( 0 4 ) 2 ( , 0 ) 1 ( , 0 2
: Conditions Order - First
* *
) 4 ( 2X -
2
2

=
+
=
+ = + = =
=
=
= + = = = = =

M and
e
M e e
e
M X L e L e L
X
M X -
-M

M
M
X
X
Fin500J Topic 4 31 Fall 2010 Olin Business School