Lagrange Multipliers Basics

R.Y.Qassim

November 2106

1. Introduction

The purpose of this brief technical note is to introduce the

basics of the application of the Lagrange multiplier method
to constrained optimisation problems. This note is based on
[1.2].

2. Constrained optimisation problem

Consider the following constrained optimisation problem.

Optimise y = y(x1, x2,...., xn) , (1)

subject to

1(x1, x2,..., xn) = 0 , (2.1)

..............................

m(x1, x2,..., xn) = 0 . (2.m)

3. Lagrange multiplier method: necessary conditions

The Lagrange multiplier method states that the optimum

occurs at values of the xs that satisfy the following set of
algebraic equations

y/x1 - 1 1/x1 -...- m m/x1 = 0 , (3.1)

..............................................................

y/xn - 1 1/xn -...- m m/xn = 0 . (3.n)

4. Solution procedure
The set of n algebraic equations (2.1)-(2.m) and and the
set of m algebraic equations (3.1)-(3.n), forming a set of
(n+m) algebraic equations, needs to be solved for the same
number (n+m) of unkowns: x1*, x2*,..., xn* and 1*, 2*,..., m*.
The asterisk on the xs na yje s indicate the values of the
xs and the s at the optimum value of the function y.

5. Application example

Consider the following example, which will be solved in a

sequence of steps.

1) Minimise f(x,y,z) = x2 + y2 + z2 ,

subject to

g(x,y,z) = x2 + y2 - z2 = 0 ,

h(x,y,z) = x 2z 3 = 0 .

2) Define the auxiliary function

L(x,y,z,,) = f(x,y,z) + g(x,y,z) + h (x,y,z)

= x2 + y2 + z2 + [x2 + y2 z2] + [x 2z 3] .

3) Solve the following system of algebraic equations.

L/x (x,y,z,,) = 2x + 2x + = 2 (1 + )x + = 0 , (5.1)

L/y (x,y,z,,) = 2x + 2y = 2(1 + )y = 0 , (5.2)

L/z (x,y,z,,) = 2z - 2z - 2 = 2(1-)z - 2 = 0 , (5.3)

L/ (x,y,z,,) = x2 + y2 z2 = 0 , (5.4)

L/ (x,y,z,,) = x 2z 3 = 0 . (5.5)

3) From equation (5.2), we have either y = 0 or = -1.

Case = -1. Then, the remaining equations reduce to

= 0 , (5.1)
4z - 2 = 0 , (5.3)

x2 + y2 z2 = 0 , (5.4)

x 2z 3 0 . (5.5)

Then, we have

(5.1) = 0, (5.3) z = 0, (5.5) x = 3, (5.4) y2 = -9.

In other words, y is complex, which is inacceptable, and

therefore case =-1 is discarded.

Case y = 0. Then, the remaining equations reduce to

2 (1 + ) x + , (5.1)

2 (1 - ) z - 2 , (5.3)

x2 z2 = 0 , (5.4)

x 2z 3 = 0 . (5.5)

From equation (5.4), we have

z = +/- x .

Subcase y = 0, z = x. Then, remaining equations reduce to

2 (1 + ) x + = 0 , (5.1)

2 (1 -) x - 2 = 0 , (5.3)

x 2x 3 = 0 . (5.4)

Then, the solution of this subcase may be written as

(x, y, z, , ) = (1, 0, -1, -3, -12).

Subcase y = 0, z = -x. Then, the remaining equations

reduce to

2 (1 + ) x + = 0 , (5.1)

-2 (1 - ) x - 2 = 0 , (5.3)
x + 2x 3 = 0 . (5.5)

Then the solution of this subcase may be written as

(x, y, z, , ) = (1, 0, -1, -1/3, -4/3).

4) We conclude that, there are two candidate solutions for

the maximum and the minimum: (3, 0, -3) and (1,0, -1),
correspoding to the objective function values of 32 and 2,
respectively; i.e., the maximum and the minimum values of
the objective function, respectively.

6. Lagrange multiplier method: sufficient conditions

For a function L (x1, x2,...,xn, 1, 2,..., m) to have a

constrained minimum (maximum)is that each root of the
polynomiaL zi, defined by the following determinantal be
positive (negative)

L11 z L12... L1n g11 g21... gm1

L21 L22-z... L2n, g12 g21...gm2

. =0 ,
(6.1)

Ln1 Ln2..., Lnn-z g1n g2n...gmn

g11 g12...g1n 0 0...0

g21 g22...g2n 0 0...0

gm1 gm2...gmn 0 0...0

where

Lij = 2L/xixj (x*, *) , (6.2)

gij = gi/xj(x*) . (6.3)

Equation (6.1), on expansion, leads to an (n-m) order

plynomial iz. If some of the roots are positive while the
others are negative, then the point x* is not am extreme
point.

7.1. Application example

Solve the following constrained maximisation problem.

Maximise f(x1, x2) = x12 x2 ,

subject to

2 x12 + 2x1 x2 = A0 = 24 .

1) The Lagrangian function is

L(x1, x2, ) = 2 x12 x2 + (2 x12 + 2 x1 x2 A0) .

2) The necessary conditions for the optimum of L may be

written as

L/x1 = 2 x1 x2 + 4 x1 + 2 x2 = 0 , (7.1)

L/x2 = x12 + 2 x1 = 0 , (7.2)

L/ = 2 x12 + 2 x1 x2 A0 . (7.3)

3) The solution of the algebraic system of equations (7.1)-

(7.3) may be written as

x1* = (A0/6)1/2 ,

x2* = (2A0/3)1/2 ,

* = -(A0/24) ,
4) Then the maximum value of f is given by

f* = (A03/54) ,

and putting A0 = 24, then the optimum solution may be

written as

x1* = 2, x2* = 4, * = -1, f* = 16 .

5) In order to verify that this s solution corresponds to the

maximum of f, we apply the sifficiency condition of Eq.
(6.1).

5) In this case, we have

L11 = 2L/x12 (x*, *) = 2 x2* + 4 * = 4 ,

L12 = 2L/x1x2 (x*, *) = L21 = 2 x1* + 2 * = 2 ,

L22 = 2L/x22 (x*, *) = 0 ,

g11 = g1/x1 (x*, *) = 4 x1* + 2 x2 = 16 ,

g12 = g1/x2 (x*, *) = 2 x1* .

6) Thus, Eq. (6.1) becomes

4 - z 2 16

2 0 z 4

16 4 0

i.e.,

2722 + 1923 = 0 ,

And

z = - 12/17 .

7) Since the value of z is negative, then the point (x 1*, x2*)

corresonds to a maximum of f.

References
[1] Rao, S.S. Engineering Optimization. Theory and Practice.
Fourth Edition, Johhn wiley, 2009.

[2] Hancock, H. theory of Maxima and Minima, FB&c Ltd.,

2016.