Rev.

9/19/2012

Chapter 5: Duality Theory

5.1 Motivation
To find bounds of the objective function value:
Ex:
m a x z 4 x1

x 2 3 x3

x1 4 x 2

s/t

U.B.

3 x1

optimal

x 2 x3

x 2 , x3 0

x1 ,

max
L.B.

a feasible solution

Q: How close is my current solution to the optimal?

LBD z z U BD ( z )

How small is my dual gap? z UBD LBD z z

Primal Bound (or Lower Bound) can be constructed using a primal feasible solution.
Q: How to find a feasible point for the LP?

0, 0, 0

z 0,

z 0 4 x1 x 2 3 x 3

1, 0, 0

z 4,

z 4 4 x1 x 2 3 x 3

0, 0, 3

z 9,

z 9 4 x1 x 2 3 x 3

Q: How good is my current solution to the optimal solution?

Lets find an upper bound of z.
Dual Bound (or Upper Bound) can be constructed using a dual feasible solution.
x1

4 x2

3 x1

x2

x3

y1 3 y 2 x1 4 y1 y 2 x 2
4

y1 0

y2 0

y 2 x 3 y1 3 y 3 Fact 1
3 0

Since x i 0

Fact 2:
y1 3 y 2

4 y1 y 2

y2

z 4 x1 x 2 3 x 3 y1 3 y 2 x1 4 y1 y 2 x 2 y 2 x 3
y1 3 y 3

Gino J. Lim, Ph.D., University of Houston

Rev. 9/19/2012

Therefore, Dual Problem of the primal LP is

m in y1 3 y 3
s / t y1 3 y 2 4

4 y1 y 2 1
y2 3
y2 0

y1 ,

Any feasible solution of this dual will provide an upper bound to the LP.

2, 3

z 11,

z 4 x1 x 2 3 x 3 11 z

1, 3

z 10,

z 4 x1 x 2 3 x 3 10 z

Q: Which upper bound is better?

Gino J. Lim, Ph.D., University of Houston

Rev. 9/19/2012

5.2 The Dual Problem

(Primal)
(Given problem)

m ax

c1 x1 c 2 x 2

s/t

a11 x1 a12 x 2

a1 n x n b1

y1

a 21 x1 a 22 x 2

a 2 n x n b2

y2

a m 1 x1 a m 2 x 2
x1 ,

cn xn

a m n x n bm

x2 ,

xn

ym

(Dual)
m in

b1 y1 b 2 y 2

s/t

a11 y1 a 21 y 2

a m 1 y m c1

a12 y1 a 22 y 2

a m1 y m c2

a1 n y1 a 2 n y 2

a mn ym cm

y1 ,

y2 ,

bn y n

ym

Example:
(P)

m ax

z 5 x1 12 x 2 4 x 3

s/t

x1 2 x 2 x 3 10
2 x1 x 2 3 x 3 8

(Pay attention to this)

x1 , x 2 , x 3 0

Write (P) in equation form:

(P)
m ax
z 5 x1 12 x 2 4 x 3 0.w1
s/t

x1 2 x 2 x 3 w1 10 ( y1 : Dual variable corresponding to the primal equation 1)

2 x1 x 2 3 x 3

( y 2 : Dual variable corresponding to the primal equation 2)

x1 , x 2 , x 3 , w1 0

(D)

m in

10 y1 8 y 2

s/t

y1 2 y 2 5
2 y1 y 2 12
y1 3 y 2 4
y1

0 y 2 free

Gino J. Lim, Ph.D., University of Houston

Rev. 9/19/2012

Example
m ax

z 5 x1 6 x 2

s/t

x1 2 x 2 5

x1 x1 x1

Substitute

x1 , x1 0

x1 5 x 2 3

4 x1 7 x 2 8

Normal

x1 free , x 2 0

Write equation form using x1 , x1

z 5 x1 5 x1 6 x 2

s/t

x1 x1 2 x 2

4 x 4 x 7 x2

x1 x1 5 x 2 w1

x1 x1 x1

m ax

w2

x1 , x1 , x 2 , w1 , w 2 0

(Dual)
m in
s/t

5 y1 3 y 2 8 y 3
y1 y 2 4 y 3 5
y1 y 2 4 y 3 5

2 y1 5 y 2 7 y 3 6

y2
0
y 3 0

y1 y 2 4 y 3 5

y 2 0, y 3 0
y1 free

Max LP
Equations

Variables

Min LP

0
0

free

0
0

free

Gino J. Lim, Ph.D., University of Houston

Variables

Equations

Rev. 9/19/2012

5.3 The weak duality Theorem

min
problem

dual gap

max
problem

Consider an LP:
<P> max c ' x
Ax b
s/t

max
s/t

c1 x1 c 2 x 2

cn xn

a11 x1

a1 n x n b1

a m 1 x1

a m n x n bm

x0

<D>

min
s/t

x1 ,

, xn 0

b'y
A' y c
y0

Theorem 5.1
Let
x x1 , x 2
y y1 , y 2

x n be a feasible solution for <P> and

y n be a feasible solution for <D>.

Then c ' x b ' y

Proof:
From dual constraint c A ' y

A y

c'x A' y 'x

c ' x y '( A x ) y ' b b ' y
c'x b'y

When c ' x y ' b , then

is an optimal solution for <P>.

y is an optimal solution for <D>.
Strong duality theorem.
x

Gino J. Lim, Ph.D., University of Houston

Rev. 9/19/2012

5.4 The strong duality theorem

Theorem 5.2
*
*
*
Let
x x1 , x 2 ,
y y1 ,
*

, x n be optimal solution for <P> and

*

y m be optimal solution for <D>.

Then c T x * b T y *
Proof:
Suppose we apply the simplex method. Assume that an optimal solution exits. The proof
is enough to show that a dual feasible solution y * satisfies c ' x * b ' y *
x 1 x n x n 1 x m

1
Z Z

C 1* C n* d 1* d

in optimal tableau

*
n

BU

= c j x *j

j B

= cjx

cjxj
*

j N

x n 1

i 1

Optimal primal objective function value.

*
j

j B

Let

cj

di

for the objective coefficients in the final tableau.

for the objective coefficients in the final tableau (slacks).
n

Then

Z Z
*

cjxj
*

j 1

wi

i 1

y i d i , i 1, 2,

Let

*
i

,m

Need to show that y * y1* , y 2* ,

c'x b'y
*

, y m is feasible for dual problem and satisfies

Z
*

cjxj

j 1

cjxj
*

j 1

*
j

xj

by
i

i 1

1. Z *

by
i

i 1

*
i

*
i

y b a
i 1

xni

c
j 1

*
i

i 1

c
j 1

*
j

i 1

ij

j 1

xi

*
y i a ij x j

*
i

xj

c'x b'y
*

Optimality

i 1

Gino J. Lim, Ph.D., University of Houston

Rev. 9/19/2012

2. c j c *j y i* a ij
i 1
m

c j c j y i a ij 0
*

By definition

i 1

y i a ij c j
*

i 1

3. y i* d i* 0

yi 0
*

Primal-dual Relationships
Based on the weak duality theorem and the strong duality theorem, following four cases
are possible:

Example:
Max 2 x1 x 2
s/t
x1 x 2 1

1
2
3
4

Primal
Optimal
Infeasible
Unbounded
Infeasible

Min
s/t

Dual
Optimal
Unbounded
Infeasible
Infeasible*
y1 2 y 2
y1 y 2 2

x1 x 2 2

y1 y 2 1

x1 , x 2 0

y1 , y 2 0

Therefore, the optimality conditions for LP (strong duality theorem)

1. Primal feasible
2. Dual feasible
3. c ' x * b ' y *
What if we know the solution for the dual (or primal) and need to recover the
corresponding primal (or dual) solution?
Use complementary slackness.

Gino J. Lim, Ph.D., University of Houston

Rev. 9/19/2012

5.5 Complementary Slackness

Theorem 5.3
Suppose that x x1 , x 2 ,

, x n is primal feasible and

y y1 , y 2 ,

x n 1 ,
y n 1 ,

, y n is dual feasible.

, x n m is primal slack variables and

, y n m is dual slack variables.

Then x and y are optimal for their respective problems if and only if (iff for short)

x j ym j x j c j

ij

i 1

y i 0, j 1, 2,

y i x n i y i a ij x j bi 0, i 1, 2,
j 1

,n

,m

Proof:
i.
From the weak duality
c'x A' y'x

xj

ij

yi x j

Equality (=) holds iff for each j 1, 2,

either x j 0
cj

or

ij

yi

xj cj

a
i

ij

,n

yi 0

x j y m j 0, j 1, 2,

ii.

Again Ax ' y b ' y

a
i

ij

x j yi

by
i

Equality (=) holds iff

either y i 0
or

bi

a
j

ij

xj

y i a ij x j bi bi 0
j 1

y i x n i 0, i 1, 2,

Gino J. Lim, Ph.D., University of Houston

Rev. 9/19/2012

For example
<P> max
s/t

5 x1 4 x 2

<D>

24 y1 6 y 2 y 3 2 y 4

min
s/t

6 x1 4 x 2 24

6 y1 y 2 y 3 5

x1 2 x 2 6

4 y1 2 y 2 y 3 y 4 4

x1 x 2 1

y1 , y 2 , y 3 , y 4

x2 2
x1 , x 2 0

Given a dual optimal solution:

y1

3
4

, y2

1
2

, y 3 y 4 0, 2 1

Find a primal optimal solution?

Gino J. Lim, Ph.D., University of Houston

Rev. 9/19/2012

5.6 The Dual Simplex Method

A dual dictionary can be constructed from a primal dictionary.
Example: in p.55

(P)

m ax

4 x1

x1 4 x 2

s/t

x1 ,

x4

1 -

x5

4 y1 -

y1 ,

y2

4
1

y2

y2

(maximization problem)

4 x1 x 2

3 x3

x1 - 4 x 2

not optim al
feasible

3 - 3 x1 x 2 -

Dual Dictionary

x3

(minimization problem)

m in y1

3 y2

y3

-4

y1 3 y 2

y4

- 1 4 y1 -

y5

-3

- m ax ( - ) - y1 3 y 2

y2

* N o entering v ariable
* N ot feasible

y2

1 x1 x 2 x 3

P =

y1 3 y 2
y1 3 y 2

x3 0

x2 ,

Primal Dictionary

m in
s/t

x 2 x3

3 x1 -

(D )

x 2 3 x3

0 4 1 3

1 -1 -4 0

3 -3 1 -1

z
x 4 y1

D = -P =

x5 y 2

y1

1
1
4
0

y2

y3
y4
y5

Which simplex method to apply?

Primal Simplex Method if:
Start with a solution that is
not optimal but feasible

Dual Simplex Method if:

Start w/ a solution that is not feasible but optimality
condition is satisfied

Optimal & feasible

z

* same objective function value: strong duality

Gino J. Lim, Ph.D., University of Houston

10

Rev. 9/19/2012

Another Example:
(P ) m ax

- x1 -

s/t

- 2 x1 -

x2

Q:

x2

- 2 x1 4 x 2 - 8

(D ) m in

4 y1 - 8 y 2 - 7 y 3

s/t

- 2 y1 - 2 y 2 - y 3

-1

- y1 4 y 2 3 y 3

-1

- x1 3 x 2 - 7
x2

x1 ,

y1 ,

y2 ,

y3

min = -max -

Write dictionaries
(P )

- x1 -

x2

x3

4 2 x1 x 2

x4

- 8 2 x1 - 4 x 2

x5

-7

(D )

x1 - 3 x 2

y4

1 - 2 y1 - 2 y 2 -

y5

1 -

- 4 y1 8 y 2 7 y 3
y3

y1 4 y 2 3 y 3

feasible "N ot optim al"!

infeasible and no m ore entering var"

(P rim al S im plex)

(D ual S im plex)

1) Dual optimality condition:

Select a basic variable as the leaving variable xk s.t
b k (or RHS) has the most negative value (k is an index, 1, 2, , m)
If all RHS values are positive, then stop.
The current solution is dual optimal
Ex). x 4 , b2 - 8 is m ost negative
2) Dual feasibility condition:
Select a non basic variable as the entering variable s.t.
m in

ex )

i non-basic variables |

x1,

ratio

ci
a ki

, a ki < 0, c i

1
2

Gino J. Lim, Ph.D., University of Houston

11

Rev. 9/19/2012

-4 -

x4

2
x3

x1

x5

12
4

x4 5 x2
1
2

-3

- 3 x2

x4

1
2

2 x2

x4 -

y3

y2

y5

y1 -

2
3

5 y1 -

y4

1
2

2 y4

y3
y3

y5

-7 -

x3

18

2 x5 7 x 2

x1

x5 3 x 2

x4

2 x5 +

x4

7 - 18 y1 - 7 y 4 - 6 y 2

y1

y3

1 -

2 y1 -

y4

y5

4 -

7 y1 - 3 y 4

y4

- 2 y 2 x5 0
- 2 y2 x2 0

y2

2 x2

x 2 0,

x1

- 4 x2

optim al

feasible
x1 7,

3 y3

x2

x3
x5

4 y4

- 1 2 y1 -

both
z

-7

optim al

feasible
y 3 1, y 5 4
y1 y 2 y 4 0
- 7 - 7

z -7

Gino J. Lim, Ph.D., University of Houston

12

Rev. 9/19/2012

5.7 A generalized simplex algorithm

Example:
- x1

m ax
s/t

4 x2
x2 4

- 2 x1 -

N ot optim al

&

N ot feasible

- 2 x1 4 x 2 - 8
- x1 3 x 2 - 7

x2 , 0

x1 ,

Phase I: Apply the dual simplex method to make the problem feasible.

- x1 4 x 2

x3

x4

-8

2 x1 - 4 x 2

x5

-7

x1 - 3 x 2

2 x1

x2

x1

enters

x4

leaves

x4

en ters

x5

lea ves

-4 -

x4 2 x2

2
x3

12

x4 5 x2

x1

x5

x3

18

x5

x1

x5 3 x 2

x4

6 2 x5

2
-3

x4 2 x2

1
2

-7

x4 -

x5

x2

2 x2
2 x2

feasible but no t optim al yet!

2 x2
U nbounded over x 2

Finally, apply the primal simplex method to find the optimal solution. This is left for your
exercise problem.

Gino J. Lim, Ph.D., University of Houston

13

Rev. 9/19/2012

Test Problems 1
P m in
s /t

x1

2 x 2 3 x3

3 x2

- x1

2 x1 -

x 2 3 x3 6
x3 4

x1 0,
D
s/t

m ax

x 2 0,

x 3 free

5 y1 6 y 2 4 y 3
- y1
3 y1 -

2 y2

y2

3 y2 y3
y1 free ,

y 2 0,

y3 0

Show that dual of the dual of the primal LP is the primal LP

Gino J. Lim, Ph.D., University of Houston

14

Rev. 9/19/2012

Test Problem 2:
3 x1 2 x 2 5 x 3

<P > m ax

x1 2 x 2

s/t

3 x1

x3

430

2 x3

460

x1 4 x 2
x1 ,
<D >

m in
s /t

x2 ,

420
x3 0

430 y1 460 y 2 420 y 3

y1

3 y2

y3 3

4 y3 2

2 y1
y1

y1 ,

2 y2
y2 ,

y3

Let the optimal solution for <D> is given as

y1 = 1,

y 2 = 2, y 3 = 0

objective function value = 1350

Find the primal optimal solution?

Gino J. Lim, Ph.D., University of Houston

15