The Strong Duality Theorem1

Adrian Vetta

1
1/39

This presentation is based upon the book Linear Programming by Vasek Chvatal

Part I
Weak Duality

2/39

Primal and Dual

Recall we have a primal linear program
Pn
max
c x
Pnj=1 j j
s.t.
a
i [m] = {1, 2, . . . , m}
j=1 ij xj bi
xj
0
j [n] = {1, 2, . . . , n}
and its dual linear program
Pm
min
i=1 bi yi
Pm
s.t.
i=1 aij yi
yi

3/39

cj
0

j [n]
i [m]

Weak Duality Theorem

The Weak Duality Theorem.

For any primal feasible solution (x1 , x2 , . . . , xn ) and any dual feasible
solution (y1 , y2 , . . . , ym ) we have
n
X
j=1

4/39

cj xj

m
X
i=1

bi yi

Proof of Weak Duality

Weak Duality Theorem.
For any primal feasible solution (x1 , x2 , . . .P
, xn ) and any P
and any dual
feasible solution (y1 , y2 , . . . , ym ) we have nj=1 cj xj m
i=1 bi yi .
Proof.

n
X

xj cj

j=1

n
X
j=1
m
X
i=1

m
X
i=1

5/39

xj
yi

m
X
i=1
n
X
j=1

yi bi

aij yi

aij xi

Part II
Strong Duality

6/39

Strong Duality Theorem

The Strong Duality Theorem.

If the primal has an optimal solution (x1 , x2 , . . . , xn ) then the dual has an
) such that
optimal solution (y1 , y2 , . . . , ym
n
X
j=1

7/39

cj xj =

m
X
i=1

bi yi

An Example

8/39

Consider the primal

maximise
subject to

4x1
x1
5x1
x1

+ x2
x2
+ x2
+ 2x2

+ 5x3 +
x3 +
+ 3x3 +
+ 3x3
x1 , x2 , x3

3x4
3x4
8x4
5x4
, x4

1
55
3
0

and its dual

minimise
subject to

y1
y1
y1
y1
3y1

+ 55y2 +
+ 5y2
+ y2
+
+ 3y2 +
+ 8y2
y1 , y2 ,

3y3
y3
2y3
3y3
5y3
y3

4
1
5
3
0

Optimal Solutions
maximise
subject to

4x1
x1
5x1
x1

+
+

x2
x2
x2
2x2

+
+
x1 ,

5x3
x3
3x3
3x3
x2 ,

+
+
+

x3

3x4
3x4
8x4
5x4
, x4

1
55
3
0

The primal has optimal solution (x1 , x2 , x3 , x4 ) = (0, 14, 0, 5) with

value 29.
minimise
subject to

y1
y1
y1
y1
3y1

+
+
+
+
+

55y2
5y2
y2
3y2
8y2
y1 ,

+
+

y2 ,

3y3
y3
2y3
3y3
5y3
y3

4
1
5
3
0

The dual has optimal solution (y1 , y2 , y3 ) = (11, 0, 6) with value 29.

9/39

Solving the Primal

10/39

Lets solve the primal by the simplex method.

maximise
subject to

4x1
x1
5x1
x1

+ x2
x2
+ x2
+ 2x2

+ 5x3 +
x3 +
+ 3x3 +
+ 3x3
x1 , x2 , x3

3x4
3x4
8x4
5x4
, x4

1
55
3
0

Put into dictionary form.

z
x5
x6
x7

=
4x1
= 1 x1
= 55 5x1
= 3 + x1

+ x2
+ x2
x2
2x2

+ 5x3
+ x3
3x3
3x3

3x4
3x4
8x4
5x4

Pivot 1
z
x5
x6
x7

=
=
=
=

1
55
3

4x1
x1
5x1
x1

+
+

x2
x2
x2
2x2

+
+

5x3
x3
3x3
3x3

3x4
3x4
8x4
5x4

Pivot (Blands Rule) Add x1 to basis. (Lowest index.)

Remove x5 from basis. (No choice.)
z
x1
x6
x7

=
=
=
=

1
55
3

z
x1
x6
x7
11/39

4(1 + x2 + x3 3x4 x5 )
x2
5(1 + x2 + x3 3x4 x5 )
(1 + x2 + x3 3x4 x5 )

= 4 + 5x2
= 1 + x2
= 50 6x2
= 4 x2

+ 9x3
+ x3
8x3
2x3

+
+

x2
x3
x2
2x2

+
+

9x4
3x4
7x4
2x4

5x3
3x4
3x3
3x3

4x5
x5
+ 5x5
x5

3x4
x5
8x4
5x4

Pivot 2

12/39

z
x1
x6
x7

=
=
=
=

4
1
50
4

+
+

5x2
x2
6x2
x2

+
+

9x3
x3
8x3
2x3

+
+

9x4
3x4
7x4
2x4

4x5
x5
5x5
x5

Pivot: Add x2 to basis. (Lowest index.)

Remove x7 from basis. (No choice.)
z
x2
x1
x6

=
=
=
=

4
4
1
50

z
x2
x1
x6

5(4 2x3 + 2x4 x5 x7 )

2x3
(4 2x3 + 2x4 x5 x7 )
6(4 2x3 + 2x4 x5 x7 )

= 24 x3
= 4 2x3
= 5 x3
= 26 + 4x3

+ x4
+ 2x4
x4
5x4

+
+
+

9x3
2x4
x3
8x3

9x5
x5
2x5
+ 11x5

9x4
x5
3x4
7x4

5x7
x7
x7
+ 6x7

4x5
x7
x5
5x5

Pivot 3

13/39

z
x2
x1
x6

=
=
=
=

24
4
5
26

x3
2x3
x3
4x3

+
+

x4
2x4
x4
5x4

9x5
x5
2x5
11x5

5x7
x7
x7
6x7

Pivot: Add x4 to basis. (No choice.)

Remove x1 from basis. (No choice.)
z
x4
x2
x6

=
=
=
=

24
5
4
26

z
x4
x2
x6

x3
x3
2x3
4x3

(5 x3 2x5 x7 x1 )
2x5
2(5 x3 2x5 x7 x1 )
5(5 x3 2x5 x7 x1 )

= 29 2x3
= 5 x3
= 14 4x3
= 1 + 9x3

11x5
2x5
5x5
+ 21x5

6x7
x7
3x7
+ 11x7

9x5
x7
x5
11x5

x1
x1
2x1
+ 5x1

5x7
x1
x7
6x7

An Optimal Primal Solution

z
x2
x4
x6

= 29 x1
= 14 2x1
= 5 x1
= 1 + 5x1

2x3
4x3
x3
+ 9x3

11x5
5x5
2x5
+ 21x5

6x7
3x7
x7
+ 11x7

The optimal solution is (x1 , x2 , x3 , x4 ) = (0, 14, 0, 5).

The optimal value is z = 29.
So the optimal primal solution is as claimed.

14/39

An Optimal Dual Solution

z
x2
x4
x6

= 29 x1
= 14 2x1
= 5 x1
= 1 + 5x1

2x3
4x3
x3
+ 9x3

11x5
5x5
2x5
+ 21x5

6x7
3x7
x7
+ 11x7

Recall that the optimal dual solution is (y1 , y2 , y3 ) = (11, 0, 6).

How does the simplex algorithm confirm this?
Look at the top row:
z = 29 x1 0x2 2x3 0x4 11x5 0x6 6x7
So the dual variables (y1 , y2 , y3 ) have values equal to the coefficients
of (x5 , x6 , x7 ) in z!

15/39

Example 2

16/39

Take the following primal and its dual

max
s.t.

5x1
2x1
4x1
3x1

+
+
+
+
x1 ,

4x2
3x2
x2
4x2
x2 ,

+
+
+
+
x3

3x3
x3
2x3
2x3

5
11
8

min
s.t.

5y1
2y1
3y1
y1

+
+
+
+
y1 ,

11y2
4y2
y2
2y2
y2 ,

+
+
+
+
y3

8y3
3y3
4y3
2y3

Recall the final dictionary was

z
x1
x3
x5

= 13 3x2
= 2 2x2
= 1 + x2
= 1 + 5x2

+
+

1x4 1x6
2x4 + x6
3x4 2x6
2x4

So the primal solution is (x1 , x2 , x3 ) = (2, 0, 1) with value 13.

The dual solution is (y1 , y2 , y3 ) = (1, 0, 1) with value 13.

5
4
3

Example 3

17/39

Take the following primal and its dual

max
s.t

5x1
x1
x1
2x1
2x1

+
+

5x2
3x2

x2
3x2
x1 ,

+
+
+
+

x2 ,

3x3
x3
3x3
2x3
x3
x3

min
s.t.

3
2
4
2
0

3y1
y1
3y1
y1

2y2
y2

3y2
y1 ,

+
+

+
y2 ,

4y3
2y3
y3
2y3
y3 ,

+
+
+

y4

2y4
2y4
3y4
y4

Recall the final dictionary was

z
x1
x2
x3
x4

= 10
32
= 29

8
= 29
= 30
29
1
= 29
+

1x5
3
29 x7
8
29 x7
1
29 x7
28
29 x7

1x6 2x7
9
29
x6 +
5
+ 29 x6
3
29
x6
3
29 x6 +

5
29 x5
6
29 x5
8
29 x5
21
29 x5

32 8 30
So the primal solution is (x1 , x2 , x3 ) = ( 29
, 29 , 29 ) with value 10.

The dual solution is (y1 , y2 , y3 , y4 ) = (0, 1, 1, 2) with value 10.

5
5
3

Part III
Proof of Strong Duality

18/39

Strong Duality Theorem

The Strong Duality Theorem.

If the primal has an optimal solution (x1 , x2 , . . . , xn ) then the dual has an
) such that
optimal solution (y1 , y2 , . . . , ym
n
X
j=1

19/39

cj xj =

m
X
i=1

bi yi

Intuition
Look at the initial dictionary:
z
x5
x6
x7

=
=
=
=

1
55
3

4x1
x1
5x1
x1

+
+

x2
x2
x2
2x2

+
+

5x3
x3
3x3
3x3

3x4
3x4
8x4
5x4

11x5
5x5
2x5
21x5

6x7
3x7
x7
11x7

And the final dictionary:

z
x2
x4
x6

=
=
=
=

29
14
5
1

x1
2x1
x1
5x1

2x3
4x3
x3
9x3

The slack variable x5 appears in one constraint in the initial dictionary.

So the only way it can appear with a coefficient of 11 in the top row
of the final dictionary is if we multiplied that constraint by 11.
Similarly we must have multiplied the x7 constraint by 6.
20/39

Intuition (cont.)
Lets verify this.
Add these constraints to the top row of the initial dictionary:
z
+
11 x5
+
6 x7
z + 11x5 + 6x7

4x1 +

x2

+ 5x3 +

= 11 (1

x1

x2

=
=

x1
x1

2x2 3x3 + 5x4 )

2x3

6
29

(3 +

Thus
z = 29 x1 2x3 11x5 6x7

21/39

x3

3x4

3x4 )

Intuition (cont.)
maximise
subject to

4x1
x1
5x1
x1

+
+

x2
x2
x2
2x2

+
+
x1 ,

5x3
x3
3x3
3x3
x2 ,

+
+
+

x3

3x4
3x4
8x4
5x4
, x4

1
55
3
0

In terms of the original LP we have

11 (x1 x2 x3 + 3x4 ) 11 1
6 (x1 + 2x2 + 3x3 5x4 ) 6 3
5x1 + x2 + 7x3 + 3x4
29
So
4x1 + x2 + 5x3 + 3x4 5x1 + x2 + 7x3 + 3x4 29

22/39

Intuition: Example 2
Look at the initial dictionary:
z
x4
x5
x6

=
=
=
=

5
11
8

5x1
2x1
4x1
3x1

4x2
3x2
x2
4x2

3x3
x3
2x3
2x3

And the final dictionary:

z
x1
x3
x5

=
=
=
=

13
2
1
1

+
+

3x2
2x2
x2
5x2

+
+

1x4
2x4
3x4
2x4

1x6
x6
2x6

The slack variable x4 appears in one constraint in the initial dictionary.

So the only way it can appear with a coefficient of 1 in the top row
of the final dictionary is if we multiplied that constraint by 1.

23/39

Intuition: Example 3
Look at the initial dictionary:
z
x4
x5
x6
x7

=
=
=
=
=

3
2
4
2

5x1
x1
x1
2x1
2x1

5x2
3x2

x2
3x2

3x3
x3
3x3
2x3
x3

And the final dictionary:

z
x1
x2
x3
x4

=
=
=
=
=

10
32
29
8
29
30
29
1
29

1x5
3
x
29 7
8
x
29 7
1
x
29 7
28
x
29 7

1x6
9
x
29 6
5
x
29 6
3
x
29 6
3
x
29 6

2x7
+

5
x
29 5
6
x
29 5
8
x
29 5
21
x
29 5

The slack variable x7 appears in one constraint in the initial dictionary.

So the only way it can appear with a coefficient of 2 in the top row
of the final dictionary is if we multiplied that constraint by 2.
24/39

Proof Idea
Does what we have seen always work? Run the simplex algorithm to find...
The values of the decision variables (x1 , x2 , . . . , xn ).
, x , . . . , x
The values of the slack variables (xn+1
n+m ).
n+2

The top row of the final dictionary

z = OPT

n+m
X

k xk

k=1

[We know k 0 for all k, and for variables in the basis k = 0.]

Set the dual variables to be yi = n+i for all i 1, 2, . . . , m.

25/39

Proof of Strong Duality

We only need to show two things:

) is feasible for the dual, i.e.

(1) That (y1 , y2 , . . . , ym
m
X

aij yi cj

j [n]

i=1

(2) It has value equal to the optimal primal value, i.e.

n
X
j=1

26/39

cj xj =

m
X
i=1

bi yi

Proof of Strong Duality (cont.)

Let
z = OPT(primal) =

n
X

cj xj

j=1

So the final dictionary states that

z

n
X

cj xj

j=1

= z

n+m
X

k xk

k=1
n+m
X

k xk

k=1

= OPT(primal)

n+m
X
k=1

e.g. z = 10 x5 x6 2x7

27/39

k xk

Proof: Some Algebra

28/39

n
X

cj xj

OPT

j=1

n+m
X

k xk

k=1

OPT

n
X
k=1

OPT

n
X

OPT

OPT

n
X

j xj

m
X

j xj

m
X

j=1

i=1

n
X

m
X

OPT

n+i xn+i

i=1

j xj

m
X
i=1

yi xn+i
yi

bi

n
X

i=1

j=1

k xk

k=n+1

j=1

n+m
X

k xk

!
bi yi

!
aij xj

j=1
n
m
X
X
j=1

i=1

!
aij yi

xj

Proof: Dual Feasibility

n
X

cj xj =

OPT

j=1

m
X

!
bi yi

i=1

n
m
X
X
j=1

!
aij yi j

i=1

But this equality is true for all choices of {x1 , x2 , . . . , xn }.

So it must be that
cj =

m
X

aij yi j

i=1

But j 0, and so we have feasibility

(2)

m
X
i=1

29/39

aij yi cj

xj

Proof: Dual Optimality

n
X

cj xj =

OPT

j=1

m
X

!
bi yi

i=1

n
m
X
X
j=1

!
aij yi

i=1

But this equality is true for all choices of {x1 , x2 , . . . , xn }.

So it must be that
OPT(primal)

m
X

bi yi = 0

i=1

Thus we have strong duality

(1)

m
X
i=1

30/39

bi yi = OPT(primal) =

n
X
j=1

cj xj

xj

Part IV
Complementary Slackness

31/39

Weak Duality

32/39

Recall for any feasible solutions {x1 , x2 , . . . , xn } and {y1 , y2 , . . . , ym }

n
X

xj cj

j=1

n
X

xj

m
X

j=1

i=1

m
X

n
X

i=1
m
X
i=1

yi

j=1

yi bi

aij yi

aij xj

Strong Duality
By strong duality, we must have equalities for the optimal solutions
}
{x1 , x2 , . . . , xn } and {y1 , y2 , . . . , ym
n
X

xj

cj

j=1

n
X
j=1
m
X
i=1

m
X
i=1

33/39

xj

yi

m
X
i=1
n
X
j=1

yi bi

aij yi
aij xj

Some Observations
Pn

j=1

xj cj

=
=
=

Pn
P

xj m
i=1 aij yi
Pj=1
m
Pn
yi j=1 aij xj
i=1
Pm

i=1 yi bi

So, by strong duality, we have

xj cj = xj

m
X

aij yi

i=1

We also have
yi

n
X

aij xi = yi bi

j=1

From these observations we obtain...

34/39

Complementary Slackness
Complementary Slackness Theorem.
A primal feasible solution (x1 , x2 , . . . , xn ) and a dual feasible solution
) are both optimal if and only if
(y1 , y2 , . . . , ym
xj > 0

m
X

aij yi = cj

j [n]

aij xj = bi

i [m]

i=1

and
yi > 0

m
X
i=1

35/39

Part V
Consequences of Strong Duality

36/39

Running Times
It is easy to check that the dual of the dual is the primal.
So solving the dual will also give an optimal primal solution.
Recall the # pivots is typically linear in the number of constraints.
The # constraints in the dual is just the # variables, n, in the primal.
Thus O(n) pivots are typically needed to solve the dual.
So solving the dual can be far quicker than solving the primal!

37/39

Dual Variables

We have seen many examples where the dual problem often has a
nice combinatorial meaning.
In economic problems often the primal variables correspond to
allocations and the dual variables to prices.

38/39

Certificates

The simplex algorithm gives us a certificate of optimality.

Given a primal solution x the dual solution y proves x is optimal.
[So always check that the x you found is optimal!]

So a certificate lets us know when to stop searching.

Certificates seem fundamental in assessing the hardness of problems....

39/39