Basic Feasible Solutions:

Recap
MS&E 211

WILL FOLLOW A CELEBRATED

INTELLECTUAL TEACHING TRADITION

TEN STEPS TOWARDS UNDERSTANDING

BASIC FEASIBLE SOLUTIONS

THESE SLIDES: ONLY INTUITION

DETAILS ALREADY COVERED

Step 0: Notation
Assume an LP in the following form
Maximize cTx
Subject to:
Ax b
x0

N Variables, M constraints
U = Set of all feasible solutions

Step 1: Convex Sets and Convex

Combinations
Convex set: If two points belong to
the set, then any point on the line
segment joining them also belongs to
the set
Convex combination: weighted
average of two or more points, such
that the sum of weights is 1 and all
weights are non-negative
Simple example: average

Convex Combination
Given points x1, x2, , xK , in N
dimensions
For any scalars a1, a2, , aK such that
Each ai is non-negative
a1 + a 2 + + aK = 1

a1x1 + a2x2 + + aKxK is a

convex combination of x1, x2, , xK

Example: Convex combination of

two points

Exercise
If w is a convex combination of x and
y, and z is a convex combination of x
and w, then must z be a convex
combination of x and y?

Example: Convex combination of

three points

Exercise
Is the set of all convex combinations of
N points always convex?

Exercise
Can any convex set be represented as
a convex combination of N points, for
finite N?

Step 2: Half spaces are

convex
aTx b, aTy b
Choose z = (x+y)/2
Then, we must have aTz b

Step 3: Intersection of convex sets

is convex
Two convex sets S, T
Suppose x, y both belong to S T
z = (x+y)/2
(1) z belongs to S since S is convex
(2) z belongs to T since T is convex

Hence, z belongs to S T
Implication: ST is convex

Step 4: Feasible solutions to LP

The set U of feasible solutions to a
Linear Program is convex
Proof: U is the intersection of Half
Spaces

Exercise
Must every convex set be bounded?

Step 5: Basic Feasible

Solution
x is a basic feasible solution to a LP,
if
(1)x is a feasible solution
(2)There do not exist two other feasible
solutions y, z such that x = (y+z)/2

ALSO known as vertex solution,

extreme point solution, corner-point
solution

Step 6: BFS and Bounded

Polytopes
If U is bounded, then any point in U is
the convex combination of its basic
feasible solutions
Use b1, b2, , bK to denote the K basic
feasible solutions

Step 7: BFS and Optimality for

Bounded Polytopes

If U is bounded and non-empty, then

there exists an optimal solution that
is also basic feasible

Step 8: BFS and Optimality for

General LPs
If an LP has a basic feasible solution
and an optimum solution, then there
exists an optimal solution that is also
basic feasible
Will call these solutions Vertex
Optimal or Optimum Basic
Feasible

Step 9: Two Implications

(1)SOLUTION OF LPs: The Simplex
method which walks from BFS to BFS
is correct for bounded polytopes
(2)STRUCTURAL CONSEQUENCES: The
Vertex Optimal Solutions often have
very interesting properties. Example:
For the matching LP, every vertex
optimal solution is integral

In Practice
Most LP Solvers return an optimum
basic feasible solution, when one
exists.
Either, they use Simplex
Or, they transform the solution that they
do find to a basic feasible solution

Hence, when we solve a problem

using Excel we get an optimum basic
feasible solution, when one exists.

Step 10: A useful property

At every basic feasible solution, at least
N constraints are tight
More specifically: Exactly N Linearly
Independent Constraints are tight
Basic solution: Not necessarily feasible,
but exactly N Linearly Independent
Constraints are tight

Exercise
You are given a LP with three decision
variables where some of the constraints
are
0 x1
0 y1
0 z1

Is it possible to add other constraints and

an objective function such that the LP has
an optimum solution but not an optimum
basic feasible solution?

LP in standard form
Minimize cTx
Subject to:
Ax = b
x0

N Variables, M constraints, assume

that the M constraints are Linearly
Independent
At any BFS, at most M variables are
strictly positive

Recap
A feasible solution is basic feasible if it is not the
average of two other feasible solutions
If the feasibility region U for a LP is bounded and
non-empty, then there exists an optimal solution that
is also basic feasible
If an LP has a basic feasible solution and an optimum
solution, then there exists an optimal solution that is
also basic feasible
Commercial LP solvers return an optimum basic
feasible solution by default, where one exists
If there are N decision variables, then a basic feasible
solution has N linearly independent constraints tight

LP Example: Transportation
Warehous
e1
Warehous
e2
Warehous
e3
DEMAND

Retailer 1

Retailer 2

Retailer 3

Retailer 4

SUPPLY

12 (c11)

13

500

(s1)

10

11

700

(s2)

10

12

14 (c34)

800

(s3)

400 (d1)

900 (d2)

200 (d3)

x11

500 (d4)
4

c x
i 1 j 1

2
2

s.t.

x
j 1

3
3

min

x34

ij

x
i 1

ij

Abstract Model

ij ij

si , i 1,2,3
d j , j 1,2,3,4

xij 0,

i, j
32

LP Example: Transportation, N warehouses, M

retailers
x11
1

min

c x

ij ij

i1 j1

2
2

s.t.

3
N

ij

j1
N

ij

xNM

si , i 1, 2,..., N
dj , j 1, 2,..., M

i1

xij 0,

i, j

At most M + N non-zero variables at a

BFS
Can we get to M + N -1?

33

LP Example: Transportation, N warehouses, M

retailers
x11
1

1
2

2
3
N

xNM

Demand = -si on the left, dj on

the right
Edges directed from left to right
Costs cij
Capacities 1

MIN COST FLOW!

Hence, BFSs are going to be integral if

demands and supplies are integral
34

LP Example: Transportation, N warehouses, M

retailers
x11
1

1
2

2
3
N

xNM

Demand = -si on the left, dj on

the right
Edges directed from left to right
Costs cij
Capacities 1

MIN COST FLOW!

No cycles at BFS (else can send flow in both

directions)
=> At most M+N-1 non-zero values (the
35
maximum number of edges in a an acyclic

Exercise
What if the demands and supplies dont
match up? For example, supply is bigger
than the demand?

Exercise
What if the demands and supplies dont
match up? For example, supply is bigger
than the demand?
x11
1

Solution: Add a fake node

2
with the residual demand,
and add 0 cost, infinite
capacity edges from all supplyN
nodes to this fake node

1
2
3

xNM

Exercise
What if there is also a cost to disposal of
excess supply?

Exercise
What if there is also a cost to disposal of
excess supply?
Solution: Add a cost on the
edges from the supply nodes
to the fake node

x11
1

1
2

2
3

Same idea works for excess

demand

xNM

THE SIMPLEX METHOD

Basic and Basic Feasible Solution

In the LP standard form, select m linearly
independent columns, denoted by the variable
index set B, from A. Solve
AB xB = b
for the dimension-m vector xB . By setting the
variables, xN , of x corresponding to the remaining
columns of A equal to zero, we obtain a solution x
such that Ax = b.
Then, x is said to be a basic solution to (LP) with
respect to the basic variable set B. The variables in
xB are called basic variables, those in xN are
nonbasic variables, and AB is called a basis.
41

Simplex Method
George B. Dantzigs Simplex Method for
linear programming stands as one of the
most significant algorithmic
achievements of the 20th century. It is
now over 60 years old and still going
strong.

x2

The basic idea of the simplex method to

confine the search to corner points of
the feasible region (of which there are
only finitely many) in a most intelligent
way, so the objective always improves
The key for the simplex method is to
make computers see corner points; and
the key for interior-point methods is to

x1

42

From Geometry to Algebra

How to make computer recognize a corner

point? BFS

How to make computer terminate and

declare optimality?

How to make computer identify a better

43

Feasible Directions at a BFS and

Optimality Test

Recall at a BFS: AB xB +ANxN = b, and xB > 0

and xN = 0 .
Thus the feasible directions, d, are the ones to
satisfy
AB dB +ANdN = 0, dN 0.
For the BFS to be optimal, any feasible direction
must be an ascent direction, that is,
cTd= cTB dB + cTNdN 0.
From dB = -(AB )-1ANdN, we must have for all dN
0,
cTd= -cTB (AB )-1ANdN +cTNdN =(cTN - c44TB (AB )-

Computing the Reduced Cost Vector

We compute shadow prices, yT = cTB (AB )-1 , or
solving a system of linear equations.

yT AB = cTB, by

Then we compute rT=cT-yTA, where rN is the reduced cost vector for

nonbasic variables (and rB=0 always).
If one of rN is negative, then an improving feasible direction is found by
increasing the corresponding nonbasic variable value

Increase along this direction till one of the basic variables becomes
0 and hence non-basic
We are left with
The process will always converge and produce an optimal solution if one
exists (special care for unbounded optimum and when two basic
variables become 0 at the same time)
45

Summary
The theory of Basic Feasible Solutions leads to a
solution method
The Simplex algorithm is one of the most
influential and practical algorithms of all time
However, we will not test or assign problems on
the Simplex method in this class (a testament to
the fact that this method has been so successful
that we can use it as a basic technology)