Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Introduction to LP
(1)
cT x
Ax b,
(2)
or as the standard form to be used in the simplex method for solving LPs
min cT x
s.t. Ax = b, x 0.
(3)
For reasons that are obscure to me, the authors of the book have chosen for minimization as the
standard optimization criterion. I will follow this unless it bothers me too much.
The rst system is usually the one that is obtained after modelling the verbal description of the
problem. It is easy to see that any such system can be transformed into an equivalent system with
only (or only ) constraints and only non-negative decision variables, or, by using slack- or
surplus variables, into a system with only = constraints and only non-negative decision variables,
obtaining the standard LP formulation.
Chapter 1 of the book gives an abundance of examples, some of which really show how LP can
capture a large variety of optimization problems. Specically, the example in Section 1.3 shows
how minimizing a piecewise linear convex function can be solved using LP, with problems concerning absolute values as a specic example. Some of the problems in Chapter 1 model as integer
LPs, a subject we will cover later in this course. Please read these modelling parts of the chapter.
A basic LP-course is heavily based on Linear Algebra. However, the insights that lead to the
solution of LP-problems and to a deeper understanding of duality are based on geometry. The
book puts a lot of emphasis on the geometrical interpretation of LP, and I will use it in this course
to show how geometric and algebraic interpretations coincide, re-deriving known theorems about
LP from another theory.
x2
10
3x1 +2x2 =18
x1 =4
2x2 =12
6
4
2
10
x1
x2
10
8
(2,6)
6
4
Z=36=3x1 +5x2
Z=30=3x1 +5x2
10
x1
1.1
Graphical solution
As a step-up let us rst have a look at a simple example that allows a graphical solution. The
following problem is extracted from Hillier and Lieberman, Introduction to Operations Research,
McGraw Hill, New York. I skip the verbal description and the modelling and switch immediately
to the LP-formulation.
maximize 3x1
s.t.
x1
3x1
x1 0,
+ 5x2
2x2
+ 2x2
4
12
18
(4)
x2 0.
Sorry for using a maximization problem here. So here we have two decision variables, which allows
a graphical description of the problem:
The feasible set of an LP in the IRn , i.e., an LP with n decision variables, is bounded by (n 1)dimensional hyperplanes. A vertex of the feasible set is the intersection of n hyperplanes of
3
1.2
Geometric interpretation
Definition 1.3 Let x1 , . . . , xk IRn and 1 , . . . , k IR+ {0} such that ki=1 i = 1 then the
k
vector i=1 i xi is called a convex combination of the vectors x1 , . . . , xk . The set of all convex
combinations of x1 , . . . , xk is called the convex hull of x1 , . . . , xk .
4
Proof. (a)(b) Suppose x is not bfs. Let I = {i|aTi x = bi } be any set of linearly independent
constraints active in x . Since |I| < n the vectors ai , i I lie in a subspace of IRn , i.e., there
is at least one vector d orthogonal aTi d = 0 to all ai , i I. Consider vectors y = x + d and
z = x d, for > 0 suciently small. y P because
aTi y = aTi x + aTi d = bi + 0 = bi , i I,
and since for i
/ I, aTi x > bi , whence for small enough aTi y bi . Similarly z P . But
1
iI
(c)(a) Take any c for which x is a unique optimal point. For any two other points y, z P we
therefore have cT y > cT x and cT z > cT x . Hence, we necessarily have cT (y + (1 )z) > cT x ,
concluding that x cannot be written as a convex combination of any two other points in P .
2
Two remarks: Firstly, two dierent basic feasible solutions are said to be adjacent on the polyhedron P if they share n1 out of the n hyperplanes or active constraints that dene each of them.
Secondly, the last theorem implies that there are a nite number of extreme points in any polyhedron P , since P is the intersection of a nite number m of halfspaces.
One of the main results of Chapter 2 concerns the existence of an extreme point.
Theorem 1.2 (2.6 in [B& T]) Given non-empty polyhedron P = {x IRn |aTi x bi , i =
1, . . . , m}, the following statements are equivalent:
a) P has at least one extreme point;
b) P does not contain a line; i.e., x P , d
= 0 IRn , IR: x + d P ;
c) There exist n vectors out of the family of a1 , . . . , am which are linearly independent.
Proof. a) c) is a direct consequence of the last theorem. I will prove b) a) and leave the
proof of c) b) to be read by yourself.
Let x P and I(x) = {i|aTi x = bi }. If {ai |i I(x)} contains n linearly independent ai s, then x
is a bfs and hence an extreme point. Otherwise, as before, they span a subspace of the IRn and
therefore there exists a vector d such that aTi d = 0, i I(x). Consider y = x + d. Clearly
aTi y = aTi x = bi i I(x). Thus I(x) I(y). Since P does not contain any line it must be
that if we decrease or increase long enough, then some other constraint than those in I(x) will
become active, say this happens at and aTj (x + d) = bj , for some j
/ I(x). Since aTj x > bj ,
T
T
T
T
aj d = bj aj x
= 0. Hence, because aj d
= 0 and ai d = 0, i I(x), we have that aj is
linearly independent of {ai |i I(x)}. This procedure is repeated until we arrive at a set of n
linearly independent active constraints, corresponding to a bfs, i.e., an extreme point of P .
2
Another result that most of you have always used but probably never seen proven rigorously.
Theorem 1.3 (2.7 in [B& T]) Given an LP-problem, the feasible polyhedron of which contains
at least one extreme point, and for which an optimal solution exists, there is always an extreme
point that is an optimal solution.
Proof. Let Q = {x IRn |Ax b, cT x = v} be the set of optimal solutions, having objective
value v. Q contains no line, since it is inside P and P has an extreme point, hence contains no
line. Thus Q also has an extreme point, say x . We show by contradiction that x is also extreme
point of P . Suppose not, then y, z P, [0, 1]: x = y + (1 )z. cT y v and cT z v
and cT x = v implies that cT y = cT z = v y, z Q and therefore x is not extreme point of
2
Q.
6
A slightly stronger result is proved along the same lines as the proof of Theorem 2.6 in the book.
Theorem 1.4 (2.8 in [B& T]) Given an LP-problem, the feasible polyhedron of which contains
at least one extreme point, then either the solution value is unbounded or there is an extreme
point that is optimal.
In my discussions of Chapters 1 and 2 I made a selection of what to treat here in class and what
to leave to yourself for study. For those who did not have a course on complexity theory I advise
to read Section 1.6 of [B& T], that shows you how to measure the running time of an algorithm
and that making a distinction between so-called polynomial running time and exponential running
time is relevant. Complexity theory is a beautiful theory, essential in the study of combinatorial
problems. In this book it only plays an essential role in Chapters 8 and 9.
There are intriguing open problems in the study of polyhedra in relation to complexity theory.
You may skip Sections 2.7 and 2.8 in the book, since the material is covered again in Chapter
4. I do advise you to study 2.4 on degeneracy.
1.3
The simplex method is a general description algorithm that solves any LP-problem instance. To
do so it rst brings it into standard form
min
s.t.
cT x
Ax = b,
x 0,
(5)
with x, c IRn , A an m n matrix and b IRm . We assume that m n and that rank(A) = m.
The latter is a rather severe restriction, but justied by Theorem 2.5 of the book, the proof of
which is easy and which I leave to yourself to read.
So far we have derived all sorts of insights about the geometry of LP-problems and especially
about their feasible sets in the general form P = {x IRn |Ax b}. All these insights are valid for
the standard form, being a special case. Almost all these insights followed from the row-structure
of the matrix A. For the standard form we can derive some more insights, based on As column
structure, and which are fundamental for the simplex method.
A bfs in LP (5) is dened by a set of n active constraints, m of which are directly imposed on us by
the m equalities Ax = b. The other n m active constraints must come from the non-negativity
constraints. Thus, in any bfs n m variables must have value 0. We call them the non-basic
variables of the bfs. The other m variables of the bfs are the basic variables. Let us denote them
by xB = (xB(1) , . . . , xB(m) ). Their values come from the subsystem BxB = b where B is the socalled basis matrix constituted of the columns AB(1) , . . . , AB(m) of A. For x to be a basic solution
it is required that these columns are linearly independent, and this is also sucient.
Theorem 1.5 (2.4 in [B& T]) Let P = {x IRn |Ax = b, x 0} with A having linearly
independent rows. x IRn is a basic solution Ax = b and there exist indices B(1), . . . , B(m)
such that
a) Columns AB(1) , . . . , AB(m) are linearly independent;
b) If i
= B(1), . . . , B(m), then xi = 0.
Proof. follows immediately from the fact that under the conditions a) and b) BxB = b has a
unique solution.
: Let x be a basic solution, hence the unique solution of the system of n active constraints:
Ax = b and (after renumbering if necessary) xi = 0 i
= B(1), . . . , B(k), for some k m. Notice that more than n m of the xi s can be 0 (degeneracy). Anyway, this satises b). Hence,
k
i=1 AB(i) xB(i) = b must have a unique solution and hence AB(1) , . . . , AB(k) are linearly independent. Since rank(A) = m, there are m k extra columns AB(k+1) , . . . , AB(m) in A that added
2
to AB(1) , . . . , AB(k) keeps the set of columns linearly independent, satisfying a).
Given the denition of adjacency of two extreme points or bfss, it is clear that in terms of the
theorem, two bfss are adjacent if
- their sets of indices B(1), . . . , B(m) dier by exactly one.
- they dier in exactly one variable being non-basic, hence also one variable being basic.
This makes that we may also call two bfss adjacent if they correspond to the same point (in the degenerate
case).
Exercises of Week 1
1.5, 1.15, 2.6, 2.15.
Next
week
Chapter