Denition of a Linear Program

Denition: A function f (x1, x2, . . . , xn) of x1, x2, . . . , xn is a linear function if and only if for some set of constants c1, c2, . . . , cn, f ( x 1 , x2 , . . . , x n ) = c 1 x 1 + c 2 x 2 + + c n x n .

Examples: x1 5x1 + 6x4 2x2 + 1 3

Non-examples: x2 1 x1 + 3x2 4x4 3 x1 x2

Linear Inequalities
Denition: For any linear function f (x1, x2, . . . , xn) and any number b, the inequalities f ( x 1 , x2 , . . . , x n ) b and f ( x 1 , x2 , . . . , x n ) b are linear inequalities. Examples: x1 + x2 4 5x1 4 0 Note: If an inequality can be rewritten as a linear inequality then it is one. Thus x1 + x2 3x3 is a linear inequality because it can be rewritten as x1 + x2 3x3 0. Even x1/x2 4 is a linear inequality because it can be rewritten as x1 4x2 0. Note that x1/x2 + x3 4 is not a linear inequality, however. Denition: equality For any linear function f (x1, x2, . . . , xn) and any number b, the f ( x 1 , x2 , . . . , x n ) = b is a linear equality.

LPs
Denition: A linear programming problem (LP) is an optimization problem for which: 1. We attempt to maximize (or minimize) a linear function of the decision variables. (objective function) 2. The values of the decision variables must satisfy a set of constraints, each of which must be a linear inequality or linear equality. 3. A sign restriction on each variable. For each variable xi the sign restriction can either say (a) xi 0, (b) xi 0, (c) xi unrestricted (urs). Denition: A solution to a linear program is a setting of the variables.

Denition: A feasible solution to a linear program is a solution that satises all constraints. Denition: The feasible region in a linear program is the set of all possible feasible solutions.

Geometry Denitions
Denition: A set of points S is a convex set if the line segment joining any 2 points in S is wholly contained in S. Fact: The set of feasible solutions to an LP (feasible region) forms a (possibly unbounded) convex set. Denition: A point p of a convex set S is an extreme point if each line segment that lies completely in S and contains p has p as an endpoint. An extreme point is also called a corner point. Fact: Every linear program has an extreme point that is an optimal solution. Corrolary: An algorithm to solve a linear program only needs to consider extreme points. Denition: A constraint of a linear program is binding at a point p if the inequality is met with equality at p. It is nonbinding otherwise. (Recall that a point is the same as a solution.)

Solutions to Linear Programs

Denition: An optimal solution to a linear program is a feasible solution with the largest objective function value (for a maximization problem). The value of the objective function for the optimal solution is said to be the value of the linear program. A linear program may have multiple optimal solutions, but only one optimal solution value. Denition A linear program is infeasible if it has no feasible solutions, i.e. the feasible region is empty. Denition A linear program is unbounded if the optimal solution is unbounded, i.e. it is either or . Note that the feasible region may be unbounded, but this is not the same as the linear program being unbounded. Theorem Every linear program either: 1. is infeasible, 2. is unbounded, 3. has a unique optimal solution value

Basic and nonbasic variables and solutions

Consider a linear programming problem in which all the constraints are equalities (conversion can be accomplished with slack and excess variables). Denition: If there are n variables and m constraints, a solution with at most m non-zero values is a basic solution. Denition In a basic solution, n m of the zero-valued variables are considered non-basic variables and the remaining m variables are considered basic variables. Fact: Any linear program that is not infeasible or unbounded has an optimal solution that is basic. Basic feasible solutions correspond to extreme points of the feasible region. Fact: Let B be formed from m linearly independent columns of the A matrix (basis), and let xB be the corresponding variables, then BxB = b .

Some facts to prove on the board:

The feasible region of an LP is convex. If an LP, maxx cT x s.t. Ax = b, x 0 has a feasible solution, it has a basic feasible solution. x is an extreme point of S = {x : Ax = b, x 0} i x is a basic feasible solution. If an LP has a nite optimal solution, then it has an optimal solution at an extreme point of the feasible set. If there is an optimal solution to an LP problem, then there is an optimal basic feasible solution.

Simplex
The simplex algorithm moves from basic feasible solution to basic feasible solution; at each iteration it increases (does not decrease) the objective function value. Each step is a pivot, it chooses a variable to leave the basis, and another to enter the basis. The entering variable is a non-basic variable with positive objective function coecient. The leaving variable is chosen via a ratio test, and is the basic variable in the constraint that most limits the increase. When the objective function has no negative coecients in the objective function, we can stop.

Simplex Example
Original LP maximize 3x1 subject to x1 2x1 4x1 + x2 + 2x3 30 24 36 0. (1) (2) (3) (4) (5)

+ x2 + 3x3 + 2x2 + 5x3 + x2 + 2x3 x 1 , x2 , x3

Standard form. z x4 x5 x6 = 3x1 + x2 + 2x3 = 30 x1 x2 3x3 = 24 2x1 2x2 5x3 = 36 4x1 x2 2x3 . (6) (7) (8) (9)

Iteration 1

z x4 x5 x6

= 3x1 + x2 + 2x3 = 30 x1 x2 3x3 = 24 2x1 2x2 5x3 = 36 4x1 x2 2x3 .

(10) (11) (12) (13)

Pivot in x1. Remove x6 from the basis. x2 4 x2 x1 = 9 4 3x2 x4 = 21 4 3x2 x5 = 6 2 z = 27 + x3 2 x3 2 5x3 2 + 4x3 3x6 4 x6 4 x6 + 4 x6 + . 2 (14) (15) (16) (17)

Iteration 2
x2 4 x2 x1 = 9 4 3x2 x4 = 21 4 3x2 x5 = 6 2 z = 27 + x3 2 x3 2 5x3 2 + 4x3 3x6 4 x6 4 x6 + 4 x6 + . 2

(18) (19) (20) (21)

Pivot in x3. Remove x5. 111 4 33 x1 = 4 3 x3 = 2 69 x4 = 4 z = x2 16 x2 16 3x2 8 3x2 + 16 + x5 8 x5 + 8 x5 4 5x5 + 8 11x6 16 5x6 16 x6 + 8 x6 . 16 (22) (23) (24) (25)

Iteration 3
111 4 33 x1 = 4 3 x3 = 2 69 x4 = 4 z = x2 16 x2 16 3x2 8 3x2 + 16 + x5 8 x5 + 8 x5 4 5x5 + 8 11x6 16 5x6 16 x6 + 8 x6 . 16

(26) (27) (28) (29)

Pivot in x2. Remove x3. x3 6 x3 x1 = 8 + 6 8x3 x2 = 4 3 x3 x4 = 18 2 z = 28 x5 2x6 6 3 x5 x6 + 6 3 2x5 x6 + 3 3 x5 + . 2 (30) (31) (32) (33)

Duality
Given a linear program (primal) max cT x Ax b x 0 The dual is min bT y AT y c y 0 x is an n -dimensional vector y is an m -dimensional vector c is an n -dimensional vector b is an m -dimensional vector A is a m n matrix

Dual Theorem
Theorem Given a primal and dual linear program, let x be the optimal solution to the primal and y be the optimal solution to the dual. Then c T x = bT y . Furthermore, the optimal dual solution can be read o as the negative of the coecients of the corresponding slack/excess variables in the nal simplex tableaux. Proof: Denote the constraints of the original LP as: xn+i = bi
n j =1

aij xj

where xn+i are the original slack variables. Solve using simplex. The nal objective function is of the form: Z = Z +
n+m

k xk c
k =1

k are the coecients of the basic variables and zero for nonbasic where c k 0 because the conditions for stopping is that of slack variables. Thus, c variables be negative and the basic variables equal zero.

Proof continued
So we have Z = Z +
n+m

k xk c
k =1

From this equation it is easy to nd the dual solution by reading the coecients of the slack variables. n+i. The claim is that yi saties Let yi = c
n j =1 n j =1

cj x j

m i=1

biyi

cj x j = Z = Z + = Z + = Z +
k =1 n j =1 n j =1 n+m

k xk c ( cj xj ) +
m i=1 m

( cn+ixn+i) yi(bi
n

( cj xj )

i=1

aij xj )

The last line follows from the denition of yi and the denitions of the slack variables in the original LP.

Proof continued
So we have that the nal objective function can be written as Z + Reordering terms, this is (Z
m i=1 n j =1 m i=1 n

( cj xj )

yi(bi

aij xj ).

biyi) +

n j =1

( cj +

m i=1

aij yi)xj

Now, recall that pivoting is just rewriting in an equivalent way, so we actually have that, for any feasible x :
n j =1

c j x j = (Z

m i=1

biyi)

n j =1

( cj +

m i=1

aij yi)xj

Because this equation must be true for an innite number of x values we must have that (look at colors) (Z and j + cj = c
m i=1

biyi) = 0.
m i=1

aij yi.

Proof continued
(Z and j + cj = c The rst of these says that Z = in other words, y is optimal. The second says that
m i=1 m i=1 m i=1

biyi) = 0.
m i=1

aij yi.

biyi

j aij yi = cj c

j 0 because these are coecients of But recall that we have said that c the objective function in the nal tableaux. Therefore,
m i=1

aij yi cj

and Y is feasible.

Complimentary Slackness
Theorem Let x be feasible in P and let y be feasible in D. Necessary and sucient conditions for optimality of x and y are
m i=1

aij yi = cj

OR

x j = 0

for all j = 1, . . . , n

AND
n j =1

aij x j = bi

OR

yi = 0

for alli = 1, . . . , m

Either constraint is tight or dual variable is zero.