LINEAR PROGRAMMING

Introduction
Development of linear programming was among the
most important scientific advances of mid-20th cent.
Most common type of applications: allocate limited
resources to competing activities in an optimal way.
Very large and wide range of applications from local to
very global ones.
Linear programming uses a mathematical model.
Linear because it requires linear functions.
Programming as synonymous of planning.

31

Introduction
Besides allocate resources to activities, any problem
described by a mathematical model can be solved.
Very efficient solution procedure: simplex method.
This method will be described next as well as the
interior-point algorithm for larger problems.
Besides these algorithms, we will also discuss
sensitivity analysis.

32

Prototype example
Wyndor Glass Co. produces glass products, including
windows and glass doors.
Plant 1 produces aluminum frames, Plant 2 produces
wood frames and Plant 3 produces glass and
assembles.
Two new products:
Glass door with aluminum framing (Plant 1 and 3)
New wood-framed glass window (Plant 2 and 3)

As the products compete for Plant 3, the most

profitable mix of the two products is needed.
33

Data for Wyndor Glass Co.

Wyndor Glass Co. Product-Mix Problem
Profit Per Batch
(Batches of 20)

Plant 1
Plant 2
Plant 3

Doors
$3.000

Windows
$5.000

Hours Used Per Batch Produced

1
0
0
2
3
2

Hours available
per week
4
12
18

34

Formulation of the LP problem

x1 = number of batches of product 1 produced per week
x2 = number of batches of product 2 produced per week
Z = total profit per week (in thousands)

maximize Z

3 x1 5 x2

subject to

and

x1
2 x2

4
12

3x1 2 x2

18

x1

0,

x2

0.

35

Graphical solution
Trial-and-error: Z = 10, 20, 36
Slope of the line is 3/5: x2

3
x1
5

1
Z
5

Feasible region:
satisfies all
constraints

and

x1

0,

x2

0.
36

Solution (using Excel)

Wyndor Glass Co. Product-Mix Problem
Profit Per Batch

Doors
$3.000

Windows
$5.000

Hours
Hours
Hours Used Per Batch Produced Used
Available
Plant 1
1
0
2
<=
4
Plant 2
0
2
12 <=
12
Plant 3
3
2
18 <=
18

Batches Produced

Doors
2

Windows
6

Range Name
Cells
BatchesProduced
C12:D12
HoursAvailable
G7:G9
HoursUsed
E7:E9
HoursUsedPerBatchProduced
C7:D9
ProfitPerBatch
C4:D4
TotalProfit
G12

Total Profit
$36.000

37

Generalizing example
Prototype example

General problem

Production capacities of plants

Resources

3 plants

m resources

Production of products

Activities

2 products

n activities

Production rate of product j, xj

Level of activity j, xj

Profit Z

Overall measure of performance Z

The most common type of LP application involves

allocating resources to activities.
38

Linear programming model

Z = value of overall performance measure
xj = level of activity j, for j = 1, 2, , n.
cj = parameters of Z related to xj.
bj = amount of resource i that is available for allocation
of activities, for i = 1, 2, , m.
aij = amount of resource i consumed by each unity of
activity j.

39

Standard form
maximize Z

c1 x1 c2 x2

cn x n

subject to
a11 x1 a12 x2

a1n xn

b1

a21 x1 a22 x2

a 2 n xn

b2

am1 x1 am 2 x2

amn xn

bm

, xn

0.

and

x1

0, x2

0,

40

Other possible forms

Minimizing rather then maximizing
minimize Z

c1 x1 c2 x2

cn xn

Some functional constraints are greater-than-or-equal

ai1 x1 ai 2 x2

ain xn

bi , for some values of i

Some functional constraints in equation form

ai1 x1 ai 2 x2

ain xn

bi , for some values of i

Deleting the non-negativity constraints for some vars

x j unestricted in sign for some values of i

41

Allocation of resources to activities

Resource Usage per Unit of Activity
Resource

Amount of
Resource
available

a11

a12

a1n

b1

a21

a22

a2n

b2

am1

am2

amn

bm

Contribution
to Z per unit
of activity

c1

c2

cn

Activity

42

Terminology for solutions

Solution: any specification of values for the decision
variables (x1, x2,, xn).
Feasible solution: solution for which all the constraints
are satisfied.
Infeasible solution: solution for which at least one
constraint is violated.
Feasible region
No feasible solution: no solution satisfies all
constraints.

43

Example: no feasible solution

If in the Wyndor Glass Co. problem the profit should
be above $50000 per week:

44

Terminology for solutions

Optimal solution: solution with the most favorable
value.
Most favorable value: largest value if maximizing or
smallest value if minimizing.
No optimal solutions: (1) no feasible solutions or,
(2) unbounded Z (see next slide).
More than one solution: see next slide.

45

Examples of solutions
More than one solution

Unbounded cost function

46

Corner-point feasible solution

Lies in a corner. Corners must have at least one
optimal solution.

47

Assumptions of linear programming

Proportionality: value of objective function Z is
proportional to level of activity xj.
Violations: 1) Setup costs

48

Violations of proportionality
2) Increase of marginal return due to economy of
scale (longer production runs, quantity discounts,
learning-curve effect, etc.)

49

Violations of proportionality
3) Decrease of marginal return due to e.g. marketing
costs: to sell more the advertisement grows
exponentially.

50

Assumptions of linear programming

Additivity: every function in a linear programming
model is the sum of the individual contributions.
Divisibility: decision variables can have any values.
If values are limited to integer values, the problem
needs an integer programming model.

Certainty: parameters of the model cj, aij and bi are

assumed to be known constants.
In real applications parameters have always some
degree of uncertainty, and sensitivity analysis must be
conducted.

51

Conclusions
Linear programming models can be solved using:
Spreadsheet as Excel Solver for relatively simple
problems
Problems with thousands (or even millions!) of decision
variables and/or functional constraints must use
modeling languages such as: CPLEX/MDL or
LINGO/LINDO
Matlab Optimization Toolbox

Linear programming solves many problems.

However, when one or more assumptions are violated
seriously, other programming techniques are needed.
52

SIMPLEX METHOD

Simplex method
Algebraic procedure with underlying geometric
concepts.
Definitions:
Constraint boundary: line of boundary of the
constraint.
Corner-point solution
Corner-point feasible solution (CFP solution)
CFP solutions are adjacent if they share n 1 constraint
boundaries.
The line segment between two CFP solutions is an edge.

54

Example revisited
Wyndor Glass Co.

55

Optimality test
Optimality test: consider LP with at least one optimal
solution. If a CFP solution has no adjacent CFP
solutions that are better, it must be an optimal
solution.
Adjacent CFP solutions for each CFP solution of
Wyndor Glass Co. problem:
CFP solution

Its adjacent CFP solutions

(0, 0)

(0, 6) and (4, 0)

(0, 6)

(2, 6) and (0, 0)

(2, 6)

(4, 3) and (0, 6)

(4, 3)

(4, 0) and (2, 6)

(4, 0)

(0, 0) and (4, 3)

56

Solving the example

Initialization: at CPF solution (0, 0).
Optimality test: conclude that (0, 0) is not optimal
Iteration 1:
1. Choose to move along the edge with faster rate (x2)
2. Stop at new constraint boundary (2x2 = 12)
3. Determine point of intersection of new set boundaries

(0,6)

Optimality test: conclude that (0, 6) is not optimal

Iteration 2: repeat steps of iteration 1. CFP sol. (2,6)
Optimality test: conclude that (2, 6) is optimal. Stop.
57

Steps of simplex algorithm

Sequence of CFP solutions:

58

Setting up the simplex method

Original form
maximize Z

Augmented form
3x1 5 x2

maximize Z

3x1 5 x2

subject to

subject to

x1
3x1

and

x1

2 x2 12
2 x2 18

0, x2

0.

(1)
(2)

x1
2 x2

x3
x4

4
12

(3)

3 x1 2 x2

x5

18

and

xj

0, j 1, 2,3, 4,5.

Convert functional inequality constraints to equivalent

equality constraints using slack variables.
59

Setting up the simplex method

Augmented solution: solution with original variables
(decision variables) and slack variables.
Example: (3, 2, 1, 8, 5)

Basic solution: augmented corner-point solution.

Example (infeasible solution): (4, 6, 0, 0, 6)

Basic feasible (BF) solution: augmented corner-point

feasible solution.

60

Basic definitions
Example problem has 5 variables and 3 non redundant
equations, and 2 (5 3) degrees of freedom.
Two variables have an arbitrary value. Simplex uses
the zero value for these nonbasic variables.
The solution for the other variables (basic variables) is
a basic solution.
Basic solutions have thus a number of properties.

61

Properties of a basic solution

Each variable is basic or nonbasic.
Number of basic variables is equal to number of
functional constraints (equations).
The nonbasic variables are set to zero.
Values of basic variables are solving system of
equations (functional constraints in augmented
form).
Set of basic variables are called the basis.
If basic variables satisfy the nonnegativity constraints
basic solution is a BF solution.
62

Final form
maximize Z
subject to
(0)

3 x1 5 x2

(1)

x1

x3

(2)

2 x2

x4

12

(3)

3x1 2 x2

x5

18

and

xj

0, j 1,

,5.

63

Tabular form (simplex tableau)

(a) Algebraic form

(0) Z 3x1 5x2

(1)

x1

+ x3

(2)

2x2

(3)

3x1 + 2x2

+ x4

(b) Tabular Form

Coefficient of:

Basic
variable

Eq.

= 0

(0)

1 -3 -5 0 0 0

= 4

x3

(1)

0 1 0 1 0 0

= 12

x4

(2)

0 0 2 0 1 0

12

+ x5 = 18

x5

(3)

0 3 2 0 0 1

18

x1 x2 x3 x4 x5

Right
side

64

Simplex method in tabular form

Initialization. Introduce slack variables. Select decision
variables to be initial nonbasic variables and slack
variables to be initial basic variables.
Initial solution of example: (0, 0, 4, 12, 18)

Optimality test. Current BF solution is optimal if every

coefficient in row zero is nonnegative. If yes, stop.
Example: coefficients of x1 and x2 are 3 and 5.

65

Simplex method in tabular form

Iteration.
Step 1: Determine entering basic variable, which is the
nonbasic variable with the largest negative coefficient.
pivot column
Basic
variable

Eq.

entering basic variable

Coefficient of:

Right
side

x1

x2

x3

x4

x5

(0)

-3

-5

x3

(1)

x4

(2)

12

x5

(3)

18
66

Iteration
Step 2: Determine leaving basic variable, by applying
minimum ratio test:

Basic
variable

Eq.

Coefficient of:

Right side

x1

x2

x3

x4

x5

(0)

-3

-5

x3

(1)

x4

(2)

12

12/2 = 6

x5

(3)

18

18/2 = 9

Ratio

pivot row
minimum

pivot number

67

Iteration
Step 3: solve for the new BF solution by using
elementary row operations.
Iteration

Basic
variable

Eq.

Coefficient of:

Right
side

x1

x2

x3

x4

x5

(0)

-3

-5

x3

(1)

x4

(2)

12

x5

(3)

18

(0)

-3

2.5

30

x3

(1)

x2

(2)

0.5

x5

(3)

-1

68

Iteration 2: Steps 1 and 2

Iteration

Basic
variable

Eq.

Coefficient of:

Right
side

x1

x2

x3

x4

x5

(0)

-3

2.5

30

x3

(1)

x2

(2)

0.5

x5

(3)

-1

Ratio

4/1 = 4

6/3 = 2

minimum

69

Complete set of simplex tableau

Iteration

Basic variable

Eq.

Coefficient of:

Right side

x1

x2

x3

x4

x5

(0)

-3

-5

x3

(1)

x4

(2)

12

x5

(3)

18

(0)

-3

2.5

30

x3

(1)

x2

(2)

0.5

x5

(3)

-1

(0)

1.5

36

x3

(1)

1/3

-1/3

x2

(2)

0.5

x1

(3)

-1/3

1/3

2
70

Other model forms

Equality constraints
In this case, artificial variables are used. The Big M
method is used.

Functional constraints in form

Surplus variables are used.

Minimization
Maximize Z.

Big M method can have two phases: 1) all artificial

variables driven to zero; 2) compute BF solutions.

71

Postoptimality analysis
Reoptimization: when the problem changes slightly,
new solution is derived from the final simplex tableau.
Shadow price for resource i (denoted yi*) is the
marginal value of i.
i.e. the rate at which Z can be increased by slightly
increasing the amount of this resource bi.

Example: recall bi values of Wyndor Glass problem.

The final tableau gives:
y1* = 0, y2* = 1.5, y3* = 1.

72

Graphical check
Increasing b2 in 1 unit, the new profit is 37.5

73

Shadow prices
Increasing b2 in 1 unit increases the profit in $1500.
Should this be done? Depends on the marginal
profitability of other products using that hour time.
Shadow prices are part of the sensitivity analysis.
Constraint on resource 1 is not binding the optimal
solution. Resources as this are free goods.
Constraints on resources 2 and 3 are binding
constraints. Resources are scarce goods.

74

Sensitivity analysis
Identify sensitive parameters (those that cannot be
changed without changing optimal solution).
Parameters bi can be analyzed using shadow prices.
Parameters ci for Wyndor example (see figure in next
slide).
Parameters aij can also be analyzed graphically.

For problems with a large number of variables usually

only bi and ci are analyzed, because aij are determined
by the technology being used.

75

Sensitivity analysis of parameters ci

Graphical analysis of
Wyndor example:
c1 = 3 can have values in
[0, 7.5] without changing
optimal
c2 = 5 can have any value
greater than 2 without
changing optimal

76

Interior-point approach
Discovery made in 1984 by Narendra Karmarkar.
Used in huge linear programming problems, beyond
the scope of the simplex method.
Starts by identifying a feasible trial solution, moving a
better one until it reaches the optimal trial solution.
Trial solutions are interior points.
Interior-point or barrier algorithms (each constraint
boundary is treated as a barrier).

77

Example
The Wyndor Glass Co. example reaches the solution in
15 iterations for changes smaller than 0.0001

78

Theory of simplex method

Recall the standard form:

maximize Z

c1 x1 c2 x2

cn x n

subject to
a11 x1 a12 x2

a1n xn

b1

a21 x1 a22 x2

a 2 n xn

b2

am1 x1 am 2 x2

amn xn

bm

, xn

0.

and

x1

0, x2

0,

79

Augmented form
maximize Z

c1 x1

cn xn cn 1 xn

cn m xn

subject to
a11 x1 a12 x2
a21 x1 a22 x2
am1 x1 am 2 x2
and

xj

0,

a1n xn xn 1 b1
a2 n xn xn 2 b2
amn xn
j 1, 2,

xn

bm

, m.

80

Matrix form (standard form)

maximize Z

cx

subject to
Ax

c [c1 , c2 ,

b1

x2

b2

xn

, cn ],

x1
,

b and x

bm

, 0

0
0
0

, A

a11
a21

a12
a22

a1n
a2 n

a m1 a m 2

amn

81

Matrix augmented form

Introduce the column vector of slack variables:
xn
xn

xs

xn

1
2

The constraints are (with I (m m) and 0 (1 (n+m)) ):

[ A, I]

x
xs

and

x
xs

82

Solving for a BF solution

One iteration of simplex:
The n nonbasic variables are set to zero.
The m basic variables is a column vector denoted as xB.
The basic matrix B is obtained from [A, I], by eliminating
the columns with coefficients of nonbasic variables.
The problem becomes: BxB = b.
The solution for basic variables and the value of the
objective function for this basic solution are:

xB

B b

and

c Bx B

c BB b

83

Revised simplex method

1. Initialization: (as original) Introduce slack variables xs.

Decision variables are initial nonbasic variables.

2. Iteration:

Step 1: (as original) Determine entering basic variable

(nonbasic variable with the largest negative
coefficient).
Step 2: Determine leaving basic variable. As original,
but computations are simplified.
Step 3: Determine new BF solution: set xB = B1b.
3. Optimality test: solution is optimal if every

coefficient in Z is 0. Computations are simplified.

84

DUALITY THEORY AND

SENSITIVITY ANALYSIS

Duality theory
Every linear programming problem (primal) has a dual
problem.
Most problem parameters are estimates, others (as
resource amounts) represent managerial decisions.
The choice of parameter values is made based on a
sensitivity analysis.
Interpretation and implementation of sensitivity
analysis are key uses of duality theory.

86

Primal and dual problems

Dual problem

Primal problem
n

maximize Z

cjxj

minimize W

j 1

bi yi
i 1

subject to

subject to

aij x j

bi , for i 1, 2,

,m

j 1

and

aij yi

c j , for j 1, 2,

,n

0 for i 1, 2,

, m.

i 1

xj

0 for j 1, 2,

, n.

and

yi

87

Duality theory
Relations between parameters:
The coefficients of the objective functions of the primal
are the right-hand sides of the functional constraints in
the dual.
The right-hand side of the functional constraints in the
primal are coefficients of the objective functions in the
dual.
The coefficients of a variable in the functional
constraints of the primal are the coefficients in the
functional constraints of the dual.

88

Primal-dual table
Primal problem
Coefficient of:

Right
side

xn

y1

a11

a12

a1n

b1

y2

a21

a22

a2n

b2

ym

an1

an2

amn

bm

Right
side

c1

c2

Dual problem

x2

Coefficients of
objective function
(minimize)

Coefficient of:

x1

cn

Coefficients of objective function

(maximize)
89

Example: Wyndor Glass Co.

Primal problem

Maximize Z

3 5

subject to
1 0
0 2
3 2

and

x1
x2

Dual problem

x1
x2

Minimize Z

y1 y2 y3

subject to
x1
x2

0
0

4
12
18

y1 y2 y3

and

y1 y2 y3

1 0
0 2
3 2

4
12 ,
18

35

0 0 0.

90

Example: Wyndor Glass Co.

Tableau representation
x1

x2

y1

y2

12

y3

18

91

Primal-dual relationships
Weak duality property: If x is a feasible solution for
the primal problem and y is a feasible solution for the
dual problem, then
cx yb
Strong duality property: If x* is an optimal solution for
the primal problem and y* is an optimal solution for
the dual problem, then
cx* = y*b

92

Primal-dual relationships
Complementary solutions property: At each iteration,
the simplex method identifies a CFP solution x for the
primal problem and a complementary solution for the
dual problem, where
cx = yb
If x is not optimal for the primal problem, then y is not
feasible for the dual problem.

93

Primal-dual relationships
Complementary optimal solutions property: At the
final iteration, the simplex method identifies an
optimal solution x* for the primal problem and a
complementary optimal solution y* for the dual
problem, where
cx* = y*b
The yi* are the shadow prices for the primal problem.

94

Primal-dual relationships
Symmetry property: For any primal problem and its
dual problem, all relationships between them must be
symmetric because the dual of this dual problem is
this primal problem.
Duality theorem: The following are the only possible
relationships between the primal and dual problems:

95

Primal-dual relationships
1. If one problem has feasible solutions and a bounded

objective function, then so does the other problem,

so both the weak and strong duality properties are
applicable.
2. If one problem has feasible solutions and an
unbounded objective function, then the other
problem has no feasible solution.
3. If one problem has no feasible solutions, then the
other problem has either no feasible solutions or an
unbounded objective function.

96

Primal-dual relationships
Relationships between complementary basic solutions
Primal basic
solution

Complementary
dual basic
solution

Both basic solutions

Primal feasible?

Suboptimal

Superoptimal

Yes

No

Optimal

Optimal

Yes

Yes

Superoptimal

Suboptimal

No

Yes

Neither feasible Neither feasible No

nor superoptimal nor superoptimal

No

Dual feasible?

97

Primal-dual relationships

98

Applications of duality theory

If the number of functional constraints m is bigger than
the number of variables n, applying simplex directly to
the dual problem will achieve a substantial reduction
in computational effort.
Evaluating a proposed solution for the primal
problem. i) if cx = yb, x and y must be optimal even
without applying simplex; ii) if cx < yb, yb provides an
upper bound on the optimal value of Z.
Economic interpretation as before

99

Duality theory in sensitivity analysis

Sensitivity analysis investigate the effect of changing
parameters aij, bi, cj in the optimal solution.
Can be easier to study these effects in the dual
problem
Optimal solution of the dual problem are the shadow
prices of the primal problem.
Changes in coefficients of a nonbasic variable. Only
changes one constraint in the dual problem. If this
constraint is satisfied the solution is still optimal.
Introducing a new variable in the objective function.
Only introduces a new constraint in the dual problem. If
the dual problem is still feasible, solution is still optimal.
100

Essence of sensitivity analysis

LP assumes that all the parameters of the model (aij, bi
and cj) are known constants.
Actually:
The parameters values used are just estimates based on
a prediction of future conditions;
The data may represent deliberate overestimates or
underestimates to protect the interests of the
estimators.

An optimal solution is optimal only with respect to

the specific model being used to represent the real
problem sensitivity analysis is crucial!
101

Simplex tableau with parameter changes

What changes in the simplex tableau if changes are
made in the model parameters, namely bb , cc ,
A A ?
Coefficient of:
Eq.

Original variables

Slack variables

Right side

(0)

(1,2,,m)

Revised
(0)
final tableau (1,2,,m)

z* c y*A c

y*

Z* y *b

A* S * A

S*

b* S * b

New initial
tableau

102

Applying sensitivity analysis

Changes in b:
Allowable range: range of values for which the current
optimal BF solution remains feasible (find range of bi
such that b* S * b 0, assuming this is the only change
in the model). The shadow price for bi remains valid if bi
stays within this interval.
The 100% rule for simultaneous changes in right-hand
side: the shadow prices remain valid as long as the
changes are not too large. If the sum of the % changes
does not exceed 100%, the shadow prices will still be
valid.

103

Applying sensitivity analysis

Changes in coefficients of nonbasic variable:
Allowable range to stay optimal: range of values over
which the current optimal solution remains optimal
(find c j y * A j , assuming this is the only change in the
model).
The 100% rule for simultaneous changes in objective
function coefficients: if the sum of the % changes does
not exceed 100%, the original optimal solution will still
be valid.

Introduction of a new variable:

Same as above.
104

Other simplex algorithms

Dual simplex method
Modification useful for sensitivity analysis

Parametric linear programming

Extension for systematic sensitivity analysis

Upper bound technique

A simplex version for dealing with decision variables
having upper bounds: xj uj, where uj is the maximum
feasible value of xj.

105