Sei sulla pagina 1di 82

Mixed Integer Nonlinear Programming

IMA New Directions Short Course on Mathematical Optimization


Jeff Linderoth and Jim Luedtke

Department of Industrial and Systems Engineering


University of Wisconsin-Madison

August 12, 2016


Introduction

Mixed-Integer Nonlinear Optimization


Mixed Integer Nonlinear Program: (MINLP)

zminlp = min
x
subject to gi (x) 0, i = 1, . . . , m
f (x)
x X, xI Z|I |

f : Rn R, g : Rn Rm smooth, sometimes convex functions.


X Rn bounded, polyhedral set, e.g. X = {x : l AT x u}
I {1, . . . , n} subset of integer variables
We may assume that f is a linear function.

NP-Super-Hard
Combines challenges of handling nonlinearities
with combinatorial explosion of integer variables
Introduction

The MINLP Family


The great watershed in
optimization isnt between If f and g are convex functions,
linearity and nonlinearity, then we have a convex MINLP
but convexity and
If f and g are not convex, then
nonconvexity.
we have a nonconvex MINLP
- R. Tyrrell Rockafellar

Optimization 101
Know the convexity properties of your instance!

Algorithms for two different classes of problems are different


We can solve significantly larger convex MINLP instances that
nonconvex MINLP instances
Algorithms designed for convex-MINLP are often used as heuristics
for nonconvex MINLPs
Introduction

Specializations

Functional Form Problem Type


gi (x) = kAi x + bi k2 piT x + qi MISOCP
gi (x) = polynomial MIPP
gi (x) = x T Qi x + piT + qi MIQP
gi (x) = aiT x bi MILP

MISOCP: Mixed Integer Second Order Cone Program


Are convex-MINLP
MIPP: Mixed Integer Polynomial Program
MIQP: Mixed Integer Quadratic Program
May be convex or nonconvex
Convex MIQP is a special case of MISOCP
If f is convex quadratic and c is an affine mapping, then there are
specialized algorithms for convex-MIQP
MILP: Mixed Integer Linear Program
Introduction

Aside MISOCP
The feasible regions of surprisingly many MINLP problems can be
expressed as MISOCP:

Hyperbolic constraints: Product of (non-negative) variables a


norm2 .
 
T
2w
w w xy , x 0, y 0 x +y
x y

Convex Quadratic: ci (x) = x T Qi x + piT x + qi t


Qi is positive semidefinite, so it has Cholesky factorization Qi = LT L
y = Lx g (x) = y T y + piT x + qi t

{(x, t) Rn+1 | x T Qx + piT x + qi t} =


Proj(x,t) {(x, y , t, p) R2n+2 | y = Lx, p + q T x + r t, p ky k2 }
Introduction

MINLP Tree
MINLP

Nonconvex Nonconvex Convex Convex


NLP MINLP MINLP NLP

MI-
Polynomial
Polynomial MISOCP SOCP
Opt.
Opt.

Nonconvex Nonconvex Convex


Convex QP
QP MIQP MIQP

MILP LP
Introduction

How Big Is It?


The Leyffer-Linderoth-Luedtke (LLL) Measure of Complexity
You have a problem of class X that has Y decision variables? What is the
largest value of Y for which one of Sven, Jim, and Jeff would be willing
to bet $50 that a state-of-the-art solver could solve the problem?

Convex Nonconvex
Problem Class (X ) # Var (Y ) Problem Class (X ) # Var (Y )
MINLP 500 MINLP 100
NLP 5 104 NLP 100
MISOCP 1000 MIPP 150
SOCP 105 PP 150
MIQP 1000 MIQP 300
QP 5 105 QP 300
MILP 2 104
LP 5 107
Introduction Software Tools and Online Resources

Solvers for MINLP


Convex Nonconvex
-ECP -BB
BONMIN ANTIGONE
DICOPT BARON
FilMINT COCONUT
GUROBI COUENNE
KNITRO CPLEX
MILANO LGO
MINLPBB LindoGlobal
MINOPT SCIP
MINOTAUR
SBB
Xpress
MIQP MIQP/convex MIQP

Open-source and commercial solvers


Introduction Software Tools and Online Resources

Software for MINLP


Convex MINLP
ALPHA-ECP, Bonmin, DICOPT, FilMINT, KNITRO, MINLP-BB,
SBB

(Convex) MIQP
CPLEX, GUROBI, MOSEK, XPRESS

(Nonconvex) MIQP
GLOMIQO

General Global Optimization


BARON, Couenne, LINDOGlobal, SCIP

Try em on NEOS
http://www.neos-server.org/neos/solvers/index.html
Introduction Basic Concepts

Simple Example
x2

(x1 2)2 + (x2 + 1)2 36


3x1 x2 6
2.5(x1 1)2 x2 4
0 x1 , x2 5
x2 Z

x1
Doubly NP-Hard!
Feasible region is not convex for two reasonsIntegrality of x2 and
nonconvexity of constraint
Introduction Basic Concepts

Q: What to Do?!

Relax!

Remove some constraints.


Solution of relaxed problem provide a lower bound on optimal solution
value
Introduction Basic Concepts

Natural Relaxation

x2

Natural relaxation is to
remove constraints xi Z i I
But this still leaves a nonconvex
NLP relaxation

x1
Introduction Basic Concepts

Relax More

x2
We need a mechanism for
relaxing the nonconvex
constraints.
Secant underestimator for
concave functions
Easy to get polyhedral
outerapproximation of
nonconvex constraints
Ideally, we would like to get the
convex hull of the feasible points
x1
Introduction Basic Concepts

Linear Objective Is Important Here!

min(x1 1/2)2 + (x2 1/2)2 x2

s.t. x1 {0, 1}, x2 {0, 1}

(x1 , x2 )
(x1 1/2)2 + (x2 1/2)2
x1

Without linear objective, optimal solution may be interior to the


convex hull convexifying may do you no good!
Introduction Basic Concepts

Algorithm Ingredients

1 Relaxation
Used to compute a lower bound on
the optimum value
Obtained by enlarging feasible set; e.g.
ignore constraints
Should be tractable and tight

2 Constraint Enforcement
Exclude solutions from relaxations not feasible to MINLP
Refine or tighten of relaxationBranching and Cutting
3 Upper Bounds
Obtained from any feasible point; e.g. solve NLP for fixed xI .
(NLP(xI )).
Other heuristic techniques have been developed
Modeling and Applications

Models are Important!

Writing good models is extremely


important in MILP
Probably more important for MINLP

More Complexity
Tradeoff between physical accuracy and mathematical tractability
Convex reformulations? Linearize expressions?

MINLP Modeling Preference


We want strong formulationsRelaxations that can be relaxed to
something that closely approximates the convex hull
We prefer linear over convex over nonconvex formulations
Modeling and Applications

Some MINLP Applications


1 Supply ChainConvex MINLP
Discrete: Fixed charges for opening facilities
Nonlinear: Nonlinear transportation costs
2 Gas/Water Network DesignNonconvex MINLP
Discrete: Pipe connections/sizes
Nonlinear: Pressure Loss
3 Pooling/PetrochemicalNonconvex MINLP
Discrete: Which process to use?
Nonlinear: Product Blending (among others)

Luedtke Theorem
97.5% of all practical MINLPs have integer variables for one of two
purposes...
1 Binary variables to make a multiple choice selection
2 Binary indicator variables that turn on/off continuous variables
and/or constraints.
Modeling and Applications Supply Chain

Uncapacitated Facility Location

Problem introduced by Gunluk, Lee,


and Weismantel (07) and classes of
strong cutting planes derived

M: Facility
N: Customer
xij : percentage of customer j N demand met by facility i M
zi = 1 facility i M is built
Fixed cost for opening facility i M
Quadratic cost for meeting demand j N from facility i M
Modeling and Applications Supply Chain

Quadratic Uncapacitated Facility Location Problem

A very simple MIQP

def
X XX
z = min ci zi + qij xij2
iM iM jN

subject to

xij zi i M, j N
X
xij = 1 j N
iM
xij 0 i M, j N
zi {0, 1} i M
Modeling and Applications Supply Chain

Base Case of Induction to Luedtke Theorem

Note that binary variables are used as indicators: zi = 0 xij = 0


If z = 1, then we need to model the epigraph of xij2

Intager Programmerz r Smrt


Mixed Integer Linear Programmers
carefully study simple problem
structures to come up with
good formulations for problems
Good formulations closely
approximate convex hull of
feasible solutions
Important to understand the
structure of a special MINLP with
indicator variables
Modeling and Applications Supply Chain

A Very Simple Structure


n o
def
R = (x, y , z) R2 B | y x 2 , 0 x uz
y
y x2

z = 0 x = 0, y 0
z = 1 x u, y x 2 z

z =1

Deep Insights
conv(R) line connecting (0, 0, 0) to y = x 2 in the z = 1 plane
Modeling and Applications Supply Chain

Characterization of Convex Hull

Deep Theorem #1
n o
R = (x, y , z) R2 B | y x 2 , 0 x uz
conv(R) = (x, y , z) R3 | yz x 2 , 0 x uz, 0 z 1, y 0


x 2 yz, y, z 0

Second Order Cone Programming


There are effective, robust algorithms for optimizing over conv(R)
Modeling and Applications Supply Chain

In GeneralGiving You Some Perspective


For a convex function f : Rn R, the perspective function
P : Rn+1 R of f is

def 0 if z = 0
P(x, z) =
zf (x/z) if z > 0
The epigraph of P(x, z) is a cone pointed at the origin whose lower
shape is f (x)
Exploiting Your Perspective
If zi is an indicator that the (nonlinear, convex) inequality
f (x) 0 must hold, (otherwise x = 0), replace the inequality with
its perspective version:

zi f (x/zi ) 0

The resulting (convex) inequality is a much tighter relaxation of


the feasible region.
Modeling and Applications Supply Chain

Perspectivizing Quadratic Uncapacitated Facility Location

def
X XX
z = min ci zi + qij xij2 yij
iM iM jN
Steps

xij zi i M, j N
1 Make Objective
X Linear
xij = 1 j N 2 Apply
iM
Perspective
xij 0 i M, j N
3 Make Giant
zi {0, 1} i M
Profit
xij2 zi yij 0 i M, j N
Modeling and Applications Supply Chain

Strength of Relaxations

zR : Value of NLP relaxation


zGLW : Value of NLP relaxation after GLW cuts
zP : Value of perspective relaxation
z : Optimal solution value

Cheers!
|M| |N| zR zGLW zP z
10 30 140.6 326.4 346.5 348.7
15 50 141.3 312.2 380.0 384.1
20 65 122.5 248.7 288.9 289.3
25 80 121.3 260.1 314.8 315.8
30 100 128.0 327.0 391.7 393.2
Modeling and Applications Supply Chain

Impact of SOCP

m = 30, n = 100
Convex MINLP, Original: 16697 CPU seconds, 45901 nodes
Convex MINLP, w/GLW Ineq.: 21206 CPU seconds, 29277 nodes
MISOCP, Perspective, 23 CPU seconds, 44 B&B nodes

Larger Instances
m n T N
30 200 141.9 63
40 100 76.4 54 Poifect
40 200 101.3 45
50 100 61.6 49
50 200 140.4 47
Modeling and Applications Water Network Design

Design of Water Distribution Networks

Goal: Design minimum cost


network from discrete pipe
diameters to meet water
demand

Sets
N nodes in network
S source nodes
A: arcs in the network
K: Pipe diameters
Modeling and Applications Water Network Design

Design of Water Distribution Networks


Variables
qij : flow pipe (i, j) A
hi : hydraulic head at node i N
dij : diameter of pipe (i, j) A, where dij {P1 , . . . , Pr }
aij : area of cross section (i, j) A
yijk : Binary variable = 1 if pipe of size k K chosen for (i, j) A

Constraints
Conservation of flow
X X
qij qji = Di , i N \ S.
(i,j)A (j,i)A

Flow bounds: (linear in a, but since aij = 0.25dij2 , nonlinear in d)

Vmax aij qij Vmax aij , (i, j) A.


Modeling and Applications Water Network Design

Design of Water Distribution Networks

Discrete pipe sizes dij {P1 , . . . , Pr }


Use multiple choice integrality to remove nonlinearity aij = dij2 /4
This is in general a good modeling practice

1 Binary variables yijk {0, 1} for k K


2 Model discrete choice as
X X
yijk = 1, and Pk yijk = dij . (i, j) A
kK kK

3 Model aij = dij2 /4 (nonlinear) as


X  
aij = Pk yijk (i, j) A
4
kK
Modeling and Applications Water Network Design

Design of Water Distribution Networks

Nonsmooth, nonlinear pressure loss as a function of flow qij along arc


(i, j) A and diameter dij :

hi hj = ij dijc1 sgn(qij )|qij |c2 (i, j) A

Another trick: disaggregate flow into flow variables for each pipe size:
X
qij = qijk
kK
(Vmax aij )yijk qijk (Vmax aij )yijk
Modeling and Applications Water Network Design

Why On Earth?

Doing this allows us to write the


(nonlinear) headloss as a f (q) = sgn(q)|q|c2
function of a single variable qijk
Often in applications, the
nonlinearity is low-dimensional
(low {1, 2})
Therefore a piecewise linear q
approximation (or relaxation)
may be sufficiently accurate
Luedtkes Theorem!
Note also that
yijk = 0 qijk = 0
Modeling and Applications Water Network Design

Modeling Piecewise-Linear Functions

Take Your Pick


SOS2 Model [Beale and Tomlin, 1970, Beale and Forrest, 1976]
Incremental Model [Markowitz and Manne, 1957]
Multiple Choice Model [Jeroslow and Lowe, 1984]
Convex Combination Model [Dantzig, 1960, Padberg, 2000]
Disaggregated Convex Combination Model [Meyer, 1976]
Logarithmic Model [Vielma and Nemhauser, 2011]
Modeling and Applications Water Network Design

Multiple Choice Model


Want to model (i)
f (q) sgn(q)|q|c2
f (q) = mi q + ci , q [Bi1 , Bi ]

B0 B1 B2 q
Introduce New Variables B3 B4
wi : = q if q [Bi1 , Bi ]
bi : = 1 if q [Bi1 , Bi ]
t f (q)

n
X Bi1 bi wi Bi bi i 1, . . . , n
q= wi ,
n
i=1 X
n 1= bi
X
t= (mi wi + ci bi ) i=1

i=1 B0 y q Bn y
Modeling and Applications Water Network Design

A Simple Trick

Instead of modeling the piecewise-linear indicator in the standard way:


n
X
B0 y q Bn y , 1= bi
i=1

Model it as
n
X
y= bi
i=1

Resulting formulation is provably stronger (locally-ideal)


All piecewise-linear functions turned on/off by an indicator have a
similar modeling trick [Sridhar et al., 2013]
It is the exact extension of the perspective reformulation to this
nonconvex case
Modeling and Applications Pooling

The Pooling Problem


The Pooling Problem
A canonical problem in the petrochemical industry
Decide how to blend (pool) feeds together to make products
meeting quality requirements at minimum cost
Feeds I Pools J Products K

Generally each product has multiple attributes


Modeling and Applications Pooling

Pooling Problem
Basic variables
xij : Flow of input i to pool j
xik : Flow of input i to product k
xjk : Flow from pool j to product k

Key Parameters
i : Concentration of attribute in feed i
`k , k : Bounds on attribute concentration in product k
Ci , Cj , Ck : Capacities on feeds, pools, and outputs
Uij , Uik , Ukj : Flow capacities (some may be zero)

Simple (Linear) Constraints


Linear Constraints: Flow balance, bounds on flows,
feed/pool/output capacity
Modeling and Applications Pooling

P-formulation
(For example) Input capacity:
X X
xi` + xij Ci i I
`L jJ

Flow Balance: X X
xi` = x`j ` L
iI jJ

Pool Quality: X X
i xi` = p` xi` ` L
iI iI

Product Quality:
!
X X X X
p` x`j + i xij = pj x`j + xij j J
`L iI `L iI
`j pj j j J
Modeling and Applications Pooling

Modeling the Quality Constraints


Q-formulation [Ben-Tal et al., 1994]
New variables qij : proportion of flow to pool j coming from feed i:
X
qij = 1
iI

qij satisfy the relations:


xij X
qij = P xij = qij xtj
tI xtj
tI
P P
Reformulate using flow balance at pool j: tI xtj = kK xjk
X
xij = qij xjk
kK
P
Concentration of attribute at pool i is then: iI i qij
Modeling and Applications Pooling

Modeling the Quality Constraints (2)


P
Concentration of attribute at pool i: iI i qij
Quality constraint on product k:
P P P
iI i xik + jJ i qij xjk
`k P P iI k
iI xik + jJ xjk

Multiply and gather terms (e.g., for upper bound):


X XX
(i k )xik + (i k )qij xjk 0
iI jJ iI

Introduce variables wijk to represent bilinear terms qij xjk :


X X
xij = qij xjk = wijk
kK kK
Modeling and Applications Pooling

Modeling the Quality Constraints (3)

Bringing it all together


X
qij = 1, j J
iI
X
xij = wijk , i I,j J
kK
X XX
(i k )xik + (i k )wijk 0, k K
iI jJ iI
X XX
(`k i )xik + (`k i )wijk 0 k K
iI jJ iI

wijk = qij xjk , i I , j J, k K


Modeling and Applications Pooling

PQ-Formulation [Tawarmalani and Sahinidis, 2002]


P
1 For each j J, multiply the constraint iI qij = 1 with xjk :
X
qij xjk = xjk
iI
P
2 For each j J, multiply the constraint: kK xjk Cj with qij :
X
qij xjk Cj qij
kK

3 Replace qij xjk with wijk :


X X
wijk = xjk , wijk Cj qj
iI kK

Much better relaxations!


Takeaway: Adding redundant constraints can significantly improve relax-
ations of nonconvex MINLPs.
Modeling and Applications Pooling

The Pooling Problem with Binary Variables

Arcs connecting the input streams to


other nodes may be used only by
paying a fixed cost.
Network Design
Ship assignment

Add binary variables zij for these arcs:

xij Uij zij

A real nonconvex, integer problem.


Modeling and Applications Pooling

Practical Advice About the Luedtke Theorem


We have seen many cases where we have continuous variable
0 x U and associated binary variable z {0, 1}.
We see many models that write the expression w = zx.

On The Horror!
Never multiply a binary variable by a
bounded continuous variable
You can linearize this:

0 w Uz
U(1 z) x w U(1 z)
Modeling and Applications Pooling

Other MINLP Applications


2. Core-Reloading
1 Portfolio ManagementConvex MIQP
Discrete: Trading Strategy
Nonlinear: Utility
2 Reactor Core ReloadingNonconvex MINLP
Discrete: Bundle Reordering
Nonlinear: Neutron Transport
4. Building Demand
3 Power Grid OperationNonconvex MINLP
Discrete: Unit Committment
Nonlinear: AC Power Flow
4 Building Co-GenerationNonconvex MINLP
Discrete: Unit Committment 5. Oil-Spill Response
Nonlinear: Ramping Constraints
5 Oil-Spill ResponseNonconvex MINLP
Discrete: Resource Allocation
Nonlinear: Fluid Flow & Chemistry
Algorithms Convex

Algorithms for Convex MINLP

Algorithms for convex MINLP are very similar to those for MILP

NLP-based branch-and-bound
Solve NLP relaxation at each node
Cutting plane based algorithms:
Use linear inequalities to outer approximate continuous relaxation
Solve LP relaxations instead of NLP relaxations
Update LP relaxations by adding cuts
Benders decomposition [Geoffrion, 1972], Outer approximation
[Duran and Grossmann, 1986], LP/NLP branch-and-bound
[Quesada and Grossmann, 1992], Extended cutting plane
[Westerlund and Pettersson, 1995], Algorithmic refinements, e.g.
[Abhishek et al., 2010, Bonami et al., 2008]
Algorithms Convex

Valid Inequalities for Convex MINLP


Many classes of inequalities for MILP have been extended/studied for
convex MINLP
Disjunctive/split cuts
[Ceria and Soares, 1999, Stubbs and Mehrotra, 1999,
Belotti et al., 2012, Bonami, 2011, Modaresi et al., 2014b,
Klnc et al., 2010, Klnc-Karzan and Yildiz, 2014]
Chvatal-Gomory rounding and mixed-integer rounding for conic
MINLP [Cezik and Iyengar, 2005, Drewes, 2009]
Conic mixed-integer rounding [Atamturk and Narayanan, 2010]
Intersection cuts [Andersen and Jensen, 2013, Modaresi et al., 2014a]
Minimal valid inequalities and cut generating functions
[Klnc-Karzan, 2013, Moran R. et al., 2012]
Closures [Dadush et al., 2011b, Dey and Moran R., 2012,
Dadush et al., 2011a]
Algorithms Nonconvex Constructing Relaxations of Nonconvex Constraints

General Approach to Nonconvex MINLP

minimize f (x) subject to g (x) 0, x X , xi Z i I


x

Use our old trick: convex (often polyhedral) relaxation!


Relax integrality as before: xi R i I
New: R() = {x X : g (x) 0}
Ensure relaxation is tractable: e.g. R() is convex
Algorithms Nonconvex Constructing Relaxations of Nonconvex Constraints

Factorable Functions and MINLP

Consider MINLP with nonconvex, factorable f (x) and g (x)

minimize f (x) subject to g (x) 0, x X , xi Z i I


x

Factorable Functions
g (x) is factorable if it can be expressed as a combination of functions
from finite set of operators O = {+, , /,, sin, cos, exp, log, | |} whose
arguments are variables, constants, or other factorable functions.

Rx
Excludes integrals =x0 h()d and black-box functions
Represented as expression trees
Algorithms Nonconvex Constructing Relaxations of Nonconvex Constraints

Expression Tree Example

* ^

x1 log x2 3

x2

Expression tree of f (x1 , x2 ) = x1 log(x2 ) + x23


Modeling languages (e.g. AMPL, GAMS) have expression tree API
Algorithms Nonconvex Constructing Relaxations of Nonconvex Constraints

Reformulation of Factorable MINLP

Reformulate factorable MINLP as



minimize xn+q


x
subject to xk = k (x)
k = n + 1, n + 2, . . . , n + q
li xi u i i = 1, 2, . . . , n + q



x X,
xi Z, i I ,

see e.g. [Smith and Pantelides, 1997]

q new auxiliary variables, xn+1 , . . . , xn+q


k is operator from O{+, , /,, sin, cos, exp, log}
Algorithms Nonconvex Constructing Relaxations of Nonconvex Constraints

Example of Reformulation of Factorable MINLP

min x1 + x1 x22
s.t. x1 x2 + sin x2 4,
x1 [4, 4] Z,
x2 [0, 10] Z.
Reformulation
min x7
s.t. x3 = sin x2 4 x1 4 0 x5 45
x4 = x1 x2 0 x2 10 400 x6 400
x5 = 4 x3 x4 1 x3 1 404 x7 404
x6 = x2 x4 40 x4 40
x7 = x1 + x6 x1 , x2 , x4 , x6 , x7 Z.

Factorization is not unique!


Integrality inherited from function and integrality on arguments
Bounds inherited from function and argument bounds
Algorithms Nonconvex Constructing Relaxations of Nonconvex Constraints

Reformulation of Factorable MINLP

Factorable form allows systematic construction of convex relaxation:


Nonconvex sets, k = n + 1, n + 2, . . . , n + q

k = {x Rn+q : xk = k (x), x X , l x u, xi Z, i I }

Let R(k ) k be a convex relaxation


minimize xn+q

x
subject to x R(k )

k = n + 1, n + 2, . . . , n + q


li xi ui i = 1, 2, . . . , n + q
x X.
... convex relaxation ... only look at simple sets!
Algorithms Nonconvex Constructing Relaxations of Nonconvex Constraints

Examples of Relaxations

Construct relaxation for each operator O{+, , /,, sin, cos, exp, log}
Odd-degree monomials, xk = xi2p+1 , see
[Liberti and Pantelides, 2003]
Bilinear functions xk = xi xj , [McCormick, 1976]
Let x = (xi , xj ), L = (li , lj ), U = (ui , uj )
get convex hull of k = {(x, xk ) : xk = xi xj , L x U}:

xk lj xi + li xj li lj xk lj xi + ui xj ui lj
xk uj xi + ui xj ui uj xk uj xi + li xj li uj

Important!
Tightness of convex hull depends on bounds li , lj , ui , uj
Algorithms Nonconvex Constructing Relaxations of Nonconvex Constraints

Examples of Relaxations

Polyhedral relaxation, R(k ), of xk = xi3 and xk = xi xj


xk xk

li xi
ui
xj

xi
Algorithms Nonconvex Constructing Relaxations of Nonconvex Constraints

Examples of Relaxations

Polyhedral relaxation, R(k ), of xk = xi2 with xi continuous/integer

xk xk

xi xi
li ui li ui
... if xi Z then add inequalities violated at xi0 6 Z
Algorithms Nonconvex Spatial Branch-and-Bound

Spatial Branch-and-Bound (BnB)

To enforce constraints violated by relaxation solution use spatial BnB


Implicit enumeration technique like integer BnB
Recursively define partitions of feasible set into two sets by changing
continuous and integer variable bounds
Solve LP (or convex NLP) relaxations ( lower bounds)
... and nonconvex NLPs ( upper bound if feasible)

Lower bounding problem at node with bounds (l, u), e.g. LP(l, u)

minimize xn+q

x
subject to x R(k ) k = n + 1, n + 2, . . . , n + q



li xi ui i = 1, 2, . . . , n + q
x X.
Algorithms Nonconvex Spatial Branch-and-Bound

Spatial Branch-and-Bound (BnB)

If LP(l, u) infeasible, then prune node.

Otherwise, x optimal solution of LP(l, u):


If x integer feasible and feasible in NLP(l, u) (hence MINLP), then
fathom node (new incumbent)
Otherwise ... branch ...

Two possible ways to branch (integer / nonlinear):


1 x not integral: xi bxi c xi dxi e like integer BnB
2 k : xk 6= k (x) nonlinear infeasible:
Choose branching variable xi from arguments of k (x)
Branch xi xi xi xi ... two subproblems
Refine convex relaxation in each branch ... tighter bounds
Algorithms Nonconvex Spatial Branch-and-Bound

Branching for Spatial Branch-and-Bound

Branching example xk = k (xi ) = (xi )2 violated

xj xj

(x^i ,x^ j ) (x^i ,x^ j )

xi xi
li ui li b ui
Algorithms Nonconvex Spatial Branch-and-Bound

Branching for Spatial Branch-and-Bound

Branching based on xi b xi b
Good performance depends on good choice of i and b
Goal: Increase both bounds LP(l , u ) and LP(l + , u + )
Strong branching, pseudocost branching, and reliability branching
generalized from convex MINLP
Other techniques (e.g., violation transfer) specific to nonconvex
MINLP

Finding continuous branching candidates xi :


xi not fixed in parent problem
xi is argument of violated function xk 6= k (x)
Algorithms Nonconvex Tightening Bounds and Relaxations

Tightening Bounds and Relaxations

Bound tightening to reduce range of bounds xi [li , ui ]


... because tighter bounds tighter relaxations smaller trees
Spatial BnB + Bound tightening = Branch-and-reduce
[Ryoo and Sahinidis, 1996, Belotti et al., 2009]

Conceptual bound-tightening procedure:


Feasible set F = {x [l, u] : g (x) 0, x X , xI Zp }
Solve 2n (global) optimization problems, given upper bound U:

li0 = min{xi : x F, f (x) U}; ui0 = max{xi : x F, f (x) U}.

... nonconvex MINLPs just as hard use relaxations:


1 FBBT: feasibility-based bound tightening
2 OBBT: optimality-based bound tightening
Algorithms Nonconvex Tightening Bounds and Relaxations

FBBT: Feasibility-Based Bound Tightening

FBBT broadly used:


Artificial intelligence community & constraint programming
NLP solvers [Messine, 2004]
MILP solvers [Savelsbergh, 1994]
Basic Principle of FBBT
Infer bounds on xi from tighter bounds on xj for j 6= i.

Example: xj = xi3 and xi [li , ui ]


Tighten interval of xj to [lj , uj ] [li3 , ui3 ]
Tightened lj0 on xj tighter li0 = 3 lj for xi
p

FBBT is efficient and fast, but convergence can be slow


Algorithms Nonconvex Tightening Bounds and Relaxations

OBBT: Optimality-Based Bound Tightening

Solving min / max xi s.t. x F (nonconvex MINLP) not practical


Instead, use current relaxation

x R()k k = n + 1, n + 2, . . . , n + q
F(l, u) = x Rn+q : li xi ui i = 1, 2, . . . , n + q .
x X

Now get bounds on xi for i = 1, . . . , n by solving 2n LPs:

li0 = min{xi : x F(l, u)}


ui0 = max{xi : x F(l, u)}

... only apply at root node, or small number of nodes


Algorithms Nonconvex Exploiting Nonconvex Structures

The Trouble with Factorable Relaxations

Factorable relaxations are primarily based on local structure Potentially


weak bounds
Example: = {(x1 , x2 ) : x12 + x22 1, 0 xi 2}

Convex hull: x1 + x2 1
Best possible factorable relaxation:

x3 = x12 (1/2)x3 x1
x4 = x22 (1/2)x4 x2
x3 + x4 1
x1 + x2 (1/2)(x3 + x4 ) 1/2

Remedy: Study relaxations of more global structures


Algorithms Nonconvex Exploiting Nonconvex Structures

A Special Structure: Nonconvex Quadratic Functions


Quadratically constrained quadratic program (QCQP)

minimize x T Q0 x + c0T x,

x
subject to x T Qk x + ckT x bk , k = 1, . . . , q



Ax b,
0 x u, xi Z, i I ,

Qk not necessarily convex nonconvex problem


Problem is N P-hard even if all variables are continuous
Many important special cases: Bilinear programming, pooling
problem, integer linear programming, max cut,. . .
Extensively studied: E.g.,
[Poljak and Wolkowicz, 1995, Sherali and Adams, 1998, Yajima and Fujie, 1998,
Burer and Vandenbussche, 2009, Burer, 2009, Burer and Letchford, 2009,
Burer and Letchford, 2012, Vandenbussche and Nemhauser, 2005,
Saxena et al., 2008, Misener and Floudas, 2012, Buchheim and Wiegele, 2012]
Algorithms Nonconvex Exploiting Nonconvex Structures

General Nonconvex Quadratic Functions

Equivalent reformulation of QCQP: introduce Xij for all i, j pairs

minimize Q0 X + c0T x,



x,X
subject to Qk X + ckT x bk , k = 1, . . . , q

Ax b,
0 x u, xi Z, i I ,




X = xx T ,

P P
where X = [Xij ] matrix, Qk X = ij [Qk ]ij Xij = ij [Qk ]ij xi xj .

X = xx T represents nonconvex constraint Xij = xi xj


... otherwise problem is linear!
Relaxing equality X = xx T gives convex (linear) relaxation
We show two approaches: RLT and SDP
Algorithms Nonconvex Exploiting Nonconvex Structures

Reformulation-Linearization Technique (RLT)


[Adams and Sherali, 1986]
Begin with standard polyhedral relaxation of Xij = xi xj :
Xij 0, Xij ui xj + uj xi ui uj , Xij uj xi , Xij ui xj
Strengthen using more structure
1 Exploiting binary variables: xi {0, 1} xi2 = xi
Add linear constraints Xii = xi
2 Multiply linear constraints (e.g.,PPQ formulation for pooling)
Multiplying xi 0 and bt atj xj 0 gives
j
X
b t xi atj xi xj 0
j

Replace xi xj with Xij yields linear inequality


X
bt xi atj Xij 0
j
Algorithms Nonconvex Exploiting Nonconvex Structures

Semidefinite Programming

Semidefinite Programming (SDP) relaxations of QCQPs


1 Relax X xx T = 0 to X xx T  0
2

1 xT
 
X xx T  0 0
xX
3 Improve by adding Xii = xi for binary xi {0, 1}
4 Additional constraints by squaring & linearizing constraints
... here H  0 means H positive semidefinite (x T Hx 0, x)

Both RLT and SDP provide good relaxations ... can also combine them
The Future of MINLP?

Some Open Research Question in MINLP

A very biased view of open research in MINLP


Strong relaxations for nonlinear structures
Implementation & software design issues
Open questions from previous slide

Going beyond traditional MINLP


Black-box functions or simulations ...
unrelaxable variables
Partial-differential equation constraints
Bilevel problems, e.g., leader-follower games
Stochastic/robust optimization

Make impact in important applications


The Future of MINLP?

Take Home Messages

Modeling is extremely important for MINLP


... strong relaxations are key
Effective techniques from MILP useful for MINLP
... more work can be done!
MINLP software maturing,
... but lags behind commercial MILP software
Linear better than convex better than nonconvex
Exploit structure whenever possible
The Future of MINLP?

Abhishek, K., Leyffer, S., and Linderoth, J. T. (2010).


FilMINT: An outer-approximation-based solver for nonlinear mixed
integer programs.
INFORMS Journal on Computing, 22:555567.
DOI:10.1287/ijoc.1090.0373.
Adams, W. and Sherali, H. (1986).
A tight linearization and an algorithm for zero-one quadratic
programming problems.
Management Science, 32(10):12741290.
Andersen, K. and Jensen, A. N. (2013).
Intersection cuts for mixed integer conic quadratic sets.
pages 3748.
Atamturk, A. and Narayanan, V. (2010).
Conic mixed-integer rounding cuts.
Mathematical Programming A, 122(1):120.
Beale, E. and Tomlin, J. (1970).
The Future of MINLP?

Special facilities in a general mathematical programming system for


non- convex problems using ordered sets of variables.
In Lawrence, J., editor, Proceedings of the 5th International
Conference on Operations Research, pages 447454, Venice, Italy.
Beale, E. M. L. and Forrest, J. J. H. (1976).
Global optimization using special ordered sets.
Mathematical Programming, 10:5269.
Belotti, P., Goez, J., Polik, I., Ralphs, T., and Terlaky, T. (2012).
A conic representation of the convex hull of disjunctive sets and conic
cuts for integer second order cone optimization.
Technical report, n. 12T-009, Lehigh University, Department of
Industrial and Systems Engineering.
http:
//www.optimization-online.org/DB_FILE/2012/06/3494.pdf.
Belotti, P., Lee, J., Liberti, L., Margot, F., and Wachter, A. (2009).
Branching and bounds tightening techniques for non-convex MINLP.
Optimization Methods and Software, 24(4-5):597634.
The Future of MINLP?

Ben-Tal, A., Eiger, G., and Gershovitz, V. (1994).


Global minimization by reducing the duality gap.
Mathematical Programming, 63(1):193212.
Bonami, P. (2011).
Lift-and-project cuts for mixed integer convex programs.
In Gunluk, O. and Woeginger, G., editors, Integer Programming and
Combinatoral Optimization, volume 6655 of Lecture Notes in
Computer Science, pages 5264. Springer, Berlin.
Bonami, P., Biegler, L., Conn, A., Cornuejols, G., Grossmann, I.,
Laird, C., Lee, J., Lodi, A., Margot, F., Sawaya, N., and Wachter, A.
(2008).
An algorithmic framework for convex mixed integer nonlinear
programs.
Discrete Optimization, 5(2):186204.
Buchheim, C. and Wiegele, A. (2012).
The Future of MINLP?

Semidefinite relaxations for non-convex quadratic mixed-integer


programming.
Mathematical Programming, pages 118.
10.1007/s10107-012-0534-y.
Burer, S. (2009).
On the copositive representation of binary and continuous nonconvex
quadratic programs.
Mathematical Programming, 120:479495.
Burer, S. and Letchford, A. (2009).
On nonconvex quadratic programming with box constraints.
SIAM Journal on Optimization, 20(2):1073 89.
Burer, S. and Letchford, A. (2012).
Unbounded convex sets for non-convex mixed-integer quadratic
programming.
Mathematical Programming, pages 126.
10.1007/s10107-012-0609-9.
The Future of MINLP?

Burer, S. and Vandenbussche, D. (2009).


Globally solving box-constrained nonconvex quadratic programs with
semidefinite-based finite branch-and-bound.
Computational Optimization and Applications, 43(2):181 195.
Ceria, S. and Soares, J. (1999).
Convex programming for disjunctive optimization.
Mathematical Programming, 86:595614.
Cezik, M. T. and Iyengar, G. (2005).
Cuts for mixed 0-1 conic programming.
Mathematical Programming, 104:179202.
Dadush, D., Dey, S., and Vielma, J. P. (2011a).
The split closure of a strictly convex body.
Operations Research Letters, 39(2):121 126.
Dadush, D., Dey, S. S., and Vielma, J. P. (2011b).
The chvatal-gomory closure of a strictly convex body.
Mathematics of Operations Research, 36(2):227 239.
The Future of MINLP?

Dantzig, G. B. (1960).
On the significance of solving linear programming problems with some
integer variables.
Econometrica, 28:3044.
Dey, S. S. and Moran R., D. A. (2012).
Some properties of convex hulls of integer points contained in general
convex sets.
Mathematical Programming, pages 1 20.
Drewes, S. (2009).
Mixed Integer Second Order Cone Programming.
PhD thesis, Technische Universitat Darmstadt.
Duran, M. A. and Grossmann, I. (1986).
An outer-approximation algorithm for a class of mixed-integer
nonlinear programs.
Mathematical Programming, 36:307339.
Geoffrion, A. M. (1972).
The Future of MINLP?

Generalized Benders decomposition.


Journal of Optimization Theory and Applications, 10(4):237260.
Jeroslow, R. and Lowe, J. (1984).
Modelling with integer variables.
Mathematical Programming Studies, 22:16784.
Klnc, M., Linderoth, J., and Luedtke, J. (2010).
Effective separation of disjunctive cuts for convex mixed integer
nonlinear programs.
Technical Report 1681, Computer Sciences Department, University of
Wisconsin-Madison.
Klnc-Karzan, F. (2013).
On minimal valid inequalities for mixed integer conic programs.
Klnc-Karzan, F. and Yildiz, S. (2014).
Two-term disjunctions on the second-order cone.
Liberti, L. and Pantelides, C. C. (2003).
Convex envelopes of monomials of odd degree.
The Future of MINLP?

Journal of Global Optimization, 25(2):157168.


Markowitz, H. M. and Manne, A. S. (1957).
On the solution of discrete programming problems.
Econometrica, 25(1):84110.
McCormick, G. P. (1976).
Computability of global solutions to factorable nonconvex programs:
Part I Convex underestimating problems.
Mathematical Programming, 10:147175.
Messine, F. (2004).
Deterministic global optimization using interval constraint propagation
techniques.
RAIRO-RO, 38(4):277294.
Meyer, R. (1976).
Mixed integer minimization models for piecewise-linear functions of a
single variable.
Discrete Mathematics, 16(2):16371.
The Future of MINLP?

Misener, R. and Floudas, C. (2012).


GloMIQO: Global mixed-integer quadratic optimizer.
Journal of Global Optimization, pages 148.
Modaresi, S., Klnc, M., and Vielma, J. (2014a).
Intersection cuts for nonlinear integer programming: convexification
techniques for structured sets.
Modaresi, S., Klnc, M., and Vielma, J. (2014b).
Split cuts and extended formulations for mixed integer conic quadratic
programming.
Moran R., D. A., Dey, S. S., and Vielma, J. P. (2012).
A strong dual for conic mixed-integer programs.
SIAM Journal on Optimization, 22(3):1136 1150.
Padberg, M. (2000).
Approximating separable nonlinear functions via mixed zero-one
programs.
Operations Research Letters, 27(1):15.
The Future of MINLP?

Poljak, S. and Wolkowicz, H. (1995).


Convex relaxations of (0-1) quadratic programming.
Mathematics of Operations Research, 20:550561.
Quesada, I. and Grossmann, I. E. (1992).
An LP/NLP based branch-and-bound algorithm for convex MINLP
optimization problems.
Computers & Chemical Engineering, 16:937947.
Ryoo, H. S. and Sahinidis, N. V. (1996).
A branch-and-reduce approach to global optimization.
Journal of Global Optimization, 8:107139.
Savelsbergh, M. W. P. (1994).
Preprocessing and probing techniques for mixed integer programming
problems.
ORSA Journal on Computing, 6:445454.
Saxena, A., Bonami, P., and Lee, J. (2008).
The Future of MINLP?

Disjunctive cuts for non-convex mixed integer quadratically


constrained programs.
Manuscript, available on Optimization Online,
(www.optimization-online.org).
Sherali, H. and Adams, W. (1998).
A Reformulation-Linearization Technique for Solving Discrete and
Continuous Nonconvex Problems.
Kluwer, Dordrecht.
Smith, E. M. B. and Pantelides, C. C. (1997).
Global optimization of nonconvex MINLPs.
Computers & Chemical Engineering, 21:S791S796.
Sridhar, S., Linderoth, J., and Luedtke, J. (2013).
Locally ideal formulations for piecewise linear functions with indicator
variables.
Operations Research Letters, 41:627632.
Stubbs, R. and Mehrotra, S. (1999).
The Future of MINLP?

A branch-and-cut method for 0-1 mixed convex programming.


Mathematical Programming, 86:515532.
Tawarmalani, M. and Sahinidis, N. V. (2002).
Convexification and Global Optimization in Continuous and
Mixed-Integer Nonlinear Programming: Theory, Algorithms, Software,
and Applications.
Kluwer Academic Publishers, Boston MA.
Vandenbussche, D. and Nemhauser, G. L. (2005).
A polyhedral study of nonconvex quadratic programs with box
constraints.
Mathematical Programming, 102:531557.
Vielma, J. P. and Nemhauser, G. (2011).
Modeling disjunctive constraints with a logarithmic number of binary
variables and constraints.
Mathematical Programming, 128(1):4972.
Westerlund, T. and Pettersson, F. (1995).
The Future of MINLP?

A cutting plane method for solving convex MINLP problems.


Computers & Chemical Engineering, 19:s131s136.
Yajima, Y. and Fujie, T. (1998).
Polyhedral approach for nonconvex quadratic programming problems
with box constraints.
Journal of Global Optimization, 13(2):151 170.

Potrebbero piacerti anche