THE EQUATION ORIENTED

STRATEGY FOR PROCESS FLOWSHEETING

Paul I. Barton
Department of Chemical Engineering
Massachusetts Institute of Technology
Cambridge, MA 02139
copyright Paul I. Barton, March 2000
The equation oriented, simultaneous or equation based strategy is the major competing
technology to the sequential modular strategy. This strategy is much easier to describe
than to realize in a practical software implementation: rather than tearing recycle
streams and solving unit models in a modular fashion, an equation oriented simulator
assembles all the equations describing a process model together and attempts to solve
them simultaneously with a general purpose multi-dimensional root finding code.
One (largely abandoned) approach to general purpose root finding is known as tearing.
This tearing at the equation level should not be confused with the tearing at the
flowsheet level performed by a sequential modular simulator. In fact, the sequential
modular approach can be best interpreted as a decomposition approach in which a two
level nested iteration is established. The outer iteration uses a tearing approach to
converge, and the inner iteration uses the unit module specific methods to converge.
Further, the variables in the outer iteration (the tear variables) are chosen so that the
inner iteration breaks down into a series of subproblems (corresponding to the unit
modules) that can be solved sequentially. Even further levels of iterative nesting are
introduced by the subroutine libraries that calculate physical and transport properties.
For example, each bubble temperature calculation during each flowsheet pass typically
requires a Newton-Raphson type iteration.
There is one further flowsheeting technology called the simultaneous modular approach.
As the name suggests, this lies somewhere between the equation oriented and
sequential modular extremes. Broadly speaking, more and more variables are moved
from the inner iteration into the outer iteration and more sophisticated techniques are
used for convergence and derivative evaluation in the outer iteration. For example, the
generalized executive controller with multiple tear streams and design specification, and
Broydens method for convergence, can be interpreted as a simultaneous modular
approach. Other researchers have proposed the use of equation oriented solution
techniques and simplified models (e.g., linear, ideal) in the outer loop, and the use of
rigorous models in the inner loop to update parameters for the simplified model. In the
case of linear models in the outer loop, this can be interpreted as a derivative evaluation
strategy. Most modern flowsheeting packages have at least options that support a
more simultaneous modular approach.
EQUATION ORIENTED FLOWSHEETING PACKAGES
In more detail, the equation oriented strategy is composed of the following steps:
(i) equations and variables for all the unit operation models in the flowsheet are
defined individually (hence the model is still decomposed in a modular fashion,
even if the solution strategy does not exploit this).
(ii) these equations and variables are then assembled together into one large system
of nonlinear equations.
(iii) additional specifications are added until the degrees of freedom for the equation
system are zero and a well posed mathematical problem remains.
(iv) the equation system is solved directly and simultaneously by a general purpose
root finding code.
Figure 1 shows how these steps are implemented in the ABACUSS II system.
Clearly, the practicality of this approach hinges on the capabilities of the root finding
code to solve a very broad class of problems. Hence, the development of such codes
has received a large amount of academic attention (of course, this is entirely unrelated
to the fact that this problem is inherently more interesting to academics!).
This approach is clearly more elegant and intellectually satisfying, and has many
potential advantages:
(i) it is much more efficient. While the sequential modular approach appeared
very efficient for simulation specifications on unidirectional flowsheets (we will see
later that the equation oriented approach can take even greater advantage of such
problems), as soon as recycles or design specifications are introduced, it becomes
prohibitively expensive by comparison due to the need for repeated passes
through the flowsheet, and the nested convergence at the unit operation level.
(ii) the artificial distinction between simulation and design specification sets is
removed. In general, provided a specification set leads to a well posed problem,
there is little difference in computation load between specification sets (unless
linearity or the block decomposition are significantly altered from specification set
to specification set). Hence, for design problems in particular, the equation
oriented approach is much more efficient.
(iii) as an equation oriented simulator really does view unit operation models just as a
set of variables and equations, it is much easier to extend the model library and
modify existing models (i.e., it is not necessary to provide a subroutine with a
system of equations and a solution procedure embedded). In some simulators this
has led to the design of high level engineering oriented simulation languages that
allow engineers to specify models merely as sets of equations and variables. These
languages are effectively high level computer programming languages designed
for this specific purpose. Note that they are primarily declarative (i.e., they are used
to specify information) as opposed to procedural (i.e., they define a sequence of
instructions to be executed). An example of a unit model coded in one of these
languages is appended to these notes.
(iv) the previous point leads into the primary advantage of this technology, and why it
has not been abandoned: it is much more readily extended to other calculations
such as dynamic simulation or flowsheet optimization. As the architecture of the
simulator completely decouples the model from particular solution routines, it can
make sets of equations and variables directly accessible to several different
numerical solution routines. Equation oriented simulators such as ABACUSS II are
the closest current technology is to realizing the process modelling environment
introduced in an earlier lecture.
(v) diagnosis of certain errors in problem formulation is much easier; e.g., if the
specification set makes the problem badly posed, but this problem is not localized
to a single unit, an equation-oriented simulator can analyze the entire equation
system for problems such as singularity.
Given all these advantages, why are we bothering to learn about the sequential
modular technology? Clearly there are disadvantages:
(i) the general purpose nonlinear equation solvers are not as robust and reliable as
the sequential modular approach. In fact, for most relatively complex models one
has to virtually know the answer in order to get ABACUSS II and its counterparts
to converge. Again, an evolutionary approach such as modelling and converging
the components of a system before attacking the overall system model can help
tremendously.
(ii) it makes much larger demands on computer resources, particularly machine
memory (big matrices need to be stored). However, from about the late 70s
onwards advances in hardware (e.g., the increasing availability of large amounts
of memory) and numerical analysis (e.g., sparse unstructured linear algebra) have
removed these objections.
In light of the first deficiency, the equation oriented approach is not currently a viable
competitor to the sequential modular approach for steady-state process simulation.
We will be using the ABACUSS II equation oriented process simulator, primarily for
dynamic simulation of chemical processes. ABACUSS II has been developed by my
students and myself.
LANGUAGE TRANSLATOR
Data Structures
User
Input
Results
dH
dt

h
IN
(t)

F
OUT,i
h
i
i

1
NC

V
MAX
V
V

N
i
v
i
i

1
NC

H

N
i
h
i
i

1
NC

F
OUT,I
N
i
N
i
i

1
NC

F
OUT ,T
F
OUT,T
0
SYMBOLIC
MANIPULATION
NUMERICAL
ALGORITHMS
SIMULATION EXECUTIVE
Figure 1: Cycle of Problem Execution in ABACUSS II
The commercial products you are most likely to encounter are Aspen Custom Modeler
(formerly called SpeedUp) and gPROMS. Speedup was developed over a period of 25
years at Imperial College in London. It first appeared in 1986 as a commercial product,
and over the next five years it was widely adopted not for steady-state simulation, but
dynamic simulation and to a lesser extent flowsheet optimization. Until the recent
commercialization of gPROMS by a spin-off company also from Imperial College, it has
not had a serious competitor in the dynamic simulation market. The equation oriented
approach is inherently better suited to dynamic simulation, and, although many have
tried, there are fundamental theoretical problems with extending the sequential
modular approach to dynamic simulation. I developed the first version of gPROMS
during my Ph.D. thesis at Imperial College. All three systems (ABACUSS II, gPROMS
and SpeedUp) share a common intellectual heritage, so once you can use one, it is
relatively easy to transfer to the others.
MULTI DIMENSIONAL NEWTONS METHOD
From the above discussion, it is clear that the heart of any equation oriented simulator
is the multidimensional root finding code. This is the key to the viability of the
technology. It is ironic, particularly in light of the effort devoted to this problem in the
70s and 80s, that at the moment, the leading commercial products rely on a slight
modification of a 17th century algorithm. As we will be using this algorithm a lot, it is
worthwhile studying it in more detail.
Basically, Newtons method comes from the Taylor series expansion of a vector of
functions in terms of a vector of variables. The solution x* is expressed as an expansion
around the current estimate x
k
. For each function i this leads to:

f f
f
x
x x
f
x
x x
f
x
x x
i n
i i
k
n
n n
k k k
( ) = ( ) + H.O.T.
= 1
k k k
x x
x x x
* * * *

1
1 1
2
2 2

( )
+
( )
+ +
( )
+

K
K
where H.O.T. denotes quadratic and higher terms in the expansion. This can also be
expressed in matrix notation as:

f x f x J x x
k
x
k
k
( ) = ( ) + ( ) +H.O.T. * *
where J is the now familiar Jacobian matrix of partial derivatives:

J =

f
x
f
x
f
x
f
x
f
x
f
x
f
x
N
N N
N
1
1
1
2
1
2
1
2
2
1
L
M
O M
L L

1
]
1
1
1
1
1
1
1
1
Each row contains the gradient vector of the corresponding function.
If the above expansion is curtailed at the first order (or linear) term, we end up with the
following linear approximation to the functions in the region of the current estimate:

f x f x J x x
x
( ( ( ) ) ) +
k
k
k
We are interested in finding the point x* at which f(x*)=0. The above linear
approximation can be used to estimate this point:

f x 0 f x J x x
x
( ) = ( ) + ( * )
+ k
k
k k 1
where x
k+1
is the updated estimate for the solution. This leads to the following system
of linear equations that define the iterative process:

J x x f x
x
k
k k k +

( )

( )
1
Example:

f x x x
f x x x x
1 1 2 2
2
2 2
2
1 2 1
2
1
2
0
2 2 0
x
x
( )
( ) +
Partial derivatives:

f
x
x
1
1
2

f
dx
x x
1
2
1 2

f
x
x x
2
1
1 2
2 2

f
x
x x
2
2
2 1
2 2
So,

J

1
]
1
x x x
x x x x
2 1 2
1 2 1
2 2 2 2
and if

x
k T
( , ) 1 2 :

2 1
2 2
1
2
0
1
1
1
2
1

1
]
1

1
]
1

1
]
1
+
+
x
x
k
k
Solve these linear equations to get a new estimate for the solution.
Algorithm: Multidimensional Newtons Method:
1. k := 0; guess x
2. evaluate all functions (or residuals) - i.e. calculate f(x
k
)
3. check for convergence. For example, if

f x ( )
k
then STOP.
4. evaluate the Jacobian (or derivative evaluation) - i.e.calculate

J
x
k
5. assemble the system of n linear equations:

J p f x
x
k
k k
( )
and solve with any suitable method for

p
k
.
6.

x x p
k k k +
+
1
:
7.

k k : + 1. Repeat from step 2.
Convergence of this iterative process is very fast close to the solution; the number of
correct significant digits will double at each iteration. Current codes can converge
several thousand equations in a few seconds. The basic Newton Method is usually
modified in the following manner:
any updated
x
k+1
must obey the physical bounds on the values of variables, so
only a fraction of the full Newton step may be taken to ensure these bounds are
not violated:

x x p
k k k +
+
1
:
where

0 1 < . This is not to be confused with trust region strategies to
improve convergence.
the systems of nonlinear equations to be solved in process flowsheeting are
usually large, sparse and unstructured, hence the use of appropriate sparse linear
equation solvers for step 5 (see Duff et al., 1986).
modifications to the iterative process are required if the Jacobian becomes
numerically singular at one or more iterates (see below).
individual elements of the Jacobian may be evaluated numerically (i.e., by a finite
difference approximation) or analytically (i.e., from an analytical expression
derived by partial differentiation of the model equations with respect to the
variable in question). In most applications, this leads to a sparse, hybrid Jacobian
composed of some numerical and analytical entries. A quasi Newton update may
be applied on successive iterations to the numerical Jacobian entries. A novel
feature of ABACUSS II is that it derives analytical expressions for most of the
partial derivatives automatically from the equations input by the user via what is
called automatic differentiation.
The success of this iterative process hinges on the quality of the initial guesses.
Newtons method is what is called a locally convergent method. Put simply, this means
that the iterative process will only converge if it is started from an initial guess within a
certain region around the solution. If the initial guess is outside this region, a solution
will not be found or the iterative process will converge to another root. Unfortunately,
there is no way to tell a priori if an initial guess is in this region of convergence.
However, the closer an initial guess is to the solution, the more likely it is that
convergence will be achieved. Hence providing good initial guesses is the key to
success.
BLOCK DECOMPOSITION - PARTITIONING AT THE EQUATION LEVEL
We have already seen how the algorithm strongly connected components can be
applied to a process flowsheet represented as a digraph to identify those minimal
subsets of units that must be solved simultaneously, and the sequence in which those
subsets must be solved. In this section we will discuss algorithmic approaches to
identifying a similar partitioning and precedence ordering applied directly to the model
equations. This can be viewed as a finer grained decomposition than that possible at the
unit operation level.
As usual, the algorithm is applied to the incidence matrix. The objective is to identify
independent equation (row) and variable (column) permutations so that the permuted
incidence matrix becomes block lower triangular with minimal blocks (e.g. the square
diagonal blocks cannot be decomposed further by row and/or column permutations).
This block lower triangular structure is illustrated in Figure 2.
As usual, the algorithm is applied to the incidence matrix. The objective is to identify a
variable (column) permutation, i.e.:

x Qy
and an equation permutation, i.e.:

g y Pf x ( ) = ( )
where Q and P are permutation matrices, such that incidence matrix corresponding to
the permuted Jacobian:

g
y
P
f
x
Q
has a block lower triangular structure (Figure 2) with minimal blocks (e.g., the square
diagonal blocks cannot be decomposed further by row and/or column permutations).
Note that a non zero entry ij in the incidence matrix is equivalent to the existence of the
partial derivative J
ij
, hence operations on the incidence matrix are sufficient to identify
this permutation.

g
y

1
]
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
Figure 2: Illustration of Block Triangular Decomposition
The advantage of discovering this permutation is that it has decomposed the system of
equations into the sequential solution of a series of subproblems. The first square
diagonal block of equations can be solved for the variables appearing in these equations
independently of all the other equations in the system (because these equations are
fully determined in this subset of the variables). Once these variables are calculated,
they can be substituted into the equations defining the second block and the second
block can be solved for the corresponding variables. The calculation sequence continues
in this manner until the whole system is solved.
To elaborate, block decomposition has the following advantages:
a large system of equations is decomposed into a sequence of smaller systems of
equations (faster computation, less computer memory required at each step).
the partial solution substituted into later blocks in the decomposition improves
robustness of the locally convergent algorithms applied to each block - this can be
viewed as having exact initial guesses for already calculated variables.
some of the blocks may be linear (and hence can be solved directly) or blocks
involving just one variable/equation (for which root finding procedures are much
more robust).
derivative evaluation for the off diagonal blocks is never required.
Users will see ABACUSS II automatically block decomposing a problem and solving the
blocks in a sequence.
Given what we have learnt already, the algorithm to derive a block decomposition is
easy to describe. We will describe a two step strategy based on the most efficient
algorithm for each step:
1. Duffs algorithm is applied to obtain a transversal. If the system is structurally
singular, terminate. Otherwise a row permutation has been identified such that
nonzero entries lie on the diagonal.
2. Use the transversal identified in step 1 to derive the directed graph representation
of the system of equations. Apply Tarjans algorithm to this graph to identify the
strongly connected components and a precedence order. The strongly connected
components are equivalent to the diagonal blocks in the block decomposition.
Clearly, the fact that Duffs algorithm is applied in step 1 implies that we get a check on
the structural singularity of the system for free. The fact that the diagonal blocks
correspond to the strongly connected components of the graph demonstrates the
essential uniqueness of the block triangular form. Permutations of equations or
variables within a block (strongly connected component) does not alter the
decomposition, and although there may be limited alternative orderings for the blocks
(precedence orderings) the fact that the blocks are strongly connected components
indicates that the number or size of blocks cannot be reduced further.
Example: solve the system of 5 nonlinear equations:

f x x
f x x x x
f x x x
f x x
f x x x
1 1 4
2 2
2
3 4 5
3 1 2
1 7
4
4 4 1
5 1 3 5
10 0
6 0
3 6 0
6 0
( )
( ) =
( ) = ( - 5) - 8 = 0
( ) =
( ) =
x
x
x
x
x
+

+
+
.
the incidence (or occurrence) matrix and step 1 (the ouput set assignment) yields:

x x x x x
f
f
f
f
f
1 2 3 4 5
1
2
3
4
5

1
]
1
1
1
1
1
1
this yields the digraph shown in Figure 3.
f
1
f
2
f
3
f
4
f
5
Figure 3: Digraph Representation of Equation System
When Tarjans algorithm is applied to this digraph, it identifies the following strongly
connected components and precedence order:

( , )
( , )
f f
f
f f
1 4
3
2 5

solution
sequence
and the corresponding permuted incidence matrix is:

x x x x x
f
f
f
f
f
1 4 2 3 5
1
4
3
5
2

1
]
1
1
1
1
1
1
We can now submit the subproblems to our Newton solver in the following sequence:
(a) solve:

x x
x x
1 4
4 1
10
3 6
+

for

x x
1 4
, { }. Note this can be done directly (in general) with a sparse linear
equation solver.
(b) substitute

x x
1 4
, { } into:

x x x
1 2
1 7
4
5 8
.
( )
and solve for

x
2
{ }. Note this can be solved by a one dimensional root finding
routine.
(c) substitute

x x x
1 4 2
, , { } into:

x x x x
x x x
2
2
3 4 5
1 3 5
6
6 0

+
and solve for

x x
3 5
,
{ }
. In fact, these equations are now linear in

x
3
and

x
5
!
Although the above example is a rather dramatic demonstration, the very high
efficiency of the algorithm to derive a block decomposition motivates its application by
default to general problems.
It is important to recognize that the above approach will at least recognize the
decompositions possible through a study of the flowsheet structure (e.g., if the
flowsheet is unidirectional, the system of equations will block decompose). A classic
example of a situation in which an equation based analysis will identify a finer
decomposition is when the mass balance of a unit or a group of units is decoupled from
the energy balance. Although this situation only arises for certain flowsheets, models,
and specification sets, the equation based analysis will solve the mass balance and then
the energy balance, whereas the unit based analysis will solve both simultaneously.
The Dulmage and Mendelsohn Decomposition
The block decomposition ideas can also be used to develop a semi-automated approach
to building a valid specification set (in a structural sense). In ABACUSS II, this feature is
called the Intelligent Degree of Freedom Analyzer.
Dulmage and Mendelsohn (1963) demonstrated that any m by n incidence matrix (i.e.,
possibly rectangular) can be transformed by row and column permutations to a
canonical decomposition composed of three parts: an over determined part, a fully
determined part and an under determined part. This is illustrated by the example
shown in Figure 4.
x
1
x
2
x
3
x
4
x
5
x
6
x
7
x
8
x
9
x
10
f
1
f
2
f
3
f
4
f
5
f
6
f
7
f
8
f
9
f
10
Over Determined Part
Under Determined Part
Fully Determined Part
Figure 4: Dulmage and Mendelsohn decomposition for an example incidence matrix
Each part gives very useful information to help debug a model. If the over determined
part exists, then the matrix in question is structurally rank deficient. In the example, we
have three equations in terms of only two variables, so a transversal cannot be
extended for these equations. In general, the equations that end up in the over
determined part are structurally inconsistent, in the sense that there are more equations
than variables involved. The corresponding system of equations will be either
inconsistent or redundant. Hence, this situation can only be rectified by deleting one or
more of the equations in the over determined part.
The fully determined part may have a finer internal structure, which is exactly the block
lower triangular decomposition discussed above. The variables in the fully determined
part are fully determined by the equations (in the structure sense), hence they should
not be selected as degrees of freedom.
If the under determined part exists, it means that a unique solution cannot be
determined for the problem as it stands; there are less equations than variables in the
under determined part, and therefore degrees of freedom that must be specified. Only
those variables that appear in the under determined part are candidates to be specified
as degrees of freedom.
An algorithm that computes the Dulmage and Mendelsohn decomposition for an
arbitrary rectangular incidence matrix is described in Pothen and Fan (1990). Suppose
that you are developing a model in ABACUSS II and that you are ready to simulate the
model. Before starting the numerical solution of a simulation, ABACUSS II uses this
algorithm to compute the Dulmage and Mendelsohn decomposition for your problem.
Note that (unwittingly) your problem, as currently stated, may have a rectangular
incidence matrix; ABACUSS II will give you guidance on how to reformulate your
problem so that it has a square incidence matrix. This analysis might lead to any of the
following feedback on your model:
if the number of unknowns is less than the number of equations, ABACUSS II will
report this. You need to specify more degrees of freedom.
if there is an over determined part, ABACUSS II will tell you that a certain number
of equations must be removed from the model. It will also suggest which equations
it is necessary to delete. For the example above, it would say delete one out of the
set of equations { f1, f2, f3 }.
if there is a under determined part, ABACUSS II will tell you that a certain number
of additional variables must be specified as degrees of freedom. It will also suggest
which variables to specify. For the example above, it would say specify one out of
the set of variables { x8, x9, x10 }. Note that one has to be a little careful with this. If
you specify more than one suggested variable at a time out of a particular group of
variables, you may introduce an over determined part in the resulting new
incidence matrix formed from deleting the columns specified. The best way to avoid
this is to work interactively with ABACUSS II, specifying only one variable from
each group at a time, and then asking ABACUSS II to compute a new Dulmage and
Mendelsohn decomposition each time. This way the advice on which variables can
be selected as degrees of freedom is always up to date.
ABACUSS II will only proceed with a simulation if the incidence matrix is square, and
the Dulmage and Mendelsohn decomposition only has a fully determined part. This
indicates that the problem is well posed in the structural sense. Clearly, this feature is
useful when constructing a model, altering the specification set, or when a structurally
singular problem is detected.
PROCEDURES
In addition to equations defined symbolically in the simulation language, most equation
oriented simulators also allow equations to be defined by PROCEDUREs. These are
effectively equations of the following functional form:

x p
i
( ) = 0 x
where

p( ) x is a function evaluated by a call to a FORTRAN subroutine, which is linked
into the simulation executable.
If the FORTRAN subroutine above performs some form of iterative calculation in order
to evaluate the function, then a nested iteration strategy is established. The outer loop
consists of all the model variables, and the inner loop consists of iterative solution of the
procedure(s) given the current iterate variables. There is a subtle point here. The
equation above is not solved by the procedure. At each step in the outer iterative loop
the following residual is evaluated:

f x p
i
k
i
k k
( ) = ( ) x x
and it will be nonzero except at the solution. In order to evaluate this residual for one
step of the outer iteration, the subroutine is called and converged (the inner loop).
However, there are no guarantees that

x
i
k
will equal the value returned by p. In the case
of, for example, a phase stability calculation, the function is defined by the global
solution of an optimization problem (minimization of Gibbs free energy), so the
procedure emdeds a numerical optimization inside the overall process simulation.
Procedures are primarily used as interfaces to a physical property subroutine library.
This is the primary manner in which physical properties are evaluated. For example, in
order to relate a vapour phase specific enthalpy to the composition, temperature and
pressure, the following procedure would be used:

H p T P
v H
v
( , ) = 0 y ,
where H
v
, y, T, P are model variables representing the enthalpy, vapour phase mole
fractions, temperature and pressure, and

p
H
v
is a hook to a FORTRAN routine that
calculates vapour phase enthalpy given mole fractions, temperature and pressure. Note
that in a transversal on the overall system, H
v
does not necessarily have to be assigned
to this equation (in fact, it is usually T). The procedure equation just establishes a
functional relationship between the four variables.
Another advantage of procedures is that they reduce the size of the process model (or
equivalently the number of variables in the outer iteration). For example, the
evaluation of k-values (vapour liquid equilibrium distribution coefficients) for an ideal
vapour phase and nonideal liquid phase uses the formula:


k T
T P T
P
i
i i
SAT
(
( )
, )
, ( )
x
x

in other words, it introduces the activity coefficients and the pure component saturation
pressures as extra variables. The use of a procedure to evaluate the k-values obviates
the need to introduce these extra variables into the outer iteration. The use of tailored
algorithms in the inner iterations (procedures) can also improve the robustness of the
overall iterative process.
In earlier days, authors have also advocated the use of procedures to represent
discontinuous or nondifferentiable functions. The need for this is less these days
because of language structures like IF equations that allow explicit declaration of such
discontinuous functions. In fact, it is highly undesirable to hide discontinuous
functions in procedures from the numerical solution routines of the simulator,
especially in dynamic simulation (Barton, 1992) (see also below).
FUNCTIONAL, STRUCTURAL AND NUMERICAL SINGULARITY
We are beginning to understand the role that the Jacobian matrix J plays in the solution
of systems of nonlinear equations. For example, any Newton family method relies on
an iteration matrix that is either the Jacobian matrix itself (evaluated at the current
values of the iterate variables x
k
), or some approximation to this Jacobian. If the
iteration matrix is singular, the system of linear equations that determines the iterate
update cannot be solved uniquely. Thus, the iterative process breaks down and another
strategy is required to continue.
In this section, we will briefly introduce the broader implications of singularity of the
Jacobian matrix. In fact, the analogy with systems of linear equations is almost exact.
For a system of linear equations:

Ax b 0 - (SI1)
the matrix of partial derivatives is clearly A itself. We already know that results
concerning the existence and uniqueness of a solution to (SI1) are based on the
singularity or nonsingularity of A. Further, we note that if (SI1) is substituted into the
Newton iteration formula, then (SI1) is solved in a single Newton step (not exactly a
stunning discovery!). In the linear case of (SI1), singular A implies a singular Jacobian
for all realizations of x.
Functional Singularity
Roughly speaking, results on the existence and uniqueness of a solution to a system of
nonlinear equations:
f(x) = 0 (SI2)
are based on the singularity (or not) of the Jacobian matrix J(x). However, singularity of
this matrix for only certain realizations of x does not imply (SI2) does not have a locally
unique solution.
Definition: functional singularity refers to the case when J(x) is singular for all
realizations of the vector x. A functionally singular system is badly posed.
Example: consider the nonlinear system (SI3):

x x
x x x x
1 2
1
2
1 2 2
2
3 0
2 0
+
+ +
(SI3)
The Jacobian matrix is:

J x ( ) =
1 1
2 2 2 2
1 2 1 2
x x x x + +

_
,

(SI4)
Note that this is a matrix of functions rather than fixed values, hence the singularity of
the Jacobian is potentially a function of the values assigned to x. Clearly, (SI4) is singular
for all x: the second row is a multiple of the first row regardless of the particular
realization of x (although the value of the multiplier will change with x). Hence (SI3) is
functionally singular, and we cannot satisfy both existence and local uniqueness of a
solution. The qualifier local refers to the fact that nonlinear systems may have multiple
(but not infinite) roots, whereas in the case of linear equations the only choices are one
or infinite solutions.
A functionally singular system is either redundant (which implies an infinite number of
solutions) or inconsistent (a solution does not exist). This provides the theoretical
conditions for these two problems which we have already discussed and illustrated in
the linear case.
What are the implications of functional singularity for the example (SI3)?
If = 9 then the second equation is exactly a function of the first equation (move the
constant to the right-hand side and then square both sides). In this case, the second
equation is redundant with the first (it does not constrain the solution further) and an
infinite family of vectors x will satisfy these equations (i.e., any x satisfying

x x
1 2
3 + ).
If 9 then no combination of values

( , ) x x
T
1 2
will satisfy both these equations and the
system is inconsistent.
In physical models, functional singularity normally implies an error in formulation of
the model, or in selection of a specification set. In the former case, the most common
mistake is overspecification of a quantity by the equations that make up a model. For
example, consider the subset model equations (SI5).

L L x i NC
x
L L
i T i
i
i
NC
i T
i
NC

1
1
1
1
K
(SI5)
Because of the first equation, the latter two equations effectively say the same thing -
the first equation makes the other two redundant with each other (i.e., the third can be
derived by substituting the first into the second). In this example, either the second or
third equation should be present in the model (to define L
T
in terms of the L
i
), but not
both simultaneously (otherwise the model will be functionally singular).
In deleting the equations that overspecify a quantity, further action may be required. If
the extra equation really was superfluous to needs, then no further action is required (in
this case, the fact that there are less D.O.F. than anticipated can warn of this problem).
On the other hand, overspecification of one quantity could mean that another quantity
is not properly defined by the system of equations and we have missed an equation in
the model formulation. This can usually be detected by structural criteria (remember:
structural singularity implies that there is a subset of equations involving less variables
than equations). If there seem to be too many D.O.F. and a structural analysis forces
one to specify a variable that should naturally be an output from the model, it is likely
that the variable concerned is not defined by and coupled to the rest of the system, and
a further equation is required.
Unfortunately, beyond the structural techniques (see above and below) I know of no
systematic technique that can help a modeller derive fully determined and functionally
nonsingular systems of equations. In my experience, the responsibility falls to the
engineer: experience, physical insight and understanding, certain tricks of the trade, and
inevitable errors lead to the final model. As an example of a trick of the trade, there are
several arguments that favour the use of:

x
i
i
NC

1
1
instead of the equivalent:

L L
i T
i
NC

1
in process models, most notably because the former formulation can avoid multiple
nonphysical roots of the equations.
Structural Singularity
This is a concept we have defined, discussed at length, and studied algorithms for
detection: all that is really necessary to add is that an incidence matrix is clearly a
structural representation of the Jacobian matrix. Hence, structural singularity implies
functional singularity, but the converse is not necessarily true (i.e., structural singularity
is sufficient for functional singularity).
Detection of Functional Singularity
The sufficiency of structural singularity implies that efficient structural algorithms (cf.
Duff) should always be applied to a model. Detection of functional singularity in this
manner is unambiguous. However, if the structural criterion is satisfied, we cannot be
sure our model is functionally nonsingular. In this case, we must resort to numerical
experimentation: the fact that the Jacobian is singular at several different realizations of
x is a good indication (but not proof) of functional singularity. On the other hand, a
single counter example of a nonsingular Jacobian is sufficient to prove functional
nonsingularity. Both the examples in the section above would not be detected by a
structural algorithm.
Numerical (Local) Singularity
A system of equations that is functionally nonsingular (and hence well posed) can still
have a Jacobian that is singular for a strict subset of realizations of x. As mentioned
above, if we are unlucky, the iterative process can yield an iterate vector in this subset.
This case is known as numerical or local singularity of the Jacobian, and is illustrated in
one dimension in Figure 5, where the Jacobian is singular at points a and b, and a locally
unique solution exists at x*.
In higher dimensions, numerical singularity is a consequence of linear dependence of
one or more of the gradient vectors of the individual functions

f
i
. The simplest example
would be a stationary point in one of the

f
i
(see Figure 5).
f(x)
0
a b x
ROOT
x
*
Figure 5: Illustration of numerical singularity in one dimension
The problem posed by numerical singularities can be illustrated by Newtons method,
for which we solve the linear system (SI6) at step k.

J p f
x x
k k
k
(SI6)
If J(x) is numerically singular at point x
k
, Gaussian elimination on

J
x
k
*
, will yield a
matrix with the structure:

1
]
1
1
1
1
1
1

1
]
1
1
1
1
1
1
p
k
(SI7)
where the number of non zero rows equals the rank r (<n) of

J
x
k , and (SI6) cannot
determine unique values for p
k
.
The solution to this problem is simply to ignore the last (n-r) rows in the factorized
matrix, to yield a r n rectangular linear system:

*
note that

J
x
k denotes a numerical realization of the matrix of functions J(x) at point x
k
.

1
]
1
1
1

1
]
1
1
1
p
k
(SI8)
and assign arbitrary values to the last (n-r) elements of p
k
, solving (SI8) for the first r
elements.
In matrix notation:

U B
p
p
b
Up Bp b
Up b Bp
1
1
1
1
[ ]

1
]
1

[ ]
+

k
k
k k
k k
2
1 2
1 2
and

p
2
k
are arbitrary values, U is an r r upper triangular matrix.
Example: consider the system in (SI9):

x x
x x x
x x
1 2
1 2 3
1
2
2
3 0
2 0
2 6 0
+
+
+
(SI9)
One root is

x ( , , ) 2 1 0
T
and the Jacobian is:

J x ( ) =
1 1 0
1 2 1
2 2 0
1

1
]
1
1
1 x
and this matrix is nonsingular at the solution. However, with the initial guess

x
0
1 1 1 ( , , )
T
(and in fact any point with

x
1
1 ) we will encounter a numerical
singularity.
First Newton iteration:

1 1 0
1 2 1
2 2 0
1
0
3
1
2
3

1
]
1
1
1

1
]
1
1
1

1
]
1
1
1
p
p
p
Gaussian elimination yields:

1 1 0
0 3 1
0 0 0
1
1
5
1
2
3

1
]
1
1
1

1
]
1
1
1

1
]
1
1
1
p
p
p
so we ignore the last equation:

1 1
0 3
0
1
1
1
1 1
0 3
1
1
1
2
3
1
2 3

1
]
1

1
]
1
+

1
]
1

1
]
1

1
]
1

1
]
1

1
]
1
p
p
p
p
p p
and take an arbitrary step in

x
3
(e.g.,

p
3
0 1 . ):

1
]
1

1
]
1

1
]
1

1 1
0 3
1
1 1
0 633 0 367 0 1
1
2
0
p
p .
. , . , . p ( )
and the iteration formula becomes:

x x p
1 0 0
+ (1.633,1.367,1.1)
T
and

J
x
0
is nonsingular, so the iterative process can proceed.
Note that a numerical singularity can occur at a root of the system of equations (

x
2
0
is the simplest example). Typically this will imply multiplicity of the root.
References and Further Reading
Barton P.I., The Modelling and Simulation of Combined Discrete/ Continuous
Processes, Ph.D. Thesis, University of London, 1992.
Dumage A.L. and N.S. Mendelsohn, Two Algorithms for Bipartite Graphs, SIAM
Journal, 11(1), pp. 183-194, 1963.
Duff I.S., A.M. Erisman and J.K. Reid, Direct Methods for Space Matrices, Oxford
University Press, 1986.
Pantelides C.C., SpeedUp - Recent Advances in Process Simulation, Computers and
Chemical Engineering, 12, pp. 745-755, 1988.
Perkins, J.D. and R.W.H. Sargent, SPEEDUP - a computer program for steady-state
and dynamic simulation and design of chemical processes, AIChE Symposium Series,
78, pp. 1-11, 1982.
Perkins J.D., Equation-oriented flowsheeting, Second Conference on Foundations
of Computer Aided Process Design, Snowmass, Colorado, 1983.
Pothen A. and C.J. Fan, Computing the Block Triangular Form of a Sparse Matrix,
ACM Transactions on Mathematical Software, 16(4), pp. 303-324, 1990.
Sargent R.W.H. and A.W. Westerberg, SPEED-UP in Chemical Engineering
Design, Trans. Institute of Chemical Engineers, 42, pp. T190-T197, 1964.
Westerberg, A.W., H.P. Hutchinson, R.L. Motard and P. Winter, Process
Flowsheeting, Cambridge University Press, 1979.
Example of High Level Equation Based Language
# ---------------------------------------------------------------------------------------------------------
# Simple extent reactor model - mass balance only
#
# Author: Paul I. Barton
# Date: 3/9/00
# Language: ABACUSS II
#
# Parameters:
#
# NoComp - number of components in the process streams
# NoReaction - number of reactions taking place
# Stoich - stoichiometric coefficients for the reaction(s)
# Key - component number for the key reactant(s)
#
# Simple specification set:
#
# Inlet flowrates plus one out of molar extent and fractional
# conversion for each reaction.
#
# Note:
#
# The molar extent and fractional conversion for each reaction are NOT
# independent.
#
# ---------------------------------------------------------------------------------------------------------
MODEL Extent_Reactor
PARAMETER
NoComp AS INTEGER
NoReaction AS INTEGER
Stoich AS ARRAY(NoReaction,NoComp) OF INTEGER
Key AS ARRAY(NoReaction) OF INTEGER
VARIABLE
Molar_Extent AS ARRAY(NoReaction) OF Positive
Fractional_Conversion AS ARRAY(NoReaction) OF Fraction
Flow_In, Flow_Out AS ARRAY(NoComp) OF Flowrate
X_In, X_Out AS ARRAY(NoComp) OF Fraction
STREAM
Inlet : Flow_In AS MainStream
Outlet : Flow_Out AS MainStream
EQUATION
# Material balance
FOR I := 1 TO NoComp DO
Flow_Out(I) = Flow_In(I) + SUM(Molar_Extent*Stoich(,I)) ;
END # for
# Define the fractional conversion for each reaction
FOR J := 1 TO NoReaction DO
Fractional_Conversion(J) =
(Flow_In(Key(J)) - Flow_Out(Key(J)))/Flow_In(Key(J)) ;
END # for
# Define mole fractions in inlet and outlet streams
Flow_In = X_In*SUM(Flow_In) ;
Flow_Out = X_Out*SUM(Flow_Out) ;
END # Extent_Reactor