An Efficient Heuristic Procedure for Partitioning Graphs

By B. W. KERNIGHAN and S. LIN

W. ",,",id.r the pToo~m "/ partiliuning Iht nOON ul " graph with "MI. 0" it.. ed9t~ intu .ub.d~ of given sizes .D "'" I. ",i"i",; .. tho ;fUm 01 th. Ul!t.. 1m ..u uIg •• cui. Th~ proo/cn II"':" •• in ,""<rill phYa:1cal ,ilu/liion!-joT aIImplt, in o.uigni,,'g tht campon",," of ekclronic circui" to circuil_ bo",d. 10 minim ... tht number of conMctiun. btlwt." boaTda.

Thi, paper p,.,enl. II heuristic ",dhOO Jor parlitioni"ll oTbi'TOTllI1Toph. which ;, bolh _gtd;"" in finding oplimal partition" and /0.1 en01.00h 10 b. prlld..<.ol ;71 ,owing large problema.

r. INTROIlUCTION

1.1 D.finilion of tht Problem

Th", paper deals with the !ollowing combinatorial problem: given a graph G with C01!Ls on il.8 edges, partition !.he node. II! G into BUbael.8 no larger than II "Wen mtu;'mum .ilt, so as to minimize t.h. total coet of the edges cut.

One important practical example of thi! problem is placing the components 01 an electronic ei reui t onto pri "ted circuit cards or subateates, so I\S to minimi ze the number of connecticns between cards. The componenta are the node! of the graph, and the circuit connections are the edges. There is some maximum number of components which may be plsced On any card. Since connections between cards have high cost compared to connections within a board, the object i~ to minimize the number of interconnections between cards.

This partitioning problem nlso arises naturally inlu) attempL to improve the p.rlging properties of prograrus lor use in computers with pe.ged memory organization. A program (lit least statically) can be thought 01 Ill! a set of connected entities. The entities might be 8UI)· routines, or procedure blocks, or single instruction and data items, depending on viewpoint and the level of detail required. The connection. between the entities might. represent possible flow or t.tBnsler of control, Or references from one entity to another. The problem is to assign the objects to "pages" of II given size so 8.5 to minirnise the number of references between objects which lie OD difJerent pages.

To pose the partitioning problem mathematically, we shall need the following definitions. Let G be u graph of n nodes, of sizes (weighte) w, > 0, i = I, ... , n. Let p be a posi tive number, sueh thal 0 < Wo =- p for all i. Let C - (coo), i, j - I, ••. ,71 be a weighted connectivity matrix describing the edge.s 01 G.

Let k be R positive integer. Ak-way pfJdilion 01 G is II act of nonempLy, psirwiee disjoint subsets of G, v, , ... , II, such that V:., ", c C. A partition is odmil8ible if

I v, I :0 p for all i,

"'here the By m bol I :r I stands for the .ut 01 a ad z, and equals the sum of the ~i.e.s of all the elements 01 z; The cu.f of 8 partition is the sumrnution 01 c" over nil ,. and j such that .- and j lire in diJiercnt subsets, The cost is lhue the sum of all external COlI!.! in the partition.

The partitioning problem we consider here i~ to find" minimal-cost admissible pnrtiticn of G.

There are three other problems which Bre equivalent to thil! one.

Fi"t, minimiaing externa) cost is equivalent to maximising internoJ C01!I becQIIlI-C the towl cost 01 all edges is ccneiact. Further, by changing the aigna 01 all c., 's, we can maximize externnl cost, or rninirni .. e internal coat.

u E%4d Solulion.

A strictly exhaustive procedure for finding the minimal cost partition ill ouen out of the question. To Bee this suppose thnL G hes " aodes of eile I \.0 be partitioned into k eubsets of ei.e 'P, w here kp ~ n. Then there !LTe (;) WRY" 01 ehccsing the lI ... t "ubaet, (.:.) W~y8 for the second, lind &0 on. Since the ordering 01 the subseL!! ie irnrnnterial, the number of CMeI! i$

For most values of n, k, and p, thie expression yields a very large nurnber; for e1Ilmple, for n - 40 and p - 10 (k - 4), it is greater lb8.0 10".

Formally the problem could also be solved as an iDt.eger linear programming problem, with" IILrge number of constraint equation.. neces.ary to express the Uniformity of the p&rtition.

BeCAUAe it seems likely that any direct approach to find.ing &n optimal solution will require an inordinate a.mount of eomput.ation, we tum to an examination of heurieucs, Heuriatic methods cal) prod uce good IIOlutiona (posslbly even an optimal solution) quickly. Often in pr""'tical applicatioTUl, several good sclutiona are 01 more value thlLn one optimal one.

The fint and fcrernost consideration in developing heuristica for combi"",torial problema of this type is findiog II procedure that is powerful and yet au.fficie.nUy faat to be practical. A process whOSll running time grow8 eXpOncoUally or ["",torially with the number of vertices 01 the graph is not likely to be practical. In most cases, a growth rate of more than the 8quo.re 01 !.he number of vertices is still not too practical. (If the running time of a procedure grcws M f(n), where" is the number of vertices involved, we 8han reler to it as an f(n)-procedure.)

1.3 Folu Sw.r"

To point out 8 rew pitfall.e, we mention some unsuccessful attempl.!! at heuristic IIOlutiona to the partit.ioning problem.

1.8.1 &ndom ,solution,

ODe tactic is simply to generate random solutions, keeping the beat seen to dllte, and terminating after some predetermined time or v .. lue is reached. This ia quite fast, although nctuaUy au n'-procedw-e. Unfortuna tel y 0 this approach is U IlM tisf ac tory for pro ble [Jl.9 of even moderate size, since there are generally few optiraal or near-optimal solutions, which thWl appear randomly "'ith very low probabilities_

• Experience with 2-way partitions for II cless of (}-l matrices of ai~e 32 X 32, for example, baa indicated that there are typically 3 to 5 optimal partitiona, out of a total 01 Jr.!) pnrtitions, giving B probability of BII= on any trial of less than 10-'.

1.3.2 MQ.% PLow-Min Cur

Another pll.rtitiorung method is the Ford and Fulkerson max flow-miD cui algorithm'. The graph is treated lIB a network in which edge cools correspond to maximum flow eapacities between pai ... 01 nodes, A cut is a separation of the nodes into two disjoint eubsets, The mAX flow-min cut theorem etates thn t the maximal flow values between any pllir of nodes i.I! equal to the minimal cut CIIpII<city of .. 11 cuts which separate the two nodes. In our terminology, " cui is II 2-wllY partition, and the cut CIIp&city is the cost of the partition. The Ford and Fulkersou algorithm finds a cui with maximBl flow, which is thus a minimal cost cut; thill represente a mini mum cost partition of the graph into two 8Uhee!.! of unspecified li~e8.

There are several diffrcultiea involved in using the Ford and Fulkerson algorithm for our partitioning problem. The most severe of these ia the lact that the algorithm hClS no provision for constraining the SiLes 01 the resultant subseta, and there seems to be 00 obvious way to extend it to include tbill. Thus if flow methods are used to perform II aplit, then furLher procesaing ia neceesary to make the resulting .ubuts the correct size. If the aubsets arc grcally diJierent in aiec, then use of thi~ algorithm will hu ve prod uced CMCntilllly no benefit. Hence in epi Le of its thcorctica I elegance, the Ford and Fulke ... on algorithm is not suitnble for thi. application. (Note however, that since iI, does find the minimal cost unconstrained 2-w1lY partition, the value it produces ill a lower bound lor solutions produced by any method.)

Reprinted with permission from Bell Sysl. Teen. J., vol. 49, no. 2, pp. 291-307. feb. 1970.

Copyright @ 1970. AT&T.

76

PARTITIONING GRAPHS

1.3.1 Chutering

A et ... of much more intuitive method, it baAed on identifying "nat.ur&! el\lJters" in the given con matrix-that ie, groupe of nodes which are .trongly connected ill sorne sense. For example, one ca.n WIll very simple heuristics [or building up elustera, bused on collecting together element.s corresponding to huge vulucs in the cost mo.trix. But lLgain these methods do nol in gencrnl include much provision for .... tidying eonstmint.s on theai.e3 of the sub5ela. nor do they provide for 'YIlLemntie assignment 01 "stragglers" (nodes which do not obviously belon~ to any pnrticular subset).

I..J.' ,,-0 piing

Lin, working on the Traveling Salesman Problem, ISee Ref. 21 C&t.egorited .. !let of methods of improving given solutions by rearranging single links, double links, triplets, und in general, ). linb_ He referred to a change involving the movemenv of ). links 8.8 a ).-.::hange. If a conliguration of the system is reached in which no ).-.::hange can be ms.de which result. in a decrease in cost, the conrigurarion is said to be "x-opt,"

For the partitioning problem, ILII &nalogou9 operation is the interchange of groups of x pointe between 1\ p~ir of sets, Thus 1\ l-change is Lhe e.xchaoge of a single point in one set with a aingle point in another set. A con Ii gu ru.ti on is then sai d La be "I-op L" if th ere ex illt.s no in terchange 01 two poicts which decresses the cost of the partition, Experiments La evaluate l~pung for 2-ws.y partitions of 0-1 matrices (32 X 32) within which .. bout one-half of Lhe elements were nonzero, show that apparently opLinu.1 values can be schieved in about 10 percent of the trials; values within 1 or 2 of the optimal enn be achieved in about 7.1 percent of C8.8e8.

It appears fruitless to extend X beyond I (l-opung ia s.lready &II n'-procedure), or to extend l-opLing experiments to partitioDB into more than two eubsets, since more powerful method! have been developed. Th esc me t hods U"e the to pic of the nex t !leet.i 008_

II, TWO-W"Y UNIFORM P"RTITIONB

l.1 In.lrOOuclilm

The .impleat p&rtitionillg problem which still contains all the aignificant features of larger problems is that of finding a m.inimal-coet partition of .. given graph of 2n vertices (of equalaize) into two !ubaeta of n vertices each, The solution of the 2-way partitionillg problem ia the subject of thia section. The solution provides the ba.si.:l for aolviog more general partitioning problems. In Section 2.6, we disc\l5! 2-way partitioDB into sets of unequal size.

Let S be " '5et of 2n poiuts, with an associated cost matrix C .. (c,,), i, j - 1, ... , 2n. We 9.S9ume without 1056 oC gcnernlity that C is s. symmetric mil. trix, and that ell .. 0 for all i. There is no a.s.sumpt.ion about nonnegativity of the C..'8. We wish to pBr1.ition S into two l!etrl A and B,uch with n points, such tllat the "external coat" T .. !:.u c ••

i. miuimised.

III essence, the method is th is: starting ",;th s.ny arbit...ry p&rtit.ion A, 8 or S, try to decrease the initial external cost T by u series of interehungcs of subsets of A and B; the subsets Sore chosen by an s.lgorithm to be described. When no further improvement is posaible, the resulting partition .-t', S' is locally minimum with respect to the algorithm. We shall indicate that the resulting pa.rtition has a fairly high probability o[ being a globally minimum partition.

This precess can then be repeated with the generation of &nother arbitrary .tarting partition A, 0, and 90 on, to obtain &II many locally minimum partilion!l II.!! we desire.

Given S and (Col). suppose A·, 0' is a minimum cost 2-wBy pe.rtition.

Let A., 8 be any arbitrary 2-way partition, Then clearly there are 8U'Dse18 X C A, Y C B with 1 X I ~ I Y I :0 n/2 such that interchanging X and Y produces A' and 8' II..!! shown below.

A· .. A."X +V B·.6-y +)(

The problem is to ideality X and Y from A and 8, without considering eJl poesible choices. The proce59 we describe finds X and Y approximately, by sequentially identifying their elements,

Let WI dd; ne for each a I A, an al.rnal cod E. by

E.- Le,. .,.

and an inla7u&l cod I. by

1 •.. E c_ . ...

Simila.r1y, define E. , I, for each b. B. Let D, .. E, - I. for all •• S; D, ia the difference between ext.e.rnal and int.e.roal c:oaLa.

Lemm a 1; C <mrider anll a , A, b • B. t / a at!d b are inINe/latLQ«l, 1M Qain (Ihal ii, 1M r«ludion in eo.1) iI precilelv D. + D. - 2c., .

Proo]: Le~. be the total COllt due to 0.11 (l<lnneeLions between A and B Lhs.t do not involve a or b_ Then

T .. I + E, + E. - e ••.

Exchange a and b; let T' be Lhe new eeet, We obtain T' - z + 1. + [. + c. ••

and ao

gain .. old coaL - new coat - r - T' - D. + D, - 2c ••.

3.2 P Jwue 1 0 pl1m i.al ion A 410rithm

In thie subsection we present the algorithm for 2-ws.y ps.rtitioo.ing. First, compute the D value!! for all elemente of S. Second, ehocee a, • A, b, • B sueh ths.t

Q, - Do. + D., - Ze .. "

ill mu::imum; a, and b, correspond to the largest possible gain from a lingle interch&nge. (We will return shortly to 1\ diacllJ9ion of how to select a, and b, quickly.) Set 0, and b, 8.8ide temporarily, and eall them a: and b: . respectively.

Third, recalculate the D value., for the elements of A - lad and (or B - fb,!. by

D:

D:

:n A - (a,1. V,B - Ib,l·

D. + 2c •• 1 -~ 21: ... , D. + Ze." - 2c .. ,

The eoreectneea of these expressiona is easily verified: the edge (%, a,J is counted II..!! intemal in D. , and it 's to be external in D: , 3.Q e, .. mus; be added twice to make Lhia correct. Similarly, Co" mu.st be subtracted twice to convert ("" b,) from external to internal,

Now repeas the second step, choosing a pair a;. b; from A - :a;1 and B - Ibn such Lhd V. - D ••. + D ••. - 2c ••.••. is maximum (a; and b: are nol considered in Lhia choice). Thus g. is the additional ",in when the points a; and b; are exchanged "" well 8.S a; and b: ; thia additional gain is maximum, given the previous choices. Set 0; and b; eside .. lso,

Continue until all nodes have been exhs.usted, identifying (CI; , b:),' . -, (0: , b~), and the corresponding mnxirnurn gaill.! U" ••• ,u •. A3 eAch (CI', b') pair is identified, it is removed [rom contention (or (urther choices 90 the size of the set., being considered decreases by I es.ch time an (a', b') is selected.

If X - (I~ j a.: , ... i 4: t Y - b:' j b;, ~ ~ ~ , b; r then. t.he decrease it, c~t when the se18 X and Yare inte.rchs.nged ilt precisely g, + Q. + , .. + g •. or course ~~ g, - O. No~ ths.L 'some or the 11,'. s.re negative, unless all s.re sero.

Choose" to maximize the partia.! sum ~:_. ,1, .. G. Now if G > O. a reduction in C<lIIt of value a CAn be ms.de by interchanging X s.nd Y, After this is done, the resulting partition is treated M the initial partition, and the procedure is repeated from the first step.

T! a - 0, we have arrived at & locs.lly optimum partition, which wc shall call " pha..~ 1 optimai parlilion. We now hue the choice of repeating with another at.&rting partition, or of trying to improve the pha.se I opti mal partition. We shall diseuse the latter option shortly. Figure 1 is a flowchart [or the phase I optimizntion procedure.

1.3 Effecliveru. •• of 1M Proeedur«

One general approach to solving problems sueh as this one i. to find the bul exchange involving ss.y ). pairs of points, for IIOme X specified in advance'. The difficulty encountered is that use of a amall va!ue or ), is not sufficient to identify good exchanges, but the computational effort required grows rapidly as ),. inereeacs,

The procedure we ho.ve described ,equenliall~ finds nil approximation La the beat exchange of ). pairs, ). is not specified in ad vance, but rs.ther is chosen to make the improvement 8.! large II..!! possible. This technique sacrifices a certs.in amount of power for s. considerable gain in speed.

Since we coostruct a sequence of glJ.ill! u. , i-I, ... , n, and find the

77

THE BELL SYSTEM TEcn N ICAL JOURNAL, FEBRUARY 1070

~(U.r:.l .L. Ap. b.i L Sp SUCH 9p. D"'L + Ob"j ~,H'.LbJ t::.. ~ ... ~n •• ulol

a~,-- b ~ ~p- ~j

AI=I"f" I -- Ap. al,_ B~., - Bp-~!

p .... p .

)-----1 UFl'O ..... u: [I .&.I,.Vf:s.

;OR Ap. 'Bp

CI"tOO'Sf ~ TO ~""');'II.AIH k

G - '[ 9l

[:I!I

Fig. 1 - Flow ~,h .. r t, of phll.:!€ 1 opt j m i IUI.! 'iOD prosed uee.

maxim!!m partial sum, lhe 11'(1<'£" doe« nlllt.rminal. immdiatoly when '0"" g, .. ""yaliue. This means that the process can sequentially identify sets for which the exchange of only a lew elements would ncluBlly increase the cost, while the interchange of the entire sets produces a net gain,

Numerous experirnenta have been perforrned to evaluate the procedure on diffe'ent types 01 eost matrices. The matrices used have ineluded (,) 0-1 matrices, with deMity of nonzero dements rll.nging Irom 5 percee 1 to SO percent, {i,; integer matrices with elements unilormly distributed on .[0, "J, k - 2, ... , 10, (ii,; matrices with clusters of known .Iit~ and binding strength. Results on .BIl of these matrices loll. ve been similar, so we shan only summnrize them here. A more extended discussion may be found in Ref. 3.

A useful measure of the power of a heuristic procedure is the probability that it fi nds an optima] solution in B single trillL Suppose tho t "P ill the probability that a phase I optimal solution found using 8. random starting partition is globally optimal. We have examined the behavior 01 this probnbi Ii ty ILS the size 01 the matrices involved is varied. Expcriments show p is around 0.5 lor matrices of siae 30 X SO, 0.2 to 0.3 lor 6CI X 6CI, and 0.05 to 0.1 lor 120 X 120. The [unctional behavior of p is approximately p(~) "" 2-"".

These vulues arc derived primarily from '0-1 matrices hnving about 51) pereont 1'8 (randomly placed). Experiments on rna triees with lower densities of I's yield larger vari nnces, but substantially identical mea" values for p.

, .• Runni"" Tim« ol tb« PToC£d .. r~

Let us define a pas« to be the opera tions involved in mnking One cycle of ident; flea lion of (a; ,b;), ... , (n:, b:J. and selection of set. X and }' to be exchanged, The total time lor a pBS! CIIn be estimated thi~ way, First the computa tion of the D values initially is an n '-procedure, since for each element 01 S, 11.11 the other elements of S must be considered. Tbe time required for updating the D values is proportional to the Dumber 01 values to be updated, so the toW upds tiDg ti me in one pBS! grows IlJl

(~ - I) + (,., - 2) + ... .+ '2 + 1 which is proportional to n'.

Tbe dcminant time factor i" the selection of the ned pair a: , b.: !.o be exchanged, The method We ha VB used to perform this searching i9 U. eort the D values so that

D •. ""= D •. O?; ••• ~ D ••

and

D" i: D,. ;;:. ... ii::; D._ ..

When 80rting ill used, only a few likely contenders for a maximum gain

Deed be considered. This i! because when scenning down the I1Ct of D-'s and Do's, il II. pair D .. ,. D .. is found whose 8um dOOl1 not exceed the maximum improvement ecen 10 far in this P""", then there canno; be another pair a. , b, with k ~ i, I <:; i. wi til II. greutcr gai 0, (assuming c., .jl: 0) nod 80 the "canning can be terminated. Til us the next pili, lor interchange is found rapidly. Sorting is an n log" operation, sn ill this method, the !.ota I time ,e'lui,cd to sort [) ,·,.1 u "" in ~ pase wi II j, t ' approxi m .. !.ely

.. log n + (n - I) log (n - II + ... + 2 log 2 which grows as n' log ".

To red uce the time 10' selection of 11." (0, II J pair, it is possible to U."", techniques which are laster than aorting, but which do not necessarily always give the maximum gain lit each stugc. For example, one method is to 1I.Un lor the largest D_ and the largest D, , and use the correspond. ing (I and b as the next interchange, This method is essentially Ii ncartime and would probably be implemented as part of the recornpu tation "I th e D val ues, It ;1I bes t Bui led lor sp a rEC m R trices, where th" probabil: ty thal G.. > 0 is arn .. H. A alight extension, invol"ing negligi blc extra cost, ill to save tile largest two or throe D-,s and D. 's, SO th,,( if the largest pair docs not give the maximum gnin (because GoO ;1I too large), thco another can be tried. Experience indicates thnt three values src aufficient in vi rtually all eeses, even (or matrices with a reIBt.ive,ly high percentage 01 nonsero entries. Use of this method reduces running time by about 30 percent in Ihe present implementation, with very 81011.11 degradation 01 power.

The number of passes required before B phase .1 cptirnal partition is achieved is small. On all matrix .i~es tested 11.1 the time of writing (up to 300 points), it haa been almost alway" from 2 to 4 passes. On the b1l..'Jis of thie experimental evidence, the number of passes i9 not shongl), dependent On the value of n.

From the foregoing cbservsuons, it is possible to esti mate the total running lime of the procedure. II we use a method which 1I.Ort. the D values at each slagc (lime proportional to n' log n), theo the runn ing time should grow 1I..'J n' log n. If II. fast-scan method is used, and the number of pf1..S$1>.8 ill constant, the running time should ha ve lin n' growth rate; thia is B lower bound.

FOT comparison, examination of all pairs of sets X and Y, lind evaluation 01 the cost. would require time proportional to

• ;g. (n)' _ ~ i-- (n)'

n~J: 2~k

_ ~ (2n)

2 n

-~ 4·(~)1

for large n. This function grows as n'''4".

Running times hsve been plotted in Fig. 2. The observed times ha ve 1I..n apparent growth rate ·of about nl.·, which isre.nsonabJy close to n'. Al though on the logan thrnic plot thia curve is close to linear OVer the rnnge n - 2CI to n .. 130, it may actually be tl' log n; insufficient. do.lII is available to check thi s. AU times are based on an irnplernen lalion in FORTRAN G on an IB.\! System 36C1 Model 65.

~.5 Improving Ih. Pha-se I Optima! Portition

In this section, we discuss n method which might be used to improve the partition produced by the phase I procedure, which may not be globally optimum. The method suggested -in this section is based heavily on experimental evidence, al though there 1I.l"C qui te plausible reasons for performing the particular set of operations. The basic idea is to perturb the locally optimal solution in wh8t we hope is an. enlightened manner, 80 th1l.1 an iteration 01 the process on the perturbed sol u lion will yield 11. [urther reduction in the tot.ul cost, H this tactic lails, nothing hss been lost excepl.jsorne computation time, since the best solution seen ·80 IBr is always aaved.

Computer results lor problems with up to 64 points euggest thll.t whenever a phase 1 optimal solution is not globally optimal, I X t .. I Y I ::::;;: «n. Roughly, this irn plies thnt il I X I and I Y I had been 9mBII compared to n/2., they would loa ve bee~ found by the process: it is on I)" larger eets wh ich arc not identified all the time.

A successful heuristic to r,nd the tOT TOOt X and Y in thi" case ;9 \.0 find B phase I optimal partltlon /0' ench 01 lhe 1I.CWi A and B, sa)' A -I A., , A .11I.nd B ..... IB, , B,I. (That is, lind near-optimal purti lion~ 01 A lind of B scpara tely.) Recombine the 4 aets into 2, say A •.. A, V B I and B. ,~ k, V B" and continue will, phase 1 optirmeation. If OUT expectation is correct, the new X and Y will be srna II, end thus readily identi r,cd by the phase I process.

78

PARTITIONING GRAPHS

'0 •

J
./
I
V"
/
~
/
/0
r- ,/
/
/
/ 0_' 10

'0

When A is split into A, ,A, and B into B. , B. there a re two ways in which the smaller sew can be recorn bined, A series of tests was mllde On mntrices of moderate size (up to 64 X 54), in which both possible recombinations were done, generati ng three phase 1 optimal values for each starting part] lion. For m a trices of size 32 X 32, the cpp a rent optimal value was observed at least once in each triple of v a lues, for a I a rge number of cases. With matrices of size 64 X 64, there were occasional failures.

It might be noted that the ext ra time involved for the recombination appro a ch is three times that required to do a completely new partition from a, random start, assuming an n' -procedure,

It ia possible to estimate whether a particular improvement Lactic ill profitable or not ill the following w a y. Suppose that some method increases the prob a bility of finding an optimal partition from p to p', while it increases the running time from I to I'. Then in .. fixed amount of time, it ill possible to do k trials of the bssie procedure, and kl/!' trials of the improved method. The corresponding probabilities of achieving an optimal solution are 1 - (I - p)' and 1 - (1 - p')"" , respectively. The improved method i~ then desirable if the seeond expression is greater th a n the first; by simple m a ni pulation, this condition becomes

1 - p' < (1 - p)"".

On the basis of the numerical values in thl.!! section, it may be useful to try the recombination method.

z.e Par:ilicnir.g i"t<> U""qua]-Si.ed Set.

It is simple to modify the procedure to parti tion !I. set S with n eillc" rnents into two sets of specified sizes n, and n,(n, + n" - n). Assume n, < n,. Then restrict the maxi rnu m number of pnirs that can be axchinged in one pasa of the procedur-e to 11, • All other operations are performed OQ all elements of each set. (The atBrti~,g partition is into two seta, of n, and n. elements respectivelv.)

Suppose we wish to psrti lion S in to two sets, such that, there a re at least II, elements and at most 11, elements in each .• ubset; n, + n, ~ n, but they o.re not specified further.

The proced ure is easily modi tied to handle this sort of constrain t by the addition of "dummy" elements, These are elements which have no connections whatsoever; that is, they have zero entries in the cost rna trix wherever they appear. Add 2n •. - n dummies eo S hILS 2n. elements, and perform the procedure on it. The resulting partition will assign the dummy elements to the t.wo subsets 80 as to minimize the external cost; lit this point the dummies are discarded, leaving a partition into two subsets th.a~ satisfy the size constraints given.

u Element. of Unequc! Site.

We 10""0 mild e the assurnpticn "0 far tha.t the elements- ("ertices) of the graph Bre !!.II of the BlIme ~ize. Thi9 requirement may be relll~ed to !!.l!!.rge extent by converting nny node of aize k > l to a clu.ster of Ie nodea of "i.e l, bound togelhcr by edges of appropriately high coat. The ."ize of tbe problem will obv-iolUlly incn:a.se proporlioo!l.lJy to lhe v .. lue of k, so i l ml'y be nece"" .. ry to ",,"rifice SOme .. ccuracy lo keep the number of generated nodea wi thi n ren.sonable bounds,

III. WULTIPL&-WAT PARTITIONS

3.1 Redudiun '" '-Wa~ Par!i!iuninq Prookm

So fsr, tbe disclJlJSiov. ba.a been concerned ex elusively with the baaie problem of performing a 2-wILY partiiion on Il eet of Zn object.&. In thill section we extend t.he technique to perform k-WIl.Y pe.r1.itiOI!!l On .. llet of I;n object.o, lUling the 2-way procedure .... tool. .

The """"llti .. 1 idea. is to Btart with some partitiov. into k !let... of size n and by repeated application of the 2-way p..,.titioning procedure to pelra of aubeeta, make the partition aa close &II poMible to being pairwise o pti mal. (Sec Lion 3.2 trea t... the ques tio n of w ha t startin g ..eta to use.) or cOUl"B<! pairwise optimality is only a v.ecessary condition for globo.! optimali~y. There mBy be aituatiollll where some complex interchaage of three or mote items from three or more IUbaeta is required to reduce a ps.irw~ optimal solution to globe.lly optimum; at the moment, no reasonable method for identifying such !let.3 U. known.

There are m pairs of 8Ubaetll to consider, so the time for one pa.&8 through all pairs ill (sssuming .. n n'-procedure) (:)n' :=::i (kn)'J2 - (number of points)'J2. In general, more ptLMell than thill will actually be required, since when t .... o sets are made optimal, thi.e may change their opti mal i ty wi th res pect to other eeta.

Experience indicates ~bat the number of p:~ i. ~1n:lll and the I'roe,·""' converges quickly. For example, our algorithm select .. (i, j) : ... ~ Ih~ next pair ofseta to be opt; mi .• ed, where either i Or i h:i.S been ehnnged since the IMt time the pair (i, j) was selected. Ilsi ng this selection process, the average number of psssea through each p:lir of set:! i, " slowly growing function of both k .. nd n. For matrice3 of size 100 or less and k < ti, the number of p!1.85e9 hsabeen leas thao 5. [The avet""ie number of passes ia computed as the ave.rage number of psira eouaidered to reach pairwise optimality, normalised by (:).J

I n any particular trial, th ere i~ • ccrrela t; Oil bet ween the number of pa.irs selected and the quality of the fine.l partition. To get a better solution requires more work.

Convergence ill rapid: two pesses account for more than 95 percent of the improvement in moat cases; the remaining P""""" contribute only .. mall further reductions. Let p(n, k) be the pro portio [I of minimum coot sol uti ous f ound for .. parti eular nand k, For k fi xed and small eo m pared to n, the functicnal behavior of p(n, k) ill similar to the case k - 2, but the actual values a.re lower. Roughly, we observe p(n, k + 1) :::::: lP(n, .k) for k in the range 2-4, and n up to 100, with eonaiderahle variation depending On the matrix being tested. For instance, for IIlI1trice.'l of size about 40, peW, 2) ~ 0.4, p(42, 3) ~ 0.2, and p(40, 4) ~ 0.1.

Another interesting quesjion ill mea.surement of how close to optimum the partitioo.s Iouad are. The solutions obtai lied by pairwise optimization have, values concentrated ill. It. narrow range. III almost all cases, the largest value found. by the procedure i.e within 4----5 percent of the amallest. M another measure, if e ill ~he mean coot of random parti tiolll! .. lid b iIt th e cos t of tb e bes t parti ti OD observed, th en virt ual) y all parti ti ons Jound have values" such that

" - b :ii O.l(c - b).

For instance, One test ease was a series of 4-way p .. rtitious of a 0--1 matri~ of size 80. Thill matrix had 1278 nonzero entries (a density of 0.2), corresponding to 639 edges in the graph. The meau vft.!ue of ran d omly chosen parti ti 0 na wa..s 480.6. Twen t y. four parti ti 011.'1 of th i.e matrix were found using the method described above, The lowest value encountered was 352 (1 time), the highest 365 (l time); the mean value WI\.'l 359.5, the median 360.

3.2 Slar!;fI1J Partiiion

T n ~bilI au bsecti on we disc U38 v ari OWl meth ods of ge nera ti ng good starting partitions, based on modifications of the basic procedure.

The pri mary reaso n for eh oosi ng good s tarti ng parti tiona ill th B t t h i8 partieul a r form of preprocessing reduces the .. mount of work required to make the system pairwise' optimal. It may also make the probability 'Of an optimal solution higher, .. [though this tendency i~ very difficult to evaluate,

Several methods for finding good multi-way starting partitions which are based On repeated application of the procedure ;welf have been investigated. The essential idea is to generate a k-wsy atarting partition by fi,."l forming 8n r-way partition, then lin .-way partition 011 each of the resulting subsets, and so on, up to I-way. (Here k - " ... I.) The parti lions foulld this way will in general be better than those wh ioh Bte completely arbitrary. A pairwise optimil5tioll stage is applied to the final set of subsets.

For example, if k is Il power of 2, then perform a 2-WBY split, then a

79

THE BELL SYSTEM TECHNICAL .JOURNAL, FEBRUARY 10'10

2-WBY "plil on each 01 these subsets, And I!O on until the desired IIi ... of subsets ;s found.

This general approach is prone 1.0 the following di.fficulty: the Iil'8t split divides the original ad in to r subset.e by trying to make the internal eonneeticne in each lunaet II-'! large l1li possible. Obviously this may conflict directly with the next 8ULge, which ie to try to divide each 8UbaCt further. Carried to scveeal level., it CAn lend to II relatively poor overall solution. In experimenta wi th 4-way partitioll3 01 matrices of Ii,,,,, up to 54 X 54, this method yields optimal lolutiona approximately II-'! of Len AS docs al&rLing with " 4-W&Y partition in the fil'8L place. In addition, Lhts method will be effective if the matrix heppena 1.0 have natural clusters of approximately the correct sise (that ie, equal W the finnl subset size).

A second method which can be used is to partition the eet, 01 kn elements into .. set of " and I< sel of (k - l)n, using the slightly modified version of the basic procedure discussed in Lhe hl'8t part of Section 2.6. The sel of " elements is eet aside, and the next 1'1 elements from the remaining (k - I) n are iden tined. Thi3 continues until k subsets have been formed; again the pairwise optirnieation technique is used to improve 011 this partition.

This method C.:I.n make 1<1'1 error in the identification of the lint set which will biM the choice of the second, and 80 on; the clJecl is mo.-I. severe for lhe esse where k is large, 80 each set is small.

The method of breaking off subsets sequentially has another potential fI.w: regardless of the .t.mting conligur&tion, it will identify approximatelv the same set each lime it is used on a particular problem, and hence little is gained by using it twice on ooe cog!. matrix, However vOlriHtions in the order of performing pairwise optimit.Rtions eon .WI produce different final partitions in general,

Limited compuLstional experience with aequentlal break~f'1 followed by pairwi!ll!: optimiution suggesl.'! that it yields soluticns which sre on the average At least as good as (and sometimes slightly better than) those provided by pairwise optimization applied to nn arbitrary k-way st.arting part-ition. Pairwise optimieation yields the optimum with a higher probability, however, because it is less suseeptible to error caused by 6 bad choice made early. For instance, in tests on the 80 point matrix mentioned previously, sequential break off yielded 4-way solutions wilh II mean value of 358_6, but the lowest value found wa.s 355. (The highest 'IV&.!! 363.) These ma:>, be eompared to 359.5, 352 and 365 Ior the slandan:! partitioning method.

Running time for lhe IW!q uential break-ofl method is lower than for straight pairwise optimization.

Insufficient data is ""I<ilable for II direct comparison between sequential break off and the method of repealed subdivision.

In all cases, the original PI"O<leOS, be it a completely n.ndom generation of scme initial configurdion, or tb e production 01 • good Blartin!!; partition, ill followed by A pairwise oplimil,ing phase. It is unlikely LIlat using better starting pnrtitiona will lc.'u:I to WOnK! results than random starts, on the average, Whether !.he possible improvement in results and running limes will justify the extra compul.:ltionaJ effort required to generate the stllrting partition depends on the ch(l..ncteristies 01 the particular e11l3.'l 01 matrices being studied.

:E

Some limited experiments were performed to compare the present procedure with I< multi-dimensional scaling technique", on .. Boolean malri" of 316 pcints, with about HOI) nonzero entries. The results indieated lhll.~ the procedure icicntifies clu~leMl well, even when no attempt is made t.o provide a good Bto..Ling purtition.

3.3 E:xpaowwn Fodor

The introduction of dummy elements "':19 mentioned in Sec Lion 2.6 II-'! R method 0/ handling p~rtiLioni!\, intosubsets 01 unequal ',iz09. This can be viewed equally well as II mllallll of introducing "slack" inLo a solution, in an attempt to get a lower overal! cost by .. llowing "expansion.' That is, 80 la.r we have treated the problem of finding a po.Tlition with B constraint on the sizes of Lhe subsets, "lid on the number 01 subsets, since given kn poi 1'1 I.'! , we have tried to find the best partiLio,,", into exactly Ie subsets 01 n pointe each. Suppose we now relax this second constraint by permitting the addition of dummy elements to incre&ae the Bi~e of the problem, and !lttempL to find the best 801utioD involving "n~ rll",wer (greater than or equal to k) of subaete, with aI mo.l n poii'll.'! in each. This solution with k 0' greater subsets will in gene .... 1 have n lower cost t.han the ecbstrained solution.

Figure 3 ebows an example in which ietrodueing slack permits a lower overall cost. Assume 11 is 3 and nil nodes lire si se L The vertical edges have cost 1 and the horizontal ones cost 2. Any partition into 2 equal 8ubseCs has a cost of at. least 3, but there is an obvious partition into 3 subsets with cost 2. Any nontrivial partition into 4 or more subeets has a cost greater than 2, so 3 subaets represents the optimal expension, It is possible to lind the minimal ccsteolutlon And tbe corresponding optimal amount of expansion a.s follows .. Suppose the problem hIlA kn points to be partitioned into Ie eets of n points each. Surting with no slack (/en points), the optimal assignment is found. Then n dummies, enougb to create one extra subset, are added, making II (Ic + 1)11 problem, and I!O on. Eventually, one subset is produced which eensists entirely of dummies. When this occurs, we take the partition with this set of dummies removed 8.1! our optimum solution.

REFEnENCES

I. Fonl, L. n .. _ntl Fulkl! . .-.on. D. R .. ,.'/0" in. NdlDOrJuj Princeton, NC"w JC"I'WY:

Princeton UDiye~t.y PrC'M, 1002. P, II.

2. LiD. S". "Comrll(.t &I,,!ion. 01 lh. Tr.vcliD, SaI •• m.D Problem," BS.T J_ ~, No. 10 (December 1005), pp.22C5-2259.

3. "~,."jgh.o, D, \Ii., "Some C"'P~ 1'."jlioniD~ Problerre R.I.t;,d 10 Prosram ~mODUlioll.," Ph.D_ Thco; •• Prineeton ljniv ... ;ly, JIOU.')' 196~. pp. 'C-126. 4. Kru.li.!. J. D., M "lli·DimeD.ioo.! SO.IiD~ by Oplimi .• ing Gecdne .. of Fit 10 • Non-Metrie Hrpoth .. is," r'vchom<I,U:c., to. No. I (Much 196t). pp. 1-27. and No_ 2 (Jlln. 196t), pp_US-129.

80