Particle Swarm Optimization Research Toolbox

Documentation Version 20110515i By George Evers

www.georgeevers.org/pso_research_toolbox.htm
Clickable Contents
ntro!"ction...................................................................................................1 #tatement o$ %"rpose.............................................................................1 &oc"mentation 'verview.......................................................................1 (oncept"al '"tline.............................................................................1 )he &oc"mentation *s a +elp ,en"...................................................2 -ormats *vailable...............................................................................2 %rere."isite )erminology.......................................................................2 &ecision Variable................................................................................2 &ecision Vector...................................................................................2 (ontrol %anel....................................................................................../ #witches............................................................................................./ #ettings.............................................................................................../ )rial.....................................................................................................0 (ol"mn 1#tatistics %ertaining to 'ne #et o$ )rials2.............................0 )able 1'ne or ,ore (ol"mns2.............................................................0 . #ystem %reparation......................................................................................5 #etting Virt"al ,emory #i3e...................................................................5 #tep 1.................................................................................................5 #teps 2 4 /.........................................................................................5 (hec5ing 6ava Version............................................................................7 #etting ,*)8*B %re$erences..................................................................7 8ine 9rapping.....................................................................................7 (omman! 9in!ows #croll B"$$er........................................................7 :ight;han! )ext 8imit.........................................................................7 #electing a -ont #"itable $or %asting into 9or!.....................................7 %ac5ing ,emory.....................................................................................7 . * G"i!e! 9al5 )hro"gh...............................................................................< nstalling the %#' )oolbox 1i.e. =n3ipping the -iles2..............................< 'pening -iles $or E!iting........................................................................< *"to;saving -ig"res................................................................................< -ig"re #witches...................................................................................< -ig"re %osition....................................................................................< =n!erstan!ing *"tomatic np"t Vali!ation.............................................> =n!erstan!ing =ser np"t Vali!ation....................................................10 i

nterpreting Graphs..............................................................................11 -ig"re ?"mbers................................................................................11 'b@ective -"nction............................................................................11 %hase %lots.......................................................................................11 %osition vs. teration.........................................................................12 *naly3ing 9or5space Variables............................................................12 $g......................................................................................................12 thresh_$or_s"cc.................................................................................12 n"m_trials_s"ccess$"l.......................................................................1/ 5........................................................................................................1/ iter_s"ccess......................................................................................1/ x........................................................................................................10 xhist..................................................................................................10 v........................................................................................................15 vhist..................................................................................................15 p.......................................................................................................1A Eval"ating %ositions..........................................................................1A $p......................................................................................................17 $........................................................................................................1< phist..................................................................................................1< l.........................................................................................................1> $histB lhistB 4 lbest_neighb_si3e.........................................................20 g.......................................................................................................22 ghist..................................................................................................22 9or5space 'rgani3ation...................................................................2/ #"mmary..........................................................................................2/ Cnowing 9here &ata is #ave!.............................................................20 -ig"res..............................................................................................20 9or5spaces......................................................................................20 *ctivating an! &e;activating Graphs....................................................2A &e;activating %hase %lotsB %osition vs. teration GraphsB an! Benchmar5 Graph.............................................................................2A *ctivating Graphs o$ %ersonal Bests 4 Global Best vs. teration.......2A *ctivating Graphs o$ -"nction Val"es vs. teration...........................27 nterpreting ?ewly *ctivate! Graphs...................................................27 -"nction Val"e vs. teration..............................................................27 %ersonal Bests 4 Global Best vs. teration........................................27 Global BestDs -"nction Val"e vs. teration........................................27 #witching Between 8best an! Gbest %#'Ds..........................................27 #witching to #tatic nertia 9eight........................................................2< (hanging the #warm #i3e....................................................................2< (hanging the ?"mber o$ terations......................................................2< *ctivating #warm )ra@ectory Graphs....................................................2< :e;"sing -ig"res...................................................................................2< ,ar5ing %ersonal Bests an! the Global Best........................................2< (hanging the 'b@ective -"nction.........................................................2< ii

*naly3ing #warm )ra@ectory.................................................................2> :e;"sing -ig"res...............................................................................2> *ctivating :egro"ping %#' 1:eg%#'2..................................................2> #etting the -ormat $or *"to;#ave! -ig"res........................................../0 *naly3ing #warm )ra@ectory o$ :eg%#'.............................................../0 *naly3ing 9or5space Variables $or :eg%#'........................................./1 #witching )ermination (riterion $rom terations to -"nction Eval"ations ............................................................................................................./1 (hanging the ?"mber o$ )rials............................................................./2 (hanging the %roblem &imensionality................................................./2 (hanging the #warm #i3e..................................................................../2 &e;activating =nnecessary +istories..................................................../2 &isabling Graphs................................................................................../2 *naly3ing the #tatistics o$ ,"ltiple )rials............................................./2 =ser np"t Vali!ation......................................................................../2 Each )rialDs &ata...............................................................................// %er$ormance ,eas"res...................................................................../0 #"mmary............................................................................................../A . ,ore &etails on #elect )opics.................................................................../A #peci$ying Gbest or 8best %#' as the (ore *lgorithm........................./A #etting the nertia 9eight..................................................................../A #etting the Velocity (lamping %ercentage.........................................../7 #electing the 'b@ective -"nction1s2....................................................../< #etting the (enter an! :ange o$ the nitiali3ation #pace...................../> =sing :eg%#' to Escape $rom #tagnation............................................00 #peci$ying the ,axim"m ?"mber o$ terations or -"nction Eval"ations .............................................................................................................02 ,axim"m per Gro"ping....................................................................02 ,axim"m over all :eg%#' Gro"pings..............................................02 ,axim"m over all ,%#' #tarts.........................................................0/ (on$ining %articles to the #earch #pace...............................................0/ Velocity :eset...................................................................................0/ %osition (lamping.............................................................................0/ Generating #tatistics *"tomatically.....................................................0/ Generating -ig"res...............................................................................00 ,ean -"nction Val"e........................................................................00 #warm )ra@ectory..............................................................................00 #aving &ata *"tomatically...................................................................05 Examining #ave! &ata.........................................................................05 ,eas"res o$ E$$iciencyE terationsB -"nction Eval"ationsB an! :eal )ime .............................................................................................................0A V. G"i!e to (on!"cting Fo"r 'wn :esearch.................................................07 *!!ing (onstraints...............................................................................0< +" 4 EberhartDs *pproach................................................................0< *n *lternative !ea...........................................................................0> Enhancing Velocity =p!ates.................................................................0> iii

Enhancing %osition =p!ates.................................................................50 Varying the nertia 9eight ?on;linearly...............................................50 (ontrol_%anel.m................................................................................50 gbest_core_loop.m 4 lbest_core_loop.m...........................................51 np"t_Vali!ation.m............................................................................51 &isplay_#ettings.m...........................................................................51 )itle_Graphs.m..................................................................................51 *!!ing Fo"r %roblem to the )oolbox...................................................51 9riting Fo"r -"nction.......................................................................51 8isting Fo"r -"nction in 'b@ectives.m...............................................52 'ther Enhancements............................................................................50 Bene$iting $rom =p!ates......................................................................50 V. (ontrib"tion #tan!ar!s..............................................................................50 -ollowing ?aming (onventions 9hen (reating ?ew Variables............50 ,inimi3ing +ar! (o!ing via G(ontrol_%anel.mH....................................55 Vali!ating np"ts via G np"t_Vali!ation.mH...........................................5A #teps $or *!!ing an *"tomatic (orrection to G np"t_Vali!ation.mH..5A &isplaying the ,ost mportant #ettings via G&isplay_#ettings.mH.......57 ,aintaining 9or5space ntegrity.........................................................5< #teps to *voi! =nintentional =sage o$ &ata.....................................5< )esting ?ew (o!e n!epen!ently o$ the %#' )oolbox..........................5> ,o!i$ying all :elevant Gbest 4 8best #cripts.......................................5> (ommenting *$ter GelseH.....................................................................5> =sing #hort;(irc"it 8ogical 'perators..................................................A0 *!!ing ?ew *lgorithms to Graph )itles................................................A0 *voi!ing %re;*llocate! Ieros...............................................................A1 &e$ining ?ew #witches 4 %rogramming Variables...............................A/ *pplying (hanges )hro"gho"t the %article #warm :esearch )oolbox. A/ V . %rogram -low............................................................................................A/ 'pen_-iles.m........................................................................................A0 (ontrol_%anel.m...................................................................................A0 'b@ectives.m........................................................................................A0 np"t_Vali!ation.m...............................................................................A0 &isplay_#ettings.m...............................................................................A0 :eg%#'_main.m...................................................................................A0 )rial_ nitiali3ations.m........................................................................A5 ,%#'.m............................................................................................A5 :eg_,etho!s_0*n!1.m....................................................................A5 :eg_,etho!_1.m..............................................................................A5 #ave_!ata_per_trial.m......................................................................AA 8oa!_trial_!ata_$or_stats.m..............................................................AA #tan!ar!_'"tp"t.m..........................................................................AA *"tosave_9or5space_%er_(ol"mn.m...............................................AA Graphs.m..........................................................................................AA V . &isclaimer................................................................................................AA V . *!!;ins....................................................................................................A7 iv

:e$erences....................................................................................................A7 *ppen!ix *E &e$initions................................................................................A< '$ %rogramming/9or5space Variables.................................................A< center_ #...........................................................................................A< center_ #0.........................................................................................A< !im....................................................................................................A< $........................................................................................................A< $g......................................................................................................A> $g_array............................................................................................A> $ghist.................................................................................................A> $ghist_c"rrent_trial............................................................................A> $ghist_mean .....................................................................................A> $ghist_over_all_trials ........................................................................70 $hist...................................................................................................70 $hist_c"rrent_trial..............................................................................70 $p......................................................................................................70 $phist.................................................................................................70 $phist_c"rrent_trial............................................................................71 g.......................................................................................................71 ghist..................................................................................................71 ghist_c"rrent_trial.............................................................................71 nternal_i...........................................................................................71 nternal_@...........................................................................................72 iter_mean_per_trial...........................................................................72 iter_s"ccess......................................................................................7/ iter_s"ccess_mean............................................................................7/ iter_tot_all_trials...............................................................................7/ 5........................................................................................................7/ l.........................................................................................................7/ lbest_neighb_si3e..............................................................................7/ lhist...................................................................................................7/ lhist_c"rrent_trial..............................................................................70 max_iter_over_all_gro"pings............................................................70 max_iter_per_gro"ping.....................................................................70 max_-Es_over_all_gro"pings............................................................70 max_-Es_per_gro"ping.....................................................................70 ,%#'_-E_co"nter.............................................................................75 ,%#'_5.............................................................................................75 np.....................................................................................................75 n"m_particles_2_eval"ate ...............................................................75 n"m_trials_s"ccess$"l.......................................................................75 'n'$$_asymmetric_initiali3ation.......................................................7A 'n'$$_$"nc_evals..............................................................................7A 'n'$$_?orm:_stag_!et.....................................................................7A 'n'$$_position_clamping..................................................................77 'n'$$_:eg%#'..................................................................................77 v

'n'$$_#"ccess$"l=ns"ccess$"l.........................................................77 'n'$$_)erminate_=pon_#"ccess......................................................77 'n'$$_v_reset...................................................................................77 '%#'_ghost_-Es_per_gro"ping.........................................................77 '%#'_ghost_-Es_:eg%#'.................................................................7< p.......................................................................................................7< phist..................................................................................................7< phist_c"rrent_trial.............................................................................7< position_matrix.................................................................................7> r1......................................................................................................7> r2.....................................................................................................7> :an!_se._start_point........................................................................7> range_ #............................................................................................7> range_ #0..........................................................................................<0 reg_$act.............................................................................................<0 :eg%#'_gro"ping_co"nter................................................................<0 :eg%#'_5..........................................................................................<0 :eg%#'_5_a$ter_each_gro"ping........................................................<1 stag_thresh.......................................................................................<1 thresh_$or_s"cc.................................................................................<1 tr"e_global_minim"m.......................................................................<2 v .......................................................................................................<2 vhist..................................................................................................<2 vhist_c"rrent_trial.............................................................................<2 vmax.................................................................................................<2 x........................................................................................................</ xhist..................................................................................................</ xhist_c"rrent_trial.............................................................................</ xmax.................................................................................................</ xmin..................................................................................................<0 '$ -"nctions.........................................................................................<0 'b@-"n_*c5ley..................................................................................<0 'b@-"n_Griewang5 ...........................................................................<0 'b@-"n_J"a!ric ...............................................................................<5 'b@-"n_J"artic_?oisy ......................................................................<5 'b@-"n_:astrigin ..............................................................................<A 'b@-"n_:osenbroc5 .........................................................................<A 'b@-"n_:osenbroc5_Engelbrecht_Van!enBergh ..............................<A 'b@-"n_#cha$$ers_$A ........................................................................<7 'b@-"n_#chwe$el ..............................................................................<< 'b@-"n_#pherical .............................................................................<> 'b@-"n_9eighte!_#phere ................................................................<> '$ -iles.................................................................................................>0 *ppen!ix BE Error (orrection........................................................................>0 @ava.lang.'"t'$,emoryErrorE 6ava heap space....................................>0 Error "sing KKL saveE =nable to write $ileM ?o s"ch $ile or !irectory >0 vi

*ppen!ix (E E$$icient ,*)8*B #hortc"ts......................................................>0 E!iting (o!e.........................................................................................>0 ?avigating............................................................................................>1 ,anaging -iles.....................................................................................>1 *ppen!ix &E (itation o$ )oolbox or &oc"mentation......................................>1 %aste;able Examples............................................................................>1

vii

Introduction
Statement of Purpose
)he %article #warm 'ptimi3ation :esearch )oolbox is c"rrently !esigne! to han!le contin"o"sB single;ob@ective optimi3ation problems. (hapter G VE G"i!e to (on!"cting Fo"r 'wn :esearchH clari$ies how a motivate! researcher co"l! a!! constraints or ma5e other improvements. will be happy to g"i!e motivate! researchers in a!!ing new $"nctionality as !one at this ,ath9or5s threa!.

Documentation Overview
Conceptual Outline
Preparation )his intro!"ction gives a ."ic5 overview o$ the !oc"mentation an! toolboxB an! the $irst chapter explains how to optimi3e yo"r system $or improve! ,*)8*B per$ormance. Usage uide )he secon! chapter explains how to "se the most pop"lar switches an! settings o$ the toolboxB an! the thir! chapter contains more !etaile! instr"ctions on vario"s topics N s"ch as how to prevent particles $rom leaving the initiali3ation space an!/or how to "se :eg%#' to escape premat"re convergence $or contin"e! optimi3ation. Contribution uide )he $o"rth chapter o"tlines how to "se the toolbox to solve "ni."e problems or to ma5e novel improvements to the algorithms implementable by the toolbox. )he $i$th chapter o"tlines a str"ct"re! $ramewor5 $or e!iting the co!e to ens"re that we all move in the same !irection. )he sixth chapter o"tlines program $low $or contrib"torsB tho"gh en! "sers nee! only concern themselves with the control panel. !cknowledgements )he seventh chapter is a !isclaimerB an! the eighth chapter cre!its contrib"tors an! provi!es lin5s to their webpages. !ppendices *ppen!ix * !e$ines variables an! $"nctions "se! by the toolbox. *ppen!ix B explains how to prevent certain errors $rom occ"rring. *ppen!ix ( contains "se$"l ,*)8*B shortc"ts. *n! *ppen!ix & contains examples o$ how to cite the toolbox an!/or !oc"mentation.

The Documentation !s a "elp #enu

)he !oc"mentation will hope$"lly answer most ."estions abo"t the toolbox an! how to "se it $or novel research. )he table o$ contents is clic5able by topic an! can be tho"ght o$ as an expan!e! help in!ex. (trl O $"nctionality can be "se! to search the contents $or 5eywor!s as wo"l! be !one within ,*)8*BDs help men". $ the !esire! topic is not a!!resse! or its !isc"ssion is $o"n! to be ins"$$icientB please email george PatQ georgeevers.org $or $"rther clari$icationR howeverB please ta5e the time to chec5 this !oc"mentation $or the answers to yo"r ."estions.

$ormats !%ailable
)his !oc"mentation is !istrib"te! with the toolbox in .p!$ $ormat since the $ree *!obe :ea!er optimi3es aesthetics better than wor! processorsB b"t .!oc $ormat is available $or those who pre$er to highlight while rea!ing. 'racleB the new owner o$ #"n ,icrosystems an! hence also o$ the 6ava $or which #"n $o"ght ,icroso$t to retain ownershipB also ma5es 'pen '$$ice N a $ree alternative to ,icroso$t an! its b"siness practices that r"ns on 9in!owsB ,acB an! 8in"x. 9hile more basic in $"nctionality an! not yet integrate! with '"tloo5B it is s"rprisingly capable an! low on b"gs $or $reewareE highly recommen! it to anyone on a b"!get.

Toolbox Compatibility
)he %article #warm 'ptimi3ation :esearch )oolbox has been veri$ie! to be compatible with ,*)8*B 2007aB 200>aB 2010a an! with 9in!ows S% an! 7. t is also tho"ght to be compatible with other recent ,*)8*B versions since all reporte! error messages have been a!!resse!.

Prerequisite Terminology
&e$initions in this chapter are not meant to be comprehensive b"t to assign basic meaning to terms "se! herein.

Decision &ariable
* !ecision variable is one o$ GnH variables to be optimi3e! by the !ecision ma5er 1i.e. the %article #warm 'ptimi3ation :esearch )oolbox "ser2. )he n"mber o$ !ecision variables to be optimi3e! !etermines the !imensionality o$ the optimi3ation problem N i.e. the !imensionality o$ the hyperc"be to be searche! by the swarm in p"rs"it o$ either a global or local minimi3er.

Decision &ector
* !ecision vector is the set o$ !ecision variables to be optimi3e! by the !ecision ma5er. n %#' literat"reB this is an n;!imensional position vector since each position ass"me! by a particle represents one can!i!ate sol"tion to the optimi3ation problem.

Control Panel
)he %article #warm 'ptimi3ation :esearch )oolbox evolve! over the co"rse o$ thesis while combating the premat"re convergence problem o$ particle swarm optimi3ation 1%#'2. :ather than har! co!ing changes to test new i!easB variables were create! whose val"es are set within in the control panel. t is essential that a research toolbox give the "ser a consi!erable !egree o$ $lexibility by which to test new i!easB which is what the extensive control panel strives to !o.

Switches
:ather than maintaining !i$$erent co!es $or !i$$erent variations o$ the %#' algorithmB switches were create! in the control panel by which to activate an! !e;activate vario"s $"nctionalities. #witches are logical variables occ"pying one byte o$ memory rather than the "s"al eight bytesB an! they chec5 ."ic5ly an! e$$iciently !"ring program exec"tion since no comp"tation is re."ire!. -or exampleB Gi$ OnOff_lbestH eval"ates to Gtr"eH when switch OnOff_lbest in the control panel has been activate! by setting it to Glogical112BH b"t it eval"ates to G$alseH when the "ser has !eactivate! the switch by setting it to Glogical102HR this partic"lar chec5 is "se! within the toolbox to !etermine whether to implement 8best or Gbest %#'.

Settings
*si!e $rom switchesB which are simply active or inactiveB the control panel also contains settings which store n"merical val"es. #ome settings store basic %#' speci$ications s"ch as swarm si3eB inertia weightB acceleration coe$$icientsB an! velocity clamping percentage. 'thers control $eat"res s"ch as the n"mber o$ trials to be exec"te!B the n"merical o"tp"t $ormatB an! the si3e o$ $ig"res generate!. )he most important parts o$ the control panel with which to become $amiliar initially areE 1i2 )he switches in s"bsections G%#' *8G': )+, #E8E() '?H an! G%#' + #)': E# )' BE ,* ?)* ?E&H o$ section G112 B*# ( #9 )(+E# 4 %#' *8G': )+, #E8E() '?BH 1ii2 )he basic %#' settings o$ section G122 %*:) (8E #9*:, *8G': )+, #E)) ?G#BH an! 1iii2 )he graphing switches in s"bsection G:EG%#' 4 %#' G:*%+ ?G #9 )(+E#H o$ section G152 :EG%#' 4 %#' G:*%+ ?G #9 )(+E# *?& #E)) ?G#H. )hese essentials will be intro!"ce! in chapter G . * G"i!e! 9al5 )hro"ghG. )he more $amiliar yo" become with the toolboxB the more settings yo" will master.

Trial
Each sim"lation is re$erre! to as a GtrialH. 9hen num_trials is set to 1 atop section G122 %*:) (8E #9*:, *8G': )+, #E)) ?G#H o$ the control panelB one trial will be exec"te! accor!ing to the settings speci$ie!.

Column 'Statistics Pertaining to One Set o( Trials)

9hen num_trials is set greater than 1B each trialDs best $"nction val"e is a"tomatically analy3e! by the %article #warm 'ptimi3ation :esearch )oolbox to calc"late an! !isplay the me!ianB meanB minim"mB maxim"mB an! stan!ar! !eviation. Each trial within this set o$ trials will "se precisely the same settingsB b"t each will "se a "ni."ely repeatable se."ence o$ pse"!o; ran!om n"mbers. )he more trials one con!"ctsB the less e$$ect any partic"lar trialDs ran!omness will have on the statisticsB an! the better the in!ication will be o$ the algorithmDs overall e$$ectiveness.

Table 'One or #ore Columns)

)he table $eat"re is only available in the $"ll version o$ the %article #warm 'ptimi3ation :esearch )oolboxB so this !isc"ssion can be s5ippe! by "sers o$ the $ree !emo version. * table containing m"ltiple col"mns is generate! when a val"e s"ch as the swarm si3eB inertia weightB velocity clamping percentageB or regro"ping $actor 1i.e. o$ :eg%#' P1Q2 is a"tomatically incremente! per col"mn. &isplaye! atop each table are the basic %#' parameters hel! constant across col"mns. &isplaye! atop each col"mn is the val"e o$ the parameter being incremente!. *n! !isplaye! within each col"mn are the statistics generate! $or the parameter combination speci$ic to that set o$ trials. ,ost basic %#' parameters can be incremente! across col"mns within a tableB while the ob@ective an! acceleration coe$$icients can be incremente! across the tables themselves. )his programming !irection was chosen to a"tomatically increment the inertia weight $or speci$ic acceleration coe$$icients within a table be$ore a"tomatically incrementing the acceleration coe$$icients between tables in p"rs"it o$ a high;."alityB general;p"rpose %#' parameter combinationB s"ch as those presente! in (hapter o$ thesis P2Q. * GtableH can generally be tho"ght o$ as a set o$ Gcol"mnsBH where each col"mn contains the statistics pro!"ce! by a partic"lar parameter combination. -or exampleB a table pertaining to acceleration coe$$icients c1 = 2.0, c2 = 2.0 might contain the statistics pro!"ce! by inertia weight = 0.71 B in which case the next col"mn wo"l! contain the statistics pro!"ce! by = 0.72 B an! so $orth. 9hen only the ob@ective $"nction is incremente!B each GtableH will contain exactly one Gcol"mnH o$ statistics since no parameter wo"l! be incremente! within each table. 0

I* S+stem Preparation
Setting Virtual Memory Size
#ince ,*)8*B is more memory;intensive than the typical 9in!ows programB c"stomi3ing the $ollowing 9in!ows options can improve per$ormance.

Step ,
1i2 1ii2 1iii2 1iv2 1v2 1vi2 (lic5 #tart $ollowe! by ,y (omp"terB %ress *lt O TvB !U to view !etailsB 8eave ,y (omp"ter open with the )otal #i3e o$ yo"r har! !rive visible as !one at the bottom le$t o$ the $ig"re belowB %ress 9in!ows O : to access the :"n !ialog boxB )ype GcalcH an! press EnterB ,"ltiply yo"r har! !riveDs )otal #i3e by 00 1i.e. by 0.00 to $in! 0VB then by 1B000 to convert $rom GB to ,B2 as !one at the top le$t o$ the $ig"re below.

&o not close ,y (omp"ter or the calc"lator "ntil the virt"al memory allocation has been c"stomi3e! below.

Steps - . /
I( running 0indows 1PB clic5E 112#tartB 122(ontrol %anelB 1/2#ystemB 102*!vance!B 5

1521%er$ormance2 #ettingsB 1A2G*!@"st $or best per$ormanceH as shown at the right o$ the image above. 172(lic5 G'CHB b"t !o not close any other win!ow yet. I( running 0indows 2 or &istaB the se."ence isE 112#tartB 122(ontrol %anelB 1/2#ystem an! #ec"rityB 102#ystemB 152G*!vance! system settingsH in the "pper le$tB 1A2*!vance!B 1721%er$ormance2 #ettingsB 1<2G*!@"st $or best per$ormanceH as shown at the right o$ the image above. 1<2(lic5 G'CHB b"t !o not close any other win!ow yet. 0hile still in the Per(ormance OptionsB clic5 the *!vance! tabB which is highlighte! at the right o$ the $ig"re belowB an! change the virt"al memory si3e to be the greater o$E 1i2 )hree times the si3e :ecommen!e! near the bottom o$ the win!owB 1ii2 0V o$ the har! !riveDs total si3e in ,B. )hen set the maxim"m si3e to the greater o$E 1i2 -o"r times the :ecommen!e! si3eB 1ii2 5V o$ the har! !riveDs total si3e in ,B. )hen clic5 #et $ollowe! by each remaining 'C b"tton.

C ec!ing "ava Version

$ yo" have a!ministrative privilegesB yo" may want to veri$y that a recent version o$ 6ava is installe! to ens"re that ,*)8*B will r"n correctly. )his can be !one simply by (trlOclic5ing www.@ava.com/en/!ownloa!/installe!.@sp an! clic5ing GVeri$y 6ava versionH. t is not necessary to have the most recent "p!ateB b"t 5eeping at least the version n"mber "p;to;!ate helps avoi! occasional b"gs.

Setting M#T$#% Preferences

3ine 0rapping
,essages generate! by G np"t_vali!ation.mH expect the "ser to ha%e 40rap lines5 acti%ated to a"tomatically $it messages to the "serDs (omman! 9in!ow. )his can be acti%ated b+ clicking 4$ile65 4Pre(erences65 and 4Command 0indow5 1"n!er the G-ontH s"b; hea!ing2 then ticking 40rap lines5*

Command 0indows Scroll 7u((er

Fo" might also want to change the G?"mber o$ lines in comman! win!ow scroll b"$$erH to 25B000 to ens"re that o"tp"ts !o not scroll o$$ the screen i$ tables o$ statistics are generate!.

Right8hand Text 3imit

* right;han! text limit can be activate! by clic5ing GPre(erencesH in the -ile men"B expan!ing the G9ditor:DebuggerH s"bmen"B clic5ing GDispla+BH tic5ing the box to GShow lineBH an! choosing a G%lacementH o$ 2;. (omments type! beyon! this col"mn in the E!itor will a"tomatically wrap to the next lineB which 5eeps comments narrow eno"gh $or pasting into a wor! processor. )he !ashe! line at col"mn 75 is also a vis"al remin!er to brea5 long lines o$ co!e onto separate lines "sing an ellipsisB which 5eeps the co!e itsel$ narrow eno"gh $or pasting.

Selecting a &ont Suitable for Pasting into 'or(

9hen pasting $rom ,*)8*BDs (omman! 9in!ow into 9or!B $ont Courier <ew preser%es spacingB an! #S #incho does the same with a narrower $ont. Both o$ these are available as stan!ar! ,*)8*B o"tp"ts under the 4$onts5 heading o$ the G-ileH men"Ds G%re$erencesH. #etting either o$ these as the ,*)8*B $ont simpli$ies the process o$ pasting into 9or!.

Pac!ing Memory
)yping Gpac5H at ,*)8*BDs comman! prompt !e$ragments the memory assigne! to ,*)8*B an! improves response time. )ype it whenever ,*)8*B has been in "se $or !ays or many ho"rs an! seems to be lagging.

II*!

uided 0alk Through

)nstalling t e PSO Toolbox *i+e+ ,nzipping t e &iles*$ter !ownloa!ing the most recent version o$ the %article #warm 'ptimi3ation :esearch )oolbox $rom www.georgeevers.org/pso_research_toolbox.htmB right;clic5 the 3ippe! $ileB an! extract all contents to G(EWH. =n3ipping to this location instea! o$ to ,y &oc"ments ens"res that wor5spaces name! transparently with the val"es o$ their containe! variables will not become problematic when combine! with the length o$ the $ile path. 1 $ yo" cannot "n3ip the $ileB yo" might $irst nee! to install a program s"ch as the $ree 7;Iip.2

Opening &iles for .(iting

Browse to the $ol!er that yo" @"st "n3ippe!B right clic5 G'pen_-iles.m, an! le$t clic5 G:"nH. )his opens the $iles most li5ely to be e!ite! an! selects G(ontrol_%anel.m as the c"rrent tabB within which yo" will speci$y precisely what sim"lations to r"n. Settings not sel(8explanator+ b+ name are described b+ accompan+ing comments. n the $"t"reB yo" can comment o"t irrelevant lines within G'pen_-iles.mH to $oc"s on partic"lar $iles to be mo!i$ie!. )he most essential settings are !isc"sse! below to intro!"ce yo" to the %article #warm 'ptimi3ation :esearch )oolbox.

#uto/saving &igures
$igure Switches
#aving $ig"res to ,*)8*BDs X.$ig $ormat is recommen!e! since this $acilitates $"t"re mo!i$ications to titlesB legen!sB $ont si3esB etc. be$ore p"blicationR whereasB $ig"res save! to other $ormats are more !i$$ic"lt to mo!i$y. 9hen switch OnOff_autosave_figs is active in s"bsection G:EG%#' 4 %#' G:*%+ ?G #E)) ?G#H o$ the control panelDs section G152 :EG%#' 4 %#' G:*%+ ?G #9 )(+E# *?& #E)) ?G#BH the %article #warm 'ptimi3ation :esearch )oolbox a"tomatically saves all $ig"res as X.$ig $iles. )his switch is active by !e$a"lt. *lso active by !e$a"lt is OnOff_autosave_figures_to_another_formatB which a"to;saves $ig"res to the a!!itional $ormat speci$ie! in setting GraphParams_autosave_format.

$igure Position
9hen saving to a $ormat other than .$igB images o$ the highest ."ality are obtaine! by maximi3ing $ig"res be$ore saving them. )he Figure_Position setting was create! $or this p"rposeB tho"gh it co"l! instea! be "se! to speci$y any non;maximi3e! location on the screen $or $ig"re generation. <

-or nowB letDs set $ig"res to $"ll;screen viewing in the control panel accor!ing to the $ollowing steps paste! $rom s"bsection G:EG%#' 4 %#' G:*%+ ?G #E)) ?G#H o$ section G152 :EG%#' 4 %#' G:*%+ ?G #9 )(+E# *?& #E)) ?G#H. Fo" can @"mp !irectly to the setting by pressing (trl O - within the control panel an! typing or pasting GFigure_PositionH. -ig"re_%osition K P1 7A 1>20 105<QR V)o set the i!eal $ig"re position $or a"tomatically V!isplaying $ig"res to yo"r screenE V112 )ype Y$ig"re112Y at ,*)8*BZs comman! prompt. V122 :esi3e the $ig"re to ass"me the si3e an! location Von the screen !esire! $or a"tomatically generate! V$ig"resE $or best !isplayB this is the maximi3e! si3e. V1/2 *t the comman! promptB type or paste Yget1$ig"re112B Z%ositionZ2Y V $ yo" maximi3e! the $ig"re in step 2B this metho! acco"nts V$or the space occ"pie! by all )as5bars s"ch as those "se! by V9in!ows an! '$$ice 1i.e. same as Yget10BZ#creen#i3eZ2Y Vexcept space is reserve! $or all men" bars $or optimal !isplay2. V102 )he res"lt is the 1S0 vector to be "se! as VY-ig"re_%ositionY above. 'nce yo" have complete! the $o"r steps aboveB letDs r"n a basis sim"lation by which to "n!erstan! the e$$ects o$ changes to be ma!e in the wal5 thro"gh. 9ith G(ontrol_%anel.mH open an! activeB please press -5 to begin an example sim"lationR this is the ."ic5est way to begin program exec"tion.

,n(erstan(ing #utomatic )nput Vali(ation

)he %article #warm 'ptimi3ation :esearch )oolbox vali!ates inp"t consistency a"tomaticallyR when the "serDs intention is in$erableB it corrects con$licting switches an! settings. )he $irst paragraph !isplaye! to the G*"tomatic np"t Vali!ationH section o$ the (omman! 9in!ow warns that switch OnOff_graph_fg_mean has been activate! in the control panelB which in!icates that the mean $"nction val"e sho"l! be graphe! vers"s iteration n"mberR howeverB we contra!ictorily speci$ie! that only one trial sho"l! be con!"cte! so that there is no mean $"nction val"e across trials. )he toolbox cannot !e$initively in$er whether to con!"ct m"ltiple trials an! graph the mean $"nction val"e or con!"ct one trial an! graph its $"nction val"e. (onse."entlyB the toolboxE 1i2 noti$ies "s o$ the !iscrepancyB 1ii2 !e;activates OnOff_graph_fg_mean a"tomatically to allow program exec"tion accor!ing to the c"rrent settingsB 1iii2 noti$ies "s o$ the action ta5enB an! 1iv2 recommen!s an alternate sol"tion in case the a"tomatic correction was not the !esire! reparation. n this exampleB since con$licting settings were not severe eno"gh to prevent !ata generationB a message warne! o$ the potential problem b"t allowe! >

program exec"tion to contin"e. 9hen a con$lict is severe eno"gh to inter$ere with inten!e! program exec"tion an! the "serDs intention cannot reliably be in$erre! to $ix the problem a"tomaticallyB a beep will be generate! along with an error message g"i!ing yo" to correct the con$licting switches or settings. )he secon! paragraph !isplaye! to the G*"tomatic np"t Vali!ationH section o$ the (omman! 9in!ow clari$ies that since phase plots an! position vers"s iteration graphs are activeB the history o$ positions has a"tomatically been activate! $rom which to constr"ct those graphs. *"tomatic changes s"ch as these ma5e the toolbox easier to "se by intelligently $ixing problems prior to program exec"tion. ?o changes nee! to be ma!e at this pointR these messages were generate! as examples.

,n(erstan(ing ,ser )nput Vali(ation

'nce G np"t_Vali!ation.mH has ma!e obvio"s corrections an! mentione! any problems tho"ght to remainB G&isplay_#ettings.mH !isplays settings $or "ser vali!ation. t can be seen $rom the G=ser np"t Vali!ationH section thatE 8best %#' will exec"te /5 iterations. ?o mechanism s"ch as position clamping or velocity reset will be employe! by which to restrain particles to remain within the initiali3ation space 1i.e. they are $ree to search $or better sol"tions o"tsi!e the space in which they are initiali3e!B which might be "se$"l $or some application problems where the initiali3ation space is nothing more than an e!"cate! g"ess as to where the global minimi3er might lie rather than being base! on physical limitations or a priori in$ormation2. Velocities will be clampe! to 50V o$ the range o$ the search spaceB which sets vmax K xmax since the initiali3ation space is centere! at the E"cli!ean origin Pi.e. vmax K 0.5Xrange K 0.5X1xmax N ;xmax2 K 0.5X12Xxmax2 K xmaxQ. *ll histories to be maintaine! are !isplaye! $or veri$ication. 'ne trial will be con!"cte!. )he swarm will consist o$ 5 particles. )he inertia weight will vary linearly $rom 0.> in the $irst velocity "p!ate to 0.0 in the $inal velocity "p!ate. )he cognitive an! social acceleration coe$$icients are each 1.0>A1<. )he two;!imensional *c5ley benchmar5 has been selecte! to allow the "ser to veri$y that the graph pro!"ce! by the %article #warm 'ptimi3ation :esearch )oolbox is correct since the Gac5ley$cnH $"nction o$ ,*)8*B contains a co"ple o$ b"gs. #ymmetric rather than asymmetric initiali3ation will be employe!. 10

$ a trial pro!"ces a $"nction val"e less than or e."al to the threshol! re."ire! $or s"ccessB 5 105 B it will be consi!ere! s"ccess$"l. )he search will not terminate "pon satis$ying the threshol! re."ire! $or s"ccess b"t will contin"e $or the $"ll /5 iterations.

%lease type GyH to con$irm that the !isplaye! settings are correctB an! letDs loo5 at some o$ the graphs that can be generate! by the %article #warm 'ptimi3ation :esearch )oolbox. %lease be patient while the graphs are generate!.

)nterpreting 0rap s
Graphs are highly important since one can more easily "n!erstan! what can be vis"ali3e!. ,*)8*BDs ability to e$$ortlessly generate power$"l graphs gives it a tremen!o"s a!vantage over lower;level programming lang"ages s"ch as (OO.

$igure <umbers
)o prevent the possibility o$ overlapping graphs sho"l! all graph types be activate! sim"ltaneo"slyB the %article #warm 'ptimi3ation :esearch )oolbox reserves a range o$ $ig"re n"mbers $or each graph type. (lic5ing on the ,*)8*B icon shows the !i$$erent ranges o$ $ig"re n"mbers "se! $or the selecte! graph types.

Ob=ecti%e $unction
-ig"re 10B000B000 shows the $"nction val"e vers"s the two !imensions being optimi3e! $or the *c5ley ob@ective $"nction. ,*)8*B !oes an excellent @ob o$ generating /;& $ig"res s"ch as this. 'b@ective $"nctions can only be graphe! in /;& 1i.e. $"nction val"e vs. two !imensions to be optimi3e!2 !"e both to geometric constraints an! the !i$$ic"lty o$ vis"ali3ing higher !imensions even within the h"man min!. #ince ob@ective behavior can change in higher !imensionsB beware that graphs in low;!imensionality may not be representative o$ ob@ective behavior in higher !imensions "nless one is attentive to the $orm"las themselves an! what can be in$erre! $rom them.

Phase Plots
Each o$ -ig"res 1B001 N 1B005 shows a phase plot $or one particle. )hese overlay a 2;& conto"r map o$ the ob@ective $"nction to show how particlesD movements are a$$ecte! by the terrain o$ the search space. )he $irst movement ta5es each particle $rom its position at k K 0 to its position at k K 1B where programming variable k co"nts the "p!ates.

11

%hase plots she! light on how particles behave in 2;& spaceB which may be "se$"l $or improving per$ormance in more problematicB higher;!imensional spaces. )he next graph type becomes "se$"l in higher !imensions.

Position %s* Iteration

Each o$ -ig"res 1 N 5 shows each !imension o$ each particleDs position vector vers"s iteration n"mber. %osition vs. iteration graphs are similar to phase plots in that they show a particleDs !imensions changing with time. Both graph types show how each particle move! iteratively $rom its starting position to its $inal positionR howeverB position vs. iteration graphs can be constr"cte! in higher !imensions since a color is assigne! to each o$ dim !imensions instea! o$ plotting !imension 2 vers"s !imension 1. )heoreticallyB position vers"s iteration graphs can han!le any n"mber o$ !imensionsR yet practicallyB graphing more than seven !imensions sim"ltaneo"sly co"l! become con$"sing !"e to overlapping lines.

#nalyzing 'or!space Variables

n this sectionB yo" will learn how to analy3e !ata pro!"ce! by the %article #warm 'ptimi3ation :esearch )oolbox. *$ter "n!erstan!ing this sectionB yo" will be able to analy3e the res"lts o$ the optimi3ation process N s"ch as each iterationDs position vectorsB personal bestsB local bests/global bestB an! the correspon!ing $"nction val"es. Fo" will also be able to analy3e the velocity vectors o$ any iteration. 9hile this !ata can be analy3e! graphicallyB some "sers will nee! to access wor5space variables !irectly to !etermine precise val"es or to write their own toolbox a!!;ins. $ yo" are only concerne! with the graphical $eat"res o$ the toolboxB yo" may s5ip this section an! @"mp !irectly to GCnowing 9here &ata is #tore!G.

(g
)he global bestDs $"nction val"e is iteratively "p!ate! an! store! to wor5space variable fg. )he best $"nction val"eB Gfg K 0.10052BH is !isplaye! in the G:es"ltsH section o$ the (omman! 9in!ow. )he same nomenclat"reB fgB is "se! regar!less o$ whether Gbest or 8best %#' is implemente! 1i.e. even tho"gh the global best is not iteratively calc"late! by 8best %#' "nless switch OnOff_ghist has been activate! to maintain a history o$ each iterationDs best position2. Fo" may type fg at the comman! prompt i$ yo" wo"l! li5e to veri$y that the val"e is correctly !isplaye!.

thresh>(or>succ
%lease type or paste thresh_for_succ at the comman! prompt. Fo" sho"l! seeE thresh_$or_s"cc K 5.0000e;005

12

)his is the val"e o$ the threshol! $or s"ccess as set in G'b@ectives.mH $or the ob@ective $"nction an! problem !imensionality selecte!. )o be consi!ere! s"ccess$"lB a trial m"st re!"ce the ob@ective $"nctionDs val"eB fgB to this threshol! val"e or better.

num>trials>success(ul
?otice the linesE ?"mber o$ #"ccess$"l )rialsE 0 ?"mber o$ =ns"ccess$"l )rialsE 1. #ince fg was not minimi3e! to the threshol! $or s"ccessB the trial was consi!ere! "ns"ccess$"l. )his is no reason $or concernB howeverB since the trial was r"n only $or /5 iterations "sing a small swarm si3e $or !emonstration p"rposes. Fo" may type or paste num_trials_successful at the comman! prompt any time to want to !etermine how many trials were s"ccess$"lB tho"gh this will generally be !isplaye! $or yo".

k
)his co"nts the n"mber o$ "p!ates. )yping k at the comman! prompt showsE 5K /0 %ositions an! velocities were "p!ate! /0 times since /5 GiterationsH were speci$ie! in the control panel via setting max_iter_per_grouping 1i.e. initiali3ation o$ the swarm O /0 ro"n!s o$ "p!ates K /5 GiterationsH2.

iter>success
?otice at the en! o$ the last line !isplaye!E iter_s"ccess K ?a?. 9or5space variable iter_success stores the iteration n"mber at which the ob@ective $"nction was s"ccess$"lly minimi3e! to thresh_for_succ. 9hen m"ltiple trials are con!"cte!B iter_success is a 1;by;num_trials vector storing the iteration o$ s"ccess $or each trial. ?a? was store! to iter_success beca"se the ob@ective $"nction was not minimi3e! to thresh_for_succ at any point !"ring the trial. ?a?Ds !enote "ns"ccess$"l trials since 0 wo"l! literally mean that at least one particle satis$ie! the threshol! $or s"ccess at initiali3ation so that no "p!ates were re."ire! to achieve s"ccess N a possible scenario sho"l! the "ser select either a large val"e $or thresh_for_succ or a very large swarm si3e. *cci!ental averaging o$ a 0 instea! o$ a ?*? when calc"lating the mean n"mber o$ iterations re."ire! $or s"ccess wo"l! pro!"ce an erroneo"s n"merical val"eR whereasB acci!ental averaging o$ a ?*? wo"l! pro!"ce a ?*?B which wo"l! clearly in!icate to the !eveloper a programming mista5e. 1/

#a$eg"ar!s s"ch as this improve the reliability o$ calc"lations per$orme! by the %article #warm 'ptimi3ation :esearch )oolbox.

x
%osition matrixB x, is "p!ate! iteratively accor!ing to the position "p!ate e."ation o$ %#'. Each row o$ x stores one particleDs position vectorE i.e. Gx11B E2H hol!s the $irst particleDs position vectorB Gx12B E2H hol!s the secon! particleDs position vectorB an! so $orth. )yping GxH at the comman! prompt !isplays particlesD $inal positions as shown below. xK ;1.71>/e;002 1.>025eO000 ;7.120/e;002 /.0715e;002 2.50A<e;001 ;1.0072e;002 ;5./025e;001 ;2.2001e;001 ;/.5571e;002 ;>.210/e;001 )he $irst particle came to rest at position vector P;0.0171>/B 1.>025QB the secon! particle came to rest at P;0.07120/B 0.0/0715QB an! so $orth.

xhist
)his is the history o$ all positions occ"pie! !"ring the search. %articlesD starting positions are ran!omly generate! !"ring the $irst iteration when "p!ate co"nter k K 0R these starting positions are containe! within the $irst dim col"mns o$ the history 1i.e. in this case col"mns 1 an! 2 since dim K 22. )yping Gwhos xhistH at the comman! prompt shows that the historyDs si3e is 5x70E this is beca"se each o$ the 5 particles recor!e! its 2;& position vector at each o$ the /5 iterations. )he last dim col"mns 1i.e. col"mns A> an! 70 in this case2 contain the $inal position matrix at iteration /5B which is e."ivalent to the $inal val"e o$ x paste! above. )he co!es in )able 1 can be "se! to access the position matrix o$ any iteration n"mber or "p!ate n"mber $rom history xhist accor!ing to the pattern shown in the $inal row. $ yo" plan to write any scripts to access !ataB yo" will want to $amiliari3e yo"rsel$ with the patterns in the tables below. *t this pointB yo" may want to paste some co!es $rom )able 1 at the comman! prompt an! analy3e the position matrices o$ the correspon!ing iterations.

10

Table ,? 9xtracting the Position #atrix o( !n+ Iteration (rom 4x ist5

Update <umber '4!5) 0 1 2 / 0 5 Iteration <umber '4! @ ,5) 1 2 / 0 5 5O1 Code (or 9xtracting Position #atrix (rom "istor+ xhist1EB xhist1EB xhist1EB xhist1EB xhist1EB xhist1EB 1E!im2 1!im O 12E12X!im22 12X!im O 12E1/X!im22 1/X!im O 12E10X!im22 10X!im O 12E15X!im22 15X!im O 12E115 O 12X!im22

?ote that xhist is only maintaine! when switch OnOff_xhist is activate! at the en! o$ the control panelDs $irst section. 9hen a variableDs history is not activate!B only the $inal val"e o$ the iteratively "p!ate! wor5space variable will be available $or analysis. -or long trialsB !e;activating "nnecessary histories improves exec"tion spee! an! re!"ces the si3e o$ a"tomatically save! wor5spaces. #im"lations can always be repeate! to generate any necessary histories i$ an exact copy o$ G(ontrol_%anel.mH is save! along with the version o$ the toolbox "se! to generate the !ata.

%
)his matrix is "p!ate! iteratively to hol! the swarmDs velocity vectors. *t the concl"sion o$ each trialB it contains the swarmDs $inal velocity vectors. )o examine particlesD $inal velocitiesB please type v at the comman! prompt. vK ;1.51A<e;002 ;>.57/7e;001 ;2.0/5>e;002 ;/.25AAe;00/ 2.770Ae;001 ;A.0505e;002 ;1.20<<eO000 <.><<>e;001 1.7>7<e;00/ ;1.0A05e;00/ )he $irst particle concl"!e! the search with velocity vector P;0.0151A<B ;0.>57/7QB the secon! with velocity vector P;0.020/5>B ;0.00/25AAQB an! so $orth as seen above.

%hist
Velocities ten! to !ecrease over the co"rse o$ the search. )he history o$ velocitiesB vhistB is "se$"l $or examining exactly how they change. %articlesD starting velocities are ran!omly generate! !"ring the $irst iteration when "p!ate co"nter k K 0R these are containe! within the $irst dim col"mns o$ the history 1i.e. in this case col"mns 1 an! 2 since dim K 22. )he last dim col"mns 1i.e. col"mns A> an! 70 in this case2 contain the $inal velocity matrix at iteration /5B which is e."ivalent to the $inal val"e o$ v paste! above. 15

)he co!es in )able 2 can be "se! to access the velocity matrix o$ any iteration n"mber or "p!ate n"mber $rom history vhist accor!ing to the pattern that becomes obvio"s in the table. *t this pointB yo" may want to paste some co!es $rom the thir! col"mn o$ )able 2 at the comman! prompt an! analy3e the velocity matrices o$ the correspon!ing iterations.

Table -? 9xtracting the &elocit+ o( !n+ Iteration (rom 4v ist5

Update <umber '4!5) 0 1 2 / 0 5 Iteration <umber '4! @ ,5) 1 2 / 0 5 5O1 Code (or 9xtracting &elocit+ #atrix (rom "istor+ vhist1EB vhist1EB vhist1EB vhist1EB vhist1EB vhist1EB 1E!im2 1!im O 12E12X!im22 12X!im O 12E1/X!im22 1/X!im O 12E10X!im22 10X!im O 12E15X!im22 15X!im O 12E115 O 12X!im22

8i5e other historiesB vhist is only maintaine! when its switchB OnOff_vhistB is active at the en! o$ the control panelDs $irst section.

p
)his matrix is "p!ate! iteratively to hol! each particleDs personal best. )o examine the swarmDs $inal personal bestsB please type p at the comman! prompt. pK 1.50AAe;002 ;1.257>eO000 ;2.00A5e;002 1.>715e;002 ;2.A7<0e;002 5.00//e;002 ;5./025e;001 ;2.2001e;001 ;/.5571e;002 ;>.210/e;001 Each row o$ p stores one particleDs personal bestE Gp11B E2H K P0.0150AAB ;1.257>Q is the personal best o$ the $irst particleB Gp12B E2H K P;0.0200A5B 0.01>715Q is the personal best o$ the secon! particleB an! so $orth.

9%aluating Positions
)o see *c5leyDs $"nction val"e $or each o$ these personal bestsB please paste 'b@-"n_*c5ley1pB np2 at the comman! prompt. )his passes into the *c5ley $"nction the matrix o$ personal bests an! re."ests that the $irst np $"nction val"es be passe! bac5R since np is the n"mber o$ particles in the swarm as set in the control panelB all 5 positions will be eval"ate!.

1A

'b@-"n_*c5ley1pB np2 ans K 0./7/1eO000 1.0052e;001 2.00//e;001 /.A1>/eO000 2.A/00eO000 )he secon! row contains the lowest $"nction val"e o$ 1.0052e;001B which correspon!s to the secon! particleDs personal best o$ P;0.0200A5B 0.01>715Q shown in p above. )he best personal best is also the global best as will be veri$ie! later. ?otice in the phase plot o$ -ig"re 1002 that the secon! particleB which event"ally pro!"ce! the best $"nction val"eB lan!e! in the general vicinity o$ the global minimi3er a$ter only two movements. -ollowing other particlesD movements along their phase plots reveals that they !i! not have s"ch a high;."ality personal best "ntil sometime a$ter their tenth movement. )his gave the secon! particle an a!vantageo"s hea! startB which helpe! it o"tper$orm the other particles over the mere /5 iterations con!"cte!. ?otice by comparing -ig"res 1 N 5 that the particles pro!"cing the best $"nction val"es 1i.e. particles 2 an! /2 ha! approximate! the global minimi3er relatively well an! re!"ce! their oscillations on both !imensions by iteration /5B while the other three particles were either still oscillating 1e.g. -ig"res 1 4 02 or moving slowly towar! the global minimi3er 1e.g. -ig"re 52. +ope$"llyB the graphs generate! by the %article #warm 'ptimi3ation :esearch )oolbox will help "s "n!erstan! an! improve the particle swarm algorithm.

(p
)he $"nction val"es o$ the personal bests are iteratively store! to wor5space variable fp. *t the concl"sion o$ a trialB it stores the $"nction val"es pro!"ce! by each particleDs personal best. )yping fp at the comman! prompt shows its contents. $p K 0./7/1eO000 1.0052e;001 2.00//e;001 /.A1>/eO000 2.A/00eO000 ?otice that this col"mn vector o$ $"nction val"es is i!entical to that generate! by 'b@-"n_*c5ley1pB np2 above. t is goo! to 5now how to eval"ate a position matrix or vectorB b"t $"nction val"es can generally be 17

analy3e! !irectly by typing the proper variable name at the comman! prompt.

(
9hile fp was the col"mn vector storing the $"nction val"e at each particleDs personal bestB f is the col"mn vector storing the $"nction val"e o$ each particleDs c"rrent or $inal position. %lease type f at the comman! prompt to see the $"nction val"e at each particleDs $inal resting position. $K 0.<></eO000 /.7257e;001 1.7702eO000 /.A1>/eO000 2.A/00eO000 9hen the search concl"!e! at iteration /5B the $o"rth an! $i$th particles ha! the same $"nction val"es store! in f as in fp. )his means that they $o"n! new personal bests in the $inal iteration as will be con$irme! below.

phist
)o examine the history o$ all personal bestsB please type phist at the comman! prompt. )hen type p an! notice that the last two col"mns o$ phist are i!entical to p since the latter is hori3ontally concatenate! to the $ormer a$ter each iteration. *s mentione! when comparing f to fpB the personal bests o$ the $inal two particles were "p!ate! in the $inal iteration since their new positions were the best they ha! $o"n! over the co"rse o$ the search. )his can now be veri$ie! by comparing the matrix o$ personal bests at iteration /0 with the $inal matrix o$ personal bests at iteration /5. )o examine the personal bests o$ iteration /0B please paste Gphist1EB 1//X!im O 12E1/0X!im22H at the comman! prompt. )hen type p to compare with the personal bests o$ $inal iteration /5. phist1EB 1//X!im O 12E1/0X!im22 ans K 1.50AAe;002 ;1.257>eO000 ;2.00A5e;002 1.>715e;002 ;2.A7<0e;002 5.00//e;002 A.7<5>e;001 ;1.12//eO000 ;/.7/A>e;002 ;>.20/7e;001

1<

pK 1.50AAe;002 ;1.257>eO000 ;2.00A5e;002 1.>715e;002 ;2.A7<0e;002 5.00//e;002 ;5./025e;001 ;2.2001e;001 ;/.5571e;002 ;>.210/e;001 )he $irst three particles 5ept the same personal bests beca"se their $inal positions !i! not pro!"ce better $"nction val"es in f than !i! their existing personal bests in fpR b"t the $inal two particles "p!ate! their personal bests since the new $"nction val"es they pro!"ce! in f were better than those alrea!y containe! in fp. *$ter "p!ating the personal bestsB fp was "p!ate! with the correspon!ing $"nction val"es. 8i5e the other histories analy3e! th"s $arB phist has 70 col"mnsE 2 !imensions $or each o$ /5 iterations. *$ter any n"mber o$ iterationsB the length o$ phist will be G1k O 12XdimH as can be in$erre! $rom )able / below. )his also applies to xhistB vhistB an! lhist N all o$ which iteratively concatenate a matrix o$ wi!th dim. *t this pointB yo" may wish to type or paste $rom the thir! col"mn o$ )able / to the comman! prompt to analy3e the matrices o$ the correspon!ing iterations.

Table /? 9xtracting the Personal 7ests o( !n+ Iteration (rom 4p ist5

Update <umber '4!5) 0 1 2 / 0 5 Iteration <umber '4! @ ,5) 1 2 / 0 5 5O1 Code (or 9xtracting Personal 7ests (rom "istor+ phist1EB phist1EB phist1EB phist1EB phist1EB phist1EB 1E!im2 1!im O 12E12X!im22 12X!im O 12E1/X!im22 1/X!im O 12E10X!im22 10X!im O 12E15X!im22 15X!im O 12E115 O 12X!im22

:emember that phist is only maintaine! when switch OnOff_phist is activate! at the en! o$ the control panelDs $irst section.

l
8ocal bests are iteratively store! to wor5space variable l. *t the concl"sion o$ a trialB l stores the best position $o"n! by each particleDs neighborhoo! over the co"rse o$ the search.

1>

lK ;2.00A5e;002 ;2.00A5e;002 ;2.00A5e;002 ;2.A7<0e;002 ;/.5571e;002

1.>715e;002 1.>715e;002 1.>715e;002 5.00//e;002 ;>.210/e;001

*s !isc"sse! "n!er s"bhea!ing GEval"ating %ositionsBH the secon! particleDs personal best o"tper$orme! those o$ the rest o$ the swarmR conse."entlyB the $irst an! thir! particles ta5e the secon! particleDs personal best to be their local best since neighborhoo! si3e 2 was speci$ie! in the control panelDs $irst section via setting lbest_neighb_si e. #ince the thir! particle ha! the secon!;best o$ all personal bestsB its position became the local best o$ the neighboring $o"rth particle. )he $i$th particle too5 its own personal best as its local best since it pro!"ce! the thir!;best o$ all personal bestsB which o"tper$orme! the neighboring $irst an! $o"rth particlesB as can be veri$ie! by typing fpB pB an! l at the comman! prompt an! examining their contents. ,atrix l is only create! when 8best %#' is exec"te! 1i.e. when switch OnOff_lbest is active in the control panelDs $irst section2B whereas global best g can be maintaine! in con@"nction with 8best %#' by activating switch OnOff_ghist to monitor how the best position change! over the co"rse o$ the search. '$ co"rseB the global best is not accesse! by the "p!ate e."ations o$ 8best %#' b"t is ma!e available merely $or analysis o$ how the best position changes in Gbest %#' vers"s 8best %#'. Fo" may veri$y that matrix g contains in each row the best o$ all personal bests by typing g at the comman! prompt.

(hist6 lhist6 . lbest>neighb>size

)o see how the local bests o$ iteration 1 !erive! $rom the positions o$ the same iterationB please type or paste G$hist1EB 12H at the comman! promptB which shows each particleDs $"nction val"e at iteration 1. )hen type or paste Gxhist1EB 1E!im2H to see the positions correspon!ing to the $"nction val"es. 8astlyB paste Glhist1EB 1E!im2H to see the local bests. )he res"lts are paste! below $or yo"r convenience. $hist1EB 12 ans K 1.570>eO001 1.<227eO001 2.12>0eO001 2.12>7eO001 1.5522eO001

20

xhist1EB 1E!im2 ans K 2.>2<<eO000 <.75/AeO000 1.2>11eO001 ;/.700<eO000 A.1A5<eO000 2./50AeO001 2.A>/0eO000 2.7<20eO001 ;0.5<07eO000 ;A.>>/5eO000 lhist1EB 1E!im2 ans K ;0.5<07eO000 ;A.>>/5eO000 2.>2<<eO000 <.75/AeO000 1.2>11eO001 ;/.700<eO000 ;0.5<07eO000 ;A.>>/5eO000 ;0.5<07eO000 ;A.>>/5eO000 ?otice that when the swarm was initiali3e! at iteration 1B the best $"nction val"e o$ 15.522 belonge! to the $i$th particleB which ha! position vector P; 0.5<07B ;A.>>/5Q. #ince the neighborhoo! si3e is set to 2 in the $irst section o$ the control panel via setting lbest_neighb_si eB the $i$th particle an! its two neighbors too5 that position vector as their local bests. )he secon!;best $"nction val"e o$ 15.70> belonge! to the $irst particleB which ha! position vector P2.>2<<B <.75/AQR the neighboring secon! particle too5 that position as its local best. )he secon! particle pro!"ce! the thir!;best $"nction val"e o$ 1<.227 at position vector P12.>11B ;/.700<QB which became the local best o$ the neighboring thir! particle. )hose three positions comprise! matrix l o$ local bests at iteration 1. )ables 0 shows how to extract $rom history fhist the col"mn vectorB fB o$ $"nction val"es correspon!ing to any iteration n"mber.

Table A? 9xtracting the $unction &alues o( !n+ Iteration (rom 4f ist5

Update <umber '4!5) 0 1 2 / 5 Iteration <umber '4! @ ,5) 1 2 / 0 5O1 Code (or 9xtracting $unction &alues (rom "istor+ $hist1EB $hist1EB $hist1EB $hist1EB $hist1EB 12 22 /2 02 5 O 12

)able 5 shows how to extract $rom history lhist the matrix o$ local bests correspon!ing to any "p!ate n"mber or iteration n"mber.

21

Table ;? 9xtracting the 3ocal 7ests o( !n+ Iteration (rom 4l ist5

Update <umber '4!5) 0 1 2 / 0 5 Iteration <umber '4! @ ,5) 1 2 / 0 5 5O1 Code (or 9xtracting 3ocal 7ests (rom "istor+ lhist1EB lhist1EB lhist1EB lhist1EB lhist1EB lhist1EB 1E!im2 1!im O 12E12X!im22 12X!im O 12E1/X!im22 1/X!im O 12E10X!im22 10X!im O 12E15X!im22 15X!im O 12E115 O 12X!im22

*s with all historiesB lhist an! fhist will only be maintaine! when switches OnOff_lhist an! OnOff_fhist respectively are active at the en! o$ the control panelDs $irst section.

g
,atrix g iteratively replicates the global best across all o$ its rows to accommo!ate matrix s"btraction in the velocity "p!ate e."ation o$ Gbest %#'. )he global best is not "s"ally maintaine! when 8best %#' is employe! 1i.e. when switch OnOff_lbest is active2R it was maintaine! $or this !emonstration only beca"se switch OnOff_ghist is active in the control panelB which activates a history o$ each iterationDs global best $or analysis. )yping g at the comman! prompt shows its contents. gK ;2.00A5e;002 1.>715e;002 ;2.00A5e;002 1.>715e;002 ;2.00A5e;002 1.>715e;002 ;2.00A5e;002 1.>715e;002 ;2.00A5e;002 1.>715e;002 t was seen earlier that the secon! particleDs personal best o$ P;0.0200A5B 0.01>715Q was the best o$ the swarm at iteration /5. (onse."entlyB it was the global best store! to each row o$ g when the search concl"!e!.

ghist
9hen switch OnOff_ghist is activate!B the global best is iteratively store! to the row o$ ghist e."al to the iteration n"mber. )yping ghist at the comman! prompt will show the $"ll history o$ global bestsE the top row is the global best $rom initiali3ation o$ the swarm at iteration 1B an! the last row stores the same position shown in g above. *s the search progresse!B both !imensions o$ the global best approache! 3ero since *c5leyDs global minimi3er is the n"ll vector. 22

*t this pointB yo" may wish to type or paste some rows o$ the thir! col"mn o$ )able / at the comman! prompt to $amiliari3e yo"rsel$ with how to analy3e the global bests o$ speci$ic iterations.

Table B? 9xtracting the 4g ist5

Update <umber '4!5) 0 1 2 / 0 5 Iteration <umber '4! @ ,5) 1 2 / 0 5 5O1

lobal 7est o( !n+ Iteration (rom

Code (or 9xtracting lobal 7ests (rom "istor+ ghist11B E2 ghist12B E2 ghist1/B E2 ghist10B E2 ghist15B E2 ghist15 O 1B E2

*s with other historiesB ghist is only maintaine! when switch OnOff_ghist is active at the en! o$ the control panelDs $irst section.

0orkspace Organization
%lease type GwhosH at the comman! prompt. $ the wor5space !oes not !isplay wellB try wi!ening or maximi3ing the (omman! 9in!ow. )yping GwhosH lists each variable in the wor5space along with its characteristics. #ince capitali3e! variables are shown $irstB variables o$ relative importance are lower case by !esign to ca"se relatively "nimportant variables to $loat to the top o$ the list o"t o$ sight. -or exampleB switches are name! G'n'$$MBH which organi3es them together in the wor5space an! p"shes them o$$ the screen when GwhosH is type!. )yping GwhoH at the comman! prompt shows all variables in the wor5space witho"t !isplaying their characteristics. Cnowing these comman!s will help yo" $in! an! analy3e important !ata.

Summar+
n this section yo" learnt that the $ollowing wor5space variables are "p!ate! iterativelyE x stores positionsB f stores the $"nction val"es pro!"ce! by xB p stores personal bestsB fp stores the $"nction val"es pro!"ce! by pB g stores the global bestB fg stores the $"nction val"e pro!"ce! by gB v stores velocitiesB an! 2/

 l stores local bests. -"rthermoreB histories o$ these variables can be activate! or !e;activate! at the en! o$ the control panelDs $irst section. )ables 1 N A show how to extract the !ata o$ partic"lar iteration n"mbers $rom activate! histories. #ince !e;activating histories improves exec"tion spee!B the copy o$ G(ontrol_%anel.mH "se! to generate any partic"lar !ata can be save! an! "se! to regenerate that !ata along with the correspon!ing histories at a later time simply by activating the appropriate switches. )his is generally pre$erable since time is more o$ the essence than excess !ata.

1nowing ' ere Data is Save(

9hen the proper switches are active as they are by !e$a"ltB $ig"res an! wor5spaces will a"tomatically be save! to the G&ataH $ol!er o$ the c"rrent !irectory. Fo" can browse to this $ol!er byE 1i2 clic5ing G#tartH at the bottom le$t o$ 9in!owsB 1ii2 clic5ing on G,y &oc"mentsH in 9in!ows S% or G&oc"mentsH in 9in!ows 7B 1iii2 typing G,*)8*BH to locate the proper $ol!er an! pressing EnterB 1iv2 browsing to the G&ataH $ol!er in the c"rrent !irectory 1i.e. the G("rrent &irectoryH is shown above ,*)8*BDs (omman! 9in!ow in what resembles the a!!ress bar o$ a browser2B an! 1v2 opening the $ol!er with the most recent !ate an! time in its name.

$igures
)he G-ig"resH $ol!er is where graphs are a"tomatically save!. Each $ig"re was save! "sing both the X.$ig $ormat an! X.png $ormats. )his is beca"se 1i2 switch OnOff_autosave_figs is active in the G:EG%#' 4 %#' G:*%+ ?G #E)) ?G#H s"bsection o$ the control panelB an! 1ii2 switch OnOff_autosave_figures_to_another_format is active along with GGraphParams_autosave_format K ZpngZG. )he p"rpose o$ saving to !"al $ormats is that i$ an X.png $ile is not to yo"r li5ingB yo" can open the correspon!ing X.$ig $ile in ,*)8*B an! man"ally mo!i$y it. -"rthermoreB sli!eshows o$ recogni3e! $ile types can easily be generate! by right;clic5ing the $irst $ile in the se."ence an! selecting G%reviewH.

0orkspaces
6"st as $ig"res are save! to the G-ig"resH $ol!erB snapshots o$ the wor5space are save! to the G9#H $ol!er. )he name o$ each save! wor5space begins with G:,[BH where [ represents the regro"ping metho! employe!. 9hen stan!ar! %#' witho"t regro"ping is "se!B save! wor5spaces begin with G:,0H. 9hen :eg%#' is activate! to liberate the swarm $rom the state o$ premat"re convergenceB save! wor5space names begin with G:,1H. G:,2H is reserve! $or $"t"re "sage. %lease browse to $ol!er G9#H.

20

R#CTrCD 'Trial Data) -ile G:,0)r1MH contains the wor5space at the en! o$ trial 1. 9hen m"ltiple trials are con!"cte!B the %article #warm 'ptimi3ation :esearch )oolbox will also save wor5spaces G:,0)r2MBH G:,0)r/MBH etc.B $or each trial con!"cte!. *ny time m"ltiple trials are con!"cte!B it is recommen!e! to have switch OnOff_!utosave_"orkspace_Per_#rial activate! atop the control panelDs $irst section to "se :*, e$$iciently by saving the wor5space at the en! o$ each trialB which both improves exec"tion spee! an! re!"ces the probability o$ memory over$low. t is recommen!e! to leave this switch active as it is by !e$a"lt. R#CCD 'Column Data) 9hen a set o$ trials completesB matrices o$ !ata are constr"cte! $rom vectors o$ trial !ata an! save! to G:,[(MH. -or exampleB the graph o$ mean $"nction val"e per iteration re."ires that a matrix o$ $"nction val"es be constr"cte! $rom each trialDs vector o$ $"nction val"es an! average! col"mn; wise. #tatistical analyses are also comp"te! $rom combine! trial !ata. (ol"mn !ata is save! a$ter each col"mn generate! to $ree "p :*,B which signi$icantly improves the spee! o$ exec"tion while signi$icantly re!"cing the probability o$ memory over$low. %ertinent col"mn !ata is reconstr"cte! be$ore saving table !ata. R#CTD 'Table Data) )able $eat"res are only available in the $"ll version o$ the %article #warm 'ptimi3ation :esearch )oolbox. )ables are most "se$"l when incrementing a %#' parameter across col"mns while hol!ing other settings static to "n!erstan! the e$$ect o$ the parameter being incremente!. *$ter each table o$ statistics generate!B the wor5space is save! to G:,[)MH. -or the $ree !emo version o$ the toolboxB this wor5space will not appear.

#ince only one trial was con!"cte! $or only one ob@ective with no val"e incremente! per col"mnB all three wor5spaces are in this case essentially the same.
!ctual 0indows n early versions o$ the toolboxB it was necessary to loa! a save! wor5space to analy3e the settings that pro!"ce! its !ata. create! transparent $ilenames to loo5 into the wor5space thro"gh the $ilenameB which allowe! me to locate the $ile o$ interest witho"t loa!ing m"ltiple wor5spaces. -or exampleB the $ollowing are enco!e! into the $ilename a$ter G:,0(H. G*c5H N the $irst three letters o$ the ob@ective name G!2H N the problem !imensionality Gnp5H N the n"mber o$ particles in the swarm Gv_cl1H N whether or not velocity clamping is active Ggr1H N the n"mber o$ gro"pings per$orme!E relevant $or :eg%#'

25

G5/5H N the n"mber o$ iterations per$orme! Gn1H N the n"mber o$ trials per$orme! Gca1.0>A2H N the val"e o$ the cognitive acceleration coe$$icient Gcb1.0>A2H N the val"e o$ the cognitive acceleration coe$$icient GvmV0.5H N the percentage o$ the range to which velocities are clampe! Gw_i0.>H N the initial val"e o$ the inertia weight Gw_$0.0H N the $inal val"e o$ the inertia weight G0.10052H N the best $"nction val"e pro!"ce! by the trial G2/*"g;10;20.5>.27H N the !ate an! time the wor5space was create!B which g"arantees "ni."eness o$ $ilenames to avoi! acci!ental overwriting o$ !ata.

9hen naming wor5spaces o$ in!ivi!"al trialsB was mostly intereste! in transparently seeing each trialDs $inal $"nction val"e. *s the %article #warm 'ptimi3ation :esearch )oolbox generate! trials a"tomaticallyB watche! the $ol!er to which !ata was being save! to monitor the $"nction val"e at the en! o$ each trialDs wor5space name. )ransparently !isplaying !ata in the $ilename was pre$erable to cl"ttering the comman! win!ow with o"tp"ts since !ata was sometimes generate! $or tens o$ tho"san!s o$ trials while a"tomatically incrementing parameters in search o$ high;."ality %#' parametersB s"ch as those p"blishe! in (hapter o$ thesis P2Q. 9hen the $"nction val"e was not being minimi3e! wellB close! the application an! $ixe! the b"g rather than wasting time. -"nction val"e G0.10052H is visible be$ore the !ate at the en! o$ wor5space G:,0)r1MHR "n$ort"natelyB to ma5e room $or itB ha! to abbreviate other portions o$ the $ilename.

#ctivating an( De/activating 0rap s

De8acti%ating Phase Plots6 Position %s* Iteration 7enchmark raph raphs6 and

n s"bsection G:EG%#' 4 %#' G:*%+ ?G #9 )(+E#H o$ the control panelDs section G152 :EG%#' 4 %#' G:*%+ ?G #9 )(+E# *?& #E)) ?G#BH please !e;activate switches OnOff_phase_plotB OnOff_graph_x, an! OnOff_graph_Ob$Fun_f_vs_%& by setting them to Glogical102H. Fo" may activate an! !e;activate switches by typing Gtr"eH an! G$alseH instea! o$ Glogical112H an! Glogical102BH b"t switching between 1Ds an! 0Ds seems easier.

!cti%ating

raphs o( Personal 7ests .

lobal 7est %s* Iteration

9hile in s"bsection G:EG%#' 4 %#' G:*%+ ?G #9 )(+E#BH please activate switches OnOff_graph_g an! OnOff_graph_pB which show how the global best an! personal bests respectively change over time.

2A

!cti%ating

raphs o( $unction &alues %s* Iteration

)o examine the $"nction val"es o$ the global best an! o$ each in!ivi!"al particleB please activate switches OnOff_graph_fg an! OnOff_graph_f respectively. ?ow press G-5H to generate !i$$erent graphs $or what will otherwise be the same trial analy3e! aboveR the changes ma!e within the control panel will be save! a"tomatically.

)nterpreting 2ewly #ctivate( 0rap s

$unction &alue %s* Iteration
)he graphs o$ $"nction val"e vers"s iteration in -ig"res 100B001 N 100B005 show how each particle sometimes steps in a poor !irection that a!versely a$$ects its $"nction val"eB b"t the overall tren! lines show progress. )he !ecreasing tren! lines are largely !"e to each particleDs iterative acceleration towar! the global best.

Personal 7ests .

lobal 7est %s* Iteration

)he graphs o$ personal bests vs. iteration in -ig"res 101 N 105B an! o$ the global best vs. iteration in -ig"re 1000B show the bests gra!"ally honing in on global minimi3er P0B 0Q. =nli5e phase plotsB position vs. iteration graphs can be generate! in m"ltiple !imensions since they graph each !imension vers"s the iteration n"mber instea! o$ graphing !imension 2 vers"s !imension 1R howeverB even position vs. iteration graphs can become !i$$ic"lt to interpret $or !imensionalities greater than 7 !"e to the n"mber o$ lines graphe! per $ig"re.

lobal 7estEs $unction &alue %s* Iteration

=nli5e each particleDs $"nction val"e in -ig"res 100B001 N 100B005B the global bestDs $"nction val"e graphe! in -ig"res 1B000B000 an! 1B000B001 never !eteriorates $rom one iteration to the next since the global best is the swarmDs memory o$ the best position !iscovere!. 'nly when the swarm is restarte! "sing Van !en BerghDs m"lti;start approach 1i.e. when OnOff_'P(O is active2 can the global bestDs $"nction val"e increase $rom one iteration to the nextB since with each restart the global best is re;initiali3e! rather than remembere! as with :eg%#'.

Switc ing %etween $best an( 0best PSO3s

n section G112 B*# ( #9 )(+E# 4 %#' *8G': )+, #E8E() '?H "n!er s"bsection G%#' *8G': )+, #E8E() '?BH !e;activate switch OnOff_lbest to activate Gbest %#'. t may be easiest to press (trl O +omeB (trl O -B type OnOff_lbestB an! press Enter to locate the setting.

27

Switc ing to Static )nertia 'eig t

9hile in s"bsection G%#' *8G': )+, #E8E() '?BH !e;activate switch OnOff_w_linear to switch $rom a linearly varying inertia weight to a static weight. )he static val"eB w, is set in section G122 %*:) (8E #9*:, *8G': )+, #E)) ?G#BH tho"gh it !oes not nee! to be change! at this point.

C anging t e Swarm Size

n section G122 %*:) (8E #9*:, *8G': )+, #E)) ?G#BH change the n"mber o$ particlesB npB $rom 5 to /. *gainB it may be ."ic5er to press (trl O - an! type Gnp.

C anging t e 2umber of )terations

?ear the en! o$ section G122 %*:) (8E #9*:, *8G': )+, #E)) ?G#BH increase the n"mber o$ iterations $rom /5 to 100 via setting max_iter_per_grouping.

#ctivating Swarm Tra4ectory 0rap s

%ress (trlO- an! type G152H to move !irectly to section G152 :EG%#' 4 %#' G:*%+ ?G #9 )(+E# *?& #E)) ?G#H. n s"bsection G:EG%#' 4 %#' G:*%+ ?G #9 )(+E#BH activate the $irst three switches an! !e;activate the remaining seven. ?otice that the comments !escribe the switches.

5e/using &igures
?ear the beginning o$ the next s"bsectionB G:EG%#' 4 %#' G:*%+ ?G #E)) ?G#BH activate switch OnOff_reuse_figuresB which closes each $ig"re be$ore generating another to avoi! the memory over$low or system !rag that co"l! otherwise res"lt $rom generating many $ig"res.

Mar!ing Personal %ests an( t e 0lobal %est

)owar! the en! o$ s"bsection G:EG%#' 4 %#' G:*%+ ?G #E)) ?G#BH notice that switches OnOff_mark_personal_bests an! OnOff_mark_global_best_alwa)s are active by !e$a"lt. )hese activate blac5 tags on the swarm tra@ectory maps !enoting where the global best an! personal bests are locate!B which $acilitates "n!erstan!ing o$ particlesD movements. 8eave these switches active.

C anging t e Ob4ective &unction

8etDs also change the ob@ective $rom *c5ley to :astriginE press (trl O En! to move to the en! o$ the control panel an! change ob$ective_id $rom 1 to 5. )here is a more !etaile! !isc"ssion o$ ob@ective selection in the next chapter. ?owB press -5 to begin exec"tionB an! con$irm that the !isplaye! settings are correct. %lease be patient while swarm tra@ectory graphs are generate!. 2<

#nalyzing Swarm Tra4ectory

)he small swarm o$ three particles prematurel) converged near the local minimi3er slightly below P0B 1Q. #ince switch OnOff_ oom_on_final_graph is active in s"bsection G:EG%#' 4 %#' G:*%+ ?G #E)) ?G#H o$ section G152 :EG%#' 4 %#' G:*%+ ?G #9 )(+E# *?& #E)) ?G#BH the $inal graph 3oome! in on the $inal swarm state. )he thir! particle approximate! the local minimi3er better than the other two particles. ?otice in the $inal $ig"re generate! to $ol!er G-ig"resH that all val"es !isplaye! on the hori3ontal axis are to be !ivi!e! by 10\/B which implies that better acc"racy was obtaine! on the vertical axis than on the hori3ontal.

Re8using $igures
#witch OnOff_reuse_figures ca"ses the same $ig"re n"mber to be re"se! $or all graphsB which prevents memory over$low when m"ltiple graphs are generate!. )his allows ."ic5 generation o$ high;."ality $ig"res regar!less o$ the n"mber o$ $ig"res generate!. )his switch was !esigne! $or "se with #warm )ra@ectory graphsB which ta5e swarm snapshots at reg"lar intervals an! overlay them on high;."alityB colore! conto"r mapsB each o$ which occ"pies :*,. 9hen "se! in con@"nction with switch OnOff_autosave_figs, each $ig"re is a"tomatically save! be$ore generating a new one. )he !ownsi!e is that $ig"res m"st be opene! to be analy3e!R howeverB this is not m"ch o$ an inconvenience i$ yo" 5now how to view them as th"mbnails. )o !o thisB open the G&ataH $ol!er an! the s"b$ol!er name! with the most recent !ate an! timeB which will be at the en! o$ the list i$ viewing by name or !ate. ?extB open the G-ig"resH $ol!er. 'n 9in!ows S%B press *lt O TVB +U to view the .png $iles as th"mbnails. 'n 9in!ows 7B clic5 the !rop;!own arrow "n!er the $ol!erDs search $iel!B an! select the !esire! th"mbnail si3e. $ yo"r operating system is wor5ing properlyB yo" sho"l! now see a"tomatically save! .png $iles serving as vis"al previews o$ the .$ig $iles to their le$t. *$ter previewing an imageB yo" can "se the right an! le$t arrow 5eys to see how the swarm progresse! over the co"rse o$ the search.

#ctivating 5egrouping PSO *5egPSO:egro"ping %#' 1:eg%#'2 regro"ps the swarm when premat"re convergence is !etecte! or when the maxim"m n"mber o$ iterations or $"nction eval"ations per gro"ping is reache!R this liberates the swarm $rom the state o$ premat"re convergenceB th"s enabling contin"e! exploration. 8etDs activate :eg%#' via switch OnOff_*egP(O in the G%#' *8G': )+, #E8E() '?H s"bsection o$ the control panelDs section G112 B*# ( #9 )(+E# 4 %#' *8G': )+, #E8E() '?H. :ememberB yo" can always locate a switch or setting ."ic5ly by pressing (trl O +omeB (trl O -B typing its nameB an! pressing Enter.

2>

?ear the en! o$ section G122 %*:) (8E #9*:, *8G': )+, #E)) ?G#BH change the maxim"m n"mber o$ iterations per gro"ping $rom 100 to 150 via setting max_iter_per_grouping. ?ear the en! o$ section G1/2 :EG%#' *8G': )+, #9 )(+E# 4 #E)) ?G#BH notice that the maxim"m n"mber o$ iterations over all gro"pings is 250 $or setting max_iter_over_all_groupings. )his speci$ies the maxim"m n"mber o$ iterations across all gro"pings o$ a :eg%#' trial. )he val"e !oes not nee! to be change! at the moment.

Setting t e &ormat for #uto/Save( &igures

#peci$y the $ormat to which $ig"res will be save! via setting GraphParams_autosave_format in s"bsection G:EG%#' 4 %#' G:*%+ ?G #E)) ?G#H o$ section G152 :EG%#' 4 %#' G:*%+ ?G #9 )(+E# *?& #E)) ?G#H. 8etDs change to bitmap $ormat by replacing GpngH with GbmpH. %lease press -5 to begin exec"tionB an! veri$y that the !isplaye! settings are correct.

#nalyzing Swarm Tra4ectory of 5egPSO

n the G&ataH $ol!er o$ the c"rrent !irectoryB browse to the $ol!er with the most recent !ate an! time in its nameE i$ viewing by name or !ate mo!i$ie!B this will be at the en! o$ the list. &o"ble clic5 the G-ig"resH $ol!erB an! !o"ble clic5 the bitmap correspon!ing to iteration 1/<R the maroon rectangle in the $ig"re !epicts the space within which the swarm regro"pe! abo"t the global best to contin"e searching $or the tr"e global minimi3er while remembering the global best o$ the previo"s gro"ping. ?otice the e$$iciency o$ regro"ping within this space vers"s restarting on the original search space o$ range 10.20 per !imension. ?ow !o"ble clic5 the $ig"re correspon!ing to iteration 17<. *$ter regro"pingB the $irst particle $o"n! a new personal best in the well that contains the global minimi3er. )ho"gh this personal best is not yet as high;."ality as the global best in the well above it !"e to its !istance $rom the center o$ the wellB it will !raw the particle bac5 to the well as i$ by c"riosity. %lease !o"ble clic5 the $ig"re correspon!ing to iteration 1<<. )he personal bests o$ the $irst an! thir! particles are now both within the well containing the global minimi3erB an! so is the global best. *lsoB the secon! particle has !iscovere! a new personal best le$t o$ the well containing the global minimi3erB which !emonstrates the exploratory bene$it o$ regro"ping the swarm. %lease !o"ble clic5 the bitmap correspon!ing to iteration n"mber 1><. )he global best an! all personal bests are now locate! within the well containing /0

the global minimi3er. 9ith only three particlesB :eg%#' has s"ccess$"lly approximate! the tr"e global minimi3er in less than 200 iterations. )he sol"tion re$inement phase is now "n!erway with some exploratory momenta. )he remaining $ig"res show the swarm re$ining its approximation o$ the global minimi3er.

#nalyzing 'or!space Variables for 5egPSO

%lease type GwhosH at the comman! promptB an! wi!en the (omman! 9in!ow as necessary to !isplay res"lts aesthetically. ?otice that $or each history maintaine! per gro"ping 1e.g. fhist, ghist, phist, vhist, an! xhist2B Gc"rrent_trialH histories were also maintaine! 1i.e. fhist_current_trial, ghist_current_trial, phist_current_trial, vhist_current_trial, an! xhist_current_trial2R each o$ these Gc"rrent_trialH histories hori3ontally concatenates each gro"pingDs history across all o$ a trialDs gro"pings. *$ter saving each gro"pingDs !ata to a temporary wor5spaceB that gro"pingDs histories are cleare! $rom the :*,. #ince Gc"rrent_trialH histories are reconstr"cte! at trialDs en! $rom these save! wor5spacesB :*, is "se! more e$$iciently than were each trialDs Gc"rrent_trialH histories maintaine! across gro"pings. %er gro"ping histories s"ch as fhist, ghist, phist, vhistB an! xhist are cleare! $rom :*, a$ter being save! to the har! !rive at the en! o$ each gro"pingR conse."entlyB at trialDs en!B they store only the histories o$ the $inal gro"ping. %lease type or paste Gwhos ghist ghist_c"rrent_trialH at the comman! prompt to compare the si3es o$ ghist an! ghist_current_trialR notice that the latter stores the global best o$ all iterations over both gro"pings o$ :eg%#'B whereas ghist stores only the global bests o$ the $inal gro"ping. Both histories contain the global best o$ the $inal iteration in the $inal rowE P2.2>55e;005B ;1./>70e;000Q. )o learn more abo"t the !etails o$ c"rrent trial histories or other wor5space variablesB please see their !escriptions in G*ppen!ix *E &e$initionsG. %lease type fg_arra) at the comman! prompt to see the $"nction val"e pro!"ce! by each gro"pingDs global best. )his shows that :eg%#' improve! sol"tion ."ality in the secon! gro"ping by liberating the swarm $rom the state o$ premat"re convergence s"$$ere! in the $irst gro"ping. :egro"ping is more e$$ective than simply restarting the search as evi!ence! by the tremen!o"sly improve! sol"tion ."ality o$ the secon! gro"pingB which stems $rom the e$$iciently si3e! regro"ping space.

Switc ing Termination Criterion from )terations to &unction .valuations

9ithin the control panelB please press (trl O +omeB (trl O -B type G$"nc_evalsBH an! press Enter to ."ic5ly $in! switch GOnOff_func_evalsH in /1

the G)E:, ?*) '? (: )E: * -': :EG%#' 4 %#' *8G': )+,#H s"bsection o$ section G112 B*# ( #9 )(+E# 4 %#' *8G': )+, #E8E() '?H. %lease activate this to switch the termination criterion $rom a maxim"m n"mber o$ iterations to a maxim"m n"mber o$ $"nction eval"ations. %ress (trl O - againB type Gmax_-EBH press Enter "ntil reaching max_F+s_per_groupingB an! set its val"e to G200000H 1i.e. 200B000 witho"t a comma2. %ress (trl O - again an! press Enter "ntil reaching setting max_F+s_over_all_groupingsB an! set this to G<00000H 1i.e. <00B000 witho"t a comma2.

C anging t e 2umber of Trials

%lease press (trl O +omeB (trl O -B type num_trialsB an! press Enter. (hange this $rom 1 to 5 to generate statistics over 5 trials. n practiceB yo" wo"l! con!"ct consi!erably more trials $or statistical reliabilityB b"t this is merely $or !emonstration.

C anging t e Problem Dimensionality

9hile in section 2B also change the !imensionality $rom 2 to /0 via setting dim to test the :eg%#' algorithm in a more !i$$ic"ltB higher;!imensional space.

C anging t e Swarm Size

#ince the problem to be solve! is now more !i$$ic"ltB increase the swarm si3eB npB $rom / to 20 on the line below dim in the same G122 %*:) (8E #9*:, *8G': )+, #E)) ?G#H section o$ the control panel.

De/activating ,nnecessary 6istories

%lease !e;activate each GOnOff_...histH switch at the en! o$ the control panelDs $irst section to !e;activate histories. #imply press (trl O -B type GOnOff_fghistBH an! set each s"ch switch to logical102.

Disabling 0rap s
%ress (trl O - in the control panelB type G152HB an! press Enter twice to move !irectly to graphing section G(5) REGPSO & PSO GRAPHING SWITCHES AND SETTINGS. )hen !e;activate switch OnOff_graphs by changing its val"e $rom logical112 to logical102.

#nalyzing t e Statistics of Multiple Trials

User Input &alidation
%lease press -5 now to begin exec"tionB which will a"tomatically save changes ma!e to the control panel. Exec"tion will ta5e a $ew min"tes since 5 long trials are being con!"cte!. n the mean timeB notice that the settings /2

!isplaye! in the =ser np"t Vali!ation section o$ ,*)8*BDs (omman! 9in!ow re$lect the changes ma!e in the control panelE :eg%#' is still applying its regro"ping mechanism to Gbest %#'. * maxim"m o$ 200B000 $"nction eval"ations have been allotte! to each gro"pingB an! regro"ping will contin"e "ntil <00B000 $"nction eval"ations are reache! "nless true_global_minimum is achieve! witho"t error be$ore then 1i.e. $"nction val"es less than 10323 cannot be !i$$erentiate! $rom the tr"e global minim"m o$ 0 by ,*)8*B2. -or all gro"pingsB premat"re convergence will be consi!ere! to have occ"rre! when the E"cli!ean swarm ra!i"s shrin5s to 0.011V o$ the gro"pingDs original swarm ra!i"sB which will trigger a"tomatic regro"ping. 'nce premat"re convergence is !etecte!B particles will be re;initiali3e! within a regro"ping space !e$ine! per !imension to be !irectly proportional to the !egree o$ "ncertainty in$erre! per !imension $rom the swarm state at premat"re convergence. )he regro"ping $actor is 1.2 !ivi!e! by the stagnation threshol!B stag_threshR this inverse proportionality ens"res that the closer particles converge be$ore regro"pingB the larger the regro"ping $actor will beB which ens"res that the regro"ping space is s"$$iciently large to $acilitate an ongoing search $or the global minimi3er. * static weight is now !isplaye! where initial an! $inal val"es were !isplaye! in con@"nction with linear variation.

9ach TrialEs Data

Each trialDs !ata is !isplaye! to the (omman! 9in!ow be$ore generating the next trialDs !ata so that any erroneo"s setting or poorly per$orming parameter combination will become obvio"s as soon as possible. 9hen per$ormance s"$$ersB ,*)8*B can be close! by clic5ing the S at the top right o$ the programB an! G(ontrol_%anel.mH can be !o"ble;clic5e! to change the settings an! try again. 9hile the !ata is being generate!B the $ollowing will explain the !ata to be !isplaye! a$ter each o$ the $ive :eg%#' trialsE fg 1i.e. the $"nction val"e o$ the global best2 will in!icate how e$$ectively the :eg%#' algorithm minimi3e! the thirty;!imensional :astrigin benchmar5. 8i5e "p!ate co"nter k, *egP(O_k initiali3es to 0 !"ring swarm initiali3ationR howeverB instea! o$ re;initiali3ing with each regro"ping as k !oesB it contin"es to increment with each iteration o$ :eg%#'. (onse."entlyB its val"e at the en! o$ each trial is 1 less than the n"mber o$ :eg%#' GiterationsH con!"cte!B which is the n"mber o$ times positions an! velocities have been "p!ate!. 'ne trialDs $inal val"e o$ *egP(O_k may be less than the $inal val"e $or other trials i$ the tr"e global minim"m is achieve! witho"t !iscernable errorB which ca"ses the search to terminate so no time is waste! in comp"tation when $"rther optimi3ation is impossibleE more speci$icallyB once error in the $"nction val"e becomes in!iscernible 1i.e. //

once f < 10323 2 so that only the !ecision variables can possibly have !iscernable error !"e to the complexity o$ the mapping $rom inp"ts to o"tp"tB the search will terminate since !ecision variables cannot be $"rther optimi3e! once error in the ob@ective $"nction becomes in!iscernible. )he thir! trial will !emonstrate this property since the tr"e global minim"m will be achieve! be$ore the trial concl"!es so that contin"e! Goptimi3ationH wo"l! be pointless. )he n"mber o$ $"nction eval"ations or iterations con!"cte! over the co"rse o$ the trial is !isplaye!. *s with *egP(O_kB this can be less than the maxim"m val"e i$ the tr"e global minim"m is achieve! witho"t errorB which will be !emonstrate! by the thir! trial. )he global bestB gB is !isplaye! a$ter each :eg%#' trial so that patterns in any problematic !imensions can be !iscovere! an! analy3e! as soon as possible. 1'nly one row o$ matrix g is !isplaye!B since each row is the same $or p"rposes o$ matrix s"btraction in the velocity "p!ate e."ation.2 )he range per !imension o$ the $inal regro"ping space is !isplaye! a$ter each :eg%#' trial since it is relate! to each !imensionDs sol"tion ."ality. (oncept"allyB since the range per !imension o$ the regro"ping space is proportional to the "ncertainty on that !imensionB overcon$i!ence on any partic"lar !imension can be co"nterpro!"ctive by preventing proper exploration. *nalyticallyB any !imension o$ the global best that is not well optimi3e! is li5ely a res"lt o$ a range so small as to have eliminate! the global minimi3er $rom the regro"ping space on that !imension. 9hen switch OnOff_k_after_each_grouping is activeB each "p!ate n"mber at which premat"re convergence was !etecte! is store! to *egP(O_k_after_each_grouping. *!!ing 1 to each col"mn o$ this row vector wo"l! pro!"ce the iteration n"mbers at which regro"ping was triggere!B with the !i$$erence being that iterations incl"!e initiali3ation o$ the swarm be$ore it begins "p!ating. 9hen switch OnOff_fg_after_each_grouping is activeB the $"nction val"e o$ each gro"pingDs global best is store! to *egP(O_fg_after_each_grouping. )he trial n"mber is !isplaye! a$ter the !ata above.

&"ring exec"tionB a progress meter will intermittently !isplay the percentage o$ calc"lations complete! an! the estimate! time o$ completion.

Per(ormance #easures
Success Rate 'nce all trials have complete!B the toolbox a"tomatically !isplays the n"mber o$ trials s"ccess$"l accor!ing to the threshol! re."ire! $or s"ccessB the n"mber o$ trials "ns"ccess$"lB an! the s"ccess rate. -or the c"rrent

/0

calc"lationsB thresh_for_succ K 100B as !isplaye! near the en! o$ the =ser np"t Vali!ation phase.
$unction 9%aluations Per(ormed *lso !isplaye! are the mean n"mber o$ $"nction eval"ations re."ire! to reach the threshol! $or s"ccess an! the mean n"mber con!"cte! per trial 1i.e. even a$ter reaching the threshol! $or s"ccess2. )he latter is less than the maxim"m o$ <00B000 $"nction eval"ations set in max_F+s_over_all_groupings beca"se the thir! trial re!"ce! the /0;& :astrigin benchmar5 to 3ero with no error !iscernable by ,*)8*B a$ter A>/B120 $"nction eval"ations 1i.e. the ob@ective $"nction was minimi3e! to less than 10323 2. )his shows the ability o$ :eg%#' to enable contin"e! progress long a$ter Gstan!ar!H %#' wo"l! have stagnate!. lobal 7est 'nce the goal o$ minimi3ing the ob@ective $"nction to its tr"e global minim"m is accomplishe!B no $"rther optimi3ation o$ the !ecision variables is possible regar!less o$ the algorithm employe!. Even tho"gh the thir! trial re!"ce! the $"nction val"e to its tr"e global minim"m o$ 0 witho"t !iscernable error 1i.e. f ] 10323 2B there was still some error !iscernable in the n;!imensional !ecision vector !"e to the complicate! mapping $rom n inp"ts to one scalar o"tp"t. *s ta"ght in high schoolB more !igits sho"l! be retaine! !"ring comp"tation than !esire! in $inal sol"tion ."alityR conse."entlyB since optimi3ation algorithms are generally applie! to $"nctions with $airly convol"te! mappings $rom inp"ts to o"tp"tB the inp"ts sho"l! retain greater !ecimal acc"racy than !esire! in the ."ality o$ the $inal sol"tion. t also $ollows that the sol"tion acc"racy possible on each !imension increases as the problem !imensionality !ecreases. Statistics -rom the best $"nction val"e per trialB the me!ianB meanB minim"mB maxim"mB an! stan!ar! !eviation are calc"late!. *s can be seen $rom the statistics generate! $or the /0;& :astrigin benchmar5B regro"ping was ."ite e$$ective at liberating the swarm $rom the state o$ premat"re convergence to $acilitate contin"e! progress. Time ReFuired *lso !isplaye! areE *verage time per trial in secon!sB Elapse! time over all trials in secon!sB teration n"mbers at which each trial satis$ie! the threshol! $or s"ccessB )ime o$ completion.

/5

7est $unction &alue per Trial )he $"nction val"e o$ each trialDs global best is store! to wor5space variable fg_final_per_trialB $rom which the me!ianB meanB minim"mB maxim"mB an! stan!ar! !eviation are calc"late!.

Fo" might want to save a bac5"p copy o$ G(ontrol_%anel.mH with these settings intact to $acilitate $"t"re :eg%#' testing.

Summary
)his wal5 thro"gh was inten!e! to intro!"ce yo" to the primary switches an! settings o$ the %article #warm 'ptimi3ation :esearch )oolbox. )he next chapter presents more !etails on several important topicsB incl"!ing a more in;!epth !isc"ssion o$ =sing :eg%#' to Escape $rom #tagnation.

III* #ore Details on Select Topics

Specifying 0best or $best PSO as t e Core #lgorit m
mprovements to %#' generally b"il! on one o$ two core algorithmsE 1i2 global best 1Gbest2 %#' or 1ii2 local or neighborhoo! 18best2 %#'. )o activate 8best %#'B set switch OnOff_lbest to logical112 "n!er the G*8G': )+,H s"bhea!ing o$ section GG112 B*# ( #9 )(+E# 4 %#' *8G': )+, #E8E() '?HH in G(ontrol_%anel.mH. -or Gbest %#'B set the switch to logical102. t is recommen!e! to "se Glogical112H an! Glogical102H rather than Gtr"eH an! G$alseH since !oing so allows switches to be activate! an! !e;activate! more ."ic5ly.

Setting t e )nertia 'eig t

)o "se a static inertia weightB !e;activate switch OnOff_w_linear in section G112 B*# ( #9 )(+E# 4 %#' *8G': )+, #E8E() '?H o$ the control panel by setting it to logical102B an! set the static val"e o$ w in section G122 %*:) (8E #9*:, *8G': )+, #E)) ?G#H. 9hen switch OnOff_w_linear is activeB the initial weightB w_iB will be "se! $or the $irst velocity "p!ateB with linear !ecrease or increase "ntil reaching the $inal weightB w_f N ass"ming the sim"lation r"ns $"ll co"rseR howeverB i$ :eg%#' or ,%#' is activate!B the $inal val"es o$ the inertia weight may not be "se! since a regro"ping or restart will be triggere! once stagnation is !etecte! to avoi! waste$"l calc"lations in a non;pro!"ctive state. )he relevant portions o$ the control panel are paste! below. )a5e care not to acci!entally change Gi$ OnOff_w_linear KK logical112H in section G122 %*:) (8E #9*:, *8G': )+, #E)) ?G#H since this merely chec5s whether initial an! $inal weights or one static weight sho"l! be loa!e! base! on the stat"s o$ switch OnOff_w_linear. /A

V^VVVVVVVVVVVVVVVVVVVVVVVVVVVVV V112 B*# ( #9 )(+E# 4 %#' *8G': )+, #E8E() '?V VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV 'n'$$_w_linear K logical102R V)"rn time;varying inertia weight on or o$$. ... V^VVVVVVVVVVVVVVVVVVVVVVVVV V122 %*:) (8E #9*:, *8G': )+, #E)) ?G#V VVVVVVVVVVVVVVVVVVVVVVVVVVV i$ 'n'$$_w_linear KK logical112 V n the case o$ time;varying inertia w_i K 0.>R Vinitial val"e "se! in $irst velocity "p!ate w_$ K 0.0R V$inal val"e "se! in $inal velocity "p!ate else Vi.e. i$ a static inertia weight is !esire! w K 0.72><0R en!

Setting t e Velocity Clamping Percentage

n n (onsi!er an example search space [ xmin , xmax ] = [100,100] B where n represents the problem !imensionalityE each !imension has a range o$ 200 centere! at 0. )he maxim"m velocityB vmax B has o$ten been ta5en in the literat"re to be some percentage o$ the maxim"m positionB xmax R howeverB this "nnecessarily creates !epen!ence on the origin o$ the search space tho"gh particle behavior sho"l! be in!epen!ent o$ the re$erence $rame chosen. #ince many real;worl! problems have search spaces that are not symmetric across the originB consi!er the same search space shi$te! 100 "nits to the " " r r r " right by changing the ob@ective $"nction $rom f 1x2 to f 1x 1002 . )he search n n space then becomes [ xmin , xmax ] = [0, 200] . &e$ining vmax as the same

percentage o$ xmax wo"l! pro!"ce twice the velocity clamping val"e as be$ore the search space was shi$te!. #"ch !epen!ence on the origin o$ the search space is easily remove! by !e$ining vmax more generally in terms o$ the range o$ the search space. 9hen the search space is centere! at the origin o$ E"cli!ean spaceB the two approaches are i!entical since the range per !imension in that case is 2 xmax so that a percentage o$ the range is also a percentage o$ xmax. )he !e$inition o$ velocity clamping "se! by the %article #warm 'ptimi3ation :esearch )oolbox is there$ore consistent with the tra!itional !e$inition $or ob@ective $"nctions whose search spaces are centere! at the origin since a percentage o$ the range is also a percentage o$ xmax . )he !e$inition employe! simply removes !epen!ence on the re$erence $rame so that /7

particles N li5e particles in physics N will behave the same way regar!less o$ the re$erence $rame by which their motions are meas"re!. -"rthermoreB some real;worl! problems have !i$$erent ranges o$ $easible val"es $or !i$$erent !ecision variables. )here$oreB the %article #warm 'ptimi3ation :esearch )oolbox !e$ines vmax as a percentage o$ the range o$ the search space per !imension. )o set vmax = 100 $or the example search space "se! aboveB simply set Gvmax_percH to 0.5 in section G122 %*:) (8E #9*:, *8G': )+, #E)) ?G#H o$ the control panelB since the range on each !imension o$ the example is 200 . )his has the e$$ect o$ setting vmax = xmax $or the original "n;shi$te! search space. vmax, j = vmax_ perc range j , where range j = xmax, j xmin, j )his more general !e$inition boils !own to the tra!itional !e$inition $or benchmar5s that "se the same range per !imensionB b"t it allows more $lexibility $or real;worl! applications whose !ecision variables may ass"me !i$$erent ranges o$ val"es. 9hen asymmetric initiali3ation is employe!B the initiali3ation spaceB x [ max , xmax ]n B !i$$ers $rom the search spaceB [ xmax , xmax ]n . n this caseB velocities 2 are clampe! to the range o$ the $"ll search spaceB which correspon!s to xmax 1i.e. the range e."als xmax N ; xmax B which e."als 2 xmax B so that a percentage o$ the range is also a percentage o$ xmax 2R this is consistent with what is generally !one in the literat"re. )he only !i$$erence is that the %article #warm 'ptimi3ation :esearch )oolbox more generally !e$ines velocities as a percentage o$ the range o$ the search space $or real;worl! application problems $or which it wo"l! be nonsensical to clamp particleDs positions base! on the val"e o$ xmax itsel$E e.g. i$ optimi3ing the !imensions o$ a so$a with length speci$ie! to lie between A$t an! < $tB clamping velocities base! on xmax rather than on the range o$ the search space along that !imension wo"l! be nonsensical since 0.5X< $t K 0 $t maxim"m step si3e per !imensionB whereas 0.5X1< $t N A $t2 K 1 $tR obvio"slyB $or a search space o$ range 2 $tB a maxim"m step si3e o$ 0 $t wo"l! be absol"tely nonsensical. ?ote that the range an! center o$ the initiali3ation space are !e$ine! per ob@ective in G'b@ectives.mH.

Selecting t e Ob4ective &unction*s'b@ectives are liste! in alphabetical or!er within $ile G'b@ectives.mBH where each is assigne! an ob$ective_id. &Ds control the or!er in which ob@ectives are teste! in the $"ll version o$ the toolboxB an! they !etermine which ob@ective will be "se! in the $ree version. $ yo" are "sing the $ree !emoB @"st /<

set the ob$ective_id in $inal section G1A2 'B6E() VE#H o$ the control panel to the i! correspon!ing to the !esire! ob@ective $"nction. )able 7 below s"mmari3es the correspon!ence between each ob@ective $"nction an! its ob$ective_id as set by !e$a"lt within G'b@ectives.mH. Fo" may change how &Ds are assigne! within that $ile i$ yo" li5e N these are merely the !e$a"lt assignments.

Table 2? Ob=ecti%e IDEs

Ob=ecti%e *c5ley Griewang5 J"a!ric ?oisy J"artic :astrigin :osenbroc5 #cha$$erDs $A #chwe$el #phere 9eighte! #phere ?? )raining 1via a!!;in2 ID 1 2 / 0 5 A 7 < > 10 11

$ yo" have p"rchase! the $"ll version o$ the %article #warm 'ptimi3ation :esearch )oolboxB yo" can test a %#' algorithm across m"ltiple ob@ectives by 1i2 setting #ableParams_num_tables in the control panelDs section G102 )*B8E #9 )(+E# 4 %*:*,E)E:#H to the total n"mber o$ ob@ectives to be teste!B 1ii2 activating switch #ableParams_OnOff_#bl_,ncr_ob$ective B 1iii2 setting ob$ective_id to 0 in the $inal section o$ the control panelB an! 1iv2 speci$ying in G'b@ectives.mH the or!er o$ the !esire! ob@ectives by assigning i! 1 to the $irst ob@ectiveB 2 to the secon!B an! so on "ntil reaching the i! correspon!ing to the n"mber o$ ob@ectives to be teste!. t is recommen!e! to test ob@ectives $rom most !i$$ic"lt to easiest so that any !i$$ic"lties will be !iscovere! ."ic5ly. )he $ree version is able to con!"ct m"ltiple trials a"tomatically $or one ob@ective at a time.

Setting t e Center an( 5ange of t e )nitialization Space

)he %#' :esearch )oolbox allows "sers to speci$y real;worl! initiali3ation spaces rather than re."iring initiali3ation to be either symmetric or asymmetric across the origin o$ n;!imensional E"cli!ean space. *s an exampleB s"ppose yo" are to $in! the optimal combination o$ vehicle lengthB wi!thB an! height to maximi3e $"el e$$iciency. *n! s"ppose that $or any reasonable combination o$ these three inp"tsB one or more vehicle types can be classi$ie! an! teste! in a sim"late! environment. )hen the c"rvat"re o$ the vehicle an! its weight can be mappe! to $rom these three !ecision variables so that only lengthB wi!thB an! height m"st be optimi3e!. $ we />

expect the i!eal length to lie somewhere between 5$t an! 15 $tB the i!eal wi!th between 1$t an! <$tB an! the i!eal height between 2 $t an! 10 $tR the range o$ the initiali3ation spaceB range_,(- wo"l! be inp"t as vector P10B 7B <Q an! the center o$ the initiali3ation spaceB center_,(- as vector P10B 0.5B AQ. )he %article #warm 'ptimi3ation :esearch )oolbox allows "sers to speci$y the center an! range o$ the initiali3ation space as vectors $or practical problems li5e this N leaving the "ser only to write his or her problem to be solve!. B"t $or ob@ectives whose initiali3ation space is the same per !imensionB the "ser can inp"t the center an! range as scalars $or convenience. -or exampleB $or the stan!ar! :astrigin ob@ective center_,(- K 0 an! range_,(- K 2X5.12 K 10.20 can be inp"tB $rom which the %article #warm 'ptimi3ation :esearch )oolbox will internally constr"ct matrices range_,( an! center_,( o$ or!er np by dim.

,sing 5egPSO to .scape from Stagnation

)he motivation behin! :eg%#' is to e$$iciently regro"p the swarm $or contin"e! progress once repeate! attractions towar! the global best have ca"se! to swarm to collapse. )his is achieve! via the $ollowing stepsE 1i2 &etect when the swarm has premat"rely converge! P1QB 1ii2 (alc"late the swarmDs "ncertainty along each !imension $rom the swarm state at premat"re convergenceB 1iii2 &e$ine the range o$ each !imension o$ the regro"ping space proportionally to the "ncertainty on that !imensionB 1iv2 (enter the regro"ping space at the global bestB 1v2 :e;initiali3e the swarm within the new regro"ping spaceB 1vi2 :e;initiali3e each particleDs personal best to be e."al to its c"rrent position as !one !"ring initiali3ation o$ the swarm with Gstan!ar!H %#'B 1vii2 :e;calc"late the velocity clamping vector "sing the range per !imension o$ the new regro"ping spaceB 1viii2(ontin"e Gbest %#' as "s"al within the regro"ping space. )his process is repeate! each time premat"re convergence is !etecte!B which enables the swarm to contin"e improving sol"tion ."ality as shown at the en! o$ #ection E * G"i!e! 9al5 )hro"gh. )o !etect premat"re convergenceB the normali3e! swarm ra!i"s criterion propose! by Van !en Bergh in his %h& thesis is "tili3e! P/Q. %articles are consi!ere! con$i!ent in their approximation along !imensions $or which no particle signi$icantly !eviates $rom the global bestR whereasB they are consi!ere! to be relatively "ncertain along any !imension $or which any n"mber o$ particles has a relatively large !eviation $rom the global best. )he "ncertainty per !imension is simply calc"late! as the maxim"m !istance o$ any particle $rom the global best along that !imension. Each !imensionDs "ncertainty is then m"ltiplie! by the static regro"ping $actor to !etermine 00

the space within which to regro"p the swarm abo"t the global best. *$ter regro"pingB Gbest %#' begins as "s"al with velocities clampe! to the same percentage o$ the new regro"ping space to which they were clampe! originallyB with the global best remembere! across gro"pings to be improve! "pon. :eg%#' wor5s well with (lercDs e."ivalent parameters P2B (trl O -B G(lercHQE constriction coe$$icients c1 = c2 = 1.49618 B an! static inertia weight = 0.72984. 'ther parameter combinations can be $o"n! to postpone stagnation in %#' witho"t regro"pingB as shown in (hapter o$ thesis P2QR howeverB empirical testing showe! that it is more bene$icial to "se (lercDs e."ivalents with :eg%#' since they pro!"ce ."ic5er convergence to the global bestB which allows the regro"ping mechanism more chances to improve sol"tion ."ality over a search o$ any $inite !"ration. n )able V;2 o$ thesisB :eg%#' achieves better per$ormance by ."ic5ly converging an! regro"ping than Gstan!ar! %#'H achieves "sing high;."ality parameters a!ept at postponing stagnation. -ollowing the steps below will lea! to e$$ective :eg%#' implementation. 112*ctivate :eg%#' by setting switch OnOff_*egP(O to logical112 in the control panelDs $irst section. 122#peci$y the maxim"m n"mber o$ $"nction eval"ations or iterations per gro"ping by pressing (trlO- an! locating either max_F+s_per_grouping or max_iter_per_grouping at the en! o$ the control panelDs secon! section 1i.e. !epen!ing on the stat"s o$ switch 'n'$$_$"nc_evals2. 8imiting each gro"ping to 100B000 $"nction eval"ations or 100B000/np iterations wor5s wellB tho"gh $or some applications another val"e between 50B000 an! 200B000 $"nction eval"ations might per$orm better. 1/2#peci$y the maxim"m n"mber o$ $"nction eval"ations or iterations over all gro"ping by pressing (trlO- an! locating either max_F+s_over_all_groupings or max_iter_over_all_groupings at the en! o$ the control panelDs thir! section 1i.e. @"st be$ore the $i$th section since the $o"rth section is not available in the $ree version o$ the toolbox2. 8imiting each trial to <00B000 $"nction eval"ations over all gro"pings or <00B000/np iterations allows eno"gh regro"pings $or the swarm to signi$icantly improve per$ormance over Gstan!ar!H %#'. 1029hile the !e$a"lt val"e o$ the stagnation threshol!B stag_threshB is recommen!e! $or general "sageB it can be change! within the control panelDs thir! section $or speci$ic applications. )his setting !etermines the sensitivity o$ the threshol! at which stagnation is consi!ere! to have occ"rre!B a$ter which regro"ping is triggere!. * larger val"e $or stag_thresh triggers regro"ping soonerR howeverB regro"ping too soon !oes not allow $or m"ch sol"tion re$inement 1a5a exploitation2B an! allowing a goo! amo"nt o$ sol"tion re$inement increases the reliability 01

o$ the "ncertainty in$erre! since particles ten! to bo"nce aro"n! ."ite a bit early in the search. )he !e$a"lt val"e o$ stag_thresh K 1.1104 allows $or a !ecent amo"nt o$ sol"tion re$inement be$ore regro"ping an! generally pro!"ces meaning$"l meas"res o$ "ncertainty. 1529hile the !e$a"lt val"e o$ the regro"ping $actorB reg_factB is recommen!e! $or general "sageB it can be change! within the control panelDs thir! section $or speci$ic applications. * larger val"e o$ reg_fact pro!"ces a larger regro"ping spaceB an! a smaller val"e has the opposite e$$ect. )he regro"ping $actor is !e$ine! to be inversely proportional to the stagnation threshol! since a smaller stagnation threshol! allows particles to collapse within closer proximity o$ each other !"ring the sol"tion re$inement phaseB which man!ates a larger regro"ping $actor by which to achieve a !ecently si3e! regro"ping space. 9ere the regro"ping $actor not !e$ine! as inversely proportional to the stagnation threshol!B tiny stagnation threshol!s wo"l! pro!"ce tiny regro"ping spacesB res"lting in an "n!esirable !epen!ency that wo"l! not $acilitate reliable escape $rom entrapping local wells. )he !e$a"lt val"e o$ 1.2/stag_thresh per$orme! well across a s"ite o$ pop"lar benchmar5s as shown in chapter $ive o$ thesis P2Q.

Specifying t e Maximum 2umber of )terations or &unction .valuations

&eci!e whether to "se a maxim"m n"mber o$ $"nction eval"ations or iterations as a termination criterion an! set switch OnOff_func_evals in the control panelDs section G112 B*# ( #9 )(+E# 4 %#' *8G': )+, #E8E() '?H accor!ingly.

#aximum per

rouping

#ettings max_F+s_per_grouping 1"se! when switch OnOff_func_evals is active2 an! max_iter_per_grouping 1"se! when OnOff_func_evals is inactive2 are locate! near the en! o$ section G122 %*:) (8E #9*:, *8G': )+, #E)) ?G#H. )hese speci$y the maxim"m n"mber o$ $"nction eval"ations or iterations allowe! by any one gro"ping o$ the swarm. ?ote that stan!ar! %#' witho"t regro"ping consists o$ only one gro"ping an! there$ore "ses these settings as its termination criteria.

#aximum o%er all RegPSO

roupings

:eg%#' regro"ps the swarm abo"t the global best when stagnation is !etecte! to contin"e searching e$$iciently rather than restarting on the original initiali3ation space. -or :eg%#'B set the maxim"m n"mber o$ $"nction eval"ationsB max_F+s_over_all_groupingsB or maxim"m n"mber o$ iterationsB max_iter_over_all_groupingsB over all gro"pings towar! the en! o$ section G1/2 :EG%#' *8G': )+, #9 )(+E# 4 #E)) ?G#H in a!!ition to setting a maxim"m per gro"ping. 02

#aximum o%er all #PSO Starts

-or m"lti;start %#' 1,%#'2B which contin"ally restarts %#' on the same initiali3ation space an! proposes the best sol"tion $o"n!B set 'P(O_max_F+s or 'P(O_max_iters towar! the en! o$ section G112 B*# ( #9 )(+E# 4 %#' *8G': )+, #E8E() '?H in a!!ition to the maxim"m per gro"ping an! possibly the maxim"m over all gro"pings 1i.e. i$ :eg%#' is activate!2. )he "ser is also given the option to speci$y a maxim"m n"mber o$ restarts via setting 'P(O_max_restarts. ,%#'B as presente! by &r. Van !en BerghB contin"ally restarts g"arantee!;convergence %#' 1G(%#'2B b"t the %article #warm 'ptimi3ation :esearch )oolbox allows the "ser to apply the m"lti; start concept to any other algorithm in the toolbox.

Confining Particles to t e Searc

Space

-or problems s"ch as the example aboveB the initiali3ation space may be nothing more than an e!"cate! g"ess o"tsi!e o$ which other val"able sol"tions might exist. +oweverB $or benchmar5s an! many real;worl! problemsB it is 5nown in a!vance that the only $easible sol"tions lie within the initiali3ation space. n s"ch casesB it is bene$icial to con$ine particles to that space. )he %article #warm 'ptimi3ation :esearch )oolbox provi!es two metho!s $or accomplishing this.

&elocit+ Reset
)his approach wor5s by resetting the velocities responsible $or p"shing particles o"t o$ bo"n!s to prevent them $rom contin"ing o"t o$ bo"n!s. t is activate! via switch OnOff_v_reset in the control panelDs $irst section "n!er s"bhea!ing G%#' *8G': )+, #E8E() '?H.

Position Clamping
!evise! another approachB which is to simply clamp particleDs positions the same way their velocities are clampe! N i.e. when any !imension is too largeB replace it with the maxim"m val"e o$ xmaxB an! when it travels too $ar in the negative !irectionB replace it with the minim"m val"e o$ xmin. )his metho! is activate! via switch OnOff_position_clamping in the control panelDs $irst section "n!er s"bhea!ing G%#' *8G': )+, #E8E() '?H. t sho"l! be note! that when asymmetric initiali3ation is "se!B the initiali3ation spaceB [ xmax / 2, xmax ]n B is !i$$erent $rom the search spaceB [ xmax , xmax ]n . n this caseB particles are con$ine! to remain within the search space rather than within the initiali3ation spaceR this is consistent with what is generally !one in the literat"re.

0enerating Statistics #utomatically

#peci$y the !esire! n"mber o$ trials to be con!"cte! per col"mn 1i.e. per set o$ trials2 via setting num_trials. Each trial is generate! "sing a !i$$erent yet 0/

repeatable se."ence o$ pse"!o;ran!om n"mbersB which is achieve! by incrementing the starting state o$ the ran!omi3er by prime n"mber 100B72> with each trial. 9hen num_trials is set to 1B the $inal $"nction val"e $or the single trial con!"cte! will be !isplaye! a"tomatically. 9hen a set o$ trials is con!"cte! 1i.e. when num_trials L 12B fg_final_per_trial will be !isplaye! a"tomatically $ollowing the me!ianB meanB minim"mB maxim"mB an! stan!ar! !eviation.

0enerating &igures
)o generate $ig"resB $irst activate main switch OnOff_graphs atop section G152 :EG%#' 4 %#' G:*%+ ?G #9 )(+E# *?& #E)) ?G#H o$ the control panel. )his is li5e the main circ"it brea5er switchE when it is o$$B all other switches are o$$. )hen select the !esire! graph types $rom the G:EG%#' 4 %#' G:*%+ ?G #9 )(+E#H s"bsection. )wo o$ the more important graph types are emphasi3e! below. $ yo" have ."estions abo"t another graph typeB please post a comment or email me.

#ean $unction &alue

'ne o$ the most important switches is OnOff_graph_fg_meanB which graphs the mean $"nction val"e o$ global best vers"s the iteration n"mber.

Swarm Tra=ector+
#witch OnOff_swarm_tra$ector) allows the "ser to watch the swarm move across a conto"r map o$ the ob@ective1s2 selecte!. )his $eat"re is $or ill"strative p"rposes an! wor5s only with two;!imensional optimi3ation problemsE i.e. problems with two inp"ts or !ecision variables to be optimi3e! N constit"ting the x an! y axes N an! a scalar o"tp"t N constit"ting what wo"l! be the 3 axis N by which to eval"ate the ."ality o$ the !ecision vector. :ather than appearing as a three;!imensional imageB a colore! conto"r map plots each $"nction val"e o$ the 3 axis "sing a color scale in which !ar5 bl"e is assigne! to the lower val"esB !ar5 re! to the "pper val"esB an! a contin"o"s color spectr"m is "se! $or val"es in between N res"lting in a $latB two;!imensionalB colore! image on which particles can more easily be seen traversing the search space. 9hen switch OnOff_contourf is activeB a smooth contin""m o$ colors will be generate!. #moothness 1i.e. acc"racy o$ the graph2 increases with the val"e o$ GraphParams_contour_map_.ualit) 1recommen!e! val"es 50 N 050 !epen!ing on whether particle movement is simply being observe! or high; ."ality $ig"res are !esire! $or presentation2. 9hen OnOff_contourf is inactiveB conto"r lines will be generate! with a n"mber on each line re$lecting the val"e o$ the $"nction at that level rather than "sing the more aesthetically pleasing colore! conto"r map.

00

#witch OnOff_mark_personal_bests mar5s the location o$ each particleDs personal best. #witch OnOff_mark_global_best_alwa)s mar5s the location o$ the global best on each conto"r map generate!B whereas OnOff_mark_global_best_on_ oomed_graph mar5s it only when 3ooming on the $inal state o$ the swarm at the en! o$ each :eg%#' gro"ping N as controlle! via switch OnOff_ oom_on_final_graph. %articlesD in!ex n"mbers can be t"rne! on an! o$$ via switch OnOff_plot_indices in the G:EG%#' 4 %#' G:*%+ ?G #E)) ?G#H s"bsection o$ G152 :EG%#' 4 %#' G:*%+ ?G #9 )(+E# *?& #E)) ?G#H. t is recommen!e! to set GraphParams_swarm_tra$_snapshot_mode to 0 since the mo!es were !evelope! in chronological or!er with each being seen as an improvement "pon the previo"s. )his "ses contmap/factor to generate a conto"r map slightly larger than the initiali3ation space since particles occasionally leave it when not restraine! "sing velocity reset.

Saving Data #utomatically

*ctivate switch OnOff_!utosave_"orkspace_Per_#rial in section G112 B*# ( #9 )(+E# 4 %#' *8G': )+, #E8E() '?H to save the wor5space a$ter each a"tomatic trial. *ctivate switch OnOff_!utosave_"orkspace_Per_0olumn to save the wor5space a$ter each set o$ trials. $ !is5 space is a concernB activate switch OnOff_!utodelete_#rial_&ata to a"tomatically !elete $iles to which single trial !ata ha! been save! once all s"ch !ata has been reconstr"cte! $or statistical p"rposes at the completion o$ all trialsB at which point reconstr"cte! !ata will be save! in the wor5space o$ each col"mn. -or the $"ll version o$ the toolboxB yo" may also save the wor5space a$ter each table generate! "sing switch OnOff_!utosave_"orkspace_Per_#able. 9hen :eg%#' is activeB yo" can save the wor5space per gro"ping via switch OnOff_!utosave_"orkspace_Per_Grouping.

.xamining Save( Data

#ave! !ata begins with G:,0H $or stan!ar! %#' witho"t regro"ping an! G:,1H $or :eg%#'B which was !esigne! to a!!ress the stagnation problem o$ %#' P2Q. 9or5spaces save! a$ter in!ivi!"al trials are !isting"ishable $rom those save! a$ter col"mns an! tables via the !esignation o$ the trial n"mber as G)r[E imme!iately $ollowing the G:,[H !isc"sse! above. -ilename G:,0)r/MH wo"l! !esignate the wor5space save! a$ter the thir! trial generate! "sing regro"ping metho! 0 1i.e. stan!ar! %#'2. G:,1(MH wo"l! !esignate the wor5space save! a$ter generating a col"mn 1i.e. set o$ trials2 pertaining to :eg%#'.

05

-ilenames are !esigne! to transparently !isplay the parameters "se! to pro!"ce the !ata containe! within the $ile so the "ser !oes not have to loa! a $ile an! examine the wor5space to locate the speci$ic $ile $or which he or she is loo5ing. -ilenames also have the !ate an! time the !ata was generate! appen!e! at the en!B which ens"res that each $ilename generate! is "ni."e an! helps the "ser locate the $ile he or she is loo5ing $or base! on when it was generate!. -or col"mn an! table !ataB val"es !isplaye! in $ilenames are pre$ace! with meaning$"l !esignators s"ch as YwY $or the inertia weight an! YcaY an! YcbY $or the acceleration coe$$icients. -ilenames $or !ata save! a$ter in!ivi!"al trials eliminate these !esignators to ma5e room $or the globally best $"nction val"e pro!"ce! per trialB which can ."ic5ly in!icate whether an algorithm is wor5ing or not by watching the $ol!er to which the !ata is being save!. 9ith time an! $amiliarityB having !ata rea!ily accessible at a glance becomes convenientB tho"gh it might seem inconvenient at $irst. -or exampleB i$ the "ser sees in the trialsZ $ilenames that the $"nction val"es being pro!"ce! are ri!ic"lo"sB he or she can imme!iately ."it an! !eb"g rather than wasting time. Excessively long $ilenames ca"se $iles to become inaccessible on 9in!ows systems "nless an! "ntil the "ser is able to shorten the nameB which wo"l! be ."ite an inconvenience as a reg"lar re."irementR conse."entlyB $ilenames are !ensely pop"late! with relevant in$ormation witho"t lengthy !escriptions as to what those val"es re$er. )his res"lts in a system that re."ires some time to ac."ire. 9hile $ilenames are generate! in a "tilitarian rather than aesthetic mannerB one can always loa! the $ile an! examine the wor5space !irectly i$ !oing so seems more convenient than !eco!ing in$ormation in the $ilenames.

Measures of .fficiency7 )terations8 &unction .valuations8 an( 5eal Time

*ccor!ing to ,erriam 9ebsterDs (ollegiate &ictionary Eleventh E!itionB an Giteration is Gthe repetition o$ a se."ence o$ comp"ter instr"ctions a speci$ie! n"mber o$ times or "ntil a con!ition is metBG which literally !oes not incl"!e initiali3ation o$ the swarm prior to the repeate! "p!ate e."ations. G-"nction eval"ation is there$ore a more transparent terminology in most cases since all eval"ations o$ positions are co"nte! N both at initiali3ation an! within the iterative loop. )his incl"!es eval"ations o$ a!!itional positions res"lting $rom any pert"rbation o$ the global best an! eval"ations o$ any a!!itional positions intro!"ce! by certain variations o$ %#' N a!!e! complexity not re$lecte! by a simple co"nt o$ iterations. -"nction eval"ations are conse."ently generally pre$erre! over iterations except possibly in graphsB where they co"l! be mislea!ing since ,*)8*B eval"ates positions one matrix at a timeR conse."entlyB the %article #warm 0A

'ptimi3ation :esearch )oolbox never "ses $"nction eval"ations on the in!epen!ent axes o$ graphs. )he "ser may choose whether to speci$y a maxim"m n"mber o$ $"nction eval"ations or iterations as a termination criterion. )o agree with the "s"al "sage o$ GiterationBH initiali3ation o$ the swarm is consi!ere! to occ"r at iteration one even tho"gh initiali3ation !oes not ma5e "se o$ the iterative "p!ating process an! conse."ently is not literally an GiterationBH as !e$ine! in the $irst paragraph o$ this sectionR conse."entlyB the $irst velocity an! position "p!ates occ"r at GiterationH two. Even tho"gh initiali3ation o$ the swarm is not literally an GiterationBH it is co"nte! towar! the maxim"m n"mber o$ iterations $or termination p"rposes. )o avoi! ambig"ityB an! to more acc"rately compare algorithms o$ !i$$erent complexitiesB it is recommen!e! to "se $"nction eval"ations as a general habit in !isc"ssionB tho"gh iterations are pre$erable $or graphs !"e to the iterative nat"re o$ optimi3ation algorithms an! the matrix "p!ates implemente! in ,*)8*B. 9hile $"nction eval"ations are generally pre$erable to iterations except in graphsB bear in min! that an algorithm re."iring more $"nction eval"ations than another is not necessarily less e$$icient in real timeB which is the "ltimate meas"re o$ an algorithmDs e$$iciency. )here are at least two reasons $or thisE 1i2 the complexity re."ire! to carry o"t a $"nction eval"ation varies across algorithms 1e.g. %#' is comp"tationally simple an! operates more ."ic5ly in real time than some other algorithms2R an! 1ii2 an algorithm "sing a smaller pop"lation si3e than another to pro!"ce the same n"mber o$ $"nction eval"ations will re."ire more iterationsB which in t"rn re."ires that more co!e be exec"te!R ass"ming a "ni$orm ."ality o$ co!e between algorithmsB exec"ting more co!e wo"l! cons"me more real time. +enceB while $"nction eval"ations are generally pre$erable to iterationsB they are still imper$ect meas"res. *n! while real time provi!es the "ltimate meas"re o$ an algorithmDs e$$iciencyB comparing real;time per$ormance across plat$orms is impractical. $ there are other topics yo" wo"l! li5e to see !isc"sse! in more !etailB please email george PatQ georgeevers.org with yo"r re."est.

I&*

uide to Conducting Gour Own Research

)his chapter will teach yo" how to apply the %article #warm 'ptimi3ation :esearch )oolbox to meet yo"r own research nee!s N whether $or solving new problemsB improving the particle swarm algorithmB or both. )o $acilitate pasting co!e into ,*)8*BB the .!oc version o$ this !oc"mentation can be !ownloa!e! $rom www.georgeevers.org/pso_research_toolbox_!oc"mentation.!oc .

07

#((ing Constraints
"u . 9berhartEs !pproach
'ne simple approach $or a!!ing constraints to the particle swarm algorithm was presente! by +" an! Eberhart in 2002 P5Q. %articles are re;initiali3e! "ntil satis$ying all constraintsB an! personal an! global bests are "p!ate! only with $easible positions. )his ens"res that all starting points an! all points o$ attraction are $easible witho"t a!!ing m"ch complexity to an otherwise comp"tationally simple algorithm. )his approach can be implemente! via the $ollowing steps. 112(reate switch OnOff_0onstraints in the control panelDs $irst section so that constraints will be implemente! only when the "ser !esires. 122(reate a new $"nctionB G(onstraints_satis$ie!.mBH to ret"rn Gtr"eH $or each position/row o$ the inp"t matrix that satis$ies all constraints an! G$alseH $or each other row. )he inp"ts sho"l! be position_matrix an! num_particles_%_evaluateB an! the o"tp"t sho"l! be a col"mn vector 1e.g. (atisfied2 o$ logical 1Ds an! 0Ds. 1/2(reate a new $"nctionB G(onstraints.mBH to correct any violations within position matrix x !"ring swarm initiali3ation. )he inp"ts sho"l! be position_matrix an! num_particles_%_evaluateB an! the o"tp"t sho"l! be the correcte! position matrixB x. G(onstraints.mH sho"l! ma5e "se o$ $"nction G(onstraints_satis$ie!.mBH which will also be "se! in step 5. 102%aste the co!e in 1a2 below to the locations speci$ie! in 1b2. 1a2 if OnOff_Constraints
"n# Constraints(x n!)

1b2a$ter Gx11EnpB 1E!im2 KH withinE ; :eg_,etho!s_0*n!1.m ; :eg_,etho!_1_regro"ping.m 152%aste the co!e in 1a2 below to the locations speci$ie! in 1b2. 1a2 if OnOff_Constraints
Constraints_satisfi"#(x n!)

1b2a$ter Gx K x O vH withinE s ; gbest_core_loop.m ; lbest_core_loop.m 1c2 a$ter Gx 1 OP(O_selections.2ox 3 45 6 OP(O_selections7.2xH withinE ; gbest_initiali3ation.m ; lbest_initiali3ation.m 1A2:eplace the co!e in 1a2 below with the co!e in 1b2 at each location speci$ie! in 1c2 so that personal bests will only be "p!ate! with positions that satis$y the constraints. 1a2 if f(Int"rna$_i) %& f!(Int"rna$_i) 1b2 if (Satisfi"#(Int"rna$_i) && (f(Int"rna$_i) %& f!(Int"rna$_i))) 0<

"n#

1!2withinE ; gbest_core_loop.m ; lbest_core_loop.m 172:eplace the co!e in 1a2 below with the co!e in 1b2 at each location speci$ie! in 1c2 so that global bests will only be "p!ate! with positions that satis$y the constraints. 1a2 f' & (in(f' (in(f)) 1b2 f' & (in(f' (in(f!)) 1c2 withinE ; gbest_core_loop.m ; lbest_core_loop.m

!n !lternati%e Idea
$ yo" wo"l! li5e to ens"re that all positions enco"ntere! !"ring the search satis$y the constraintsB yo" co"l! implement the $ollowing approachR howeverB it has not been !emonstrate! to o"tper$orm the approach above 1i.e. it has not even been teste! yet2. 112(reate switch OnOff_0onstraints in the control panelDs $irst section so that constraints will only be implemente! when the "ser !esires. 122(reate a new $"nctionB G(onstraints.mBH to correct any violations $o"n! within position matrix x !"ring swarm initiali3ation. )he inp"ts sho"l! be position_matrix an! num_particles_%_evaluateB an! the o"tp"t sho"l! be the correcte! position matrixB x. 1/2%aste the co!e below to the locations speci$ie! in 1a2 an! 1b2.
if OnOff_Constraints Constraints(x n)(_!arti*$"s_+_",a$)at" "n# -an# !ossi.$/ ,0)

1a2 a$ter Gx K x O vH withinE ; gbest_core_loop.m ; lbest_core_loop.m 1b2 a$ter Gx11EnpB 1E!im2 KH withinE ; :eg_,etho!s_0*n!1.m ; :eg_,etho!_1_regro"ping.m )his alternative approach corrects all new positions that violate constraintsR howeverB $ail"re to correct velocities responsible $or p"shing particles o"t o$ bo"n!s co"l! res"lt in repeate! violationsR there$oreB a thoro"gh implementation o$ this approach wo"l! acco"nt $or positions $irstly an! velocities secon!lyB both o$ which co"l! be correcte! by G(onstraints.mH.

.n ancing Velocity ,p(ates

$ yo"r research involves enhancing the velocity "p!ate e."ationB yo" can mo!i$y it as $ollows. 112(reate switchB OnOff_89our(witch:, in the $irst section o$ G(ontrol_%anel.mH. 122%aste the co!e below over the co!es speci$ie! in 1a2 an! 1b2. 0>

1a2 Gv K wXv O c5Xr5.X1p ; x2Oc%Xr%.X1g ; x2H within Ggbest_core_loop.mH an! 1b2 Gv K wXv O c5Xr5.X1p ; x2Oc%Xr%.X1l ; x2H within Glbest_core_loop.mH

if OnOff_1o)rS2it*3 -Ins"rt /o)r ,"$o*it/ )!#at" "4)ation 3"r".0 "$s" 5i.". stan#ar# ,"$o*it/ )!#at" "4)ation , & 26, 7 *86r8.6(! 9 x)7*+6r+.6(-' or $0 9 x) "n#

.n ancing Position ,p(ates

$ yo"r research enhances the position "p!ate e."ationB yo" can mo!i$y it as $ollows. 112(reate switchB OnOff_9our(witch, in the $irst section o$ G(ontrol_%anel.mH. 122:eplace Gx K x O vH in Ggbest_core_loop.mH an! Glbest_core_loop.mH with the $ollowingE
if OnOff_1o)rS2it*3 -Ins"rt /o)r !osition )!#at" "4)ation 3"r".0 "$s" 5i.". stan#ar# !osition )!#at" "4)ation x & x 7 , "n#

Varying t e )nertia 'eig t 2on/linearly

Control>Panel*m
$ yo"r research involves non;linear variation o$ the inertia weightB the $irst step is to create new switch OnOff_w_nonlinear_variation in G(ontrol_%anel.mH. -or sa5e o$ organi3ationB please create the new switch below OnOff_w_linear. )hen change
if OnOff_2_$in"ar 5i.". if in"rtia 2"i'3t is ti("9,ar/in' 2_i & :.;< 5initia$ ,a$)" )s"# in first ,"$o*it/ )!#at" 2_f & :.=< 5fina$ ,a$)" )s"# in fina$ ,"$o*it/ )!#at" "$s" 5i.". if a stati* in"rtia 2"i'3t is #"sir"# 2 & :.>+;?=< "n# if OnOff_2_$in"ar @@ OnOff_2_non$in"ar_,ariation 5i.". if in"rtia 52"i'3t is ti("9,ar/in' 2_i & :.;< 5initia$ ,a$)" )s"# in first ,"$o*it/ )!#at" 2_f & :.=< 5fina$ ,a$)" )s"# in fina$ ,"$o*it/ )!#at" "$s" 5i.". if a stati* in"rtia 2"i'3t is #"sir"# 2 & :.>+;?=< "n#

in section G122 %*:) (8E #9*:, *8G': )+, #E)) ?G#H to

to allow the "ser to speci$y the initial an! $inal val"es witho"t creating new programming variables "nnecessarily.

50

gbest>core>loop*m . lbest>core>loop*m
*$ter Gi$ OnOff_w_linearH in these two $ilesB a!! an Gelsei$ OnOff_w_nonlinear_variationH to implement the $orm"la by which w is to be "p!ate!.

Input>&alidation*m
#ince linear an! non;linear variation cannot occ"r sim"ltaneo"slyB please a!! a chec5 to G np"t_Vali!ation.mH s"ch as the $ollowing. n a!!ition to being a co"rtesy to the "serB this explicit chec5 will prevent "nexpecte! behavior.
if OnOff_2_non$in"ar_,ariation && OnOff_2_$in"ar "rror(-ASin*" t3" in"rtia 2"i'3t *annot ." ,ari"# .ot3 A... A$in"ar$/ an# non9$in"ar$/ !$"as" #"9a*ti,at" "it3"r A... AOnOff_2_non$in"ar_,ariation or BOnOff_2_$in"ar in t3" A... APSO ACGORITHD SECECTION s).s"*tion of t3" *ontro$ A... A!an"$Es s"*tion B(8) FASIC SWITCHES & RECATED ACGORITHDIC A... ASETTINGS.A0) "n#

Displa+>Settings*m
*t each occ"rrence o$ Gi$ OnOff_w_linearH within G&isplay_#ettings.mBH a!! a correspon!ing Gelsei$ OnOff_w_nonlinear_variationH to !isplay an appropriate message !"ring the =ser np"t Vali!ation phase o$ program exec"tion.

Title> raphs*m
Below each occ"rrence o$ Gi$ OnOff_w_linearH within G)itle Graphs.mBH a!! an Gelsei$ OnOff_w_nonlinear_variationH to !isplay the same title "n!er Gi$ OnOff_w_linearH except with the G8inearly Varying nertiaH entry change! to yo"r li5ing.

#((ing 9our Problem to t e Toolbox

0riting Gour $unction
<aming Con%ention $ yo" are "sing the %article #warm 'ptimi3ation :esearch )oolbox to solve an application problem or wo"l! li5e to a!! a new benchmar5 to the s"iteB $irstly write the new ob@ective into the toolbox as a $"nction. -eel $ree to begin with a $ile s"ch as G'b@-"n_*c5ley.mH as a templateB b"t remember to save the $"nction as a new $ile. )he $irst line o$ this new $ile is shown belowB where G?ameH is "se! generically.

$"nction P$Q K 'b@-"n_?ame1position_matrixB n"m_particles_2_eval"ate2 Each ob@ectiveDs name begins with G'b@-"n_H to organi3e ob@ectives together when viewe! by name in their containing $ol!er. )he $ilename sho"l! be the 51

same as the $"nction nameE in this case G'b@-"n_?ame.mH. :egar!less o$ the nameB the $"nction will be assigne! $"nction han!le Ob$Fun !"ring program exec"tionB which allows the toolbox to "se one generic name $or all ob@ectives as in the co!e paste! below $rom Ggbest_core_loop.mH. $ K 'b@-"n1xB np2
Inputs to be !ccepted #ince generic $"nction han!le Ob$Fun is "se! internallyB all ob@ectives m"st accept the same inp"tsE 1i2 a matrix o$ positionsB an! 1ii2 the n"mber o$ positions/rows to be eval"ate!. n the example line paste! aboveB the $"ll position matrixB x, is inp"t $or eval"ationR the secon! inp"t arg"ment speci$ies that all np rows o$ the inp"t matrix are to be eval"ate! 1i.e. the $"ll swarm si3e2. )ho"gh Gsi3e1position_matrixB 12H co"l! be calc"late! iteratively within the $"nction rather than passing in the n"mber o$ rows to be eval"ate!B the approach "tili3e! len!s itsel$ to e$$icient comp"ting by eliminating the nee! to repeate!ly calc"late 5nowle!ge alrea!y possesse! by the toolbox.

9hen eval"ating a single position vectorB pass in a 1 $or the n"mber o$ rows to be eval"ate! as shown in the co!e belowB which eval"ates the ."ality o$ a m"tate! global bestB gmB within Ggbest_core_loop.mH when switch OnOff_0auch)_mutation_of_global_best is active. $m K 'b@-"n1gmB 12
lobal &ariables #ome ob@ectives also ma5e "se o$ the problem !imensionalityB dimB which is the n"mber o$ col"mns in the inp"t matrix/vector. -or this reasonB dim is !eclare! global in the control panel an! is accessible by any $"nction that also !eclares it globalB which eliminates the ine$$icient nee! to iteratively calc"late Gsi3e1position_matrixB 22H within the $"nction. )he *c5ley $"nctionB G'b@-"n_*c5leyBH is one example o$ a $"nction that ma5es "se o$ global variable dim.

$ yo" nee! to access any other wor5space variable within yo"r $"nctionB yo" can simply !e$ine it to be globalE 1i2 where it is create! by the toolboxB an! 1ii2 within the new $"nction. -or exampleB dim is !eclare! global in the control panel imme!iately prior to its creation so that it can be accesse! by G'b@-"n_*c5ley.mBH which also !eclares it globalR the same can be !one $or any other wor5space variable an! $"nction to grant imme!iate access to variables in the wor5space that !o not change when the $"nction is calle!.

3isting Gour $unction in Ob=ecti%es*m

'nce yo" have a!!e! yo"r problem to the %article #warm 'ptimi3ation :esearch )oolboxB the next steps areE 1i2 assign it an ob@ective &B an! 1ii2 52

speci$y a reasonable initiali3ation space within G'b@ectives.mBH which lists all ob@ectives in alphabetical or!er. %lease $in! the alphabetical location $or yo"r ob@ective an! paste the co!e below into G'b@ectives.mH. elsei$ ob@ective_i! KK 12 ob@ective K Z?ameFo"r'b@ective+ereZR i$ 'n'$$_asymmetric_initiali3ation range_ #0 K 2.5AR Vasymmetric P2.5AB 5.12Q center_ #0 K /.<0R Vi.e. 1lower bo"n! O "pper bo"n!2/2 else range_ #0 K 2X5.12R VP;5.12B 5.12Q center_ #0 K 0R V#peci$y the initial center_ # o$ the initiali3ation space 1n"ll vector V!e$a"lt $or each particle2. en! i$ 'n'$$_#"ccess$"l=ns"ccess$"l i$ !im KK /0 thresh_$or_s"cc K 0.01R elsei$ !im KK2 thresh_$or_s"cc K 10\1;72R en! en!
<ame *$ter pasting the co!e above into G'b@ectives.mBH :eplace Z?ameFo"r'b@ectiveZ with the name yo"r ob@ective as yo" wo"l! li5e it to be !isplaye! by the toolbox 1i.e. witho"t G'b@-"n_H at the beginning2.

Initialization Space #peci$y the center an! range o$ the initiali3ation space. $ each !imension to be optimi3e! has the same range o$ potential val"es 1e.g. as is the case with simple benchmar5s2B then the range an! center o$ the initiali3ation space can be inp"t as scalarsB which tells the toolbox to "se the same val"es per !imension. +oweverB i$ !i$$erent !imensions represent !i$$erent variables to be optimi3e! N each o$ which has its own range o$ val"es N a !i$$erent range o$ val"es can be speci$ie! per !imension by !esignating the range an! center o$ the initiali3ation space as vectorsR an example problem o$ this type is given in chapter G E ,ore &etails on #elect )opicsH in section G#etting the (enter an! :ange o$ the nitiali3ation #paceH.
Threshold (or Success #peci$y a threshol! o$ s"ccess $or each problem !imensionality o$ interest. )he higher the problem !imensionalityB the more !i$$ic"lt the problem will become. -or this reasonB the threshol! $or s"ccess sho"l! increase with !imensionality while acco"nting $or the !i$$ic"lty o$ the partic"lar problem. )he threshol! $or s"ccess merely !etermines the s"b@ective $"nction val"e at which a trial sho"l! be consi!ere! s"ccess$"lR yet !espite its s"b@ective

5/

nat"reB thresh_for_succ complements the generate! statistics by showing the n"mber o$ trials per$orming worse than the val"e o$ interestB which gives an i!ea o$ how s"ch o"tlying trials a$$ect the meanB stan!ar! !eviationB an! me!ian.
Check (or Redundant Id !ssignment 8astlyB chec5 that no other problem is assigne! the same ob@ective i! within G'b@ectives.mBH an! set ob$ective_id at the en! o$ the control panel to the same val"e assigne! to yo"r problem within G'b@ectives.mH.

Ot er .n ancements
$ yo"r research involves more than @"st a!!ing constraintsB mo!i$ying the velocity "p!ate e."ationB mo!i$ying the position "p!ate e."ationB a!!ing yo"r problem to the %article #warm 'ptimi3ation :esearch )oolboxB an!/or training *??DsB yo" are welcome to email me $or g"i!ance. will be happy to provi!e g"i!ing hints as time permits to help yo" a!! the nee!e! $"nctionality to the %article #warm 'ptimi3ation :esearch )oolbox as !one at this ,ath9or5s threa!.

%enefiting from ,p(ates

ntegrating improvements into the %#' :esearch )oolbox will allow the contrib"tor to bene$it $rom $"t"re $ixes an! enhancements ma!e to the toolbox. #imply email me any improve! co!e with !etaile! comments !escribing its $"nctionalityB an! will a!! it to the toolbox as time permits. )he "ser comm"nity wo"l! appreciate yo"r contrib"tionB an! $ormal ac5nowle!gement will be ma!e.

&* Contribution Standards

&ollowing 2aming Conventions ' en Creating 2ew Variables
*s m"ch as possibleB please name new programming %ariables so that they will be grouped together when 4whos5 is t+ped at the command prompt 1e.g. switches begin with G'n'$$_BH graph settings begin with GGraph%arams_BH table settings begin with G)able%arams_BH :eg%#' settings begin with G:eg%#'_BH G(%#' settings begin with GG(%#'_BH an! m"lti;start settings begin with G,%#'_H. #ince capitali3e! names !isplay be$ore lower case names when GwhosH is type! at the comman! promptB capitali3e! names are p"she! o$$ the screenR hence switches6 internal %ariablesB an! the li5e sho"l! begin with capital letters6 while outputs an! settings to $re."ently re$erence! 1e.g. np, dim, w2 sho"l! begin with lower8case letters to be more easily accessible. Please use transparent names $or all programming variables to help "sers easily ma5e sense o$ the wor5space an! co!e* -or exampleB a$ter 50

writing an! testing the co!e to calc"late the n"mber an! percentage o$ trials s"ccess$"lB replace! all occ"rrences o$ G?sH with num_trials_successful an! G?"H with num_trials_unsuccessful. )his gro"pe! them in the wor5space with the total n"mber o$ trialsB num_trialsB an! ma!e their meaning obvio"s to all "sers. Be$ore creating new variablesB pleaseE 1i2 press *lt O TeB iU within ,*)8*B to G-in! -ilesBH 1ii2 type the name yo" are thin5ing o$ assigning to the new variable into the G-in! $iles containing textH $iel!B 1iii2 $or the G8oo5 inH $iel! GBrowseH to the $ol!er containing the most recently "p!ate! version o$ the %article #warm 'ptimi3ation :esearch )oolboxB 1iv2 change the Gsearch typeH at the lower le$t to G,atches whole wor!BH an! 1v2 press Enter. $ no res"lts are !isplaye!B the variable name is available $or "se.

Minimizing 6ar( Co(ing via :Control;Panel+m<

Please create a switch (or new code so that it will run onl+ when acti%ated* %lease name switches Y'n'$$_P:elevantly_?ame!_#witchQY "sing transparent names. %lease activate switches "sing syntax Y'n'$$_P:elevantly_?ame!_#witchQ K logical112Y an! !e;activate by setting Y'n'$$_P:elevantly_?ame!_#witchQ K logical102Y to enable the "ser to activate or !e;activate switches simply by changing 1 to 0 or vice versa. )his is more e$$icient than changing Gtr"eH to G$alseH or vice versa. %lease !escribe the switchDs $"nctionality with a comment. %lease a!! yo"r name an! email a!!ress a$ter switches yo" create so that $"t"re "sers will 5now who to contact with ."estions 1e.g. 'n'$$_PFo"r#witchQK logical112 V6on&oe_6onZs=niversity.e!"B 6on &oeB 200>2. I( +ou (ind +oursel( regularl+ changing a hard8coded %alue6 tr+ creating a %ariable whose %alue can be set in 4Control>Panel*m <+ -or exampleB in a!!ition to saving $ig"res to .$ig $ormat to $acilitate mo!i$ication within ,*)8*B prior to p"blicationB it became !esirable to eliminate the nee! to man"ally save $ig"res to other $ormats. )o speci$y the !esire! $ormat witho"t har! co!ing itB parameter GraphParams_autosave_format was intro!"ce! in the control panelDs section GGraphing #witches 4 %arametersH. )o avoi! wor5space cl"tterB it is only create! when switch OnOff_autosave_figures_to_another_format is active. Please take care to place new programming %ariables in the appropriate sections o$ G(ontrol_%anel.mH. )he control panel is !ivi!e! into the $ollowing sectionsE 1i2 Basic #witches 4 %#' *lgorithm #electionB 1ii2 %article #warm *lgorithm #ettingsB 1iii2 :eg%#' *lgorithm #witches 4 #ettingsB 1iv2 )able #witches 4 #ettingsB 1v2 :eg%#' an! %#' Graphing #witches 4 #ettingsB an! 1vi2 'b@ectives. 55

,ost o$ the settings pertaining to the ob@ective $"nction are set within G'b@ectives.mBH which is calle! $rom section G1A2 'B6E() VE#H o$ G(ontrol_%anel.mH. )his script is 5ept separate $rom G(ontrol_%anel.mH since it is re;"se! when ob@ectives are a"tomatically incremente! by the pro$essional version o$ the toolbox.

Vali(ating )nputs via :)nput;Vali(ation+m<

Y np"t_Vali!ation.mY is calle! to chec5 the vali!ity an! consistency o$ "ser; speci$ie! switches an! settings. 9hile !eb"gging new co!eB yo" may enco"nter "ninten!e! parameter combinations that lea! to errors or "nexpecte! behavior. 0hen a contradictor+ combination o( switches and settings in HControl;Panel+mH produces or could concei%abl+ produce unexpected beha%ior6 please write a check within HInput>&alidation*mH to either 1i2 prevent exec"tion altogether an! !isplay a message !irecting the "ser to correct the con$licting inp"tsB or 1ii2 correct the $lawe! parameter a"tomatically by in$erring the "serDs intention an! !isplay a message stating what change was ma!e. *n example o$ the secon! case is !isplaye! below. i$ 'n'$$_graph_$ 44 `'n'$$_$hist V $ the $"nction val"e o$ each Vparticle is to be graphe! 1instea! o$ only the $"nction val"e Vo$ the global best2B 'n'$$_$hist K logical112R Vmatrix Y$histBY which maintains in each Vrow the $"nction val"e o$ the correspon!ing particleB m"st be Vmaintaine!. !isp1PZBeca"se yo" have activate! Y'n'$$_graph_$BY which graphsZ... Z the $"nction val"e vers"s iteration $or every Z... Zsingle particleB matrix Y$histY m"st be maintaine!R Z... Zthere$oreB Y'n'$$_$histY has been activate! $or yo"r convenience.ZQ2 $print$1ZWrZ2 V nsert a carriage ret"rn be$ore $"t"re o"tp"ts. en!

Steps (or !dding an !utomatic Correction to 4Input>&alidation*m5

8ocate the proper location $or the Gi$ statementH. -or exampleB graphing switches are only create! in the wor5space when parent switch OnOff_graphs is active. #o the example co!e above appears a$ter that parent switch has been chec5e! since the attempt to chec5 the stat"s o$ non;existent switches wo"l! terminate exec"tion with an error. Generate a message appropriate to the severity o$ the con$lict. =se the G!ispH comman! to merely !isplay a ca"tion o$ which the "ser sho"l! be aware. )he co!e above exempli$ies this "sage. 5A

 )o in!icate a more "rgent messageB GwarningH can be "se! instea! o$ G!ispH. )his @"st a!!s G9arningEH be$ore the message to be !isplaye!. =se GerrorH to !irect the "ser to con$licting inp"ts an! prevent exec"tionB which is appropriate when the con$licting inp"ts are severe eno"gh to re."ire correction be$ore exec"tion. )his wo"l! be "se! when the "serDs intentions cannot acc"rately be in$erre! as they can be in the example co!e paste! above. )o !isplay a !etaile! messageB please "se ellipses 1i.e. GMH2 as !one in the example co!e paste! above to in!icate to ,*)8*B that the message contin"es on the $ollowing line. )his creates a message that wraps nicely within the "serDs (omman! 9in!ow regar!less o$ the win!owDs si3e. *$ter each G!isp1M2H or Gwarning1M2BH please type Y$print$1ZWrZ2Y to a!! a carriage ret"rn so that any other messages will not be r"n together.

Displaying t e Most )mportant Settings via :Display;Settings+m<

-or each new algorithm or enhancementB please !isplay the most important settings $or "ser veri$ication in script G&isplay_#ettings.mBH which is calle! a$ter G np"t_Vali!ation.mH. -or exampleB when :eg%#' is activate! via switch OnOff_*egP(O in the control panelDs $irst sectionB the $ollowing lines o$ G&isplay_#ettings.m will !isplay relevant settings $or "ser veri$ication be$ore program exec"tion. Beca"se the "ser is given !irect control o$ so many settings in the control panelB it is "se$"l to !isplay the most important settings $or veri$ication prior to exec"tion. i$ :eg_,etho! KK 1 !isp1Z:egro"ping %#' 1:eg%#'2 "tili3ingZ2 elsei$ :eg_,etho! KK 2 !isp1Z:egro"ping ,etho! 2 "tili3ingZ2 Vnot as reliable as :eg%#'E co!e Vnot yet per$ecte! en! ... i$ 'n'$$_$"nc_evals i$ :eg_,etho! `K 0 Vi.e. a regro"ping %#' will be "se! !isp1Pn"m2str1max_-Es_per_gro"ping2B Z -EZZs maxim"m Z... Z1per gro"ping2ZQ2 !isp1Pn"m2str1npXceil1max_-Es_over_all_gro"pings/np22... Z -EZZs maxim"m 1total over all gro"pings2ZQ2 else Vi.e. :eg_,etho! KK 0 selects stan!ar! %#' !isp1Pn"m2str1max_-Es_per_gro"ping2B Z -EZZs maxim"mZQ2 en! else Vi.e. i$ iterations are to be "se! instea! o$ $"nction eval"ations 57

i$ :eg_,etho! `K 0 Vi.e. a regro"ping %#' will be "se! !isp1Pn"m2str1max_iter_per_gro"ping2B Z iterations Z... Zmaxim"m 1per gro"ping2ZQ2 !isp1Pn"m2str1max_iter_over_all_gro"pings2B Z iterations Z... Zmaxim"m 1total over all gro"pings2ZQ2 else V1i.e. :eg_,etho! KK 0 $or the stan!ar! case witho"t Vregro"ping2 !isp1Pn"m2str1max_iter_per_gro"ping2B Z iterations maxim"mZQ2 en! en!

Maintaining 'or!space )ntegrity

Ygbest_core_loop.mY an! Glbest_core_loop.mH were written as scripts a$ter tests showe! scripts to be more e$$icient than $"nctions regar!less o$ whether variables were passe! in or !eclare! as globals. )he common wor5space is convenient an! e$$icientB b"t the "ser sho"l! ta5e care to maintain an organi3e! wor5space. t is better $or an error to terminate exec"tion beca"se a variable has been cleare! than to allow $or the possibility o$ a variable being "se! "nexpecte!lyE i$ in !o"btB clear the variable. *n! always clear the variable a$ter its $inal "sage "nless yo" want it to remain in the wor5space $or event"al analysis.

Steps to !%oid Unintentional Usage o( Data

&ariables should be cleared when the+ are no longer needed to 1i2 maintain an organi3e! wor5spaceB 1ii2 re!"ce both the si3e o$ the a"tomatically save! wor5spaces an! the time re."ire! to save themB an! 1iii2 eliminate the possibility o$ an incorrect val"e being "se! $or another p"rpose sho"l! the same basic variable name acci!entally be "se by another contrib"tor. )his can be !one by typing GclearH $ollowe! by the name o$ the variables to be cleare!. )o clear a global variableB type Gclear globalH instea! o$ GclearH an! !eclare the variable global again on the $ollowing line. &ariables should be re8initialized where appropriate 'e*g* with each new table6 column6 regrouping6 or automatic restart o( the algorithm) to ens"re that each variable initiali3es with a $resh slate with each new exec"tion. -or exampleB the variables pertaining to each trial are initiali3e! within G)rial_ nitiali3ations.mHB an! variables pertaining to :eg%#' are initiali3e! within G:eg%#'_main.mH "n!er hea!ing G5Initia$iG" HR"'PSO_(ainH Iaria.$"s5H. 8ist any new variables that yo" have create! in the &e$initions o$ *ppen!ix * so that other "sers will "n!erstan! their p"rpose. )his will help other contrib"tors avoi! "sing the same name $or another p"rpose so that yo"r $"nctionality contin"es wor5ing as planne!. #imply email me any e!its to be a!!e! to the !oc"mentation.

5<

Testing 2ew Co(e )n(epen(ently of t e PSO Toolbox

t is important to avoi! overcon$i!ence in oneDs ability to write $lawless co!e. ?ew bloc5s o$ co!e sho"l! be teste! in!epen!ently o$ the overall toolbox to ens"re that they wor5 as expecte! since slight !eviations $rom expecte! behavior might go "nnotice! in overall per$ormance. 'ne healthy approach is to write new co!e in a new $ileR whether as a $"nction or as a scriptB it will be easier to !eb"g this way. * test $ile can then be "se! to 1i2 pass test val"es $or all expecte! variables into a $"nction an! !isplay the o"tp"tsB or 1ii2 create the expecte! variables in the wor5space i$ a script is being teste! instea! o$ a $"nction. 9hen testing a scriptB this test $ile sho"l! state Gclear allH near the top to ens"re that previo"sly create! val"es !o not poll"te the integrity o$ yo"r tests. #aving these test $iles an! organi3ing them well will help when testing $"t"re mo!i$ications. $ the new co!e applies to the iterative portion o$ the toolbox 1e.g. Ggbest_core_loop.mH or Glbest_core_loop.mBH it may be more e$$icient to paste the co!e !irectly into the calling script once it has been veri$ie! to wor5 correctlyR howeverB this sho"l! be not !one "ntil the co!e has been thoro"ghly teste!.

Mo(ifying all 5elevant 0best = $best Scripts

-or clarity o$ tho"ghtB separate scripts are maintaine! $or the Gbest an! ;best variants o$ %#'. #ome contrib"tions may not apply to all scripts N $or exampleB any pert"rbation or other treatment o$ the global best is irrelevant to ;best %#'B which "tili3es a local or neighborhoo! best rather than a global best. *$ter in!epen!ently testing contrib"tions to iterative portions o$ the co!eB please be s"re to apply them to all o$ the $ollowing $iles as relevantE 1. gbest_core.mB 2. lbest_core.mB /. gbest_initiali3ation.mB 0. lbest_initiali3ation.m 5. gbest_core_loop.mB an! A. lbest_core_loop.m.

Commenting #fter :else<

9hen "sing an Gi$B elseH se."enceB please paste a comment a$ter each GelseH clari$ying to which Gi$H statement it correspon!s. )his improves the e$$iciency o$ e!iting since one can more easily navigate to the relevant section to be e!ite! witho"t nee!ing to re$er bac5 to the correspon!ing Gi$H statement. -or exampleB at the time o$ this entryB Ggbest_core.mH containe! the $ollowingE 8ine /0E i$ 'n'$$_,%#' ... 5>

8ine 100E 8ine 2/AE

else Vi.e. i$ `'n'$$_,%#' ... en!

n the E!itorB hovering the c"rsor over Gi$H on line /0 shows a clic5able lin5 at the bottom le$t o$ the E!itor to the correspon!ing Gen!H on line 2/A. #imilarlyB hovering the mo"se over GelseH or Gen!H pro!"ces a clic5able text box at the top le$t o$ the E!itor !isplaying the line n"mber o$ the beginning Gi$H statementR howeverB the line n"mber correspon!ing to GelseH is nowhere shownB so that its location can only be estimate! to lie abo"t hal$ way between Gi$H an! Gen!H. (onse."entlyB commenting GelseH lines helps 5eep co!e organi3e! an! rea!ableB which is bene$icial !"ring e!iting. +ope$"llyB )he ,ath9or5s will expan! the !isplay boxes in the $"t"re to list all relevant Gelsei$H an! GelseH lines rather than @"st the beginning an! en!ing linesB tho"gh the c"rrent $"nctionality is !e$initely a goo! start.

,sing S ort/Circuit $ogical Operators

In logical checks6 please use 4..5 (or 4and< and 4II5 (or 4or5 since these chec5 the secon! expression only when the $irst expression !oes not itsel$ !etermine the res"lt. -or example i$ 1something KK 12 aa 1something_else KK 02 !isp1PZ)here is no nee! to chec5 the secon! expression since Z... Zthe $irst eval"ates to ):=E. )o maximi3e e$$iciencyB list Z... Z$irst the con!ition most li5ely to be ):=E.ZQ2 en! i$ 1something KK 02 44 1something_else KK 12 !isp1PZ)here is no nee! to chec5 the secon! expression sinceZ... Zthe $irst eval"ates to -*8#E. )o maximi3e e$$iciencyB list Z... Z$irst the con!ition most li5ely to be -*8#E.ZQ2 en! ,*)8*B allows "se o$ G4H an! GaBH b"t these are generally expecte! to be less e$$icient than G44H an! GaaH since they chec5 s"bse."ent expressions regar!less o$ the stat"s o$ the $irst.

#((ing 2ew #lgorit ms to 0rap

Titles

*$ter contrib"ting new $"nctionality to the toolboxB please a!! a relevant title to the a"to;titling co!e as $ollowsE 112%aste at the beginning o$ G)itle_Graphs.mH the text below "sing the act"al name o$ a new switchE i$ 'n'$$_?ew#witch else Vi.e. ` 'n'$$_?ew#witch en! 122 +ighlight the existing titling co!eB A0

1/2 102 152 1A2 172 1<2 1>2 wo"l!

%ress (trl O Q to right in!ent itB %ress (trl O S to c"t itB (lic5 a$ter the new GelseH line an! paste the co!e that yo" @"st c"tB (lic5 a$ter the new Gi$H line an! paste the co!e againB ,a5e a note o$ the line the c"rsor is c"rrently onB %ress (trl O +omeB +ighlight the series o$ characters common to each title that yo" li5e to mo!i$yE s"ch as GZ &imensionsRZQBH 1102 %ress (trl O +B 1112 )ype the text with which to replace the common characters N s"ch as GZ &imensionsRZQB Z,y ?ew *lgorithm s Enable!bZBH 1122 %lace the c"rsor over the :eplace b"tton an! tap "ntil reaching the line n"mber note! in step 172 1!onDt worryB yo" can press (trl O I to "n!o i$ yo" happen to go too $arE hope$"llyB ,ath9or5s will intro!"ce the ability to replace all occ"rrences between speci$ie! line n"mbersB b"t $or now the tapping is only a minor inconvenience2B an! 11/2 %ress (trl O # to save the a"to;titling co!e with yo"r new title incl"!e!.

#voi(ing Pre/#llocate( >eros

,*)8*B s"ggests pre;allocating the !imensions o$ vectors an! matrices rather than having them grow within a loop. 9hile this wo"l! be possible when %#' is to be exec"te! $or a speci$ic n"mber o$ iterationsB the exact n"mber o$ iterations is not 5nown in a!vance $or all histories in all cases. *s examplesB 1i2 :eg%#' e$$iciently regro"ps the swarm when stagnation is !etecte! rather than simply exec"ting a pre;speci$ie! n"mber o$ iterationsB an! 1ii2 'pposition;Base! %#' 1'%#'2 ran!omly selects other positions to be eval"ate! so that i$ a maxim"m n"mber o$ $"nction eval"ations is "se! as a termination criterionB there is no !irect translation to the n"mber o$ iterations to be con!"cte!. *!!itionallyB examining the str"ct"res o$ histories at completion to be s"re they are as expecte! is a "se$"l !eb"gging practice that o$$ers extra insight into whether the program has $"nctione! as expecte!. -"rthermoreB pre;allocating a matrix or vector with 3eros co"l! be !angero"s $or an optimi3ation toolbox since 3ero is the optimal $"nction val"e $or many ob@ectivesB an! the n"ll vector is the i!eal !ecision vectorR henceB a b"g in the program co"l! erroneo"sly interpret a pre;allocate! 3ero N not overwritten as expecte! N to mean that per$ect s"ccess was achieve! when it act"ally was not. (onse."entlyB rather than ris5ing the poll"tion o$ !ata with val"es not act"ally generate! by the algorithm N especially 3eros N this toolbox saves !ata to wor5spaces !"ring exec"tion an! reconstr"cts it to generate statistics. 9hen :eg%#' is enable!B this prevents matrix histories $rom becoming excessively long so that growing within a loop is not a ma@or A1

problem. ("rrentlyB the toolbox can a"tomatically save an! reconstr"ct the !ata o$ each trial an! o$ each gro"ping within each trial 1i.e. the latter $or :eg%#' only2. -or vectors whose $inal lengths are not !eterminable in a!vanceB the length co"l! be incremente! by a val"e s"ch as 200 each time an increase! length becomes necessary instea! o$ iteratively increasing the length by 1 each time a new val"e is store!. +oweverB when this approach is compare! to saving !ata per 200 iterations an! reconstr"cting the $"ll vector at the en! o$ the trialB a"tomatic saving an! reconstr"ction is the clear winner since it clears :*, at each chec5point rather than searching $or more contig"o"s :*, by which to increment a vectorDs length per 200 iterations. &"e to the iterative nat"re o$ the %article #warm 'ptimi3ation :esearch )oolboxB which re."ires that matrix an! vector histories be written to rapi!lyB incrementing the length o$ a vector by any $inite si3e when its pre;allocate! length has been reache! is less a!vantageo"s than simply saving the !ata to $ree memory an! reconstr"cting the vector at the en! o$ that trial. )here$oreB to anyone consi!ering improving the ."ality o$ the toolbox via pre;allocation o$ vectors whose lengths are not act"ally 5nowable in a!vanceB please consi!er a"tomatically saving the !ata every x iterations rather than a"tomatically incrementing the vector by si3e x when its pre;allocate! length is reache!. ?ote that G:eg%#'_save_gro"ping_!ata.m an! G:eg%#'_loa!_gro"ping_!ata.mB which are "se! to save an! reconstr"ct !ata per gro"pingB co"l! be improve! to save stan!ar! %#' !ata every 200 iterations or so an! reconstr"ct it at the en! o$ each trial. )his wo"l! $"rther improve the spee! o$ exec"tion $or lengthy trials while stillE 1i2 allowing str"ct"res o$ histories to be examine! $or consistency with other val"es s"ch as the n"mber o$ iterations con!"cte!B an! 1ii2 preventing the potential poll"tion o$ !ata with pre;allocate! val"es in the presence o$ an ina!vertent b"g. Beca"se this improvement wo"l! apply both to algorithms whose primary vectors have pre!ictable lengths as well as to those o$ "npre!ictable lengthsB it is an all;encompassing improvement o$ higher val"e than pre; allocating the lengths o$ vectors having pre!ictable lengths. -or relatively short vectors whose lengths are alwa)s 5nowable in a!vance 1i.e. regar!less o$ the algorithm employe!2B a goo! option is to pre;allocate "sing ?a?Ds 1i.e. Gnot a n"mberH2 since ?a?Ds prevent the possibility o$ !ata becoming poll"te! with val"es not generate! by the toolbox. )his was !one $or iter_success, which meas"res the iteration n"mber at which each trial achieves s"ccess. ts length is exactly e."al to the n"mber o$ trials to be con!"cte! as speci$ie! by the "ser in the control panel via setting num_trials. ?otice that the "se o$ ?a?Ds in iter_success prevents the possibility o$ a miscalc"late! val"e being ret"rne! in iter_success_meanB which ret"rns the average iteration n"mber at which s"ccess was achieve! N i.e. since no 3ero or other n"mber can acci!entally be applie! to the mean in A2

the event that s"ccess is not achieve!. )his re!"ces the probability o$ error an! improves the integrity o$ the toolbox. =sing ?a?Ds $or pre;allocation is sa$er than pre;allocating with n"merical val"es since it ac5nowle!ges the possibility o$ mista5en co!e an! prevents any s"ch mista5en co!e $rom ret"rning incorrect val"esB since !ata cannot become poll"te! with val"es that were not act"ally generate! by the algorithm.

Defining 2ew Switc es = Programming Variables

n a!!ition to clearly commenting new co!eB it is important to list new variables in *ppen!ix * o$ this %ro!"ct &oc"mentation to enable others to easilyE 1i2 e$$iciently re$erence each programming variable to "n!erstan! its $"nctionalityB an! 1ii2 avoi! "sing the same name $or new variables N a mista5e that co"l! ca"se some !ata to be overwritten an! other !ata to be "se! $or "ninten!e! p"rposes.

#pplying C anges T roug out t e Particle Swarm 5esearc Toolbox

$ yo" !eci!e at some point to impro%e the name o( a new programming %ariable or (ile contrib"te!B yo" can highlight the text to be change!B press *lt O TEB U to -in! -ilesB browse to the !irectory to G8oo5 inBH an! clic5 G-in!H. )his will show each containing $ile an! the line n"mber o$ each occ"rrence. Fo" can then !o"ble;clic5 the $irst occ"rrence o$ the variable in each $ileB press (trl O + to replace other occ"rrences within the same $ileB chec5 the Gwhole wor!H box i$ applicableB type the new name o$ the variable or $ile to replace the ol! nameB an! press *lt O : to replace each occ"rrence $o"n! 1or *lt O ? to s5ip an occ"rrence witho"t replacing it2. =sing *lt O : is sa$er than "sing *lt O * to replace all occ"rrences since there o$ten exist occ"rrences that sho"l! not be replace! $or one reason or another. )he $ile is then e$$iciently close! by pressing GEscBH (trl O # to save the changesB an! (trl O 9 to close the $ile. )his can also be "se! to h"nt !own an! !eb"g any mal$"nctioning programming variable or @"st $"lly "n!erstan! how any variable is "se!. *gainB please "se transparent names to help other "sers "n!erstan! the p"rposes o$ $iles an! programming variables. t is o$ten convenient to initially write co!e "sing short names that are easy to type an! replace them with more transparent names once the co!e is complete. 1,athwor5s !oes not c"rrently o$$er the ability to a"tomatically replace whole wor!s within all $iles o$ a !irectory/toolboxB b"t have receive! con$irmation o$ an interest to !evelop it in $"t"re versions.2

&I* Program $low

)his is a basic overview an! may be expan!e! "pon re."est. A/

Open;&iles+m
)his is an optional $ile that opens $iles speci$ie! within it be$ore selecting G(ontrol_%anel.mH as the c"rrent tab. )o "se the $ileB right clic5 G'pen_-iles.mH an! clic5 G:"nBH which opens all $iles except those commente! o"t within the $ile. G'pen_*ll_-iles.mH $"nctions the same way except that no $iles are commente! o"t within it so that it opens all $iles.

Control;Panel+m
*ll program settings are speci$ie! within the control panel except $or the initiali3ation space an! threshol! $or s"ccessB which are speci$ie! in G'b@ectives.mH.

Ob4ectives+m
'nce the switches an! settings o$ G(ontrol_%anel.mH have been loa!e!B the initiali3ation spaceB threshol! $or s"ccessB an! ob@ective name are loa!e! $or the ob$ective_id speci$ie! at the en! o$ the control panel.

)nput;Vali(ation+m
9hen con$licting switches or settings are !etecte!B the %article #warm 'ptimi3ation :esearch )oolbox attempts to in$er the "serDs intentions an! correct con$licts a"tomatically. ?oti$ications o$ corrections ma!e are !isplaye! !"ring the G*"tomatic np"t Vali!ationH phase. 9hen a con$lict is !etecte! b"t the "serDs intention cannot be in$erre! reliablyB the "ser will be noti$ie! o$ the nat"re o$ the con$lict an! how to $ix it rather than being le$t to in$er as m"ch $rom the error message that wo"l! otherwise be generate! by ,*)8*B. )he more $amiliar yo" become with a"tomatic changesB the more convenient they become. -or exampleB when activate :eg%#' via switch OnOff_*egP(OB !onDt bother activating OnOff_<orm*_stag_det since 5now it will be !one $or me a"tomatically.

Display;Settings+m
9hen the a"tomatic inp"t vali!ation phase completesB the most relevant switches an! settings are !isplaye! $or veri$icationB which allows yo" to !etect be$ore exec"tion begins any setting that yo" may have $orgotten to change.

5egPSO;main+m
*$ter veri$ying the correctness o$ !isplaye! settingsB primary $ile G:eg%#'_main.mH exec"tes the $ollowing $iles accor!ing to the switches as they are set a$ter the inp"t vali!ation phase. A0

Trial>Initializations*m
-or each trial con!"cte!B G:eg%#'_main.mH initiali3es the histories an! co"nters pertinent to each trial an! resets the initiali3ation space.

#PSO*m
$ switch OnOff_'P(O is activeB Van !en BerghDs m"lti;start %#' will be exec"te! by G,%#'.mBH which repeate!ly restarts G:eg_,etho!s_0*n!1.mH 1i.e. an! G:eg_,etho!_1.mH i$ OnOff_*egP(O is active2 "ntil termination criterion 'P(O_max_F+sB 'P(O_max_iters, or 'P(O_max_starts is reache!. $ switch OnOff_'P(O is inactiveB G,%#'.mH will not be exec"te!.

Reg>#ethods>J!nd,*m
-ollowing initiali3ationB each trial begins. -or all %#' varietiesB this co!e exec"tes whichever o$ Ggbest_core.mH or Glbest_core.mH below is relevant base! on the stat"s o$ switch OnOff_lbest.
gbest>core*m )his exec"tes Ggbest_initiali3ation.mH to ran!omly initiali3e positions an! velocities within the initiali3ation space an! repeats Ggbest_core_loop.mH "ntil any one o$ the $ollowing termination criteria is satis$ie!E max_F+s_per_grouping i$ OnOff_func_evals is activeB max_iter_per_grouping i$ OnOff_func_evals is inactiveB 'P(O_max_F+s i$ OnOff_func_evals an! OnOff_'P(O are activeB 'P(O_max_iters i$ OnOff_func_evals is inactive an! OnOff_'P(O is activeB thresh_for_succ i$ OnOff_(uccessful=nsuccessful an! OnOff_#erminate_=pon_(uccess are activeB true_global_minimum, in which case contin"e! comp"tation wo"l! be "seless. lbest>core*m )his exec"tes Glbest_initiali3ation.mH to initiali3e the swarm an! then repeats Glbest_core_loop.mH "ntil a termination criterion speci$ie! $or the gro"ping is satis$ie! as liste! "n!er Ggbest_core.mH above.

Reg>#ethod>,*m
9hen :eg%#' is activate! via switch OnOff_*egP(O in s"bsection G%#' *8G': )+, #E8E() '?H o$ the control panelDs $irst sectionB Van !en BerghDs normali3e! swarm ra!i"s termination criterion !etects the onset o$ premat"re convergenceB an! the swarm is regro"pe! to $acilitate contin"e! progress.

A5

&ata is store! to the activate! historiesB an! G:eg_,etho!_1_regro"ping.mH is repeate! "ntil satis$ying one o$ the $ollowing termination criteria speci$ie! $or the trialE max_F+s_over_all_groupings i$ OnOff_func_evals is activeB max_iter_over_all_groupings i$ OnOff_func_evals is inactiveB 'P(O_max_F+s i$ OnOff_func_evals an! OnOff_'P(O are activeB 'P(O_max_iters i$ OnOff_func_evals is inactive an! OnOff_'P(O is activeB thresh_for_succ i$ OnOff_(uccessful=nsuccessful an! OnOff_#erminate_=pon_(uccess are activeB true_global_minimum, in which case contin"e! comp"tation wo"l! be "seless.

Sa%e>data>per>trial*m
9hen switch OnOff_!utosave_"orkspace_Per_#rial is activeB the wor5space is a"tomatically save! at the completion o$ each trial.

3oad>trial>data>(or>stats*m
9hen all trials have complete!B each trialDs wor5space variables are loa!e! an! analy3e! to calc"late statistics an! mean $"nction val"es $or graphing p"rposes.

Standard>Output*m
)he stan!ar! o"tp"ts o$ interest are !isplaye!R these are in a!!ition to any o"tp"ts activate! by speciali3e! switches.

!utosa%e>0orkspace>Per>Column*m
9hen switch OnOff_!utosave_"orkspace_Per_0olumn is activeB the wor5space is save! a$ter each set o$ trials.

raphs*m
9hen switch OnOff_graphs is activeB "ser;speci$ie! graphs are generate!.

&II* Disclaimer
&"e to the magnit"!e o$ the %article #warm 'ptimi3ation :esearch )oolbox an! changes ma!e since generating thesis !ataB it is possible that "n5nown b"gs might exist. ?o g"arantee is ma!e that the co!e is $ree o$ $law. t is yo"r responsibility to vali!ate the acc"racy o$ !ata be$ore p"blishing it or otherwise asserting its acc"racy. ?either ,*)8*B nor any other mathematical tool sho"l! be "se! blin!lyE e.g. 1i2 ,*)8*BDs !e$inition o$ the Gmo!eH $"nction pro!"ces mo!e1P1B 500B 500.0001B 500.5B 501Q2 K 1B which is consi!ere! by some to be a mislea!ing Gmeas"re o$ central ten!encyH that re!e$ines Gmo!eH as the most repeate! AA

or lowest n"mber in a list rather than more stably reporting a ?a? or treating every n"mber as the mo!e since each GrepeatsH exactly 3ero timesB an! 1ii2 ,*)8*BDs Gac5ley$cnH $"nction pro!"ces $"nction val"es that are everywhere !i$$erent than those pro!"ce! by the tra!itional *c5ley $"nction N partly !"e to a closing parenthesis place! in an "northo!ox positionB which again res"lts in a mathematical re!e$inition o$ a common $"nction where the "ser expects no s"ch a re!e$inition. )his is not to somehow !isparage ,*)8*BB which is a won!er$"l toolB b"t to ill"strate clearly that no mathematical tool is $ree o$ $lawB an! that all mathematical tools sho"l! be !istr"ste! to a healthy !egree. #imilarlyB the %article #warm 'ptimi3ation :esearch )oolbox sho"l! not be "se! blin!ly or as a s"bstit"te $or har! wor5R ratherB it is meant to be contrib"te! to an! improve! by other analytical an! har!;wor5ing researchers. )he %article #warm 'ptimi3ation :esearch )oolbox is tho"ght to be a goo! starting point $or yo"r researchB an! it is hope! that yo" will contrib"te to it. George Evers !oes not g"arantee the acc"racy o$ !ata pro!"ce! by the %article #warm 'ptimi3ation :esearch )oolboxB tho"gh it is tho"ght to be acc"rate an! pro!"ces !ata comparable to that $o"n! in other papers when parameters are set i!entically.

&III* !dd8ins
(ontrib"tors to the %#' :esearch )oolbox (omm"nity;Base! &evelopment %ro@ect are motivate! researchers who give bac5 to the pro@ect. << Training !dd8in )his a!!;in by )ricia :ambharose allows ,*)8*BDs ne"ral networ5 toolbox to call the %#' :esearch )oolbox so that any o$ its incl"!e! algorithms can be harnesse! $or training p"rposesE this is m"ch more power$"l than simply "tili3ing basic Gbest %#' $or training p"rposes.

Re(erences
P1Q G. Evers an! ,. Ben GhaliaB G:egro"ping %article #warm 'ptimi3ationE * ?ew Global 'ptimi3ation *lgorithm with mprove! %er$ormance (onsistency *cross 'b@ectivesBH %--> ,+++ ,nternational 0onference on ()stems, 'an, and 0)bernetics, in press. 1p!$2 P2Q G. EversB G*n *"tomatic :egro"ping ,echanism to &eal with #tagnation in %article #warm 'ptimi3ationBH ,.#. thesisB )he =niversity o$ )exas N %an *mericanB E!inb"rgB )SB 200> 1p!$2 P/Q -. van !en BerghB G*n *nalysis o$ %article #warm 'ptimi3ersBH %h& thesisB &epartment o$ (omp"ter #cienceB =niversity o$ %retoriaB %retoriaB #o"th *$ricaB 2002

A7

P0Q S. +" an! :. (. EberhartB Y#olving constraine! nonlinear optimi3ation problems with particle swarm optimi3ationBY in Proceedings of the (ixth "orld 'ulticonference on ()stemics, 0)bernetics and ,nformatics 4(0, %--%7 B 'rlan!oB 2002. P5Q G. Venter an! 6. #obies5iB Y%article swarm optimi3ationBY in Proceedings of the /?rd !,!!@!('+@!(0+@!A(@!(0 (tructures, (tructural &)namics, and 'aterials 0onferenceB &enverB ('B 2002.

!ppendix !? De(initions
t might be help$"l to print the !e$initions o$ wor5space variables $or ."ic5 re$erence while "sing the toolbox.

Of Programming?'or!space Variables
)o re."est the !e$inition o$ a!!itional programming variablesB or i$ any existing !e$inition seems ambig"o"s or lac5ingB please email george PatQ georgeevers.org.

center>IS
-or Gstan!ar!H %#' an! the initial gro"ping o$ :eg%#'B the center o$ the initiali3ation space is center_,(-, which is set in G'b@ectives.mH. Each new regro"ping space is centere! at the global best o$ the previo"s gro"pingB which is accomplishe! via line within G:eg_,etho!_1_regro"ping.mH.
*"nt"r_IS & '

center>ISJ
)his is the initial center o$ the initiali3ation space as set by the "ser in 'b@ectives.m. t may be set as a scalar when all !imensions have the same centerB or as a vector when any !imension has a "ni."e centerR an example o$ the latter is given in section #etting the (enter an! :ange o$ the nitiali3ation #pace o$ chapter G E ,ore &etails on #elect )opicsH.

dim
)his is the !imensionality o$ the problem being solve!. t is static except in the pro$essional version o$ the toolbox when a"to;incrementing ob@ectivesB in which case the !imensionality is allowe! to change with the ob@ective i$ the "ser wo"l! li5e.

(
(ol"mn matrix f is generate! by passing any n"mber o$ rows $rom position matrix x 1i.e. "s"ally the n"mber o$ particlesB np2 into a ob@ective $"nction an! re."esting the $"nction val"e $or those particles. -or exampleB G'b@-"n_*c5ley1xB np2H ret"rns the *c5ley $"nction val"e o$ each particle in the swarm. Each row o$ A<

f hol!s the $"nction val"e o$ the position vector in the correspon!ing row o$ x. )yping G'b@-"n_:astrigin1x1/B E2B 12H wo"l! ret"rn only the $"nction val"e o$ the thir! particle in position matrix x.

(g
#calar wor5space variable fg stores the $"nction val"e o$ the global best an! is "p!ate! any time a better global best is $o"n!. *t the concl"sion o$ the searchB the variable stores the $"nction val"e o$ the best position occ"pie! by any particle at any time !"ring the search.

(g>arra+
)his stores the $"nction val"e o$ each gro"pingDs global best when :egro"ping %#' 1:eg%#'2 is "se! to help the swarm escape $rom the state o$ premat"re convergence to contin"e searching $or better sol"tions. :eg%#' is m"ch more e$$ective than simply restarting the search.

(ghist
)his row vector stores the $"nction val"e o$ the global best thro"gh each iteration $or one gro"ping o$ :eg%#' or $or one trial o$ stan!ar! %#' 1i.e. since stan!ar! %#' !oes not regro"p the swarm an! conse."ently "tili3es only one gro"ping2. )o enable the historyB activate switch OnOff_fghist in the G%#' + #)': E# )' BE ,* ?)* ?E&H s"bsection at the en! o$ the control panelDs $irst section. )o access the best $"nction val"e $o"n! thro"gh iteration 5B $or exampleB type G$ghist152H at the comman! prompt.

(ghist>current>trial
9hen switch OnOff_*egP(O is active in the G%#' *8G': )+, #E8E() '?H s"bsection o$ the control panelDs $irst section an! switch OnOff_fghist is active in the G%#' + #)': E# )' BE ,* ?)* ?E&H s"bsectionB the %article #warm 'ptimi3ation :esearch )oolbox hori3ontally concatenates each gro"pingDs history o$ the $"nction val"e o$ the global bestB fghistB to constr"ct the history over all gro"pings con!"cte! !"ring the :eg%#' trial. )he best $"nction val"e $o"n! thro"gh any iteration can be extracte! $rom fghist_current_trial in the same $ashion as $rom fghist.

(ghist>mean
9hen switch OnOff_fghist is active in the G%#' + #)': E# )' BE ,* ?)* ?E&H s"bsection o$ the control panelDs $irst section an! num_trials is greater than 1 in section G122 %*:) (8E #9*:, *8G': )+, #E)) ?G#HB the %article #warm 'ptimi3ation :esearch )oolbox comp"tes the mean $"nction val"e per iteration across all trials by averaging col"mn;wise matrix fghist_over_all_trials. 9hen switch OnOff_graph_fg_mean is active in section G152 :EG%#' 4 %#' G:*%+ ?G #9 )(+E# *?& #E)) ?G#BH the graph o$ A>

mean $"nction val"e per iteration is constr"cte! $rom row vector fghist_mean.

(ghist>o%er>all>trials
9hen switch OnOff_fghist is active in the G%#' + #)': E# )' BE ,* ?)* ?E&H s"bsection o$ the control panelDs $irst section an! num_trials is greater than 1 in section G122 %*:) (8E #9*:, *8G': )+, #E)) ?G#HB the %article #warm 'ptimi3ation :esearch )oolbox vertically concatenates each trialDs row history o$ the $"nction val"e o$ the global bestB fghist. )he $irst trialDs history is store! to the $irst rowB the secon! trialDs history to the secon! rowB etc. )his allows the constr"ction o$ fghist_meanB $rom which to constr"ct the graph o$ mean $"nction val"e per iteration.

(hist
)his history o$ each iterationDs $"nction val"es is activate! via switch OnOff_fhist at the en! o$ the control panelDs $irst section. ,atrix fhist stores each iterationDs col"mn matrix o$ $"nction val"es $or one gro"ping o$ :eg%#' or $or one trial o$ stan!ar! %#'. )able 0 in the wal5 thro"gh shows how to extract the col"mn matrix o$ any iterationDs $"nction val"es $rom the history.

(hist>current>trial
9hen switch OnOff_*egP(O is active in the G%#' *8G': )+, #E8E() '?H s"bsection o$ the control panelDs $irst section an! switch OnOff_fhist is active in the G%#' + #)': E# )' BE ,* ?)* ?E&H s"bsectionB the %article #warm 'ptimi3ation :esearch )oolbox hori3ontally concatenates each gro"pingDs history o$ $"nction val"es per particleB fhistB to constr"ct the history o$ $"nction val"es over all gro"pings con!"cte! !"ring the :eg%#' trial. )he matrix o$ $"nction val"es correspon!ing to any iteration can be extracte! $rom fhist_current_trial in the same $ashion as $rom fhist in )able 0 o$ section G*naly3ing 9or5space VariablesH in chapter G E * G"i!e! 9al5 )hro"ghH.

(p
(ol"mn vector fp stores the $"nction val"e o$ each particleDs personal best an! is "p!ate! iteratively.

(phist
)his matrix stores the $"nction val"es o$ the personal bests thro"gh each iteration $or one gro"ping o$ :eg%#' or $or one trial o$ stan!ar! %#' 1i.e. since stan!ar! %#' !oes not regro"p the swarm an! conse."ently "tili3es only one gro"ping2. )o enable the historyB activate switch OnOff_fphist in the G%#' + #)': E# )' BE ,* ?)* ?E&H s"bsection at the en! o$ the control panelDs $irst section. )he col"mn vector o$ $"nction val"es o$ the personal bests thro"gh any iteration can be extracte! $rom $phist as !one $or fhist in )able 0 o$ section G*naly3ing 9or5space VariablesH in chapter G E * G"i!e! 9al5 )hro"ghH. 70

(phist>current>trial
9hen switch OnOff_*egP(O is active in the G%#' *8G': )+, #E8E() '?H s"bsection o$ the control panelDs $irst section an! switch OnOff_fphist is active in the G%#' + #)': E# )' BE ,* ?)* ?E&H s"bsectionB the %article #warm 'ptimi3ation :esearch )oolbox hori3ontally concatenates each gro"pingDs history o$ the $"nction val"e o$ the personal bestsB fphistB to constr"ct the history over all gro"pings con!"cte! !"ring the :eg%#' trial. )he col"mn vector o$ $"nction val"es o$ the personal bests thro"gh any iteration can be extracte! $rom $phist_current_trial as !one $or fhist in )able 0 o$ section G*naly3ing 9or5space VariablesH in chapter G E * G"i!e! 9al5 )hro"ghH.

g
)his stores the global best o$ the swarm whenE 1i2 Gbest %#' is exec"te! by !e;activating switch OnOff_lbestB or 1ii2 8best %#' is exec"te! an! switch OnOff_ghist is activate! in the control panel to maintain a history o$ each iterationDs global best $or analysis 1i.e. tho"gh the global best is not "se! in the "p!ate e."ations o$ 8best %#'2. )he global best is store! to each row o$ g to $acilitate matrix s"btraction in the velocity "p!ate e."ation o$ Gbest P(OR g11B E2 can be "se! to access the global best itsel$ witho"t repetition across all np rows o$ the matrix.

ghist
9hen switch OnOff_ghist is activeB the global best is iteratively store! to the row o$ ghist correspon!ing to the iteration n"mber per gro"ping o$ :eg%#' or per trial o$ stan!ar! %#' 1i.e. since stan!ar! %#' "tili3es only one gro"ping2. )he top row stores the global best $rom initiali3ation o$ the swarm at iteration 1B an! the last row stores the global best o$ the $inal iteration o$ the gro"ping.

ghist>current>trial
9hen switch OnOff_*egP(O is active in the G%#' *8G': )+, #E8E() '?H s"bsection o$ the control panelDs $irst section an! switch OnOff_ghist is active in the G%#' + #)': E# )' BE ,* ?)* ?E&H s"bsectionB the %article #warm 'ptimi3ation :esearch )oolbox vertically concatenates each gro"pingDs history o$ global bestsB ghistB to constr"ct the history o$ global bests over all gro"pings con!"cte! !"ring the :eg%#' trial. )he global best o$ any iteration can be extracte! $rom ghist_current_trial as !one with ghist in section G*naly3ing 9or5space VariablesH o$ chapter G E * G"i!e! 9al5 )hro"ghH.

Internal>i
)his is one o$ two co"nters "se! $or vario"s p"rposes at vario"s locations in the sense o$ G$or i K 1E10H. -or exampleB the $ollowing co!e "ses ,nternal_i 71

to graph each particleDs position vers"s iteration n"mber accor!ing to the "serDs pre$erre! graphing styles an! to generate an appropriate legen!.
if OnOff_'ra!3_x for Int"rna$_i & 8Jn! K if #i( && > !$ot(8J8JsiG"(x3ist +)L#i( x3ist(Int"rna$_i 8J#i(J#i(6(M 7 !$ot(8J8JsiG"(x3ist +)L#i( x3ist(Int"rna$_i +J#i(J#i(6(M 7 !$ot(8J8JsiG"(x3ist +)L#i( x3ist(Int"rna$_i NJ#i(J#i(6(M 7 !$ot(8J8JsiG"(x3ist +)L#i( x3ist(Int"rna$_i =J#i(J#i(6(M 7 !$ot(8J8JsiG"(x3ist +)L#i( x3ist(Int"rna$_i 5J#i(J#i(6(M 7 !$ot(8J8JsiG"(x3ist +)L#i( x3ist(Int"rna$_i OJ#i(J#i(6(M 7 !$ot(8J8JsiG"(x3ist +)L#i( x3ist(Int"rna$_i >J#i(J#i(6(M 7 $"'"n#(Ax8A Ax+A AxNA Ax=A Ax5A AxOA Ax>A ACo*ationA "n# K "n# "n#

8)) A9r6A) 8)) A99'6A) 8)) AJ.6A) 8)) A9.*6A) 8)) A9(6A) 8)) A99/6A) 8)) AJM6A) AEastO)tsi#"A)

(o"ntersB graph settingsB switchesB an! other variables that are not generally analy3e! a$ter exec"tion are capitali3e! so that they will be !isplaye! $irst when GwhosH is type! at the comman! prompt. )his $loats them to the top o$ the list an! o$$ o$ the screen so that !ata which is expecte! to be o$ interest once the sim"lation1s2 concl"!e are conveniently gro"pe! together on the screen $or e$$icient analysis. )his topic is !isc"sse! $"rther in section Y-ollowing ?aming (onventions 9hen (reating ?ew VariablesY o$ chapter YV. (ontrib"tion #tan!ar!sY. )o avoi! con$"sionB G,nternal_H is pre$ixe! to $"rther organi3e the wor5space by clearly !elineating variables that only ma5e sense to the program $rom an internal perspective rather than $rom the perspective o$ the "ser a$ter exec"tion.

Internal>=
)ogether with ,nternal_iB this is one o$ two co"nters "se! $or vario"s p"rposes at vario"s locations in the sense o$ G$or @ K 1E10H. -or exampleB the $ollowing co!e o$ the J"a!ric benchmar5 "ses it to increment the problem !imensionality as !imensions are s"mme! across.
for Int"rna$_P & 8J#i( f & s)((!osition_(atrix(8Jn)(_!arti*$"s_+_",a$)at" 8JInt"rna$_P) "n# +).Q+ 7 f<

iter>mean>per>trial
)he mean n"mber o$ iterations con!"cte! per trial is store! to iter_mean_per_trial. #ome trials might re."ire less iterations than others i$ switch OnOff_#erminate_=pon_(uccess is activeB in which case the swarm will stop searching $or better sol"tions once the acc"racy speci$ie! in thresh_for_succ has been satis$ie!. 72

iter>success
)his stores the iteration n"mber at which each trial reaches s"ccess. 9hen m"ltiple trials are con!"cte!B it is a vector whose length e."als the n"mber o$ trials con!"cte!. 9hen iter_success is !isplaye! as ?a?B the correspon!ing trial $aile! to minimi3e the $"nction val"e to the "ser;speci$ie! threshol! $or s"ccessB thresh_for_succ.

iter>success>mean
)his stores the average iteration n"mber at which the threshol! $or s"ccess was reache!. t is calc"late! by s"mming the real parts o$ iter_success an! !ivi!ing by the n"mber o$ s"ccess$"l trials. =ns"ccess$"l trials are not consi!ere! in this comp"tation so that iter_success_mean is the mean n"mber o$ iterations re."ire! $or s"ccess$"l trials to re!"ce the $"nction val"e to thresh_for_succ or better.

iter>tot>all>trials
)his co"nts the n"mber o$ iterations con!"cte! over all trials to comp"te the mean n"mber o$ iterations con!"cte! per trialB iter_mean_per_trial.

k
)his co"nts the n"mber o$ "p!ates that have occ"rre!. nitiali3ation o$ the swarm occ"rs at k K 0 since no velocity or position "p!ate has yet occ"rre!. )he $irst swarm movement ta5es each particle $rom its position at k K 0 to its position at k K 1B which can be seen by examining the phase plots that are generate! when switch OnOff_phase_plot is active in the control panel. 1:et"rn to wal5 thro"gh2

l
)his is the matrix o$ local bestsB which is "p!ate! iteratively when 8best %#' is activate! via switch OnOff_lbest in the control panelDs $irst section. *t the en! o$ each trialB it contains the best position $o"n! by each particle over the co"rse o$ the search.

lbest>neighb>size
)his is the neighborhoo! si3e as set in the control panelDs $irst section "n!er s"bsection G%#' *8G': )+, #E8E() '?H. t is only create! in the wor5space when parent switch OnOff_lbest is active.

lhist
)his matrix history o$ each iterationDs local bests is activate! via switch OnOff_lhist at the en! o$ the control panelDs $irst section. )able 5 in the wal5 thro"gh shows how to extract any iterationDs matrix $rom this history per gro"ping o$ :eg%#' or per trial o$ stan!ar! %#' 1i.e. since stan!ar! %#' "tili3es only one gro"ping2. 7/

lhist>current>trial
9hen switch OnOff_*egP(O is active in the G%#' *8G': )+, #E8E() '?H s"bsection o$ the control panelDs $irst section an! switch OnOff_lhist is active in the G%#' + #)': E# )' BE ,* ?)* ?E&H s"bsectionB the %article #warm 'ptimi3ation :esearch )oolbox hori3ontally concatenates each gro"pingDs history o$ local or neighborhoo! bestsB lhistB to constr"ct the history o$ local bests over all gro"pings con!"cte! !"ring the :eg%#' trial. )he local bests o$ any iteration can be extracte! $rom lhist_current_trial as !one with lhist in )able / o$ section G*naly3ing 9or5space VariablesH o$ chapter G E * G"i!e! 9al5 )hro"ghH.

max>iter>o%er>all>groupings
)his is the maxim"m n"mber o$ iterations allowe! in one :eg%#' trial. #ee max_iter_per_grouping $or more in$ormation.

max>iter>per>grouping
9hen regro"ping is activate! via switch OnOff_*egP(OB the maxim"m n"mber o$ iterations allotte! per gro"ping is speci$ie! via setting max_iter_per_grouping. $ it is set e."al to max_iter_over_all_groupingsB regro"ping will only be triggere! when stagnation is !etecte! accor!ing to $orm"las 0.1 ; 0.0 o$ thesis P2Q. )he "ser also has the option o$ setting max_iter_per_grouping less than max_iter_over_all_groupings to prompt swarm regro"ping even i$ a stray particle is still wan!ering abo"t the search space a$ter a consi!erable n"mber o$ iterations. )his seems to improve general per$ormance an! is a5in to a gro"p o$ in!ivi!"als ma5ing a !ecision rather than being paraly3e! by in!ecision sho"l! one in!ivi!"al still !issent a$ter a consi!erable amo"nt o$ time. 9hen Gstan!ar!H %#' is "se! witho"t regro"pingB max_iter_per_grouping is the maxim"m n"mber o$ iterations to be con!"cte! by the trial.

max>$9s>o%er>all>groupings
9hen switch OnOff_func_evals is activeB a maxim"m n"mber o$ $"nction eval"ations is monitore! instea! o$ a maxim"m n"mber o$ iterations. )his setting speci$ies the maxim"m n"mber o$ $"nction eval"ations allowe! in one :eg%#' trial. #ee max_F+s_per_grouping $or more in$ormation.

max>$9s>per>grouping
9hen switch OnOff_func_evals is activeB a maxim"m n"mber o$ $"nction eval"ations is monitore! instea! o$ a maxim"m n"mber o$ iterations. 9hen regro"ping is activate! via switch OnOff_*egP(OB the maxim"m n"mber o$ $"nction eval"ations allotte! per gro"ping is speci$ie! via setting max_F+s_per_grouping. $ it is set e."al to max_F+s_over_all_groupingsB regro"ping will only be triggere! when stagnation is !etecte! accor!ing to $orm"las 0.1 ; 0.0 o$ thesis P2Q. )he "ser also has the option o$ setting 70

max_F+s_per_grouping less than max_ F+s_over_all_groupings to prompt swarm regro"ping even i$ a stray particle is still wan!ering abo"t the search space a$ter a consi!erable n"mber o$ $"nction eval"ations. )his seems to improve general per$ormance an! is a5in to a gro"p o$ in!ivi!"als ma5ing a !ecision rather than being paraly3e! by in!ecision sho"l! one in!ivi!"al still !issent a$ter a consi!erable amo"nt o$ time. 9hen Gstan!ar!H %#' is "se! witho"t regro"pingB max_F+s_per_grouping is the maxim"m n"mber o$ $"nction eval"ations to be con!"cte! by the trial.

#PSO>$9>counter
*t the en! o$ each ,%#' startB 'P(O_F+_counter a!!s the amo"nt o$ $"nction eval"ations !one in that start to its previo"s val"eB which was initiali3e! to 3ero in G)rial_ nitiali3ations.mH. 'P(O_F+_counter is "se! by the relevant while hea!ers within G:eg_,etho!_1.mH an! either Ggbest_core.mH or Glbest_core.mH to ens"re that the maxim"m n"mber o$ $"nction eval"ationsB 'P(O_max_F+sB speci$ie! in the control panel is not s"rpasse!E since the val"e o$ 'P(O_F+_counter is not "p!ate! "ntil the en! o$ each startB while hea!ers a!! the n"mber o$ $"nction eval"ations !one within the c"rrent start to 'P(O_F+_counter to !etermine the total n"mber o$ $"nction eval"ations !one over all starts.

#PSO>k
)his is 0 when the swarm is initiali3e! an! increments with each ens"ing iteration o$ ,%#' when switch OnOff_'P(O is active in the control panelZs $irst section.

np
)his stores the n"mber o$ particles or swarm si3e. t is set in section G122 %*:) (8E #9*:, *8G': )+, #E)) ?G#H o$ the control panel. t is static except in the pro$essional version o$ the toolbox when a"to;incrementing the swarm si3e to $in! high;."ality parameter combinations as !one $or (hapter o$ thesis P2Q.

num>particles>->e%aluate
9hen calling an ob@ective $"nctionB this is the internal variable that speci$ies the n"mber o$ positions to be eval"ate!. t will typically be either 1 or np 1i.e. the n"mber o$ particles in the swarmB which is the same as the n"mber o$ rows in position matrix x2 an! "s"ally ass"mes the latter val"e to eval"ate each position in the swarm.

num>trials>success(ul
9hen m"ltiple trials are con!"cte!B this co"nts the n"mber o$ trials $or which a $"nction val"e less than or e."al to thresh_for_succ is pro!"ce!. 75

OnO((>as+mmetric>initialization
)his switch activates asymmetric initiali3ationB which mo!i$ies the initiali3ation space by setting xmin K xmax/2 on benchmar5s instea! o$ xmin K ;xmax. )he i!ea is to increase the !i$$ic"lty o$ locating the global minimi3er. )his tests the e$$ectiveness o$ the optimi3ation algorithm at $in!ing sol"tions that might ina!vertently be excl"!e! $rom the initiali3ation space sho"l! the algorithm be applie! to problems whose sol"tions are "n5nown.

OnO((>(unc>e%als
9hen switch OnOff_func_evals is active in the G%#' &*)* )' BE :E)* ?E& 4 #*VE&H s"bsection o$ the control panelDs section G112 B*# ( #9 )(+E# 4 %#' *8G': )+, #E8E() '?HB $"nction eval"ations will be "se! as a termination criterionR when inactiveB iterations will be monitore! as a termination criterion instea!. )he maxim"m n"mber o$ $"nction eval"ations or iterations per gro"ping is set at the en! o$ section G122 %*:) (8E #9*:, *8G': )+, #E)) ?G#H via settings max_F+s_per_grouping or max_iter_per_grouping respectivelyR these speci$y the maxim"m n"mber allowe! per :eg%#' gro"ping or per stan!ar! %#' trial 1i.e. since a stan!ar! trial consists o$ only one gro"ping2. )he maxim"m n"mber o$ $"nction eval"ations or iterations over all gro"pings within a :eg%#' trial is set at the en! o$ section G1/2 :EG%#' *8G': )+, #9 )(+E# 4 #E)) ?G#H via settings max_F+s_over_all_groupings or max_iter_over_all_groupings respectively.

OnO((><ormR>stag>det
Van !en BerghZs normali3e! swarm ra!i"s criterion terminates the search when particles are all near the global bestB which is a goo! in!ication o$ stagnation P/Q. *ccor!ing to ."ic5 empirical testing !one !"ring thesisB monitoring particlesZ proximities to !etect stagnation seems more e$$ective than monitoring the n"mber o$ iterations $or which the $"nction val"e $ails to improveB which stan!s to reason since monitoring the so"rce o$ a problem is more e$$icient than waiting $or an "n!esirable si!e e$$ect to occ"r ; the logic behin! preventive me!icine an! $irewalls. #witch OnOff_<orm*_stag_det can be activate! in s"bsection Y)E:, ?*) '? (: )E: * -': :EG%#' 4 %#' *8G': )+,#Y in the control panelZs $irst section. :eg%#' regro"ps the swarm abo"t the global best when stagnation is !etecte! instea! o$ terminating the searchB which allows the $"nction val"e to improve over time. $ switch OnOff_*egP(O is activate! in con@"nction with switch OnOff_<orm*_stag_detB the swarm will regro"p within a regro"ping space !e$ine! accor!ing to the !egree o$ "ncertainty in$erre! per !imension $rom the !istrib"tion o$ particles when stagnation was !etecte! P1Q.

7A

OnO((>position>clamping
)his prevents particles $rom leaving the search space by clamping their positions to P;xmaxB xmaxQ the same way velocity clamping restricts velocities to P;vmaxB vmaxQ.

OnO((>RegPSO
9hen stagnation is !etecte!B :eg%#' regro"ps the swarm abo"t the global best $or contin"e! progress instea! o$ terminating the search or wasting comp"tational timeR this allows the $"nction val"e to improve over time. :eg%#' can be activate! via switch OnOff_*egP(O in s"bsection Y%#' *lgorithm #electionY o$ the control panelZs $irst section. *n example o$ the e$$ectiveness o$ :eg%#' is given at the en! o$ (hapter Ds wal5 thro"gh.

OnO((>Success(ulUnsuccess(ul
9hen switch OnOff_(uccessful=nsuccessful is activate! in s"bsection Y:EG%#' 4 %#' #=((E## ,E*#=:E#Y o$ the control panelZs $irst sectionB each trial will be classi$ie! as s"ccess$"l or "ns"ccess$"l. n some casesB "sers may only be intereste! in the statistics res"lting $rom the trials con!"cte! an! may not be intereste! in classi$ying trials as s"ccess$"l or "ns"ccess$"lR b"t the $eat"re can be a "se$"l way o$ trac5ing o"tliers since any $"nction val"e larger than the threshol! $or s"ccess as set in Y'b@ectives.mY will be classi$ie! as "ns"ccess$"lB thereby essentially co"nting the n"mber o$ o"tliers.

OnO((>Terminate>Upon>Success
n s"bsection Y:EG%#' 4 %#' #=((E## ,E*#=:E#Y o$ the control panelZs $irst sectionB i$ switch OnOff_#erminate_=pon_(uccess is activate! along with OnOff_(uccessful=nsuccessfulB trials will terminate early "pon minimi3ing the $"nction val"e to thresh_for_succB the threshol! re."ire! $or a trial to be consi!ere! s"ccess$"l.

OnO((>%>reset
)his sets to 3ero any velocity component responsible $or p"shing a particle o"tsi!e the search space an! recalc"lates the position so that particles remain within the search space. )o my 5nowle!geB the $irst wor5 on this topic was !one by Gerhar! Venter P5Q.

OPSO>ghost>$9s>per>grouping
)his co"nts in Ggbest_initiali3ation.mH an! Ggbest_core_loop.m the a!!itional $"nction eval"ations !one by 'pposition;Base! %#' 1'%#'2 an! the (a"chy m"tation o$ global best. '%#' an! the (a"chy m"tation can be "se! separatelyB an! the co"nt will still be correct. t also co"nts in Glbest_initiali3ation.m an! Glbest_core.m the a!!itional $"nction eval"ations !one by '%#' witho"t (a"chy m"tation o$ global best since 8best %#' !oes not ma5e "se o$ the global best. 77

t is initiali3e! to 3ero within Ggbest_initiali3ation.m or Glbest_initiali3ation.mBH !epen!ing on the core algorithm selecte!.

OPSO>ghost>$9s>RegPSO
OP(O_ghost_F+s_*egP(O a!!s over all gro"pings across a :eg%#' trial the val"es ret"rne! by OP(O_ghost_F+s_per_grouping a$ter each gro"ping. *$ter all :eg%#' gro"pingsB it contains the n"mber o$ a!!itional $"nction eval"ations con!"cte! by '%#' an! the (a"chy m"tation o$ global bestB which also wor5s i$ only one o$ '%#' or the (a"chy m"tation is enable!. t is initiali3e! within G:eg_,etho!s_0*n!1.mH as GOP(O_ghost_F+s_*egP(O K OP(O_ghost_F+s_per_groupingH an! accr"es within G:eg_,etho!_1.mH as GOP(O_ghost_F+s_*egP(O K OP(O_ghost_F+s_*egP(O O OP(O_ghost_F+s_per_groupingH. t is then "se! within the relevant while hea!er o$ Ggbest_core.m or Glbest_core.m to ens"re that the maxim"m n"mber o$ $"nction eval"ations allotte! over all gro"pings o$ a :eg%#' trial is not s"rpasse!E G1k_*egP(O O k O 12Xnp O OP(O_ghost_F+s_*egP(O2 ] max_F+s_over_all_groupingsH.

p
)his is the matrix o$ personal bestsB which is "p!ate! iteratively. *t the en! o$ a trialB it contains the best position $o"n! by each particle over the co"rse o$ the search.

phist
)his history o$ each iterationDs matrix o$ personal bests is activate! via switch OnOff_phist at the en! o$ the control panelDs $irst section. )able / in the g"i!e! wal5 thro"gh shows how to extract the matrix o$ any iteration $rom this history per gro"ping o$ :eg%#' or per trial o$ stan!ar! %#' 1i.e. since stan!ar! %#' "tili3es only one gro"ping2.

phist>current>trial
9hen switch OnOff_*egP(O is active in the G%#' *8G': )+, #E8E() '?H s"bsection o$ the control panelDs $irst section an! switch OnOff_phist is active in the G%#' + #)': E# )' BE ,* ?)* ?E&H s"bsectionB the %article #warm 'ptimi3ation :esearch )oolbox hori3ontally concatenates each gro"pingDs history o$ personal bestsB phistB to constr"ct the history o$ personal bests over all gro"pings con!"cte! !"ring the :eg%#' trial. )he personal bests o$ any iteration can be extracte! $rom phist_current_trial as !one with phist in )able / o$ section G*naly3ing 9or5space VariablesH o$ chapter G E * G"i!e! 9al5 )hro"ghH.

7<

position>matrix
9hen calling an ob@ective $"nctionB this internal variable stores the position or positions passe! in $or eval"ation. t "s"ally ass"mes the $"ll matrixB x, to eval"ate each position in the swarmR howeverB any single position or s"bset o$ x may be passe! in as well 1e.g. a pert"rbation o$ the global best2.

r,
)his is the np by dim matrix o$ pse"!o;ran!om n"mbers "se! in the cognitive term o$ the velocity "p!ate e."ation shown below. t is m"ltiplie! element;wise by the !i$$erence between the matrix o$ personal bestsB pB an! the position matrixB x. #atricized &elocit+ Update 9Fuation v ( k + 1) = v ( k ) + c1r1 ( k ) ( p ( k ) x ( k ) ) + c2r2 ( k ) ( g ( k ) x ( k ) )

r)his is the np by dim matrix o$ pse"!o;ran!om n"mbers "se! in the social term o$ the velocity "p!ate e."ation shown above. t is m"ltiplie! element; wise by the !i$$erence between the matrix o$ personal bestsB pB an! the position matrixB x.

Rand>seF>start>point
)his controls the state o$ the ran!omi3er at which each trial begins toE 1i2 ens"re $"t"re repro!"cibility o$ !ataB an! 1ii2 "se a "ni."e se."ence o$ ran!om n"mbers $or each trial a"tomatically con!"cte! by the toolbox.

range>IS
9hen each trial is initiali3e! 1i.e. within G)rial_ nitiali3ations.mH2B the "ser; speci$ie! scalar or vector range_,(- is replicate! across each row o$ range_,( to accommo!ate simple matrix s"btractionE this conversion is !one internally to enable the "ser to speci$y the range in the simplest terms possible. -or each regro"ping o$ :eg%#'B range_,( is calc"late! per !imension proportionally to the !egree o$ "ncertainty in$erre! $rom the swarm state when premat"re convergence is !etecte!. )he range o$ each new regro"ping space is per !imension the minim"m o$E 1i2 the original range o$ the initiali3ation space an! 1ii2 the pro!"ct o$ the regro"ping $actor an! the maxim"m !eviation o$ any particle $rom global best along that !imension. )he latter is accomplishe! via within G:eg_,etho!_1_regro"ping.mH.
ran'"_IS & r"!(at((in(r"'_fa*t6(ax(a.s(x 9 ')) ran'"_IS:) n! 8)

7>

range>ISJ
)his is the initial range o$ the initiali3ation space as set by the "ser in 'b@ectives.m. t may be set as a scalar when all !imensions have the same range or as a vector when any !imension has a "ni."e rangeR an example o$ the latter is given "n!er section #etting the (enter an! :ange o$ the nitiali3ation #pace o$ chapter G E ,ore &etails on #elect )opicsH. =nli5e variable range_,(B which is "p!ate! with each regro"pingB range_,(- is "nmo!i$ie! !"ring program exec"tion.

reg>(act
$ switch OnOff_*egP(O is activeB the swarm will regro"p within a regrouping space !e$ine! accor!ing to the degree o( uncertaint+ in(erred per dimension $rom the !istrib"tion o$ particles when stagnation is !etecte! P2Q. )he regrouping (actor is m"ltiplie! by each !imensionDs uncertaint+ to !etermine the regrouping spaceB while clamping each dimensionEs range to its initial %alueB as shown in $orm"la 0.A o$ thesisB which is accomplishe! via the $ollowing co!e within G:eg_,etho!_1_regro"ping.mH.
range_IS & r"!(at(min(reg_fact6max(abs(x - g)), range_IS0) n! 8)

'nce the range o$ the regro"ping space has been !e$ine! as shown aboveB particles are regro"pe! abo"t the global best accor!ing to $orm"la 0.7 o$ thesis. )he velocity clamping vector is then calc"late! accor!ing to $orm"la 0.10 in proportion to the range per !imension o$ the new regro"ping spaceB an! that maxim"m velocity vector is "se! to re;initiali3e velocities as shown in $orm"la 0.11. %ersonal bests are then re;initiali3e! as shown in $orm"la 0.12B b"t the global best is remembere! to be improve! "pon. )he search then contin"es as "s"al "ntil stagnation is !etecte! again.

RegPSO>grouping>counter
9hen :eg%#' is activate! via switch OnOff_*egP(O in the control panelB *egP(O_grouping_counter is initiali3e! to 1 within G:eg_,etho!s_041.mH an! incremente! by 1 with each regro"ping per$orme! by G:eg_,etho!_1_regro"ping.mH.

RegPSO>k
$ :eg%#' is activate! via switch OnOff_*egP(O in the control panelDs $irst section to liberate the swarm $rom the state o$ premat"re convergence when it is !etecte!B *egP(O_k is initiali3e! to 0 !"ring swarm initiali3ation an! incremente! with each ens"ing iteration. ts val"e at the en! o$ each trial is 1 less than the n"mber o$ iterations con!"cte! since initiali3ation o$ the swarm co"nts as an iteration b"t not as an "p!ate.

<0

*egP(O_k may be less $or one trial than $or others i$ that trial terminates early as a res"lt o$ the tr"e global minim"m being achieve! witho"t !iscernable error 1i.e. f < 10323 cannot be !i$$erentiate! $rom tr"e 3ero by ,*)8*B2.

RegPSO>k>a(ter>each>grouping
9hen only one :eg%#' trial is con!"cte!B this is a vector storing the iteration n"mbers at which premat"re convergence was !etecte!R a!!ing 1 to each col"mn wo"l! pro!"ce a vector storing the iteration n"mbers at which regro"ping occ"rre! since initiali3ation o$ the swarm co"nts as an iteration b"t not as an "p!ate. 9hen m"ltiple trials are con!"cte!B this is a matrix storing each trialDs vector in chronological or!er.

stag>thresh
*ctivating switch OnOff_<orm*_stag_det ca"ses the search to terminate when the normali3e! swarm ra!i"s becomes smaller than stag_thresh, which is a goo! in!ication that the swarm has stagnate! P/Q. $ switch OnOff_*egP(O is active conc"rrentlyB the swarm will regro"p within a regro"ping space !e$ine! accor!ing to the !egree o$ "ncertainty in$erre! per !imension $rom the !istrib"tion o$ particles when stagnation is !etecte! P2Q. #tagnation is !etecte! accor!ing to $orm"las 0.1 N 0.0 o$ thesisB which are implemente! within Ygbest_core.mY an! Ylbest_core.mY via termination criterion Ymax1s.rt1s"m11x ; g2.\2B 2222/s.rt1s"m11range_ #11B E22.\222 L stag_threshY.

thresh>(or>succ
9hen the best $"nction val"e pro!"ce! by a trial is less than or e."al to the threshol! $or s"ccessB thresh_for_succB the trial will be consi!ere! s"ccess$"l. )he val"e o$ thresh_for_succ is set within G'b@ectives.mH an! sho"l! be appropriate to both the ob@ective an! its problem !imensionality. 9hen switches OnOff_(uccessful=nsuccessful an! OnOff_#erminate_=pon_(uccess are activeB each trial will terminate "pon reaching the threshol! $or s"ccess instea! o$ attempting $"rther optimi3ation. )he threshol! !etermines the s"b@ective val"e at which a trial will be consi!ere! s"ccess$"l. t can be "se$"l in con@"nction with statistics since it shows the n"mber o$ trials per$orming worse than the $itness val"e o$ interestB which gives an i!ea o$ how s"ch trials a$$ect the mean an! me!ian $itness val"esB th"s complementing the stan!ar! !eviation.

<1

true>global>minimum
)he tr"e global minim"m is speci$ie! in section G1A2 'B6E() VE#H o$ the control panel so that any trial minimi3ing the $"nction val"e to the lowest val"e possible will terminate rather than spen!ing excess time in comp"tation. 'nce the n"mber o$ !ecimal places carrie! internally by ,*)8*B all agree entirely with true_global_minimumB no $"rther optimi3ation is possible. '$ co"rseB when many !ecision variables map thro"gh a complex $orm"la to a $itness val"eB the !ecision variables will not be optimi3e! to the same n"mber o$ !ecimal places as the $itness val"e itsel$.

%
Each row o$ velocity matrix v stores a particleDs velocity. -or exampleB Gv11B E2H hol!s the velocity vector o$ the $irst particleB Gv12B E2H hol!s the velocity vector o$ the secon! particleB an! so $orth. ,atrix v is "p!ate! iteratively accor!ing to the velocity "p!ate e."ation o$ %#'R so at the completion o$ a trialB it stores the $inal resting place o$ each particle. ,atrix v is initiali3e! within Ggbest_iteration_0.mH or Glbest_initiali3ation.mBH !epen!ing on whether Gbest or 8best %#' is selecte! accor!ing to switch OnOff_lbest. t is "p!ate! in Ggbest_core_loop.mH or Glbest_core_loop.mH.

%hist
)his history o$ all iterationsD velocity matrices is activate! via switch OnOff_vhist at the en! o$ the control panelDs $irst section. )able 2 in the wal5 thro"gh shows how to extract the position matrix o$ any iteration $rom this history per gro"ping o$ :eg%#' or per trial o$ stan!ar! %#' 1i.e. since stan!ar! %#' "tili3es only one gro"ping2.

%hist>current>trial
9hen switch OnOff_*egP(O is active in the G%#' *8G': )+, #E8E() '?H s"bsection o$ the control panelDs $irst section an! switch OnOff_vhist is active in the G%#' + #)': E# )' BE ,* ?)* ?E&H s"bsectionB the %article #warm 'ptimi3ation :esearch )oolbox hori3ontally concatenates each gro"pingDs history o$ velocity vectorsB vhistB to constr"ct the history o$ velocity vectors over all gro"pings con!"cte! !"ring the :eg%#' trial. )he velocity vectors o$ any iteration can be extracte! $rom vhist_current_trial as !one with vhist in section G*naly3ing 9or5space VariablesH o$ chapter G E * G"i!e! 9al5 )hro"ghH.

%max
)his is the matrix o$ maxim"m velocities "se! $or velocity clamping. t is !e$ine! as
,(ax_!"r*.6ran'"_IS

within G:eg_,etho!s_0*n!1.mH an! G:eg_,etho!_1_regro"ping.mH. $ the same range is "se! on all !imensionsB vmax will be the same on all <2

!imensions. B"t i$ the range on one !imension is larger than the range on another by any scale $actorB its maxim"m velocity will be larger by the same scale $actor so that the same $raction o$ the search space can be combe! per !imension per iteration. 9hen :eg%#' is "tili3e!B each !imension o$ the range o$ the regro"ping space is !e$ine! proportionally to the !egree o$ "ncertainty on that !imension at stagnation. #ince the range per !imension variesB so !oes the maxim"m velocity per !imension.

x
Each row o$ position matrix x stores a particleDs position. -or exampleB Gx11B E2H hol!s the position vector o$ the $irst particleB Gx12B E2H hol!s the position vector o$ the secon! particleB an! so $orth. ,atrix x is "p!ate! iteratively accor!ing to the position "p!ate e."ation o$ %#'R so at the completion o$ a trialB it stores the $inal resting place o$ each particle. ,atrix x is initiali3e! within G:eg_,etho!s_0*n!1.mH an! "p!ate! in Ggbest_core_loop.mH or Glbest_core_loop.mBH !epen!ing on whether Gbest or 8best %#' is selecte! accor!ing to switch OnOff_lbest.

xhist
)his history o$ each iterationDs position matrix is activate! via switch OnOff_xhist at the en! o$ the control panelDs $irst section. )able 1 in the wal5 thro"gh shows how to extract the position matrix o$ any iteration $rom this history per gro"ping o$ :eg%#' or per trial o$ stan!ar! %#' 1i.e. since stan!ar! %#' "tili3es only one gro"ping2.

xhist>current>trial
9hen switch OnOff_*egP(O is active in the G%#' *8G': )+, #E8E() '?H s"bsection o$ the control panelDs $irst section an! switch OnOff_xhist is active in the G%#' + #)': E# )' BE ,* ?)* ?E&H s"bsectionB the %article #warm 'ptimi3ation :esearch )oolbox hori3ontally concatenates each gro"pingDs history o$ position vectorsB xhistB to constr"ct the history o$ position vectors over all gro"pings con!"cte! !"ring the :eg%#' trial. )he position vectors o$ any iteration can be extracte! $rom xhist_current_trial as !one with xhist in section G*naly3ing 9or5space VariablesH o$ chapter G E * G"i!e! 9al5 )hro"ghH.

xmax
9hen velocity reset or position clamping is activate! to prevent particles $rom leaving the search spaceB xmax hol!s the maxim"m val"es that particles are not allowe! to excee!. 9hen each !imension o$ the search space has the same range an! centerB as is the case $or pop"lar benchmar5s an! many real;worl! application problemsB xmax is scalar an! all !imensions are clampe! to lie within PxminB xmaxQ. -or application problems having a !i$$erent range o$ val"es on some !imensions than on othersB s"ch as $or the </

example problem in section G#etting the (enter an! :ange o$ the nitiali3ation #paceBH xmax is a vector storing the maxim"m val"e allowe! per !imension.

xmin
9hen velocity reset or position clamping is activate! to prevent particles $rom leaving the search spaceB xmin hol!s the minim"m val"es that particles are not allowe! to excee!. 9hen each !imension o$ the search space has the same range an! centerB as is the case $or pop"lar benchmar5s an! many real;worl! application problemsB xmin is scalar an! all !imensions are clampe! to lie within PxminB xmaxQ. -or application problems having a !i$$erent range o$ val"es on some !imensions than on othersB s"ch as $or the example problem in section G#etting the (enter an! :ange o$ the nitiali3ation #paceBH xmin is a vector storing the minim"m val"e allowe! per !imension.

Of &unctions
)his section is a wor5 in progress. $ yo" have any ."estions or recommen!ationsB please email george PatQ georgeevers.orgB where yo" may also re."est to be noti$ie! o$ $"t"re "p!ates.

Ob=$un>!ckle+
)his eval"ates the position1s2 passe! into the *c5ley $"nction. )he $irst inp"t arg"ment is the position vector or matrix to be eval"ate!. :ather than iteratively re;calc"lating the n"mber o$ particles $rom the !imensionality o$ the position matrix each time the $"nction is calle!B this in$ormation is simply passe! into the $"nction via the secon! inp"t arg"ment. )he o"tp"t is the col"mn matrix o$ $"nction val"es correspon!ing to each particle/row eval"ate!. )he ob@ective $"nction is !e$ine! as $ollows.

0.2 r f ( x = 20 + e 20e

j =1

x2 j

cos(2 x j e
j =1

32 x j 32
Ob=$un> riewangk
)his eval"ates the position1s2 passe! into the Griewang5 $"nction. )he $irst inp"t arg"ment is the position vector or matrix to be eval"ate!. :ather than iteratively re;calc"lating the n"mber o$ particles $rom the !imensionality o$ <0

the position matrixB this in$ormation is simply passe! into the $"nction via the secon! inp"t arg"ment. )he o"tp"t is the col"mn matrix o$ $"nction val"es correspon!ing to each particle/row eval"ate!. )he $"nction to be minimi3e! is shown below.
n n xj x2 r j f (x = 1+ cos 4000 j j =1 j =1 600 x j 600

Ob=$un>Kuadric
)his eval"ates the position1s2 passe! into the J"a!ric $"nction. )he $irst inp"t arg"ment is the position vector or matrix to be eval"ate!. :ather than iteratively re;calc"lating the n"mber o$ particles $rom the !imensionality o$ the position matrixB this in$ormation is simply passe! into the $"nction via the secon! inp"t arg"ment. )he o"tp"t is the col"mn matrix o$ $"nction val"es correspon!ing to each particle/row eval"ate!. Each particleDs position vector is iteratively mappe! to a scalar $"nction val"e as $ollows. 2 j n

r f (x = x j j =1 k =1 100 x j 100

Ob=$un>Kuartic><ois+
)his eval"ates the position1s2 passe! into the noisy J"artic $"nction. )he $irst inp"t arg"ment is the position vector or matrix to be eval"ate!. :ather than iteratively re;calc"lating the n"mber o$ particles $rom the !imensionality o$ the position matrixB this in$ormation is simply passe! into the $"nction via the secon! inp"t arg"ment. )he o"tp"t is the col"mn matrix o$ $"nction val"es correspon!ing to each particle/row eval"ate!. Both in!ivi!"ally an! collectivelyB particles attempt to minimi3e the $orm"la below.

<5

n r f ( x = random[0,1 + i xi4 i =1

1.28 x j 1.28
Ob=$un>Rastrigin
)his eval"ates the position1s2 passe! into the :astrigin $"nction. )he $irst inp"t arg"ment is the position vector or matrix to be eval"ate!. :ather than iteratively re;calc"lating the n"mber o$ particles $rom the !imensionality o$ the position matrixB this in$ormation is simply passe! into the $"nction via the secon! inp"t arg"ment. )he o"tp"t is the col"mn matrix o$ $"nction val"es correspon!ing to each particle/row eval"ate!. #hown below is the $orm"la by which each particleDs $"nction val"e is !etermine!.
n r f ( x = 10n + ( x 2 j 10 cos(2 x j j =1

5.12 x j 5.12
Ob=$un>Rosenbrock
)his eval"ates the position1s2 passe! into the :osenbroc5 $"nction. )he $irst inp"t arg"ment is the position vector or matrix to be eval"ate!. :ather than iteratively re;calc"lating the n"mber o$ particles $rom the !imensionality o$ the position matrixB this in$ormation is simply passe! into the $"nction via the secon! inp"t arg"ment. )he o"tp"t is the col"mn matrix o$ $"nction val"es correspon!ing to each particle/row eval"ate!. %ersonal an! global bests are "p!ate! base! on how well they minimi3e the $ollowing $orm"la. n 1 2 2 2 j +1 j j j =1

r f ( x = ( 100( x

+ (1 x

30 x j 30
Ob=$un>Rosenbrock>9ngelbrecht>&anden7ergh
)his eval"ates positions accor!ing to the version o$ the :osenbroc5 $orm"la "se! in Van !en BerghDs %h& thesis. )he $irst inp"t arg"ment is the position vector or matrix to be eval"ate!. :ather than iteratively re;calc"lating the <A

n"mber o$ particles $rom the !imensionality o$ the position matrixB this in$ormation is simply passe! into the $"nction via the secon! inp"t arg"ment. )he o"tp"t is the col"mn matrix o$ $"nction val"es correspon!ing to each particle/row eval"ate!. ?otice that every s"mman! pro!"ce! by an even val"e o$ $ in the :osenbroc5 $orm"lation above is excl"!e! by the $orm"lation below. )he $irst s"mman!s o$ the two $orm"lations are i!entical 1i.e. $or @ K 12B b"t the $orm"lation below excl"!es every other s"mman! pro!"ce! by the $orm"lation above since the $irst s"bscript is always evenB whereas the $irst s"bscript above co"nts all integers $rom 1 thro"gh 1n N 12. Given problem !imensionalityB nB the $orm"lation below s"ms abo"t hal$ as many s"mman!s as the $orm"lation above an! is there$ore a less complicate! $orm"la. n /2 2 2 2 2j 2 j 1 2 j 1 j =1

r f ( x = ( 100( x x

+ (1 x

2.048 x j 2.048
Ob=$un>Scha((ers>(B
)his eval"ates the position1s2 passe! into the two;!imensional #cha$$erDs $A $"nction. )he $irst inp"t arg"ment is the position vector or matrix to be eval"ate!. :ather than iteratively re;calc"lating the n"mber o$ particles $rom the !imensionality o$ the position matrixB this in$ormation is simply passe! into the $"nction via the secon! inp"t arg"ment. )he o"tp"t is the col"mn matrix o$ $"nction val"es correspon!ing to each particle/row eval"ate!. )he $orm"la by which each particleDs $"nction val"e is !etermine! $rom the two !imensions to be optimi3e! is shown below.

sin( x + x 0.5 r f ( x = 0.5 + 2 2 (1.0 + 0.001( x12 + x2

2 1 2 2

100 x j 100

<7

Ob=$un>Schwe(el
$ormulation Used b+ PSO Research Toolbox )his eval"ates the val"e1s2 o$ the position1s2 passe! into the #chwe$el $"nction. )he $irst inp"t arg"ment is the position vector or matrix to be eval"ate!. :ather than iteratively re;calc"lating the n"mber o$ particles $rom the !imensionality o$ the position matrixB this in$ormation is simply passe! into the $"nction via the secon! inp"t arg"ment. )he o"tp"t is the col"mn matrix o$ $"nction val"es correspon!ing to each particle/row eval"ate!.

Each positionDs $"nction val"e is !etermine! accor!ing to the $orm"la below.

n r f1 ( x = 418.982887272433799807913601398n x j sin j =1

( x)
j

500 x j 500
#chwe$elDs crests an! tro"ghs increase in magnit"!e with !istance $rom the origin so that there is no global minim"m or global minimi3er when the search space is "nbo"n!e!R conse."entlyB particles m"st be con$ine! to the search space to prevent them $rom $in!ing positions o"tsi!e the $easible search space pro!"cing better $"nction val"es. )his is especially important since the global minimi3er o$ the bo"n!e! search space is $airly near its e!ge. )o restrict particles to the search spaceB "se position clamping or velocity reset. 9hen the search space is bo"n!e!B the global minim"m o$ 0 occ"rs at vector 020.>A<70AXones11B !im2.
Other $ormulation #ome a"thors "se the $ollowing $orm"lation instea!B which pro!"ces the same global minim"m o$ 0 at vector ;020.>A<70AXones11B !im2B which is per !imension the a!!itive inverse o$ the global minimi3er o$ the $irst $orm"lation.
n r f 2 ( x = 418.982887272433799807913601398n + x j sin j =1

xj

500 x j 500
)he %article #warm 'ptimi3ation :esearch )oolbox "ses the $irst $orm"lation to pro!"ce a global minimi3er that is positive on each !imensionB b"t both $orm"lations are o$ e."ivalent mathematical !i$$ic"lty by symmetry across r r the $ axisE i.e. since f1 ( x = f 2 ( x . )he rea!er is merely ca"tione! that both $orm"lations are in circ"lation $or the p"rpose o$ "n!erstan!ing which global <<

minimi3er pro!"ces the global minim"m o$ 0. *s a simple mnemonicB #chwe$elDs global minimi3er is per !imension opposite in sign to that seen in the $orm"la itsel$.

Ob=$un>Spherical
)his eval"ates the position1s2 passe! into the #phere $"nctionB which is contin"o"sB strictly convexB an! "nimo!al. )he $irst inp"t arg"ment is the position vector or matrix to be eval"ate!. :ather than iteratively re; calc"lating the n"mber o$ particles $rom the !imensionality o$ the position matrixB this in$ormation is simply passe! into the $"nction via the secon! inp"t arg"ment. )he o"tp"t is the col"mn matrix o$ $"nction val"es correspon!ing to each particle/row eval"ate!. )he cost $"nction to be minimi3e! is !e$ine! as $ollows.
n r f ( x = x2 j j =1

100 x j 100
9hether the search space is bo"n!e! or "nbo"n!e!B the global minim"m o$ 0 occ"rs at the n"ll vector.

Ob=$un>0eighted>Sphere
)his eval"ates the position1s2 passe! into the weighte! #phere $"nctionB which is contin"o"sB convexB an! "nimo!al. )he $irst inp"t arg"ment is the position vector or matrix to be eval"ate!. :ather than iteratively re; calc"lating the n"mber o$ particles $rom the !imensionality o$ the position matrixB it is simply passe! into the $"nction via the secon! inp"t arg"ment. )he o"tp"t is the col"mn matrix o$ $"nction val"es correspon!ing to each particle/row eval"ate!. Each position vector enco"ntere! by a particle is translate! to a $"nction val"e accor!ing to the $orm"la below.
n r f ( x = j x 2 j j =1

5.12 x j 5.12
9hether the search space is bo"n!e! or "nbo"n!e!B the global minim"m o$ 0 occ"rs at the n"ll vector.

<>

Of &iles
$ yo" are intereste! in seeing this section !evelope!B please email george PatQ georgeevers.orgB where yo" may also re."est noti$ication o$ $"t"re "p!ates. -or nowB please visit chapter G%rogram -lowBH which !escribes the most important $iles.

!ppendix 7? 9rror Correction

4ava+lang+OutOfMemory.rror7 "ava eap space
)his error was enco"ntere! when generating many swarm tra@ectory graphs. )o a!!ress the problemB intro!"ce! switch OnOff_reuse_figuresB which ca"ses all graphs to be generate! within the same $ig"re N res"lting in a $rame;by;$rame vi!eo. Each $rame can be save! a"tomatically by activating switch OnOff_autosave_figs in the G:EG%#' 4 %#' G:*%+ ?G #9 )(+E#H s"bsection o$ the control panel.

.rror using @@A save7 ,nable to write fileB 2o suc (irectory

file or

9in!ows concatenates a $ileDs !irectory an! $ilename to "ni."ely i!enti$y its location when savingB copyingB or movingB which can become problematic when both a $ile an! its containing !irectory have lengthy names. )his happens more o$ten than it sho"l! since ,y &oc"ments is "nnecessarily $ar $rom the root !irectoryB which pro!"ces "nnecessarily long !irectory names $or most save! content. 9in!ows 7 ma5es some progress in this regar! by removing G&oc"ments 4 #ettingsH $rom the !irectory nameB b"t long $ol!er names within the ,*)8*B s"b$ol!er o$ ,y &oc"ments may still ca"se problems. )his error can be voi!e! altogether simply by "n3ipping the %#' :esearch )oolbox !irectly to G(EWH. $ that is inconvenientE 1i2 close ,*)8*B so that each $ol!er comprising the c"rrent !irectory can be rename!B 1ii2 highlight problematic $ol!ers with lengthy names one at a timeB an! 1iii2 press -2 to assign a shorter name s"ch as G%#':)H.

!ppendix C? 9((icient #!T3!7 Shortcuts

.(iting Co(e
(trl (trl (trl (trl (trl (trl O O O O O O P Q or )ab G : ) + &ecrease in!ent ncrease in!ent Go to line [ (omment line =ncomment line #earch or repl 1Mo$ highlighte! text2 1Mo$ highlighte! text2

1i.e. c"rrent line or highlighte! lines2 1i.e. c"rrent line or highlighte! lines2 1also wor5s in 9or!B 9or! %a!B ?otepa!B an! ?otepa!

>0

ace (trl (trl (trl (trl (trl O O O O O * ( % I F #elect *ll (opy %aste =n!o :e!o

2avigating
(trl O 0 #witch to (omman! 9in!ow (trl O #hi$t O #witch to E!itor 0 (trl O %age =p )ab "p 1between open tabs in E!itor2 (trl O %age )ab &own 1between open tabs in &own E!itor2

Managing &iles
(trl O ? (trl O ' (trl O 9 *lt O -0 (trl O # ?ew E!itor tab 'pen (lose E!itor tab (lose app #ave 1also wor5s in 9or!B 9or! %a!B ?otepa!B an! ?otepa!OO2 1also wor5s in 9or!B 9or! %a!B ?otepa!B an! ?otepa!OO2 1also wor5s with 9in!ows $ol!ers an! ?otepa! OO tabs2 1also wor5s in 9or!B 9or! %a!B ?otepa!B an! ?otepa!OO2 1also wor5s in 9or!B 9or! %a!B ?otepa!B an! ?otepa!OO2

!ppendix D? Citation o( Toolbox or Documentation

9hen citing the toolboxB please incl"!e the version n"mber to ens"re the repro!"cibility o$ !ataB an! please incl"!e =:8 GhttpE//www.georgeevers.org/pso_research_toolbox.htmH. 9hen citing the !oc"mentationB please incl"!e the version n"mber to avoi! any potential con$"sion sho"l! anyone else try to loo5 "p the same in$ormation in an ol!er version. *lsoB please incl"!e =:8 Gh!!p"//www.#eor#ee$ers.or#/pso_research_!oo%&ox_'oc(men!a!ion.'oc).

Paste/able .xamples
*$ers, +., PSO Research Toolbox (,ersion 20--mm'' , ../. !hesis co'e, 2011, 0 h!!p"//www.#eor#ee$ers.or#/pso_research_!oo%&ox.h!m 1 *$ers, +., PSO Research Toolbox Documentation (,ersion 20--mm'' , ../. !hesis co'e 'oc(men!a!ion, 2011, 0 h!!p"//www.#eor#ee$ers.or#/pso_research_!oo%&ox_'oc(men!a!ion.p'2 1

>1