[65] [66]

Schittkowski, K., QLD: A FORTRAN Code for Quadratic Programming, Users Guide, Mathematisches Institut, Universitt Bayreuth, Germany, 1986. Miele, A., and Losi, A., Sequential Gradient-Restoration Algorithm for Mathematical Programming Problems with Direct and Functional Inequality Constraints, AAR-203, Rice University, Houston, Texas, U.S.A., 1986. Users Guide for SGRA -Version 0.1, Universit di Cassino, Italy, in preparation. Lasdon, L. S., and Waren, A. D., Generalized Reduced Gradient Software for Linearly and Nonlinearly Constrained Problems, in Design and Implementation of Optimization Software, H. Greenberg, ed., Sijthoff and Noordhoof, pubs, 1979. Chuang, A., An Extendible Framework for Genetic Algorithms in the Ptolemy Environment, Master Thesis, EECS Department, University of California at Berkeley, September 1995. Cohen, G., Optimization by Decomposition and Coordination: A Unified Approach, IEEE Transactions on Automatic Control, vol. AC-23, No. 2, April 1978. Cohen, G., Auxiliary Problem Principle and Decomposition of Optimization Problems, JOTA: vol. 2, No. 3, November 1980. Batut, J., and Renaud A., Daily Generation Scheduling Optimization with Transmission Constraints: A New Class of Algorithms, IEEE Transactions on Power Systems, Vol. 7, No. 3, pp. 982-989, August 1992. Luenberger, D. G., Linear and Nonlinear Programming, 2nd. Ed., Addison-Wesley, 1984. Contreras, J., Losi, A., Russo, M., and Wu, F. F., DistOpt, a Tool for Modeling and Performance Evaluation of Distributed Optimization Environments, submitted to IEEE PES96 SM. Contreras, J., Losi, A., Russo, M., and Wu, F. F., An Application of DistOpt to Power Systems Optimization Problems, submitted to IEEE PES96 SM. Lun, S. M., Wu, F. F., Xiao, N., and Varaiya, P., NetPlan: An Integrated Network Planning Environment, Int. Workshop on Modeling Analysis and Simulation of Computer and Telecomm. Systems, Jan., 1993.

[67]

[68]

[69]

[70] [71]

[72] [73]

[74] [75]

101

[52] [53] [54]

Wildberger, A. M., Automated Management for Future Power Networks: A LongTerm Vision, Public Utilities Fortnightly 132, 20 (Nov), pp. 38-41. Wang, X., and McDonald, J. R., Modern Power System Planning, McGraw-Hill, 1994. Romero, R., Gallego, R. A., and Monticelli, A., Transmission System Expansion Planning by Simulated Annealing, IEEE Transactions on Power Systems, vol. 11, No. 1, February 1996. Introduction to Scilab, Scilab Group, INRIA Meta2 Project/ENPC Cergrene, Unit de recherche de Rocquencourt, France, 1996. Youssef, H. K., and Hackam, R., New Transmission Planning Model, IEEE Transactions on Power Systems, Vol. 4, No. 1, pp. 9-18, February 1989. Siddiqi, S. N., and Baughman, M. L., Value-based Transmission Planning and the Effects of Network Models, IEEE Transactions on Power Systems, Vol. 10, No. 4, pp. 1835-1842, November 1995. Harsanyi, J., Rational Behavior and Bargaining Equilibrium in Games and Social Sciences, Cambridge University Press, NY, 1977. Shehory, O., and Kraus, S., Coalition Formation Among Autonomous Agents: Strategies and Complexity, Lecture Notes in Artificial Intelligence No. 957, From Reaction to Cognition, pp. 57-72, C. Castelfranchi and J. P. Muller (Eds.), Springer-Verlag, Berlin, Germany, 1993. Young, H. P., Handbook of Game Theory, Volume 2, Aumann, R. J., and Hart, S. (Eds.), Chapter 34: Cost Allocation, pp 1194-1230, Elsevier Science, B.V., 1994. Billera, L. J., and Heath, D. C., Allocation of shared costs: a set of axioms yielding a unique procedure, Mathematics of Operations Research, vol. 7, No. 1, February 1982, pp. 32-39. Klusch, M., Using a cooperative agent system FCSI for a context-based recognition of interdatabase dependencies, Proceedings CIKM-94 Workshop on Intelligent Information Agents, Gaithersburg, 1994. Aumann, R. J., and Myerson, R., Endogenous formation of links between players and of coalitions: an application of the Shapley value, in The Shapley value: essays in honour of LLoyd S. Shapley, Roth, R. B., A. E., Chapter 12, pp. 175-278, Cambridge University Press, Cambridge, England, 1988. Lawrence, C., Zhou, J. L., and Tits, A. L., Users Guide for CFSQP Version 2.2, University of Maryland, College Park, U.S.A., Jan. 1995.

[55] [56] [57]

[58] [59]

[60] [61]

[62]

[63]

[64]

100

[40] [41] [42]

Brennan, T. J. et al., A shock to the system: restructuring Americas electricity industry, Washington D.C., Resources for the Future, 1996. Bohn, R. E., Caramanis, M. C., Tabors, R. D., Schweppe, F. C., Spot Pricing of Electricity, Boston: Kluwer Academic Publishers, 1988. Baldick, R., and Kahn, E., Transmission Planning Issues in a Competitive Economic Environment, IEEE Transactions on Power Systems, vol. 8, no. 4, November 1993, pp. 1497-1503. Vojdani, A. F., Imparato, C. F., Saini, N. K., Wollenberg, B. F., and Happ, H. H., Transmission Access Issues, IEEE Transactions on Power Systems, vol. 11, no. 1, February 1996, pp. 41-51. Smith, V. L., Regulatory Reform in the Electric Power Industry, Economic Science Laboratory on-line working paper, University of Arizona, Tucson, AZ, http://www.econlab.arizona.edu/elecref.eps. McCabe, K., Rassenti, S., and Smith, V., Designing a Uniform-price Double Auction, Chapter 11 in Friedman and Rust (eds.), The Double Auction Market, Addison-Wesley, 1993. Wilson, R., Activity Rules for a Power Exchange, Proc. of the Second Annual Research Conference on Electricity Industry Restructuring, University of California Program on Workable Energy Regulation (POWER), Berkeley, CA, March 1997. Hughes, W. R., and Felak, R., Bridging the gap between theory and practice of transmission pricing, in Electricity Transmission Pricing and Technology, Einhorn, M., and Siddiqi, R., eds., Kluwer Academic Publishers, Boston, pp. 2558, 1996. Marangon Lima, J. W., Pereira, M. V. F., and Pereira J. L. R., An Integrated Framework for Cost Allocation in a Multi-owned Transmission System, IEEE Transactions on Power Systems, Vol. 10, No. 2, pp. 971-977, May 1995. von Martial, F., Coordinating Plans of Autonomous Agents, in Lecture Notes in Artificial Intelligence vol. 610, Springer-Verlag, Berlin, Germany, 1992. Santa Fe Institute (sfi@santafe.edu). 1996. ftp://ftp.santafe.edu/pub/swarm/, or from WEB site: http://www.santafe.edu/projects/swarm. Wildberger, A. M., Introduction & Overview of Artificial Life -Evolving Intelligent Agents for Modeling & Simulation, Proceedings of the 1996 Winter Simulation Conference, ed. J. M. Charnes, D. J. Morrice, D. T. Brunner, and J. J. Swain, pp. 161-168.

[43]

[44]

[45]

[46]

[47]

[48]

[49] [50] [51]

99

[26]

Hobbs, B. F., and Kelly, K. A., Using game theory to analyze electric transmission pricing policies in the United States, European Journal of Operational Research 56, North-Holland, Elsevier Science Publishers B.V., pp. 154-171, 1992. von Neumann, J., and Morgenstern, O., Theory of games and economic behaviour. 2nd. Edition, Princeton, NJ, Princeton University Press, 1947. Gillies, D. B., Some theorems on n-person games. Ph.D. Dissertation, Department of Mathematics, Princeton University, Princeton, N.J., 1953. Varian, H. editor, Economic and Financial Modeling with Mathematica, Telos, Springer-Verlag, Santa Clara, CA, 1993. Aumann, R. J., and Maschler, M., The bargaining set for cooperative games. In M. Dresher, L. S. Shapley, and A. W. Tucker (Eds.), Advances in game theory. Princeton, N.J.: Princeton University Press, 1964. Davis, J. H., and Maschler, M., The kernel of a cooperative game. Naval Research Logistics Quaterly, 1965, 12, pp. 223-259. Schmeidler, D., The nucleolus of a characteristic function game. SIAM Journal of Applied Mathematics, 1969, 17, pp. 1163-1170. Shapley, L. L., A value for n-person games. In H. W. Kuhn, and W. Tucker (Eds.), Contributions to the theory of games II. Annals of Mathematical Studies No. 28. Princeton, N.J.: Princeton University Press, 1953. Komorita, S. S., An equal excess model of coalition formation. Behavioral Science, 1979, 24, 369-381. Stearns, R. E.,Convergent transfer schemes for n-person games. Transactions of the American Mathematical Society, 1968, 134, pp. 449-459. Wu, L. S., A dynamic theory for the class of games with nonempty cores. SIAM Journal of Applied Mathematics, 1977, 32, pp. 328-338. Rudnick, H., Palma, R., Cura, E., and Silva, C., Economically Adapted Transmission Systems in Open Access Schemes -Application of Genetic Algorithms, IEEE Transactions on Power Systems, vol. 11, No. 3, pp. 1427-1440, August 1996. Moen, J., Electric utility regulation, structure and competition. Experiences from the Norwegian electric supply industry, Norwegian Water Resources and Energy Administration, Aug. 1994. Ruff, L., Stop Wheeling and Start Dealing: Resolving The Transmission Dilemma, The Electricity Journal, v. 7, no. 5, pp. 24-43, June 1994. 98

[27] [28] [29] [30]

[31] [32] [33]

[34] [35] [36] [37]

[38]

[39]

[12] [13] [14]

Chao, H., and Peck, S., A Market Mechanism For Electric Power Transmission, Journal of Regulatory Economics; 10:25-59, 1996. Hogan, W., Contract Networks for Electric Power Transmission, Journal of Regulatory Economics; 4:211-242, 1992. Garver, L. L., Transmission Net Estimation Using Linear Programming, IEEE Transactions on Power Apparatus & Systems, vol. PAS-89, n.7, pp. 1688-1697, Sept./Oct. 1970. Dusonchet, Y. P., and El-Abiad A.H., Transmission Planning Using Discrete Dynamic Optimization, IEEE Transactions on Power Apparatus and Systems, vol. PAS-92, pp. 1358-1371, 1973. Lee, S. T. Y., Hocks, K. L., and Hnyilicza, E., Transmission Expansion of BranchAnd-Bound Integer Programming With Optimal Cost-Capacity Curves, IEEE Transactions on Power Apparatus and Systems, vol. PAS-93, n.5, pp. 1390-1400, 1974. Bennon, R. J., Juves, J.A., and Meliopoulos, A. P., Use of Sensitivity Analysis Automated Transmission Planning, IEEE Transactions on Power Apparatus and Systems, vol. PAS-101, n.1, pp. 53-59, 1982. Monticelli, A., Santos Jr., A., Pereira, M. V. F., Cunha, S. H., Parker, B. J., and Praa, J. C. G., Interactive Transmission Network Planning using a Least-Effort Criterion, IEEE Transactions on Power Apparatus & Systems, vol. PAS-101, n.10, pp. 3919-3925, October 1982. Yoshimoto, K., Yasuda, K., and Yokohama, R., Transmission Expansion Planning Using Neuro-Computing Hybridized with Genetic Algorithm, Proceedings of the 1995 International Conference on Evolutionary Computing, ICEC95, pp. 126-131. Walton, S., Establishing Firm Transmission Rights using a Rated System Path Model, The Electricity Journal, pp. 20-33, October 1993. DistOpt 0.1 Users manual, U.C. Berkeley, EECS Department, in preparation. Ptolemy Reference Manual, V. 0.6., EECS Department, U.C. Berkeley, 1996. Rasmusen, E., Games and Information, Basil Blackwell Inc., New York, 1989. Kahan, J. P., and Rapoport, A., Theories of Coalition Formation, Lawrence Erlbaum Associates, Publishers, Hillsdale, NJ, 1984. Colman, A. M., Game Theory and its Applications, 2nd. Edition, Butterworth Heinemann Ltd., Oxford, England, 1995.

[15]

[16]

[17]

[18]

[19]

[20] [21] [22] [23] [24] [25]

97

Bibliography
[1] Oliveira, G. C., Costa, A. P. C., and Binato, S., Large Scale Transmission Network Planning Using Optimization and Heuristic Techniques, IEEE Transactions on Power Systems, Vol. 10, n.4, pp. 1828-1833, November 1995. Gately, D., Sharing The Gains From Regional Cooperation: A Game Theoretic Application To Planning Investment In Electric Power, International Economic Review, vol. 15, n. 1, pp. 195-208, February, 1974. Tsukamoto, Y., and Iyoda I., Allocation of Fixed Transmission Cost to Wheeling Transactions by Cooperative Game Theory, IEEE Transactions on Power Systems, vol. 11, No. 2, pp. 620-629, May 1996. Chattopadhyay, D., An Energy Brokerage System with Emission Trading and Allocation of Cost Savings, IEEE Transactions on Power Systems, vol. 10, No. 4, pp. 1939-1945, November 1995. Ketchpel, S. P., Coalition Formation Among Autonomous Agents, Proceedings of MAAMAW-93. Klusch, M., Utilitarian coalition formation between information agents for a cooperative discovery of interdatabase dependencies, in S. Kirn and G. OHare (Eds.), Cooperative Knowledge Processing, 1996, Springer-Verlag, London, 1996. See also the web site: ftp://ftp.informatik.uni-kiel/pub/kiel/ideas. Klusch, M., and Shehory, O., Coalition Formation Among Rational Information Agents, Proceedings of MAAMAW-96, Eindhoven, LNAI Series vol. 1038:204217, Springer Verlag. Klusch, M., Shehory, O., A polynomial Kernel-oriented coalition algorithm for rational information agents, Proc. 2. International Conference on Multi-Agent Systems ICMAS-96, Kyoto (Japan), AAAI Press, 1996. Shehory, O., and Kraus, S., A Kernel-oriented model for coalition formation in general environments: Implementation and results, Proceedings AAAI-96, Portland, OR, 1996. Wu, F.F., Varaiya. P., Coordinated Multilateral Trades for Electric Power: Theory and Implementation, POWER report PWP-031, University of California Energy Institute, June 1995. Bushnell, J., and Stoft, S., Transmission and Generation Investment in a Competitive Electric Power Industry, Power Report PWP-030, University of California Energy Institute, May 1995. 96

[2]

[3]

[4]

[5] [6]

[7]

[8]

[9]

[10]

[11]

6.3. Concluding Remarks

The electric utilities are experiencing radical changes with the upcoming deregulation process. Traditional methods to solve power systems problems need to be updated and adapted to the new times coming ahead. New software environments are urgently needed to understand players interaction and to model new market structures in the future. Isolationist operating and planning policies will soon become obsolete, and the need for a decentralized but coordinated structures is clear. To address these issues, the approach provided in this thesis can be applied to a new deregulated environment to simulate not only the interaction between the players, but also to implement innovative policies with the help of our multiagent environment. A decentralized approach to power systems problems, like the one we propose, will have to combine individual decision making with a coordinated environment. Communication among the players and fast algorithms to solve large scale problems will be a must. We also believe that Ptolemy, with its communications capability, modularity, graphical and external interfaces, and object-oriented structure is an ideal software to model new power system paradigms in the future.

95

Chapter 6 Conclusions and Future Work

6.1. Contributions
Through the previous chapters we have outlined a new method to simulate coalitions of players in the transmission planning game and to allocate the expansion costs that any expansion creates. The approach that we have followed to solve the problem combined a multiagent environment and a cooperative game theory framework to form coalitions and allocate costs respectively. To test our theoretical formulation of the transmission planning problem, we have also implemented a new software environment to solve this and other problems in the power systems and optimization areas.

6.2. Future Research

Future work will add the Bilateral Shapley Value coalition scheme and other coalition formation schemes to the existing IDEAS [6] framework; in particular kernel transfer schemes (see Chapter 2, Section 8). A more detailed study of the coalition stability properties of our Bilateral Shapley Value recursive cost allocation procedure is forthcoming. At the same time, new algorithms are being added to DistOpt, and new applications of DistOpt in decentralized planning are considered. On the other hand, a genetic algorithm implementation of individual agents expansion planning using [68] is forthcoming, and dynamic (more than one year) expansions are under study. We are currently integrating the coalition formation and cost allocation algorithms in the Ptolemy environment. The interested reader can see some of the capabilities of this graphical software framework in power systems applications [73, 74]. 94

Applications to the optimal power flow problem in [74] show the effectiveness of the software package.

93

Figure 5-9 : IEEE-57 Bus Case Subproblem 2 Maximum Error

Figure 5-10 : IEEE-57 Bus Case Subproblem 3 Maximum Error

5.7. Conclusions
We set out a research infrastructure with the goal of tackling optimization problems arising from power system planning and operations. A generic structure for these problems based on decomposition and co-ordination for power system applications was constructed. The methods are designed to be compatible with vastly enhanced computing power offered by current technology of parallel and distributed computing. The software implementation utilizes recent advances in object-oriented software that is flexible, extensible, and conducive to cooperative development. The object-oriented methodology is especially suited for the emerging design approach in power industry. The distributed optimization research environment is a power tool to handle a variety of projects, to develop innovative solution approaches, and to solve critical emerging problems in power systems.

92

solve the problem can be less if we divide the problem in very small subsystems, because the problems are smaller now and they take less time to solve. For an asynchronous execution, the objective function approaches its final value faster than with the synchronous execution, but, to grant convergence, parameter -see (5-2)- has to be significantly low [74].

Figure 5-8 : IEEE-57 Bus Case Subproblem 1 Maximum Error

91

Figure 5-6 : IEEE-57 Bus Case Subproblem 2 Objective Function

Figure 5-7 : IEEE-57 Bus Case Subproblem 3 Objective Function Figure 5-8, Figure 5-9, and Figure 5-10 show the value of the error: maximum discrepancy between the values of neighboring quantities [ error = max i ] -see (5-1)- veri sus the computational time. Note that no matter how many subsystems the problem is divided into, the APP always ensures convergence to a solution, although more subsystems usually require more APP iterations, for the same problem. However, the time to 90

5.6. An Optimal Power Flow Application

We study the performance in executing the optimization functions for the IEEE-57 bus network. In particular, we study the Optimal Power Flow (OPF) problem, for which a fullfledged model is considered (with no model simplifications), solved without any P-Q decomposition. For evaluating the performance, the following aspects are considered: value of objective function maximum discrepancy between the values of the duplicated variables The algorithm we adopt for solving the various optimization problems is a first-order one, namely SGRA [66], for which an adequate interface is provided in the Solver palette. For space reasons, we only consider a three subsystems example of the IEEE-57 bus case, with a synchronous execution, and an Arrow-Hurwicz core function. The computational time, i.e., the number of total APP iterations is 300; the number of iterations of the SGRA algorithm depends on each case. In Figure 5-5, Figure 5-6, and Figure 5-7, the value of the objective function for the three subsystems versus the computational time is reported.

Figure 5-5 : IEEE-57 Bus Case Subproblem 1 Objective Function 89

5.5. DistOpt Simulation

A simulation case is run through a universe; this is built starting with the specification of the number of subproblems by selecting the appropriate block from the mathematical building palette. Then, the choice of a particular solver has to be made for each subproblem. This is done by grabbing the icon representing that algorithm from the Solvers palette and dragging it to the appropriate ports of the subproblem block [73]. Then, according to the needs of the information exchange of the subproblems, input/output ports of the icons representing neighboring subproblems have to be graphically connected. A universe for the simulation of the solution of a problem split into three subproblems is shown in Figure 5-4. The simulation is then started (the universe is run), and the user is requested via Tcl/Tk window interface what kind of core function K to use (Arrow-Hurwicz, Uzawa-like, or other supplied), and the value of some parameters influencing the convergence (computed values are suggested). Parameters such as synchronization technique may be easily entered and changed with pop-up menus. The evaluation of alternative ways of solving the subproblems with different algorithms is easily carried out by connecting the icon of the desired algorithm to the appropriate port of the subproblem block. The behavior of the communication system that cares for the data exchange between subproblems can be easily simulated by using the blocks already developed for simulating communication networks [75]. The ease of such operation comes from the modularity, flexibility, and extendibility of the Ptolemy environment, with its block-oriented graphical construction of the applications.

88

Figure 5-3 : APP_3 Galaxy

Figure 5-4 : Final3 Universe 87

APP_# Galaxy takes care of the initialization of the solution (by reading the information contained in the user-supplied file, and by sending to Measures galaxy and to SubProblem star the appropriate information on the methods of the optimization problem to be solved, as well as on the splitting of the problem). APP_# chooses the core function and the value of several parameters of the computation scheme, as well. Measures galaxy models the front-end between the computing facility and the real world. It receives initialization information from APP_#, and reads data from a file, representing the data/measurements from which the subproblem has to be solved. It has a state, the name of the input file (which has a default value), that can be modified by the user. SubProblem star, which receives initialization information and data from Measures, updates the local copy of the Lagrange multipliers associated with the coupling constraints (the prices), interfaces with the solution algorithms, and exchange information with the neighboring subproblems. In Figure 5-2 and Figure 5-3, we can observe more details of the Measures and APP_3 galaxies.

Figure 5-2 : Measures Galaxy

The overall picture of an instance of DistOpt universe that solves an optimization problem by splitting it into three subproblems can be seen in Figure 5-4.

86

once they have the information from the other solvers (and their own updates), they will incorporate new information immediately. Asynchronous methods and partially synchronous methods are also provided in this version of DistOpt. The modularity of design in the Ptolemy environment allows the implementation of instances of the DistOpt universe for the user to built her own examples. All the blocks (galaxies in Ptolemy) pertaining to the DistOpt environment are contained in the corresponding Ptolemy palette as shown in Figure 5-1.

Figure 5-1 : DistOpt Palette

85

and IEEE*bus_dat respectively. In any particular application, the user simply has to change the input file, without relinking, in order to solve a different problem (though it has to be of the same type). The user can define her own application by adding it to Methods.cc and recompiling again. The second level of abstraction represents the decomposition-coordination approach, and it uses the Auxiliary Problem Principle (APP) scheme. Now, the program duplicates the variables that belong to at least two subproblems and calculates the number of duplications (couplings) for the whole optimization problem. The APP scheme is used to implement the decomposition-coordination algorithm. The user is prompted via Tcl/Tk interface to decide which core function she desires to use to solve the problem. The core function is a function that mimics the objective function (see [70]) and it is built-in the APP algorithm. After that, the user is prompted to choose parameters values to ensure convergence of the algorithm. The program calculates these values, but the user can change them interactively via Tcl/Tk script. In particular, an accurate evaluation of is calculated in the spectral_radius.cc file (see [70] for details). If the new value is out of bounds, the program will pop up a warning message before continuing. The third level of abstraction represents the interface with the solvers. There exists a library of different solvers that can be used for different applications. If the problem is linear, the available solvers are: DECFSQPSolver.pl, DELSGRG2Solver.pl, and DESGRASolver.pl. If the problem is of quadratic nature: DEQProgSolver.pl (based on Schittkowskis QLD), DEQPMATLABSolver.pl, DECFSQPSolver.pl, DELSGRG2Solver.pl, and DESGRASolver.pl. To solve OPF problems: DECFSQPSolver.pl, DELSGRG2Solver.pl, and DESGRASolver.pl. There is one more solver, although in experimental phase. It is called DEMIQPMATLABSolver.pl: It is a Mixed Integer Quadratic Programming solver based on the MATLAB script setconst_qp.m, and (sometimes) cannot take advantage of decomposition schemes, because convergence is not always guaranteed. It can be tested with the files problem*_MIQP.dat provided in this distribution. Also, the synchronization method to exchange information among subproblems is up to the user. By default, the solvers will exchange information in a synchronous mode, i.e., 84

5.4. DistOpt Structure

DistOpt [21, 73, 74] is a distributed optimization application built in the Ptolemy environment as a universe. In DistOpt, blocks are modeled in the event-driven model of computation Discrete Event (DE) of Ptolemy. DE stars act as event-processing units, which receive particles from the outside, process them, and generate output events after a usergiven latency. A particle carries a time stamp, generated by the block which produced the particle, and represents an event corresponding to a change in a system state. The DE scheduler processes events in chronological order, until the global time reaches a userspecified stop-time. A star is executed (fired) whenever a new event appears at the input portholes; after execution, output events may be generated at the output portholes of the star. The user is referred to Chapter 3: Running DistOpt of [21] for a detailed description of the DistOpt universe user interface. In this Section, we explain the overall structure that supports the software implementation. There are three levels of abstraction implemented in the overall design of DistOpt. The first level of abstraction represents the mathematical formulation of the optimization problem. The user has to provide objective function and constraint coefficients in one input file. The file contains the number of variables, number of equality and inequality constraints, and the coefficients of all the constraints and objective functions. The number of variables and constraints that belong to each subproblem (and their respective indexes) are also included. It can happen that some constraints in a particular subproblem contain variables from some other subproblems. It is possible because the user has decided to split the number of variables, but she also splits the constraint set, and some of the variables may need to be duplicated. Three applications are available by simply changing the file Methods.cc, and relinking. In particular: linear programming, quadratic programming, and optimal power flow problems (non-linear quadratic programming application in power systems). The input files are of the type problem*_LP.dat, problem*_quad.dat,

1. Ptolemy Version 0.6, released on April 1996, incorporates a new syntax manager: Tycho, that can be used as a text editor, or as a GUI within Ptolemy. There are other features in the new release, like the use of itcl2.0 (an object-oriented version of Tcl/Tk), and new Tcl/MATLAB and Tcl/Mathematica interfaces for scripted runs and numeric/ symbolic parameter calculations.

83

extensible. Users can create new component models, new design process managers, and even entirely new programming environments. The core of Ptolemy is an object-oriented toolkit and library that makes few assumptions about the system to be modeled; rather, standard interfaces are provided for generic objects and more specialized, applicationspecific objects are derived from these. In short, Ptolemy can be used to model complex heterogeneous systems. Ptolemy achieves its goals using the principle of polymorphism. The Ptolemy kernel, written in C++, defines the basic classes that allow the components of the system to function together; application-specific objects are derived from these. Information hiding and data abstraction are key to the design, so that is easy to modify; a key goal is that the system be extensible in many dimensions without the need to modify the kernel. Each model of computation in Ptolemy is called a Domain, an extensible library of functional blocks. In Ptolemy, an overall description of a system is decomposed into software modules called Blocks, the basic units of computation. Graphically, a block is an icon with terminals, corresponding to its input/output Portholes. Blocks may have State variables, which serve as user-settable parameters or as named variables that blocks can modify and users can monitor. The blocks are invoked at run time in an order determined by a Scheduler, and exchange data among themselves as they execute. The scheduler determines the operational semantics of a network of blocks, so schedulers are different in each domain. There are two types of blocks: the Star and the Galaxy. The star is elemental, in the sense that is implemented by user-provided code. A galaxy is a block which internally contains stars as well as possibly other galaxies. The galaxy is thus a construct for producing a hierarchical description of the simulation. A Universe is a complete program, or application. Data passes between blocks in discrete units called Particles, which may be of many types -integer, real, complex, fixed-point, matrix, or a general structure (a data packet, for example). To get started, you can use the Ptolemy interactive graphical interface (pigi) or the Ptolemy interpreter (ptcl)1. The interpreter has a textual command language based on Tcl. Both interfaces are extensible using Tcl, and the graphical interface is extensible using the associated X-Window toolkit Tk.

82

pv

k+1

= pv + ( v

k+1

k+1

In this last scheme, the updating of p u and p v can be done in parallel. Equation (5-5) always holds if: pu = pv
0 0

(5-8)

The convergence results hold for the synchronous execution of the different computations the algorithm is made of; if a processor is assigned the tasks regarding a given subproblem, namely the solution of the minimization problem and the updating of the local copy of the prices, at the iteration k it would need the value k-1 of the neighboring variables to solve the problem, and the value k of the same variables to update the prices. The main advantages of an APP approach are the following. First, the APP allows one to replace the master problem by a sequence of auxiliary problems and, taking this opportunity, to give particular desirable features to the latter problems (well conditioning, decomposability, etc.). Also, the APP approach not only allows one to give general proofs of convergence for a large class of algorithms (Newton, Projected Gradient, Uzawa, ArrowHurwicz, etc.), but it gives systematic guidelines for designing new algorithms. As far as decomposition-coordination is concerned, sequential decomposition (see [69]) seems to be useful for solving problems of a size larger than that one can deal with using the largest computer facilities available nowadays. Parallel decomposition may be a way of achieving fast computation in on-line situations (for example, process control).

5.3. Ptolemy Environment

Ptolemy [22], programmed in C++, is an extensible object-oriented environment intended as a general framework in which system specification, simulation, and design are possible. Several environments have been built, including dataflow-oriented graphical programming for signal processing, discrete-event modeling of communication networks, register-transfer-level circuit design, synthesis environments for embedded software, and design assistants for hardware/software codesign. The Ptolemy system is fundamentally 81

k+1

= p + ( u

k+1

,v

k+1

) = p + (u

k+1

k+1

It is apparent that the two minimization problems in (5-4) can be solved in parallel, provided that the values of the duplicated variables and the multipliers of the previous iteration are known. Let us look in deeper detail into the coordinators task, represented in (5-4) by the updating of the Lagrange multipliers p associated with the coupling constraints . Note that this updating is the solution of the coordinators problem represented by the maximization of the dual function of the minimization problem. For the particular form of the coupling constraints , the coordinators task can be duplicated in such a way that the minimization subproblems to be solved in subspaces U and V have exactly the same form. To do so, let vectors pu and p v be defined such that: p = pu = p v Combining (5-5) and (5-4), the two-level iterative scheme (5-4) can be written as: min K 1 ( u ) + J' u ( u , v ) K' 1 ( u ), u + p u + c ( u v ), u u uU
k k k k k k k+1

(5-5)

(5-6)

min K 2 ( v ) + J' v ( u k, v k ) K' 2 ( v k ), v + p v + c ( u k v k ), v v k + 1 k vV pu

k+1

= p u + ( u = p v ( u
k

k+1

,v ,v

k+1

) = pu + ( u ) = pv ( u
k

k+1

v v

k+1

pv

k+1

k+1

k+1

k+1

k+1

and then min K 1 ( u ) + J' u ( u , v ) K' 1 ( u ), u + p u + c ( u v ), u u uU min K 2 ( v ) + J' v ( u , v ) K' 2 ( v ), v + pv + c ( v u ), v v vV pu

k+1 k k k k k k k k k k k k k+1

(5-7)

k+1

= pu + ( u 80

k+1

k+1

adding convex/concave terms which can be chosen to be separable. Under some conditions, the new problem (after applying the APP) has the same saddle-point as the original problem. A description of an iterative algorithm to find the saddle point of this new problem can be found in [70]. By choosing the APP terms to be decomposable, the central step of this iterative algorithm decomposes into independent parallel minimizations, followed by Lagrange multiplier updates which involve only local information transfers. After each step, the processing elements exchange coordination information and re-solve the subproblems. The idea is that several steps of this distributed iterative algorithm may produce a solution faster than a more traditional centralized algorithm. It is useful to our purposes to explicit the iterative scheme form (5-2), such that (5-1) can be put in the following equivalent form: min J ( u, v ) u U, v V subject to ( u, v ) = u v = 0 , where v is the duplication of some of the variables u of the original problem; this duplication can be made according to the wanted splitting of the problem into subproblems. It is noteworthy that the technique of variable duplication generates couplings between variables of different subsystems that are always linear equalities, and then affine; in other words, by this technique, we are always able to formulate problems where the decomposition is made such that the coupling constraints are exactly of the type the APP requires. With a core function K additive with respect to u and v [ K ( u, v ) = K 1 ( u ) + K 2 ( v ) ], the application of the scheme (5-2) to problem (5-3) yields: min K 1 ( u ) + Ju ( u , v ) K1 ( u ), u + p + c ( u , v ), u u uU min K 2 ( v ) + Jv ( u , v ) K2 ( v ), v + p + c ( u , v ), v v vV 79
k k k k k k k k k k k k k+1

(5-3)

(5-4)

k+1

solved with the following two-level iterative scheme [69] ( , stands for scalar product): min K ( u ) + J( u ) K( u ), u + p + c ( u ), ( u ) u uU p
k+1 k k k k k+1

(5-2)

= p + ( u

k+1

where the auxiliary function K, named the core function, can be given particular characteristics to allow a parallel solution of the auxiliary minimization problem; in particular, K has to be additive with respect to the chosen variable decomposition. The global convergence of the iterative scheme is granted upon the following conditions: J is convex, with derivative J Lipschitz with constant A; is affine, with Lipschitz constant ; K is strongly convex, with constant b, and Lipschitz derivative; 0 < < 2c 0 < < b ( A + c ) . The restriction to the convex case allows us to have necessary and sufficient optimality conditions, and also to obtain proofs for global convergence (i.e., starting from any initial guess). Of course, anybody can use the algorithm in practical non-convex cases, but then the necessary convergence conditions are only met in the limit. Before using the APP, we duplicate some (or all) of the problem variables, so as to decompose the constraints into purely local subsets. This produces an artificial decomposition and an increase in the number of variables. In order to make sure the solutions of the new problem and the original one coincide, we introduce consistency constraints which force all duplicated copies of a variable to be equal at the solution. These equality constraints are the only non-local constraints in the resulting transformed problem. Then we apply the Auxiliary Problem Principle to the problem of finding a saddle-point of the Augmented Lagrangian [72] of the transformed optimization problem. Informally, the APP involves linearizing any non-separable terms in the Augmented Lagrangian, and 78
2

By contrast, DistOpt uses a particular methodology called the Auxiliary Problem Principle [69] (APP), that consists of decomposing a large problem into several subproblems, where each one can be solved using any optimization technique (decomposition), given that some of the subproblems share common variables and need to exchange their values during the iterative solving process (coordination). The advantage of using the APP method shows when dealing with large problems where the speed of convergence and the computation time can be extremely scattered, and often very large. DistOpt takes advantage of the convergence properties [70] that the APP has, and splits a large problem into many subproblems that are easier to solve and faster to converge to the optimum. The last but not least advantage that DistOpt offers is the versatility in the decomposition phase, where the user is free to use many optimization techniques from the solvers palette. This very fact makes DistOpt a very powerful software environment.

5.2. The Auxiliary Problem Principle and Problem Transformation

As explained in the preceding Section, the Auxiliary Problem Principle (APP) allows one to find the solution of a problem (minimization problem, saddle point problem, etc.) by solving a sequence of auxiliary problems. There is a wide range of possible choices for these problems, so that one can choose suitable algorithms to make them easier to solve. The frame is that of convex mathematical programming, although several applications of the APP [71] have extended the convex range to non-convex scenarios. The general structure of an optimization problem can be expressed as follows: min J ( u ) uU subject to ( u ) = 0 , where u is the vector of optimization variables, J(u) is convex, U is a closed convex feasibility set for the quantities, representing typically both equality and inequality constraints, ( u ) is affine, and J represents the criterion of the optimization problem. (5-1) can be 77

(5-1)

Chapter 5 DistOpt: A Distributed Optimization Software Environment

5.1. Introduction
DistOpt [21, 73, 74] is a distributed software environment for solving large scale optimization problems developed at the Department of Electrical Engineering and Computer Sciences, University of California, Berkeley. At the present time, the software tools that have been integrated in DistOpt for solving such problems include: Ptolemy [22]: A platform for heterogeneous simulation and prototypes, CFSQP [64]: A C code for solving constrained optimization problems, a C version of QLD [65]: A FORTRAN Code for Quadratic Programming, SGRA [66]: A Sequential Gradient and Restoration Algorithm, LSGRG21[67]: A Generalized Reduced Gradient software, and MATLAB2: An environment for high-performance numerical computations. The main objective of DistOpt is to solve a variety of big size distributed optimization problems that can arise in several fields, from network planning to power systems or transportation networks. Traditionally, large scale optimization problems have been solved using classical techniques. Newtons method and its variations still stand as the main methodologies to solve any nonlinear problem and the Simplex method is still widely applied in linear programming. In the last few years, some other techniques have emerged, like Karmarkars method, Interior Point method, or Genetic Algorithms, although recent advances are more devoted to improve the speed of convergence than to develop new methods. The interested reader is referred to [68] for a Genetic Algorithm implementation in the Ptolemy environment.

1. Copyright 1979-1986 by Winward Technologies, Inc., and Optimal Methods, Inc. 2. MATLAB is a trademark of The MathWorks, Inc.

76

egy that a player follows. For instance, agent 6, knowing his value in the game, will team either with 2 or with 4 to reduce her own final cost. We can recall that in our decentralized environment the offers are only bilateral, building up to larger coalitions. By building only 2 member coalitions we are missing some combinations that may be more beneficial to the agents. Therefore, not all combinations are considered, like in a SV centralized environment, like in a SV centralized environment, but the coalition formation algorithm terminates in polynomial time. It can be inferred that the skill of the agents skill to choose the right coalition is crucial to get the largest slice of the allocation pie. By relinquishing his right to choose the right coalition, an agent may end up with a value that does not reflect his true power in the game. Therefore, a decentralized approach like the one we propose, will reflect agents strengths, and it will also set a scheme for negotiation. Finally, were we stopping the negotiation process, we could do it at any time, and calculate the BSVs for the current scheme. This in turn provides a new approach in power transmission planning for selforganizing regional cooperations within reasonable time.

75

Figure 4-4 : Core, Nucleolus, and Shapley Value Calculations Comparing this result with the three decentralized simulations of the previous Section, we can observe that there is a clear mismatch between agents 3 and 6 allocations. The reason is that, when using pure SVs to allocate values, we consider all agents combinations: 3 becomes a valuable agent for some coalitions: {3,5}, {1,3,5}, and {1,2,3,6}; but 6 is a worthless agent: {2,5,6}, {4,5,6}1. On the other hand, SVs are suited for superadditive environments, which is not the case here. By contrast, using BSVs, only the best combinations will form, and the concepts of valuable and worthless will depend on the strat-

1. See Table 4-1; a valuable player for a coalition refers to her positive marginal contribution.

74

73

information amongst coordinator and agents, neither among the agents themselves. The coordinators role becomes more important in a centralized approach, then. He not only calculates joint values, but also Shapley values for all the agents, given all possible combinations, as shown in Table 3-2. Were we implementing that scheme, these would be the allocation values: (16.85,-45.86,54.73,-19.41,-42.48,-93.81). Figure 4-3 shows it graphically.

Figure 4-3 : BSV versus SV Allocations Note that the game that we are simulating does not have a core in general, so that the only measure available to the modeler is the Shapley value, whose calculation was performed in Mathematica, as it can be seen in Figure 4-4. The reader can observe the core, nucleolus, and Shapley value calculations in the same figure, as calculated using the software program provided in [29].

72

Figure 4-2 : Overall Coalition Formation and Cost Allocation Procedure A second simulation was run, since player 6 is indifferent whether to team with 2 or 4 at the beginning of the game. The coalition process was: [{1},{2},{3-5},{4-6}] -> [{1-35},{2-4-6}] -> [{1-2-3-4-5-6}]. The allocated values were given by: (21.25,-49.375,6.25,55,-33.75,-19.375). The third simulation did not started from a grand coalition scheme, its order was: [{1},{26},{3},{4},{5}] -> [{1-2-6},{3},{4},{5}] -> [{1-2-3-6},{4},{5}]. The allocation was (still total of -130 monetary units): (22.5,-48.75,15,-60,-40,-18.75). For all simulations, agents 1 and 3 receive monetary units for their contribution to the welfare of the system, while the remaining agents must contribute, although less that what they would pay on their own. This is a clear incentive for all players to play this game.

4.5. Discussion
The main advantage that a centralized approach offers to the expansion game is its simplicity. Using a central planner, we do not need a synchronized algorithm to exchange 71

Figure 4-1 : Cost Allocation Procedure

70

calculate value allocations via BSVs.: BIM (or Backward Induction Method). The process is as follows: ROUND 1: founders: {1-2-3-5-6} and 4 for {1-2-3-4-5-6}, the grand coalition BIM: bsv { 12356, 4 } ( 12356 ) = 1 2v ( 12356 ) + 1 2 ( v ( 123456 ) v ( 4 ) ) = 75 BIM: bsv { 12356, 4 } ( 12356 ) = 1 2v ( 4 ) + 1 2 ( v ( 123456 ) v ( 12356 ) ) = 55 ROUND 2: founders: {1} and {2-3-5-6} for {1-2-3-5-6} BIM: bsv { 1, 2356 } ( 1 ) = 1 2v ( 1 ) + 1 2 ( bsv { 1, 2356 } ( 1, 2356 ) v ( 2356 ) ) = 12.5 BIM: bsv { 1, 2356 } ( 2356 ) = 1 2v ( 2356 ) + 1 2 ( bsv { 1, 2356 } ( 1, 2356 ) v ( 1 ) ) = 87.5 ROUND 3: founders: {2-6} and {3-5} for {2-3-5-6} BIM: bsv { 26, 35 } ( 26 ) = 1 2v ( 26 ) + 1 2 ( bsv { 26, 35 } ( 26, 35 ) v ( 35 ) ) = 68.75 BIM: bsv { 26, 35 } ( 35 ) = 1 2v ( 35 ) + 1 2 ( bsv { 26, 35 } ( 26, 35 ) v ( 26 ) ) = 18.75 ROUND 4: founders: {2} and {6} for {2-6}; {3} and {5} for {3-5} BIM: bsv { 2, 6 } ( 2 ) = 1 2v ( 2 ) + 1 2 ( bsv { 2, 6 } ( 2, 6 ) v ( 6 ) ) = 49.375 BIM: bsv { 2, 6 } ( 6 ) = 1 2v ( 6 ) + 1 2 ( bsv { 2, 6 } ( 2, 6 ) v ( 2 ) ) = 19.375 BIM: bsv { 3, 5 } ( 2 ) = 1 2v ( 3 ) + 1 2 ( bsv { 3, 5 } ( 3, 5 ) v ( 5 ) ) = 10.625 BIM: bsv { 2, 6 } ( 6 ) = 1 2v ( 6 ) + 1 2 ( bsv { 2, 6 } ( 2, 6 ) v ( 2 ) ) = 29.375 The process ends here, because the values for individual agents are found: (12.5,49.375,10.625,-55,-29.375,-19.375). Note that only the first round uses BSVs in their classical sense, i.e., in the second and subsequent steps of the BIM, the total value to split is given by the previous BSV, instead of the value. Figure 4-1 and Figure 4-2 depict the entire cost allocation and coalition formation processes for this example.

69

However, there is no proof that the individual rationality condition expressed in Equation 4-10 implies individual rationality of single agents in C . Note that Equation 4-7 is always met when the grand coalition is formed, since the superadditive cover is also equal to the coalition value (cost) in that case. If it is the case that the grand coalition is not formed, Equation 4-7 is also met, because final coalitions are indifferent whether to team or stay solo: thus the maximum (minimum) possible value (cost) is obtained, and it is also equal to v ( 123456 ) . Note that in this particular case, Equation 4-9 is equal to zero. The reader can check this assertion for the cases: [{1-2-3-4-6},{5}] and [{1-2-3-6},{4},{5}]. Therefore, we can guarantee for a general environment that the proposed algorithm meets the SOF that any cooperative fair cost allocation game has to satisfy. For comparison with other definitions of SOFs, the reader is referred to Section 5.1.2. of [24]. It is also possible to determine coalition stability using the propensity to disrupt concept provided in [2]. Gately defined this concept as the ratio between how much a set of players in a coalition would lose if player i refused to cooperate to how much i would lose if she refused to cooperate. For a bilateral coalition, the ratio is equal to 1 for both players, as seen in (4-9). For any step of the algorithm, the players are fairly rewarded, since their respective propensities to disrupt are equal.

4.4. Simulation Results

We have run three simulations for the Garver 6 bus test case implementing the BSV recursive cost allocation procedure. In this Section, we explain in detail the first simulation, for which the final resulting coalition is the grand coalition: {1,2,3,4,5,6}, and the coalition creation order is as follows: [{1},{2-6},{3-5},{4}] in the first round, [{1},{2-3-5-6},{4}] in the second, [{1-2-3-5-6},{4}] in the third, and [{1-2-3-4-5-6}] in the fourth and last round. Once the grand coalition is created we follow our backward induction algorithm to 68

 where v ( N ) is the superadditive cover of a coalition: P v ( N ) = max v ( P j ) (4-8) P j = 1 That is, coalition P divides itself up into subcoalitions in such a way that the sum of the values of the subcoalitions is a maximum. Clearly, for superadditive games, the superadditive cover of a coalition is the value of the coalition. Equation 4-6 states that any negotiation group can command as a fair share at least the value it would obtain were it to bind itself to a coalition. Equation 4-7 defines the entity being divided into fair shares as the maximum joint payoff available in the game, i.e., the superadditive cover of the grand coalition. In our case, the backward induction algorithm to allocate values (costs) is more specific, because a coalition structure CS = { C 1, , C n } , and its sequence of bilateral assignments (of coalition founders) ce = ( ce CS ( C 1 ), , ce CS ( C n ) ), ce CS ( C i ) = 2 is given for each coalition formation round, i.e., the history of bilateral assignments for each coalition structure is known. In particular, our coalition formation algorithm allows only bilateral coalitions with individually rational founders in each round. Thus, given two players a and b as founders of C in some coalition structure ( CS, ce CS ) (with

ce CS ( C ) = { a, b } ) it holds that: 1 bsv { a, b } ( a ) v ( a ) = bsv { a, b } ( b ) v ( b ) = -- ( v ( a, b ) v ( a ) v ( b ) ) 0 2 (4-9)

It is guaranteed that at any step, agents a and b , as founders of the created coalition, are individually rational, and the subgame is superadditive, as shown in Equation 4-9. Thus, given a partition P = { P 1, P 2 } with a bilateral assignment ce , we have that: for each P j P , C k ce ( P j ) : bsv { P j } ( C k ) v ( C k ) (4-10)

67

4.3. Coalition Stability

Following the decentralized coalition formation algorithm shown in the previous Section, it remains to be proved that the resulting coalition(s) is (are) stable. To say that a coalition is stable means that none of its members desires to leave the group to obtain a better payoff. In our particular 6 bus example, there are only three sets of coalitions that can divide the total cost in an optimal way: 1) Grand coalition: [{1-2-3-4-5-6}], 2) [{1-2-3-4-6},{5}], and 3) [{1-2-3-6},{4},{5}] All of them divide a total value (cost) of (-)130, but the negotiation process in this game is not unique. The stability method proposed by Aumann and Myerson [63] is applicable for some cases; however it does not provide a general rule for stability. It is particularly valuable in cases where the games present specific properties, as shown in Chapter 5 of [63]. For a more general criterion, we use a cooperative standard of fairness (SOF), as presented in Chapter 5 of [24]. Quoting Kahan and Rapoport, a SOF for a game is a vector valued function with t elements = ( 1 ( P ), 2 ( P ), , t ( P ) ) , defined for each partition P = ( P 1, P 2, , P t ) of the n players into t negotiation groups ( 1 t n ) . P j is a nonempty negotiation group in the partition P , and j ( P ) designates the fair share of that group P j according to the rules generating the SOF. The SOF function ( P ) is assumed to satisfy two conditions: j ( P ) v ( P j ) for any P and all Pj P 1 ( P ) + 2 ( P ) + + t ( P ) = v ( N ) for all P (4-6) (4-7)

66

for each C coaltuple [ 1 ] BCL C do for each ( S, bsv C ( S ) ) coaltuple [ 2 ] do bcu ( S, bsv C ( S ) ) ; for each ( { a i }, u ) ARUL do if a i = a x then begin recutil:= u ; break; end; bcu(S: Coalition, bsv: real); begin if S > 1 then begin v ( S ) := bsv ; for each K F S do begin ru := bsv S ( K ) ; if K > 1 then bcu ( K, ru ) else ARUL := ARUL { ( K, ru ) } ; end end else ARUL := ARUL { ( S, bsv ) } ; end; Two examples with the 6 bus example are provided in the Simulation Results Section of this Chapter.

65

Coalition {5,6} {1,2,6} {1,3,5} {1,4,6} {1,5,6} {2,3,6} {2,4,6}

Cost 183 60 20 60 183 60 150

Coalition {2,3,4,6} {2,3,5,6} {3,4,5,6} {1,2,3,4,6} {1,2,3,5,6} {1,2,4,5,6} {2,3,4,5,6} {1,2,3,4,5,6}

Cost 120 100 161 90 80 272 160 130

Table 4-1. Coalition Expansion Costs This recursive division of Bilateral Shapley Values within each coalition can be written as follows: 1. BCL C := { ( C, { ( C 1, bsv Ck ( C 1 ) ), ( C 2, bsv Ck ( C 2 ) ) } ) ;C C } 2. F S := F S = ce C ( S ) = { S 1, S 2 } 3. ARUL := { ( { a i }, u ) ;u , a i A } where BCL is a set containing the history of coalition structures and monetary profits, F is the set of founders, and ARUL is the set of agents utility distribution. Agent a x of coalition entity C i in a bilateral coalition C k C, ce C ( C k ) = { C i, C j } receives a monetary reward ru x of the joint utility bsv Ck ( C i ) for C i : ru x := recutil ( C, BCL C, a x ) based on the functions recutil(C: Coalition Structure, BCL C : SetCoalTuple, a : Agent): real; begin ARUL := ; (4-5)

64

Starting from the grand coalition, we divide the team into the two founding members: [{12-3-5-6},{4}], and split the total value of -130. Let bsv { i, j } ( i ) be the value allocated to agent i using the Bilateral Shapley Value rationale: bsv { 12356, 4 } ( 2356 ) = 1 2 v ( 12356 ) + 1 2 ( v ( 12356, 4 ) v ( 4 ) ) bsv { 12356, 4 } ( 4 ) = 1 2 v ( 4 ) + 1 2 ( v ( 12356, 4 ) v ( 12356 ) ) (4-1) (4-2)

where v ( i ) is the value of coalition i , as per Table 4-1, when we reverse signs. Going backwards one more step, we find the following values for the next subdivision of [{1-23-5-6}] into their founders [{1}] and [{2-3-5-6}]: bsv { 1, 2356 } ( 1 ) = 1 2 v ( 1 ) + 1 2 ( bsv { 1, 2356 } ( 1, 2356 ) v ( 12356 ) ) bsv { 1, 2356 } ( 2356 ) = 1 2 v ( 2356 ) + 1 2 ( bsv { 1, 2356 } ( 1, 2356 ) v ( 1 ) ) (4-3) (4-4)

The process is followed until the values for individual agents are found: (12.5, -49.375, 10.625, -55, -29.375, -19.375), adding up to -130 total final value. Note that in the second and subsequent steps of the backward algorithm, the total value to split is given by the previous Bilateral Shapley Value, as shown in Equation 4-3 and Equation 4-4, by using bsv { 1, 2356 } ( 2356 ) instead of v ( 1, 2356 ) . This backward process can be used as a general calculation scheme for every agent utility in its coalition.

Coalition {2} {4} {5} {6} {2,6} {3,5} {4,6}

Cost 90 60 40 60 90 40 60 63

Coalition {2,5,6} {3,5,6} {4,5,6} {1,2,3,6} {1,2,4,6} {1,2,5,6} {1,4,5,6}

Cost 334 101 304 30 120 273 243

The solution that we suggest will make use of bilateral agreements based on Shapley values, that are beneficial to the individuals, but we will also consider the history of the coalition formation algorithm shown in the previous Chapter in order to reward or penalize players that benefit or harm the overall system. In our game, a valuable player will be rewarded accordingly to the benefit that he adds to the system and vice versa, a worthless player will be penalized. This Chapter is devoted to finding such an allocation scheme.

4.2. The Backward Induction Algorithm

Defining a coalition formation scheme is not enough to solve the transmission expansion game. Even if all agents agree on the final coalition arrangements, the issue of allocating the total cost among them remains yet unsolved. To address this issue we propose a solution ( C, ( x 1, , x n ) ) for a given cooperative game ( A, v ) in general environments1 using BSVs. Our method is based on a special algorithmic calculation of each agents utility x i ( i { 1, , m } ) which is not necessarily bsv , if a i C k , and C k is a founder of C in the coalition structure CS . In the following, we will denote coalitions with more than one agent as (multi-parties) players. To illustrate the algorithm let us return to our familiar 6 bus example test case. We know from the previous Chapter, that the grand coalition was formed in the first simulation, where we assumed that whenever an agent or multi-parties player was undecided, he tossed a fair coin to decide. The resulting final coalition for the first simulation was the grand coalition: {1,2,3,4,5,6}, with the following formation stages: [{1},{2-6},{3-5},{4}] in the first round, [{1},{2-3-5-6},{4}] in the second, [{1-2-3-5-6},{4}] in the third, and [{1-2-3-4-5-6}] in the fourth and last round. Now, we will follow a backward-induction algorithm method to calculate value (cost) allocations via BSVs.
1. General environments allow for both, superadditive as well as subadditive games. A game ( A, v ) is superadditive iff C = C 1 C 2 P ( A ) : v ( C ) v ( C 1 ) + v ( C 2 ) ; it is subadditive iff it is not superadditive.

62

Chapter 4 A Cost Allocation Method: Backward Induction Algorithm

4.1. Introduction
While cost allocation is an interesting problem, its relation to game theory is not obvious. What does the division of costs (benefits) has to do with game theory? The answer is that cost allocation is a kind of game in which costs (and benefits) are shared among different parts of an organization. The organization wants an allocation mechanism that is efficient, equitable, and provides appropriate incentives to its various parts. Cooperative game theory provides the tools for analyzing these issues. Moreover, cooperative game theory and cost allocation are closely intertwined in practice [60]. Some central ideas in cooperative game theory have been presented in Chapter 2: core, kernel, and Shapley value standing out as the most relevant solution concepts. However, new solutions are required in a decentralized environment where the players are independent decision makers whose decisions have important effects on the rest of them. A decentralized framework, where the entities cooperate with the net effect of all individual actions converging to the best overall result, is the answer. A Multiagent System (MAS) is our proposed solution, as introduced in Chapter 3, Section 3 of this dissertation. However, the answer is not obvious, as Billera et al. point out in [61]; they propose a unique solution, somewhat in the Shapley value fashion. We do not postulate such a solution in general, because the new power systems environment will not be centralized, and a more sophisticated solution is required to guarantee a global optimum with decentralized decision makers.

61

players attain via negotiation with BSVs. Furthermore, we also consider the players in a network structure, so that a link between some players is certainly easier to build than with other players. In other words, the grid and its players are intertwined, and one players decision affects the other ones, and so their willingness to build a link with this player. Our framework is the Multiagent System (MAS) environment, the coalitions are based on rules that are based on power system criteria, and the decentralization (and coordination) of all decisions fit the new deregulated environment in power systems planning.

60

if with 4: bsv { 12356, 4 } ( 12356 ) = 1 2 v ( 12356 ) + 1 2 ( v ( 12356 ) v ( 4 ) ) = 75 4: solo: v ( 4 ) = 60 ; if with 12356: bsv { 12356, 4 } ( 4 ) = 1 2 v ( 4 ) + 1 2 ( v ( 12356, 4 ) v ( 12356 ) ) = 55 Status: 12356 prefers 4; 4 prefers 12356. Coalitions after round 4: 12356 teams with 4. FINAL RESULT: Grand coalition 123456 is formed! A second simulation was run, since player 6 was indifferent to team with 2 or 4 at the beginning. The sequential process was: [1, 2, {3-5}, {4-6}] -> [{1-3-5}, {2-4-6}] -> [{12-3-4-5-6}]. Note that not all simulations drove the coalition to a grand coalition scheme; here is one example: [1, {2-6}, 3, 4, 5] -> [{1-2-6}, 3, 4, 5] -> [{1-2-3-6}, 4, 5]. This was a case where the sum of the costs of the final coalitions were also equal to the grand coalition total cost.

3.8. Discussion
On the whole, the theories on coalition formation work well, but for very specific examples. They all follow game theory prescriptions of rationality to some extent at least. Kahan and Rapoport [24] provided a critical review of ten experiments of three-person cooperative games played in characteristic function form. Interesting results are also provided by dynamic coalition formation models, as seen in Chapter 2, Section 8. In particular, Komoritas Equal Excess Theory is the only one that introduces dynamics to the game, unifying both coalition formation and cost allocation aspects in the same framework. In the light of these results, for the transmission expansion problem, we propose that the dynamics of the coalition formation must be determined by individual benefits that the 59

if with 35: bsv { 26, 35 } ( 26 ) = 1 2 v ( 26 ) + 1 2 ( v ( 26, 35 ) v ( 35 ) ) = 75 35: solo: v ( 35 ) = 40 ; if with 26: bsv { 26, 35 } ( 35 ) = 1 2 v ( 35 ) + 1 2 ( v ( 26, 35 ) v ( 26 ) ) = 25 ; if with 1: bsv { 1, 35 } ( 35 ) = 1 2 v ( 35 ) + 1 2 ( v ( 1, 35 ) v ( 1 ) ) = 30 4: solo: v ( 4 ) = 60 ; if with 26: bsv { 2, 46 } ( 4 ) = 1 2 v ( 4 ) + 1 2 ( v ( 2, 46 ) v ( 26 ) ) = 60 Status: 1 prefers 35; 26 is undecided between 1 and 35 (tossing a fair coin, he prefers 35); 35 prefers 26; 4 is undecided between solo and 26 (tossing a fair coin, he prefers 26). Coalitions after round 2: 26 teams with 35. ROUND 3: players: 1, 2356, and 4 1: solo: v ( 1 ) = 0 ; if with 2356: bsv { 1, 2356 } ( 1 ) = 1 2 v ( 1 ) + 1 2 ( v ( 1, 2356 ) v ( 2356 ) ) = 10 2356: solo: v ( 2356 ) = 100 ; if with 1: bsv { 1, 2356 } ( 2356 ) = 1 2 v ( 2356 ) + 1 2 ( v ( 1, 2356 ) v ( 1 ) ) = 90 4: solo: v ( 4 ) = 60 ; if with 2356: bsv { 2356, 4 } ( 4 ) = 1 2 v ( 4 ) + 1 2 ( v ( 23456 ) v ( 2356 ) ) = 60 Status: 1 prefers 2356; 2356 prefers 1; 4 is undecided between solo and 2356 (tossing a fair coin, he prefers 2356). Coalitions after round 3: 26 teams with 35. ROUND 4: players: 12356 and 4 12356: solo: v ( 12356 ) = 80 ; 58

ROUND 1: players: 1, 2, 3, 4, 5, and 6 1: no choice 2: solo: v ( 2 ) = 90 ; if with 6: bsv { 2, 6 } ( 2 ) = 1 2 v ( 2 ) + 1 2 ( v ( 2, 6 ) v ( 6 ) ) = 60 3: solo: v ( 3 ) = 0 ; if with 5: bsv { 3, 5 } ( 3 ) = 1 2 v ( 3 ) + 1 2 ( v ( 3, 5 ) v ( 5 ) ) = 0 4: solo: v ( 4 ) = 60 ; if with 6: bsv { 4, 6 } ( 4 ) = 1 2 v ( 4 ) + 1 2 ( v ( 4, 6 ) v ( 6 ) ) = 15 5: solo: v ( 5 ) = 40 ; if with 3: bsv { 3, 5 } ( 5 ) = 1 2 v ( 5 ) + 1 2 ( v ( 3, 5 ) v ( 3 ) ) = 40 ; if with 6: bsv { 5, 6 } ( 5 ) = 1 2 v ( 5 ) + 1 2 ( v ( 5, 6 ) v ( 6 ) ) = 81.5 6: solo: v ( 6 ) = 60 ; if with 2: bsv { 2, 6 } ( 6 ) = 1 2 v ( 6 ) + 1 2 ( v ( 2, 6 ) v ( 2 ) ) = 30 ; if with 4: bsv { 4, 6 } ( 6 ) = 1 2 v ( 6 ) + 1 2 ( v ( 4, 6 ) v ( 4 ) ) = 30 ; if with 5: bsv { 5, 6 } ( 6 ) = 1 2 v ( 6 ) + 1 2 ( v ( 5, 6 ) v ( 5 ) ) = 101.5 Status: 1 has no choice; 2 prefers 6; 3 is undecided between solo and 5 (tossing a fair coin, he prefers 5); 4 prefers 6; 5 is undecided between solo and 3 (tossing a fair coin, he prefers 3); 6 is undecided between 2 and 4 (tossing a fair coin, he prefers 2). Coalitions after round 1: 2 teams with 6; 3 teams with 5. ROUND 2: players: 1, 26, 35, and 4 1: solo: v ( 0 ) = 0 ; if with 26: bsv { 1, 26 } ( 1 ) = 1 2 v ( 1 ) + 1 2 ( v ( 1, 26 ) v ( 26 ) ) = 15 ; if with 35: bsv { 1, 35 } ( 1 ) = 1 2 v ( 1 ) + 1 2 ( v ( 1, 35 ) v ( 35 ) ) = 10 26: solo: v ( 26 ) = 90 ; if with 1: bsv { 1, 26 } ( 26 ) = 1 2 v ( 26 ) + 1 2 ( v ( 1, 26 ) v ( 1 ) ) = 75 ; if with 4: bsv { 26, 4 } ( 26 ) = 1 2 v ( 26 ) + 1 2 ( v ( 26, 4 ) v ( 4 ) ) = 90 ; 57

Coalition {2} {4} {5} {6} {2,6} {3,5} {4,6} {5,6} {1,2,6} {1,3,5} {1,4,6} {1,5,6} {2,3,6} {2,4,6}

Cost 90 60 40 60 90 40 60 183 60 20 60 183 60 150

Coalition {2,5,6} {3,5,6} {4,5,6} {1,2,3,6} {1,2,4,6} {1,2,5,6} {1,4,5,6} {2,3,4,6} {2,3,5,6} {3,4,5,6} {1,2,3,4,6} {1,2,3,5,6} {1,2,4,5,6} {2,3,4,5,6} {1,2,3,4,5,6}

Cost 334 101 304 30 120 273 243 120 100 161 90 80 272 160 130

Table 3-2. Coalition Expansion Costs1

The first simulation starts with the individual agents 1 to 6 creating lists of preferences. We assume for the first two simulations that whenever a tie occurs between going solo or teaming, teaming is preferred. The final resulting coalition is {1,2,3,4,5,6}, and the order of coalition formation is the following: [1, {2-6}, {3-5}, 4] in the first round, [1, {2-3-56}, 4] in the second, [{1-2-3-5-6}, 4] in the third, and [{1-2-3-4-5-6}] in the fourth and last round. The step-by-step run of the algorithm is as follows:
1. E.g., the value (cost) for the grand coalition is -130 (130). 1. If a coalition is not included in Table 3-2, it means that its cost is zero.

56

Figure 3-4 : Overall Coalition Formation Framework

Finally, even if common resources are shared (the lines), it does not mean that all agents can access them. In the case of multi-parties players, for example, load at bus 2 and generation at bus 6 forming a single agent, we disregard all lines connecting agent 2-6 to other agents. For single agent expansion plans, the agent pays in full for all the lines that it needs to meet its own load at a minimum cost.

3.7. Simulation Results

We have run a coalition formation simulation for the Garver 6 Bus test case implementing the Bilateral Negotiation algorithm. Cost functions, as per Section 3.4, are shown in Table 3-2. Note that when using SVs and BSVs, values are negative to reflect positive expansion costs in monetary units. Note that the respective coalition values of this (subadditive) game are negative, reflecting the utility of a coalition in a cost-oriented view1.

55

tions. These lists will change whenever step 3.6.3. is called again, with new multi-parties players acting as agents1.

3.6.4. Bilateral Negotiation Phase

Each agent looks at the head of his ordered list, and extends an offer to the preferred partner. The offer consists of sending the partners BSV: the value that he would attain for collaborating with the sender. If it happens that the sender receives also a message from the preferred partner, and they both find it is beneficial to join, they do. They create a multiparties player that will behave like one agent from then on. Every other agent is also informed, for him to erase the members of the multi-parties players from his own preference list. The coalition formation process is repeated by all agents and multi-parties players, starting from Step 3.6.2, until no more coalitions are possible. If no coalition is possible at one particular step, the agents look at the second best partner; if still not possible, to the third, etc., until reaching the end of the list. Figure 3-4 shows the overall coalition formation framework. There are several features of this negotiation algorithm that are only relevant to expansion planning. First, this is a general (non superadditive) environment, thus the grand coalition will not necessarily be formed. Second, previous negotiation algorithms used a utilitarian coalition building scheme, where the agents have to satisfy their tasks via cooperation, thus increasing their benefits. The utilitarian coalition formation in the transmission expansion domain is done in a cost-oriented view2.

1. This means that the agents in the coalition will vote for a representative agent that will be the spokesperson for this coalition in the future. 2. Related work can be found in the DAI area for task-oriented domains with cost functions by Rosenschein and Zlotkin.

54

that agents 1 and 3 are self-sufficient, because the load is met by the existing bus generation. However, agents 2, 4, and 5 need to use extra lines to meet their own demand. Finally, agent 6 needs to be attached to the network, not to become isolated. These facts are reflected in their self-values. In the case of different initial settings, agents 3 and 5 being a single initial agent for instance, costs would change, reflecting different initial conditions.

3.6.2. Communication Phase

Once each agent has calculated its own self-cost, it is time to determine the joint cost that he will have when cooperating with another agent. Unfortunately, an agent is not necessarily aware of the environment surrounding him, thus a coordinator is needed to gather this information. This is certainly the case of transmission expansion planning, where the entire network is not known completely by any player. In order to calculate the BSVs, the agents need to send all their proposed line addition(s) to this central figure, and then, receive the adequate number of new lines for requested coalitions, via messages sent by the coordinator to all of them. It is possible that two agents reach an agreement that is satisfactory to both of them, but detrimental to the security of the system. That is why the role of an independent coordinator is needed to check network reliability and quality of service. Usually, a power flow subject to security constraints should be enough. On the other hand, agents freely exchange self-costs with each other.

3.6.3. BSV Calculation Phase

Now the agents know their own self-values, and every possible cost with other coalition partners via Step 3.6.2. After getting these messages from the coordinator, they proceed to calculate BSVs if teaming with another agent. Then, the agents determine individually rational1 lists of preferred agents: ordered list of local agentss BSVs for two-entity coali1. Individual rationality means that the agent wants to have a new value that is, at least, as good as the one that he could attain alone.

53

lition formation that they provided is also useful in the power transmission planning environment. Thus, to formulate our problem using BSVs, let us define our framework first. Let CS P ( A ) be a coalition structure on a given set of agents A = { a 1, , a m } , where C i C j A is a (bilateral) coalition of disjoint (n-agent) coalitions C i and C j ( n 0 ) . The Bilateral Shapley Value for some coalition C i in a bilateral coalition C is 1 1 defined as bsv { Ci, Cj } ( C i ) := -- v ( C i ) + -- ( v ( C ) v ( C j ) ) . Both coalitions C i and C j are 2 2 called founders of C , and v ( C ) denotes the self-value of coalition C 1. It can be seen that the founders will get half of their local contribution, and the other half stemming from cooperative work with the other entity. The second term of the BSV expression reflects the strength of each agent concerning his contribution, therefore avoiding the free-rider2 problem, so common in transmission expansion value allocation schemes. The BSV is a particular case of the Shapley value concept, because they create a fair distribution of resources among two agents only. Our coalition formation method is based on the approach followed by Klusch and Shehory in [7]. A summary of the steps in the coalition formation process is given in the following subsections.

3.6.1. Self-Calculation Phase

Each individual agent gathers information to determine its self-cost. Calculation of the self-cost determines the monetary cost of line expansion for individual agents, following the three axioms from the previous Section. It is possible that some of the players are unwilling to use new lines on their own, and this fact is reflected by a cost of zero. For the 6 Bus Garver example, where each individual agent is attached to a bus, we can observe

1. Note that bsv { C, } ( C )

= v ( C ) , and v ( ) := 0 .

2. The free-rider concept addresses the issue of new agents that take advantage of the work done by the existing ones without paying them any compensation.

52

ley value is a cooperative game theory solution concept that calculates a fair division of a common utility (money, resources, etc.) among the members of a coalition. It can be defined as the weighted average of marginal contributions of a member to all possible coalitions in which it may participate. It assumes that the game is superadditive (see Section 2.3), and that the grand coalition is formed. The reader is referred to Section 2.6 and [33, 58] for a detailed explanation of necessary conditions to calculate a meaningful Shapley value. The mathematical expression of the Shapley value, i , is given as follows:
n

1 i = -n where i = player s = coalition of players q = size of a coalition n = total number of players

--------- [ v ( s ) v ( s i ) ] c(s)
q=1 is

(3-4)

v(q) = characteristic function (cost savings) associated with coalition q. c(q) = number of coalitions of size q containing the designated player i, given by, ( n 1 )! c ( q ) = ------------------------------------( n q )! ( q 1 )! (3-5)

In order to avoid the exponential complexity of a Shapley value calculation, Ketchpel introduced the so-called Bilateral Shapley Value (BSV) [5]1. Klusch and Shehory [6, 7, 62] adapted this approach for a completely decentralized and bilateral negotiation process among rational information agents using these values. In particular, the algorithm for coa1. Similar work has been done by Kraus and Shehory [59]. In contrast to our work, they did not provide a solution for cooperative games and a given coalition structure with other than 2-agents coalitions.

51

UNITS 120 240 240

4
180

2
240 100

UNITS 120 240 240

4
180

Figure 3-3 : Two Examples of Autonomous Coalitions

For the 6 bus case, the set of feasible autonomous coalitions is as follows: 1 agent: {2}, {4}, {5}, and {6} 2 agents: {2,6}, {3,5}, {4,6}, and {5,6} 3 agents: {1,2,6}, {1,3,5}, {1,4,6}, {1,5,6}, {2,3,6}, {2,4,6}, {2,5,6}, {3,5,6}, and {4,5,6} 4 agents: {1,2,3,6}, {1,2,4,6}, {1,2,5,6}, {1,4,5,6}, {2,3,4,6}, {2,3,5,6}, and {3,4,5,6} 5 agents: {1,2,3,4,6}, {1,2,3,5,6}, {1,2,4,5,6}, and {2,3,4,5,6} grand coalition: {1,2,3,4,5,6}

3.6. Four Phases of the Coalition Formation Algorithm

The use of decision techniques to analyze DAI problems, like the one posed in the previous Section, started in the present decade. However, the Shapley value (SV) is known since the seminal work by Lloyd Shapley (see Chapter 2, Section 6, and [33]). The Shap50

although these would be the atomic agents -the minimal ones. We also assume that any set of generation units and loads attached to the same bus belong to a single agent. For our familiar 6 bus example, we have a maximum of 6 agents, corresponding to the 6 generation/load entities attached to the buses. A coalition of agents is a set of agents consisting of: at least one generator, one load, and one transmission line. There are three axioms that a coalition has to satisfy: 1.- Generator(s) must meet the demand, i.e., the total generator output in the coalition must be greater than or equal to the load. 2.- New and existing line(s) thermal limits can not be exceeded when running a power flow for the new coalition. 3.- There must be one or more transmission lines, either existing or possible candidates, connecting all the buses in the coalition. These three axioms create what we call autonomous coalitions, because they can try their own expansion plans without having to necessarily negotiate with any other similar entity. Figure 3-3 shows some examples of feasible autonomous coalitions for the 6 bus case. It can be anticipated from the set of axioms that some of the possible coalitions may be ruled out. In particular, if there is a single bus in the system, with either a generator, a load, or both, it can not be considered a coalition, because it lacks at least one transmission line. Nevertheless, we will assume that single agents will pay for line investments if necessary, so that it is not necessary for them to own transmission lines at the beginning of the expansion game. Also, if two buses meet the first axiom, and there are no line candidates that can connect them, the second axiom is violated. Finally, if not all buses are in the coalition connected to each other, the third axiom is violated.

49

tion rescheduling (generator outputs 1, 3, and 6 are 50, 165, and 545 MW respectively). The optimal solution has a cost of 200 monetary units, and circuit additions are: n 26 = 4 circuits, n 35 = 1 circuit, and n 46 = 2 circuits. Should we allow generation rescheduling, i.e. the real power generation ranging from 0 to the maximum generation available (150, 360, and 600 MW respectively), the optimal cost would be 1301 monetary units, and circuit additions would be: n 26 = 3 circuits, and n 35 = 2 circuits. In the remainder of the thesis we will assume that generation rescheduling is possible, disregarding dynamic expansion plans for more than one year shot, as is the case in some models [56, 57]. Note that these solutions correspond to what is defined in cooperative game theory as grand coalition structure. In the next Section we will analyze the rules of the transmission expansion game and the construction of feasible coalitions.

3.5. Coalitions in Expansion Planning

In this Section we are going to define what is the game that is played in a transmission expansion process (from a cooperative game theory standpoint), who are the players (agents2), and what is a coalition in expansion planning. The purpose of the game is the expansion of the transmission network, with the minimum possible cost, subject to constraints given in (3-1), (3-2), and (3-3), and with a fair allocation of the total cost among the agents. An agent in the game can be either a generator, a load, or an independent third party (for example, an independent company who owns transmission lines) that is physically attached to a bus. A typical agent in this context is regarded as an independent entity: a customer or group of customer loads, a generator or a set of generators, or a combination of both. For simplicity, we do not consider for now fractional bus loads or generators,
1. Note that 130 is not the best possible result; the optimum value is 110, as shown in [54]. This deficiency is due to the simplicity of the model, that does not explore all possible line combinations. 2. We will follow a Multi-Agent System (MAS) terminology, where players are called agents.

48

into account the effect of the power transmission cost, the line candidate with the largest power flow is most effective in the expanded network1.

240 80

Bus 5
100

Bus 1

4 UNITS 30 30 30 60

100 4 UNITS 60 60 120 120 40 80 100 100

Bus 3

Bus 2
240 100

3 UNITS 120 240 240

Bus 6
180

Bus 4

LEGEND Existing line Possible line addition Figure 3-2 : Six Bus Problem

We have coded the transmission expansion problem in the Scilab computing environment [55], and the solution obtained is the same as the one provided by Garver without genera1. See [53], pp. 394-400, for a very detailed explanation.

47

1 2 min -- c j P j 2
j=1

(3-1)

B + K P D = P B L A P L

(3-2) (3-3)

Bus: from/to 1/2 1/4 1/5 2/3 2/4 2/6 3/5 4/6 5/6

Cost (units) 40 60 20 20 40 30 20 30 61

Suscept. (1/ ) 2.50 1.67 5.00 5.00 2.50 3.33 5.00 3.33 1.64

Capacity (MW) 100 80 100 100 100 100 100 100 78

Table 3-1. 6 Bus System Circuit Data

where c j is the cost of adding line j to the network, P j is the active power (in p.u.) flowing through the added line j , M is the number of possible new lines, B is the matrix whose elements are the imaginary parts of the nodal admittance matrix of the existing network, is the phase angle vector, K
T

is the transpose of the node-branch connection

matrix, P D is the flow vector for possible lines, P is the nodal injection power for the overall network, BL is a diagonal matrix whose elements are branch admittances, and P L is the branch active power vector. The minimization algorithm is run recursively until there are no overloads in the system. Since the objective function, Equation 3-1, has taken 46

Finally, Shebl [51] is using a genetic algorithm to simulate combinations of simple strategies for a small number of agents that act as traders in a buy-sell game for bulk power transactions. He is able to model power pool arrangements, not only bilateral trading. Currently, he is extending his trading model to include futures trading and retail as well as wholesale contracts.

3.4. Simplified Network Expansion Model

The objective of a mathematical model of a transmission expansion planning problem is minimizing the capital and operating costs associated with the system expansion over the planning horizon. The constraints associated with this model are the physical and economical constraints that are important when attempting to expand a utility system at minimum cost, and yet meet all economic and demand restrictions that are placed upon the system. To formulate our problem, we will follow the simple model that Garver [14] used to solve a six bus (a bus is equivalent to a node in communication networks) system, as shown in Figure 3-2. Note that the dotted lines represent possible line additions, and the straight lines are existing lines connecting buses. The buses in the network are numbered from 1 to 6: buses 1 and 3 have both generation and load supplies; 2, 4, and 5 are pure loads, and 6 is a new generation bus that needs to be connected to the network. The generation upper limits for buses 1, 3, and 6, and line flows upper limits for existing lines are also shown. The reader is referred to [14, 53] for further details. When ranking possible additions to the system, we follow the heuristic approach suggested by [18, 53], such that the original linear programming problem is transformed into a quadratic problem subject to linear constraints. The general formulation of the expansion problem can be expressed as:

45

the dynamics of the coordination and communication, it is a mere snapshot of a MAS. As we can see in the figure, first the agents develop their own plans autonomously. Then, they transfer their individual plans to each other: either via a coordination agent or a blackboard system. As a result of the coordination process, the individual plans are reconciled; each agent adapts its own plan because of the existence of other agents with their own plans. The result can be seen as a set of plans being integrated into one global plan; although individual plans still exist. Several software packages have been developed to simulate MASs. In particular, there are two that deserve our attention. IDEAS [6] is an interactive development environment for the specification and simulation of agent systems that is currently applied to simulate coalition formation in transmission planning. The other package is called SWARM [50]. SWARM is a toolkit for the study of complex adaptive systems using MAS discrete simulation. The basic unit of a SWARM simulation is an agent. Simulations consist of groups of many interacting agents, called swarms, which may, themselves, be grouped into more comprehensive swarms. The SWARM software package is currently used in a power systems research project initiated by Ramesh in 1996, as cited in Wildberger [51]. Ramesh is investigating a multiagent approach to contingency analysis. He is building individual agent solvers to solve the problem (still modeled as constrained nonlinear flow); each of the agents attacks the solution in a different way. He expects that cooperation and competition will achieve faster convergence than an agent alone. Wildberger [52] suggests not only using the MAS model to solve specific power system problems, but also to simulate the entire North American power grid. He has two purposes: simulate the network as a MAS for real-time distributed control, and perform what-if studies and computer experiments to get insight into the evolution of the power industry under various forms of organization, including various degrees of deregulation, competition, and unbundling of services. 44

3) Coordinating Plans of Autonomous Agents We will assume that the agents are intelligent, autonomous (each agent wants to solve its individual problem first) problem solvers. An autonomous agent has its own goals, intentions, capabilities and knowledge. The agents will also be able to incorporate the actions of other agents as part of their plans (multiagent planning). They will broadcast their plans (always telling the truth) before execution in order to allow other agents greater access to their anticipated future behavior.

AGENTS make individual, partial PLANS

negative and positive relationships Reaction/Proposal Communication/negotiation Plan modification

until free of conflicts and chances for beneficial combinations until consensus

Coordinated Plans

Figure 3-1 : Coordinating Plans for Agents

The scheme of coordinating individual plans in a MAS is illustrated in Figure 3-1, as shown in von Martials book [49]. This figure is somewhat simplified, because it neglects 43

transmission investment requirements. Under some simplifying assumptions, their scheme reduces to a modified MW-mile rule. Tsukamoto et al. [3] have proposed a methodology to allocate the cost of transmission network facilities if there are wheeling transactions according to the transmission usage pattern. They incorporate MW-mile method considering economies of scale.

3.3. Multiagent Environments in Power Systems

Any intelligent behavior is supposed to have the ability to coordinate and communicate. Coordination is a way of adapting to the environment. In the area of Distributed Artificial Intelligence (DAI), there is a branch called Multiagent Planning that studies intelligent behavior of autonomous entities called agents within specific environments. An agent is a fuzzy notion that can designate either a physical entity (human, computer, etc.) or a formal entity (process, program). A Multiagent System (MAS) is a structure given by an environment together with a set of artificial agents able to act on an environment and possibly to collaborate with each other and/or external agents. In any multiagent environment, the agents coordinate by modifying their intentions; the reason to coordinate are the intentions of other agents. Communication is the way by which information relevant for coordination is exchanged. The agent communicates their intentions, which then will be negotiated among them. The agents can interact with each other at two different levels: indirectly through the plans they intend to execute, and directly by communicating with each other. As a consequence of this, we are mainly concerned with (see [49]): 1) Planning in Distributed Systems 2) Communication and Negotiation among Autonomous Agents: either to communicate plans, or to reach agreements via negotiation.

3. The MW-mile method allocates costs of transmission plant according to the product of line length and MW flow, thereby incorporating distance in the rate.

42

In California, the rules for expansion of the grid are structured around the following broad points (see the CPUC homepage at http://www.cpuc.ca.gov): - The Independent System Operator (ISO) should not build or own transmission facilities, to assure its independence. He should not take a financial interest, or be part of generation, transmission or distribution. Thus, the owner of the transmission grid has the ultimate obligation to build. - The rules for transmission system expansion will depend upon whether the expansion project is less expensive than the cost of other alternatives, or, alternatively, if it is driven solely by reliability needs. In the first case, a marginal cost-based usage charge resulting from congestion will send appropriate signals for transmission expansion, enabling the expansion that is economically beneficial. In the latter case, the Transmission Owner (TO) is obligated to build if market participants decide it is justified and are willing to pay. To implement these broad principles, several procedures have been proposed: - Determination of need of expansion, either for economic or reliability reasons - State Approval and RTGs coordination - Obligation to Build - Cost Recovery Approval 7) Cost Allocation and Recovery: The question of how to allocate line investment costs among the participants in a fair way is still open. Several allocation rules have been proposed in the literature [47], such as postage stamp1, contract path2, short term marginal costs, long term marginal costs, and MW-mile3. In particular, Marangon Lima et al. [48] have proposed that each participant share has to be proportional to its impact on system
1. The postage stamp rates are a flat amount per kW. 2. The contract path method is based upon the assumption that a wheeling transaction is confined to flow along a specified electrically continuous path through the wheeling companys transmission system.

41

6) Responsibilities and Obligations: Under the current regulatory environment, building of the transmission facilities is the obligation of utilities that provide electricity to customers in that region. In the future, RTGs will be set up to determine the best alternatives for transmission additions, but this issue is still under debate. Some proponents of regulatory reforms, like Smith et al. [44, 45], suggest that the transmission network would be owned jointly by retail power merchants, commercial and industrial customers, and, perhaps, by the generating companies (case of New Zealand). The network as a joint venture would be an operating company, run as shared resource cost center, not as a profit center. It would be constituted as a competitively ruled property right system defined by the government, and not as ordinary shared ownership corporation. The property right rules would have the objective of providing incentives for prudent investment, maintaining competition as follows. Capital and maintenance cost for the transmission system would be shared by the owners in proportion to their installed capacities to withdraw power from, or to inject into, the network. Large customers, consortia of small customers, and new generators would also have access to the grid, subject to the payment of their direct connection costs and their share of grid capital and maintenance costs. Capacity expansion of any part of the grid would be the responsibility of any user or consortium of users willing to make the investment. In turn, the user(s) would obtain rights to the increased capacities made possible by the expansion. Users who are not part of the capacity expansion will have no rights to block the expansion or demand compensation if the result is to shift the supply or demand for power in favor of others. But they will be free to join the joint venture, to share the creation of more favorably located new grid rights. Network capacity rights can be brought, sold, rented or leased subject only to antitrust laws that apply to any other industry. Smith et al. have even run simulations of their own auction algorithm [45]; some other authors, like Wilson, propose a different auction mechanism [46].

40

2) Externalities: externalities occur when the actions of one group of economic agents have impacts on parties who are not directly participating in the given activity and who are not compensated for its effects. They can be negative, where the impacts upon involved third parties result in higher costs and losses. One example of negative externality is the cost increase created by unintended power flows unrelated to a contract path. Externalities can also be positive, where a benefit is produced for parties not involved in a particular transaction. In a similar fashion, a synergy can be defined as the non-linear effect of two or more projects or factors, where the sum of their benefits considered separately is not equal to the benefits of all projects considered together. 3) Economies of scale: an economy of scale describes situations where quantity counts: where the average cost of producing an item falls as the number produced increases. The more of an item that a company makes, the less each item costs to make on average. One typical example of economies of scale is a new projected transmission line triggered by a specific generation project. Since the incremental costs of additional transmission capacity are low, it may make economic sense to invest in additional capacity beyond the current need of the specific project. 4) Lumpiness of investment: Traditionally, line investments have not been systematically applied to the overall transmission network. In the past, certain areas of the network were upgraded by the line owners with an absolute control over time and location of the investment. However, in a future open access environment, investments will be more complex: line ownership issues and investment side effects will arise. 5) Reliability under TOA: Utilities will not be able to build facilities using the same criteria for wheeling transactions as has been done before. With TOA, more simultaneous transactions are expected to be going on across the transmission network. New techniques must be developed to evaluate simultaneous transfer and to measure its impact on the reliability of the entire grid. 39

casted competitive electric utility industry, the diverse parties using the transmission system will be subject to new challenges. Even new figures, like the power marketer, an entity that will make money on transmission transactions, guaranteeing security of the grid, are expected to appear. The new competitive environment will have generators looking for their own individual economic interests, while sharing an open access network. The development of transmission will be seen differently by each of the participants, and while a particular capacity expansion may be of interest to one party, there will be others that may be against it. To alleviate that problem, setting up of Regional Transmission Groups (RTGs) and sharing of information among not only other utilities, but also other interest groups, like Independent Power Producers (IPPs), Exempt Wholesale Generators (EWGs) or customer groups, has been suggested. At the same time, Transmission Open Access (TOA) is already changing the traditional concepts and approaches to providing electric service. TOA refers to the regulatory construct (rights, obligations, operational procedures, economic conditions) enabling two or more parties to use a transmission network that belongs totally or in part to another party or parties. As a result of all these issues, conflicts over transmission planning are due to happen, and they can be typified according to their origin as follows [42, 43]: 1) Radial and network line expansion: radial connections involve the initial connection of two participants where there was no prior interconnection, or the strengthening of a corridor between two participants. Network projects involve the reinforcement of the power grid. Capacity is usually added over an extended period in complex patterns between many individual pairs of nodes in the network. A line can have both radial and network characteristics, and that may complicate the classification of any expansion projects.

38

The commission will also adopt the establishment of a wholesale pool called the Power Exchange (PX). This PX, and independent entity separate from the ISO, will provide a market for electric power with hourly and half-hourly prices published to electric consumers and other participants such as investors and power marketers. This marketplace will be reliable, fast and based upon simple trading algorithms, but recognizing security constraints. Purchasing from and selling to the PX will be voluntary except for investorowned utilities who will bid all their generation output to the PX. In the new market organization, California three largest IOUs are to be divided into affiliated generation companies (gencos), which will be unregulated, and distribution companies (distcos) that will remain regulated. Ownership of transmission assets will pass to the distribution companies, although control of these assets will be held by the ISO. The gencos will be required to sell, and distcos required to purchase, all of their power through the PX. Any other power producers or customers may also participate in the PX on a voluntary basis. The PX is considered one of potentially many schedule coordinators compiling load balanced power transactions that would then be submitted to the ISO. Eventually, the commission foresees the figure of an Independent System Planner (ISP) that will effect transmission enhancements for the good of the system. As wheeling and other forms of competition arise, however, transmission planning decisions, access policies, and prices for transmission services will be subject to more scrutiny both by regulators and by competing parties vying for access to limited capacity. Therefore, conflicts over transmission planning will arise, and they are the subject of our next Section.

3.2. Transmission Planning under the Open Access Paradigm

Traditionally, transmission system planning has been performed by the vertically integrated firm, with limited regulatory review of transmission capacity additions. In the fore37

a problem. The CMLT model defines the notion of multilateral trade as a generalization of a bilateral trade. A multilateral trade is a trade involving two or more parties in which the sum of generation minus the sum of consumption losses is equal to its share of losses. The party that arranges the trade is called a broker. The broker may be a generator or consumer involved in the trade but may also be an unrelated third party. But, in order to relieve the congestion or to ensure security of operation, it is essential to have coordinated trades involving three or more parties [10]. Wu et al. have proved in [10] that even without a central authority, participants in the coordinated multilateral trade can reach either an optimal point or a trade that is feasible and profitable. In the United States, and, in particular, California, the new market structure has been subject to some preliminary ruling policies, as can be seen in California Public Utilities Decision 95-12-063 of Dec. 20, 1995, and in the modifications specified in D.96-01-009, of Jan. 10, 19961. Under the new plan, the investor-owned electric utility industry in California will be restructured to allow for wholesale and retail competition beginning in 1998. An independent system operator will operate the transmission systems that at present are owned and operated by the three largest investor owned utilities (IOUs) in the state: PG&E, Southern California Edison Co., and San Diego Gas & Electric Co. The new regulation foresees that the ISO will act as an independent, statewide transmission system operator that will control and operate the states transmission system. The ISO is supposed to ensure the feasibility of all the aggregated scheduled transactions and, if necessary, ration transmission access in a non-discriminatory way. Although the utilities will retain ownership of their transmission facilities, the ISO will have operational control of the facilities.

1. For more information regarding the basic California plan, see the California Public Utilities Commission (CPUC) homepage at http://www.cpuc.ca.gov.

36

In the poolco model, the ISO performs all the system coordination functions plus the operation of a centralized spot market for electricity. The claim of the poolco proponents is that a competitive market equilibrium can be achieved, but it has been subject to controversy. The pool-based transmission bidding model explicitly incorporates the network externality impacts into the competitive trading mechanism. In this model, Chao and Peck have designed a market mechanism for electric power transmission that consists of tradable transmission capacity rights and a trading rule that induces a socially optimal allocation. Their approach allocates transmission rights based on contract paths, but sets explicit trading rules which recognize the externality impacts. According to this model, a transmission capacity right entitles its owner to the right to send a unit of power through a specific transmission line in an specific direction, and to collect economic rents associated with the transmission line according to the trading rule. A fixed set of transmission capacity rights are issued for each link (i,j), and these rights are tradable. This entitlement mechanism only needs to define transactions between the base node n and every other node in the network. The trading rule consists of a set of coefficients B ( t ) = { ij ( t ) 1 i, j, k n } , where the value ij represents the quantity of transmission capacity rights on line (i,j) that a trader needs to acquire in order to inject one additional unit of power at node k in the network, and deliver ( 1 k ( t ) ) unit of power at base node n1. The dynamic trading process is (Lyapunov) stable and converges to a competitive equilibrium. The coordinated multilateral trade model (CMLT) completely separates the power system operation from the trading market functions. Economic efficiency is obtained through distributed decisions, and neither centralization (poolco) nor feasibility (bilateral trades) are
k k

1. Let k ( t ) describe the system marginal transmission loss attributable to the injection of power at node k.

35

responsible for ensuring that schedules for use of the transmission system are feasible, balancing electricity demand and supply, operating the transmission system in real time, and ensuring that all standards of transmission service are satisfied. The ISO should not have any financial interest in the source of generation in order to guarantee open access to the transmission grid. This method provides economic incentives to find the most economic arrangements but its fundamental flaw is the lack of coordination among the independent trades; this can lead to a violation of transmission network constraints [10]. Facing that problem, the proponents of bilateral trading propose to grant the ISO with the authority to determine the safe level of generation and the power to curtail contracts. The poolco model coordinates all electricity transactions in a single official spot market called the poolco. Generators sell power in this spot market1 and distribution companies purchase it from this market. All traders submit hourly bids for marginal cost of supply or marginal value of demand2 and the ISO (the poolco also performs the functions of an ISO) dispatches the market so as to minimize the cost of electricity supply, as reflected by the bids, subject to security constraints, and sets nodal prices at the marginal bid price for generation under optimal dispatch. To hedge fluctuations in the nodal price differences, the poolco model defines Transmission Congestion Contracts (TCCs): contracts that pay the right holder the price difference between the nodes specified by that right. Similarly, Contracts for Differences (CFD) enable buyers and sellers to enter into a financial bilateral contract that mitigates the risk of price variation. CFDs are financial contracts where sellers compensate buyers when the spot price exceeds the contract price, and buyers compensate sellers when the spot price is below the contract price.

1. A spot market is a market where sellers have their commodities on hand, and the goods are delivered immediately: on the spot. 2. [41] is an excellent reference to understand spot pricing in electric markets.

34

generators look for their own individual economic interests, while sharing an open access network, is very different. The development of transmission is seen differently by each of the participants, and while a particular capacity expansion may be of interest to one party, they will be others that may be against it [37]. Transmission planning in a competitive economic environment is clearly an emerging complex issue [42]. Because electric utilities are interconnected via the transmission network, together they can provide improved reliability and lower overall generation costs. With the current thrust towards a competitive generation market with new independent market entrants, the correct pricing of transmission is crucial in providing the right signals to the marketplace. Pioneering work on transmission pricing was done by Schweppe et al. Their conclusion was that the short term value (i.e. price) of transmission services across a line is the difference of spot prices between the two nodes connected by that line [41]. The result illustrated that alternative geographic-based pricing methodologies such as the postage stamp rates1 do not provide the correct market signals. Among the many proposals that have been put forward, aiming at a new transmission pricing scheme, four models have emerged as possible new electric market structures: bilateral contracting [38], poolco [13, 39], pool-based transmission bidding [12], and coordinated multilateral trading [10]. Whether one or a mix of these market models will prevail as the future model is still uncertain. Bilateral trading model is based on the principle that free market competition is a route to economic efficiency. In this model, suppliers and consumers independently arrange contracts setting the amount of generation and consumption under specific supply contracts. The model introduces the key concept of Independent System Operator (ISO). The ISO is
1. The postage stamp rate method assumes that the entire transmission system is used in a wheeling transaction, irrespective of the actual transmission facilities that carry the wheel.

33

Chapter 3 A Coalition Formation Algorithm in Transmission Planning

3.1. New Policies in a Deregulated Environment
The electricity industry is undergoing a deregulation process where the next regulatory policies and industry future structure are still under debate1. From a traditional regulated monopoly, the electric utility industry is shifting to a newer model, where competition is allowed. Unbundling generation, transmission, and distribution for effective and efficient competition, aims to break the current monopoly, but the way in which the electricity system blocks (generators, separate grid owners, jointly owned power pool, and consumers) will interact with each other is still under discussion. There are three main reasons that are largely responsible for the difficulties of implementing competition [40]: Unlike other network industries, such as natural gas and telecommunications, the flows of electricity across the network of interconnected power lines cannot be directed Electricity is a unique commodity in that it must be produced largely upon demand so that blackouts dont occur Development of efficient markets for competitive power will require regulatory reforms and coordination by both state and federal regulatory agencies In the traditional vertically integrated and regulated electric utility, generation and transmission expansion plans are coordinated so that the network does not restrict optimal dispatch, the network itself is not overloaded, and specific desirable technical conditions are met. The environment in an increasing competitive deregulated electrical sector, where

1. [40] is an excellent introduction to the new deregulated environment.

32

Transfer schemes cannot be regarded as complete theories of coalition formation for nperson games in characteristic form, but they provide some insight of the dynamics of the bargaining process. One shortcoming of this approach is the fact that transfer schemes proceed from payoff vector to payoff vector within the same coalition structure, and that seriously limits the practical applicability of these techniques.

2.9. Summary
We have introduced the basic tools of n-person cooperative game theory throughout this Chapter. The terms payoff vector, coalition structure, characteristic function and imputation have been defined and applied in static solution concepts such as: core, stable set, bargaining set, kernel, nucleolus and Shapley value. Towards finding a general theory of coalition formation, we have presented the classical equal excess theory model, and several transfer schemes for the bargaining set, kernel and core. In the next two chapters we will combine an innovative coalition formation scheme (borrowed from Distributed Artificial Intelligence) with a new recursive cost allocation implementation, and we will show its application for Power Systems Transmission Planning.

31

excesses -see (2-17)- and maximum surpluses -see (2-18)-. If s kl is the maximum surplus of player k over player l , then a demand function can be defined as: d kl = min [ ( s kl s lk ) 2, x l ] , if s kl > s lk , d kl = 0 ,otherwise where k and l are members of the same coalition. To insure convergence, the transfer is limited by the demand, i.e., 0 < d k . The transfer scheme for the core is due to Wu (1977) [36], and it is a variation of the technique used by Stearns in 1968. She proved that a sequence of payoff vectors will always converge to the core if two conditions are set: 1) If a payoff is not coalition rational (i.e., has no positive excess), it needs a correction. Given a payoff vector x at a stage j , if it is objected by some coalition S with positive excess with respect to it: e ( S, x ) > 0 , the members of S can negotiate a new payoff vector x
j+1 j

(2-32)

that insures that x

j+1

( S ) > v ( S ) by defining an S-correction such that: e ( S, x ) + ----------------- , for i S , s

j

j+1 xi

j xi j

(2-33)

xi

j+1

= x i otherwise.

2) If a payoff is not group rational -see (2-3)-, because, for example the claims for payoff are too high, an N-correction is equally applied to each players payoff: e ( N, x ) -----------------n
j

j+1 xi

j xi

(2-34)

where n is the total number of players and N is the set containing all players. With these two corrections, Wu proved that starting from an arbitrary payoff vector, this could be altered in subsequent stages by applying corrections as needed, to converge to an element of the core if the latter is nonempty. 30

Although there are no coalition formation theories that can be empirically tested so far, the existence of transfer schemes is known since 1968 (Stearns [35]). Theses schemes can be used to construct sequences of payoff configurations for n-person games. A transfer scheme can be defined as a sequence of proposals, where players start negotiations at some arbitrary allocation of a given coalition structure, and proceed sequentially to transfer money from one player to another according to some formula. A transfer scheme can be expressed therefore as a sequence of payoff configurations ( x ;S ), ( x ;S ), , ( x ;S ), in which the payoff vectors x can change, but the coalition structure S remains the same. The change in payoff vectors is due to the transfer, of size , from player l to player k at stage j ; given some ( x , S ) , j = 0, 1, , the next member of the sequence ( x
j+1 j 0 1 j

, S ) is given by xk xl
j+1

= xk + , = xl , = x i for all i k, l .
j j

j+1

(2-31)

xi

j+1

Transfer schemes can be applied to several of the static payoff allocation theories we have seen, in particular to the bargaining set, kernel, and the core. The transfer scheme for the bargaining set is based on the idea of objection -see (2-15)- and counterobjection -see (216)- such that for a payoff allocation: if within a coalition S , a player k has a justified objection against another member l of the same coalition, then player l makes the (minimum) necessary transfer to a member of S (not necessarily player k ) so that player k no longer has a justified objection against player l . Stearns proved that this sequence will converge to a payoff configuration that is in the bargaining set. For the kernel, there are different conditions for the transfer scheme, although converging to the kernel as proved again by Searns (1968). Now, the transfer scheme employs 29

Based on these expectations, each player builds a priority list and coalitions are formed whenever expectations are jointly maximized for all players in the coalition. If no coalition is formed, the expectations in round r are used for the next round, r+1. For a player i and coalition S, the highest expectation A ( i, S ) is given by: A ( i, S ) = max
TS r r

[ E ( i, T ) ]

(2-29)

Now, to create a new expectation for round r+1, each player claims its best expectation from round r, and the remainder of the sum of these claims is divided into equal parts, i.e.:
r+1

( i, S ) = A ( i, S ) + v ( S )

jS

A ( j, S )
r

s .

(2-30)

Then, each player has a new expectation for each coalition he is a member for the next round of negotiations. Unlike most of the theories we examined earlier, equal excess makes predictions about coalition structures by ranking coalitional preferences based on players expectations for a given round. However, several problems that arise are that some of the definitions are ambiguous, there is no algorithmic formula for E ( i, S ) given only the game and round number, and the construction of all possible alternative coalitions is not clear, except for very simple games. That is why Komoritas theory is a good step towards a general theory of coalition formation, but the ambiguity of some terms makes it not very useful.
r

2.8. Dynamic Coalition Formation Models: Transfer Schemes

The previous theoretical models, except for Komoritas Equal Excess Theory, were static in nature. They did not address the negotiation dynamics among the players. They all assumed that the final coalition or coalitions were already created and then allocated the total payoff according to different rules. 28

2.7. Equal Excess Theory Model

So far we have been concerned with models that predict the allocation of payoffs among the members of a coalition once it has been formed. Historically, cooperative game theory has studied this aspect in more detail, until approximately twenty years ago. A newer field of study emerged in the last decades, and it was called coalition-prediction theory. From the practical point of view, the question now was which particular coalitions are going to form. The strong requirement that was imposed historically on the characteristic function was its superadditivity property -see equation (2-2)-. However, in real life, the grand coalition, which was the implicit result of a superadditive game, is not always formed, and there is a need to address the question of which particular coalitions will form. Among the theories of coalition formation, Equal Excess Theory (Komorita, 1979) [34] stands out because it also deals with payoff division. At the beginning of the game, all players expect equal shares in any coalition that they are part of, and they also want to maximize their own benefit. For the next rounds, the theory predicts that each player expects a share equal to the payoff in the best available coalition the player may potentially join. As the negotiation proceeds, players expectations change because of the different coalitions that they may join. Any excess that is left over after the players receive what they expect is divided equally among them. More formally, assume that each player enters the game assuming equal share of each coalition he belongs to, and define a round r = ( 0, 1, ) as the rth discrete stage of the bargaining process where each player has an expectation of what he will obtain from each coalition he is member of. This expectation is called E ( i, S ) subsequently. Komorita also assumed that and S N .
r iS

E ( i, S )
r

= v ( S ) r

27

xi = where

S i S

pn ( S ) [ v ( S { i } ) v ( S ) ]

(2-27)

S ! ( n S 1 )! p n ( S ) = --------------------------------------n!

(2-28)

and S is the number of players in S , and for n 1 , n! = n ( n 1 )2 ( 1 ) ( 0! = 1 ) . For the already familiar three person game, the resulting Shapley value is: {50, 45, 40}, very similar to the nucleolus solution. The interpretation of equation (2-27) is that player is reward should be the expected amount that player i adds to the coalition made up of the players who are present when he or she arrives, given that equation (2-28) is the probability that when player i arrives, the players in the subset S will be present. With his derivation of the value of a game, Shapley presented a combinatorial procedure for playing it. In his procedure, a single players starts the game alone. Then, players are added, one at a time, until n players have been admitted and the grand coalition is formed. The order of arrival is given by a chance mechanism, as shown in (2-28), that considers all permutations of the players to be equiprobable. Each player, after joining the coalition, receives the full amount that his entry has added to the coalition, as shown in (2-27). The Shapley value has some attractive features, one of which is the fact that it yields a unique point for every game and coalition structure. The axiomatic approach based on somewhat weak assumptions provides a good framework and relatively easy calculations. However, it does not reflect the strengths of the players in the game, like the core or the bargaining set, and it is not suitable for decentralized games. Nevertheless, an extension of it, the bilateral Shapley value, is a viable alternative, as will be seen in the next Chapter.

26

e min x ( i ) ; i = 1, 2, 3 e min x ( 1 ) + x ( 2 ) v ( 12 ) e min x ( 1 ) + x ( 3 ) v ( 13 ) e min x ( 2 ) + x ( 3 ) v ( 23 ) x ( 1 ) + x ( 2 ) + x ( 3 ) = v ( 123 )

(2-22) (2-23) (2-24) (2-25) (2-26)

2.6. Shapley Value

The Shapley value is an a priori worth to each player of playing a game that is a function of the characteristic function v and any coalition structure S . For any characteristic function, Shapley (1953) [33] showed that there is a unique payoff vector x = ( x 1, x 2, , x n ) satisfying the following axioms: Axiom 1: Relabeling of players interchanges the players payoffs. In other words, the value of a game does not depend on the names of the players. Axiom 2: The sum of the worth of each player of every coalitionn in a given coalition struci= ture equals the value of the coalition, i.e., group rationality:

xi
i=1

= v( N ) .

Axiom 3: If v ( S { i } ) = v ( S ) holds for all coalitions S , then the Shapley value has x i = 0 : if player i adds no value to the coalition, he receives a payoff of 0 from the Shapley value. Axiom 4: Let x be the Shapley value vector for game v , and let y be the Shapley value vector for game v . Then, the Shapley value vector for the game ( v + v ) is the vector x + y . If Axioms 1-4 are valid, Shapley proved the following result: The payoff (or reward) of the ith player ( x i ) is given by

25

The nucleolus is a solution concept introduced by Schmeidler in 1969 [32]. It is a concept related to the kernel, because it also uses the idea of an excess. It combines the idea of stability of the core and the equity of the Shapley value, as we will see in the next Section. One advantage of the nucleolus is that every game has one and only one nucleolus, and unless the core is empty, the nucleolus is in the core. The basic concept is to find an imputation that makes the most unhappy member of a potential coalition happier than the most unhappy member of a potential coalition under any other imputation; in other words, the nucleolus is the imputation for which the maximal excess is minimized. It can be expressed as max e min , where e min = min x i v ( S ) , over all coalitions S i S

(2-21)

The nucleolus for the three person example shown in Figure 2-1 is the point: {55,45,35}, and it is inside the core, as it can be seen in Figure 2-2. The corresponding equations are shown below:

Figure 2-2 : The Nucleolus of a Three Person Game.

Maximize e min subject to:

24

e ( S, x ) = v ( S )

iS

xi

(2-17)

i.e., it is the amount by which the worth of a coalition exceeds its preferred payoff, or, in other words, the total amount that the prospective members of coalition S collectively gain or lose (depending on the sign of the excess) if they withdraw the imputation x to form coalition S. Now, consider two distinct players k and l within a coalition S, and all alternative coalitions that include k but exclude l. Each of these coalitions will have an excess given by equation (2-17); choose the largest of these excesses and call it the maximum surplus of player k over player l, i.e.: s kl = max e ( S, x ) (2-18)

S k S, l S

The next step is a comparison of the maximum surpluses of player k over player l and vice versa. If both s kl > s lk and x l > v ( l ) (2-19)

player k is said to outweigh player l with respect to x for coalition S. If neither player outweighs the other, the two are in equilibrium. Equilibrium conditions exist between the two players if any of the following three conditions is satisfied: s kl = s lk or s kl > s lk and x l = v ( l ) or s kl < s lk and x k = v ( k ) (2-20)

Note that equilibrium is defined only for pairs of players who are members of the same coalition. The definition of kernel follows immediately from the above; the kernel is the set of all imputations x such that any two players k, l S are in equilibrium for any S. In short, the kernel solution reaches equality of strengths by all pairs of players within a common coalition. 23

iY

yi

= v ( Y ) ; and y k > x k , y i x i for all i Y ; k, l S ; k Y ; l Y

(2-15)

In the objection ( y ;Y ) against l, player k can gain more in the new coalition Y, without l, and his claim is reasonable because y is possible through v ( Y ) , and each member of coalition Y gets at least as much as what he could obtain in the original imputation x . Now, being ( y ;Y ) an objection of player k against player l, as defined by equation (2-15), for a coalition Z, and an allocation of z of its value v ( Z ) to its members, the pair ( z, Z ) is called a counterobjection to the objection ( y ;Y ) if:

iZ

zi

= v ( Z ) ; z i x i for all i Z ; and z i y i for all i ( Y Z ) ; l Z ; k Z (2-16)

By making a counterobjection to players k objection, player l claims that he can maintain al least a payoff equal to x by giving each of the other members of the coalition Z at least their payoff. Moreover, if any member of Z is also a member of coalition Y proposed in the objection, that player gets at least what he could have received in ( y ;Y ) . An objection is said to be justified if no counterobjection to it exists. Otherwise, it is said to be unjustified. If we also define that a bargaining point of a game has the property that for each pair of players i, j, any objection of i against j can be met by a counterobjection by j against i, the bargaining set is the set of all bargaining points.

2.5. Excess Theories: Kernel and Nucleolus

The kernel as a solution concept was introduced by Davis and Maschler in 1965 [31], and it is based on pairwise comparisons of all players in a game in terms of the excess payoff that one of the players could have by forming a coalition that excludes the other. Therefore, we need to define first what is the excess of a coalition. By the excess of a coalition S at an imputation x we mean:

22

tion concept more generally applicable than the core. That proposal is called the von Neumann-Morgenstern solution or the stable set. The stable set is based in the concept of dominance, that is explained as follows. One imputation is said to dominate another if there is a subset of players who all prefer the first to the second and can enforce it by forming a coalition. Mathematically, imputation y = ( y 1, y 2, , y n ) is said to dominate imputation x = ( x 1, x 2, , x n ) if there exists some subset S of players for which the following two inequalities are satisfied: y i > x i for every i S ; (2-13) (2-14)

yi v ( S ) for i S .

The first inequality states that every player in the subset S prefers the payoff offered by the imputation y ; the second inequality states that members of the subset S have the power to form the coalition and they are in a position to enforce their preference, because the value of the game to the coalition is at least as high as the sum of the payoffs with imputation y . Turning back to the original stable set solution by von Neumann and Morgenstern (1947), a solution of the game has to satisfy two conditions: Internal stability: None of the imputations in the stable set dominates any other one inside the set. External stability: There is at least one member of the stable set that dominates any other imputation outside the set. The bargaining set is a concept originally due to Aumann and Maschler (1964) [30]. To properly define this concept, we need to introduce the terms objection and counterobjection, similarly to the concept of domination in stable sets. Let S be a coalition containing at least two players k and l: given an imputation x that does not satisfy player k, for a coalition Y and an allocation of y of its own value v ( Y ) among its members, the pair ( y ;Y ) is called an objection of k against l via coalition Y if: 21

payoffs to all of its members. However, there are many games without a core, coreless games, or empty core games, precisely because (2-7) is not satisfied.

Figure 2-1 : The Core of a Three Person Game.

When the core is nonempty, the cooperative demands of every coalition can be granted, but when the core is empty, at least one coalition will be dissatisfied. Shapley and Shubik (1973) noted that a game with a nonempty core is sociologically neutral, i.e. every cooperative demand by every coalition can be granted, and there is no need to resolve conflicts. On the other hand, in a coreless game, the coalitions are too strong for any mechanism to satisfy every coalitional demand. However, a core set with too many elements is not desirable, and it has little predictive power [24]. Imputations in the core, where they exist, have a certain stability, because no player or subset of players has any incentive to leave the grand coalition. But since many games have empty cores, the core fails to provide a general solution for n-person games in characteristic form1. Von Neumann and Morgenstern (1944) [27] proposed a different solu-

1. Any game can be expressed either in extensive (spanning tree) form or strategic form (matrix), more suited for noncooperative n-person games, or in characteristic form (using Characteristic functions), as is shown in this Chapter.

20

More precisely, the core, introduced by Gillies in 1953 [28], is the set of imputations satisfying individual, collective, and coalitional rationality:
i=n

v(N ) =

xi
i=1

(Group rationality)

(2-5) (2-6) (2-7)

x i v ( { i } ) (for each i N ) (Individual rationality), and

xi v ( S ) for all i S , for all S N

(Coalitional rationality)

We can also define the core as the set of all undominated imputations. For a simple superadditive 3-person game: 1, 2and 3, this may be expressed as the following set of inequalities (let v ( 1 ) = v ( 2 ) = v ( 3 ) = 0 in this particular case): x(1) 0 , x(2) 0 , x(3) 0 , x ( 12 ) v ( 12 ) , x ( 23 ) v ( 23 ) , x ( 123 ) v ( 123 ) . x ( 1 ) + x ( 2 ) + x ( 3 ) = v ( 123 ) (2-8) (2-9) (2-10) (2-11) (2-12)

Equation (2-12) is equivalent to the familiar Pareto-optimum, which requires that no members share can be increased without decreasing the share to another member. Giving numerical values to the inequalities, such that: v ( 12 ) = 90 ; v ( 13 ) = 80 ; v ( 23 ) = 70 ; v ( 123 ) = 135 , the core restricted to coalition (123) is a triangle bounded by the points in the payoff space (65,55,15), (65,25,45), and (35,55,45). Figure 2-1, generated in Mathematica1 [29], exhibits this triangle pictorially. This solution concept has remained popular ever since its introduction in 1953 by Gillies. It is the simplest and most persuasive of all cooperative solution concepts. Essentially, it consists of the set of imputations which leave no coalition in a position to improve the
1. Mathematica is a trademark of Wolfram Research, Inc.

19

v( S T ) v( S) + v(T ) , whenever S and T are disjoint coalitions of players. Games in which at least one possible coalition can increase the total payoff of its members are called essential, and those in which there is no coalition that improves the total payoff are called inessential. Mathematically, an essential game is one in which at least one of the superadditive inequalities (2-2) is strict. Assuming superadditivity and collective rationality, the grand coalition will form at the end of the game. However, how will the players divide the joint payoff? The division of the joint payoff v ( N ) , represented by the payoff vector x = ( x A, x B, , x n ) is not evident. A payoff vector will not be a reasonable candidate for a solution unless it satisfies
i=n

v(N ) =

xi
i=1

(Group rationality)

(2-3) (2-4)

x i v ( { i } ) (for each i N ) (Individual rationality)

If x satisfies (2-3) and (2-4), we say that x is an imputation. Equation (2-3) states that any reasonable payoff vector must give all the players an amount that equals the amount that a grand coalition would attain. Equation (2-4) says that player i must receive a payoff al least as large as what he can get by himself: ( v { i } ) .

2.4. Core, Stable Set, and Bargaining Set

The notions of group and individual rationality introduced in equations (2-3) and (2-4) suggest that there may be a solution of the game including all possible coalitions of players as well. If we add coalition rationality to an imputation, i.e. rationality for every subset of players, we have defined a new solution concept: the core.

18

S i S j = for all i j , and

(2-1)

Sj

= N.

These conditions state that each player belongs to one and only one of the m nonempty coalitions within the coalition structure, and also specifies that none of the players in any coalition m is connected to other players not in the coalition, finally, the mutually exclusive union of all coalitions m forms the grand coalition.

2.3. Characteristic Function and Imputation

Von Neumann and Morgenstern (1944) [27] introduced the term characteristic function, denoted v , for the first time. More formally, we can define that: Definition: For each subset S of N, the characteristic function v of a game gives the biggest amount v ( S ) that the members of S can be sure of receiving if they act together and form a coalition, without any help from other players not in S. A restriction on this definition is that the value of the game to the empty coalition is zero, that is, v ( ) = 0 . A further requirement that is generally made is called superadditivity. Superadditivity can be expressed as follows: v ( S T ) v ( S ) + v ( T ) for all S, T N such that S T = (2-2)

This means that the total payoff for the grand coalition is collectively rational, because the total payoff to the players is always as much as what they would get individually. This suggests the following definition. Definition: A game in characteristic function form consists of a set P = { P 1, , P N } of players, together with a function v , defined for all subsets of P , such that

17

linear utility, transferable utility, comparable utility, or indeed any utility whatsoever [pp. 14-15]. The implication of this argument is that we can calculate any solution of a game in units of money, provided there is no distortion of the solution that would have been obtained had we used the utilities. Although players in a game are autonomous decision makers, they may have an interest in making binding agreements in order to have a bigger payoff at the end of the game. This agreement or partnership is the basic ingredient of the mathematical model of a cooperative game, and it is called a coalition. Mathematically, a coalition is a subset of the set of players N and we can denote it by S. To form a coalition S, it is required that agreements take place involving all players in the future coalition S. Whenever all players approve joining in a new entity called coalition, we can say that the new coalition is formed. Joining a coalition S also implies that there is no possible agreement between any member of S and any member not in S (set N-S). In short, the essential feature of a coalition is its foundational agreement that binds and reconstitutes the individuals as a coordinated entity. We denote a specific coalition by a concatenation of its members, for example, coalition AB will refers to players A and B acting as one decision making unit. The grand coalition of all n players will be referred as coalition N; in a game of n players there are 2
N

possible coalitions. The empty coalition

is a coalition made up of no members (the null set ). A coalition structure is a means of describing how the players divide themselves into mutually exclusive coalitions. Any exhaustive partition of the players can be described by a set S = { S 1, S 2, , S m } of the m coalitions that are formed. The set S is a partition of N that satisfies three conditions: S j , j = 1, , m , 16

Cooperative games allow players to split the gains from cooperation by making side-payments: transfers among themselves that change the prescribed payoffs. Cooperative game theory incorporates commitments and side-payments in the solution concept, which can be very sophisticated, unlike noncooperative game theory. The distinction between the two types of games is therefore a modeling one. In the next sections of this Chapter, we will provide an overview of n-person cooperative games only, following two excellent references: Kahan and Rapoport [24], and Chapter 8 from Colman [25]. For noncooperative games, the reader is referred to [26], where the rules of a noncooperative game are explained in detail for a power transmission pricing scenario.

2.2. N-Person Cooperative Games

Any game terminates in an end-state called an outcome. The quantitative representation of a players outcome at the end of a game is called a payoff. When a game terminates, each player i receives a payoff, that we can denote as x i . The collection of payoffs to all players may be expressed as the row vector x = ( x A, x B, , x n ) of each players payoff, in alphabetical order. Such a vector is termed a payoff vector. We will consider the first assumption posited by Aumann (1967) regarding the relationship of players utilities to payoffs expressed in units of a real, desirable, infinitely divisible medium of exchange (i.e., money): The utility of money to each player is fixed for any given game, and all players prefer more money to less money [24]. This assumption was justified by Aumann (1967), and is articulated in [24] as follows: To represent preferences between actual sums of money,...,utilities are not needed, as the dollar amount is a perfectly good measure for this purpose. Therefore, we may calculate [any solution] and the intuitive validity of the result is not based on any consideration of

15

Chapter 2 Cooperative Game Theory: An Overview

2.1. Introduction
Game theory is a branch of mathematics that is concerned with the actions of individuals who are conscious that their actions affect each other [23]. In everyday life all of us face decision making, and most of the times, we have to make rational decisions using limited information. One typical example of practical game theory is the famous prisoners dilemma. In this example, two prisoners are being interrogated separately. If both confess, they are sentenced to eight years in prison; if both cooperate, each is sentenced to one year. If just one confesses, he is released, but the other prisoner is sentenced to ten years. What difference would it make if the two prisoners could talk to each other before making their decisions? A cooperative game is a game in which the players can make binding commitments, as opposed to a noncooperative game, in which they cannot [23]. This definition draws the traditional distinction between the two theories of games, cooperative and noncooperative, but the difference is in fact in the modeling approach. Both theories consider players: individuals who make decisions, actions: choices that the players can make, and the rules of the game. However, they differ in the kinds of solution concepts employed. Cooperative game theory seeks equity and fairness; noncooperative game theory has solution concepts based on players maximizing their own utility functions.

14

tency. A test example is provided, although the user should be able to define her own problem following the instructions of the users guide. DistOpt is the result of a joint U.C. Berkeley-Universit degli Studi di Cassino research project initiated by Professor Felix F. Wu at U.C. Berkeley, EECS Department.

1.4. Thesis Outline

The dissertation is organized according to the following scheme: Chapter 2 introduces the basic concepts of cooperative game theory from a general mathematical standpoint. Chapter 3 introduces the new deregulated environment in Power Systems and presents a new algorithm for decentralized coalition formation among transmission expansion players. Chapter 4 presents a new backward induction algorithm to allocate costs among the players, once the coalition formation process has ended. Chapter 5 presents a new decentralized software environment called DistOpt, and its application to Power Systems problems. Chapter 6 summarizes the major contributions of this dissertation and suggests directions for future research.

13

 Defining the rules that a coalition of players has to meet to become an autonomous coalition in the expansion game Defining the role of an expansion coordinator, an arbitrator entity whose attributes are exclusively regulatory and safety enforcing, and not related to the economics of the game Using a cooperative game theory concept, the Bilateral Shapley Value (BSV), for the players to make offers to potential expansion partners Introducing a new framework, the multi-agent system, in which we can allocate costs once the negotiation process has ended. To do that we have developed a new algorithm: we call it the backward induction algorithm. This algorithm takes into account the previous history of coalition formation and rewards valuable players according to their contribution to cost reduction. It uses BSVs to allocate costs in a recursive fashion, i.e., it starts from the final coalition that was formed, splits the final cost among its two founding members using BSVs, and proceeds to allocate the calculated costs recursively until the first stage of the negotiation is reached. This scheme calculates both the total cost and its division among the players. A flexible software environment, suitable for the assessment of different proposals and the modeling of new methodologies of cooperation and coordination in a decentralized context was needed. We have developed a new software system, named DistOpt [21] that is based on an existing package developed at UC Berkeley, called Ptolemy [22]. DistOpt is a distributed software environment for solving large scale optimization problems developed at the Department of Electrical Engineering and Computer Sciences, University of California, Berkeley. DistOpt is an extremely flexible simulation software suitable to solve large scale optimization problems. The modularity of its approach allows the user to change the values of many parameters simultaneously with the execution of the program, and rerun the simulation after that. Also, the number of subproblems and the convergence parameters can be interactively selected, provided that some acceptable limits are calculated by the program. Tcl/Tk is used to interface with DistOpt whenever the user decides to change a certain value or parameter, or when DistOpt points out a mistake or inconsis12

well known cooperative game theory concept. Gatelys approach was a centralized one, where there was a central planner taking the allocation decisions. Gatelys main contribution was the creation of a new concept, the propensity to disrupt. Roughly speaking, the propensity to disrupt of a player is the ratio of how much the other players would lose if that player refused to cooperate to how much she would lose if it refused to cooperate. He used cooperative game theory concepts: core, imputation and Shapley Value, to allocate expansion costs in the most fair way possible.

1.3. Thesis Contributions

We believe that the power system structure that will emerge after the restructuring process will be characterized by three principles: decentralization, cooperation and coordination. To address these issues, we have developed a framework that introduces a new paradigm of competition and cooperation in power systems: a methodology that is able to benefit the grid as a whole, but letting the economic decisions to the players. In this dissertation, we have tried to apply this principles to transmission planning, but the same paradigm can be used to model other issues in power systems operations and planning1. The framework that we have chosen is based on cooperative game theory. We have modeled the power system as a multi-agent system, where the players are able to take rational autonomous decisions that benefit the system as a whole. We have set the rules of the cooperative game, and our model has determined how the coalitions of players are formed and the way the total cost can be allocated among them. In detail, this has been our contribution: Proposing what is the minimum cost that every individual player has to face at the beginning of the game when expanding on her own

1. Coordinated multilateral trades in power system operations is a perfect example [10].

11

Phase II: The final network is reinforced to take into account the effect of single contingencies: the algorithm initially simulates the effect of the removal of circuit elements selected from a list of severe contingencies. Very recently, Yoshimoto el al. [19] presented an approach for solving transmission expansion planning based on neuro-computing hybridized with a genetic algorithm. They used a 0-1 Hopfield Neural Network to solve the combinatorial transmission expansion planning problem, and a genetic algorithm that improves the solution accuracy obtained from the Neural Network model. From a policy standpoint, Walton [20] has proposed the use of a rated system path model as an approach to defining firm transmission rights as part of the transmission planning process. The rated system described in [20] is an adaptation of system planning and design procedures in the Western System within the WSCC area. Walton suggested that with his rated system path, by examining the contribution each facility makes to the simultaneous transfer capability of the system, a more appropriate division of firm rights can be made. Therefore, the transmission right of every party involved in a transaction of power is based upon the same rules used to design system additions that increase transfer capability, so that the physical effects of one partys schedules on another party are not ignored. Finally, in a game theoretic approach to the expansion problem, Gately was the first author concerned not only with the transmission expansion cost, but also with the distribution of the gains from cooperation between two or more players of the so called expansion game. His analysis concerned regional cooperation in planning investment in electric power, in the particular case of four states in southern India. He proved that the formation of coalitions of players in a transmission expansion investment scenario is a valid approach to solve the expansion problem,. In [2], he was concerned with regional cooperation in planning investments and cost allocation using the Shapley Value (SV), a 10

types. The result is a nonlinear staircase function with unequal steps. In addition, for each discrete value of capacity, the optimal combination of line types is determined. 2) The method works directly with discrete line additions. the single-stage expansion problem is formulated as a zero-one integer program which is solved by a modified branch-and-bound method with optimal ordering of variables. 3) A screening algorithm is included which reduces the size of the problem by eliminating those rights-of-way which are ineffective, through the use of sensitivity coefficients. 4) A DC load flow model is used, which results in fast computation and reasonable accuracy. Single contingencies can also be handled readily. The real innovation in [16] stems from the use of sensitivity coefficients to compare the effectiveness of different line additions and the use of screening algorithms to discard rights-of-way that are not in the optimal solution. Garver, by adding the most effective line at each step of his expansion algorithm, and Lee and Hnyilicza, were using sensitivity analysis concepts to determine new line additions. However, Monticelli et al. [18] developed a new interactive software tool for long term transmission system expansion planning that is totally based on sensitivity analysis. They created analysis tools where the user can add or remove any circuit of the network and evaluate its impact in terms of flows, overloads, etc. The algorithm that they used for static studies proceeds as follows: Phase I: Automatic network reinforcement until all overloads have been eliminated and all disconnected buses with non-zero injections have been connected. The ranking of new additions is based on a least-effort criterion that takes into account the pattern of flow distribution in the network. The program will list the overload reduction or increase in each overloaded circuit. Finally, the user can rank the n best candidates for addition in accordance with the least-effort criterion. A DC power-flow program is needed. 9

From a different standpoint, a cooperative game theory approach to allocate the expansion costs among the users of the lines was first developed by Gately in his seminal paper [2]. And more recently, a novel technique that uses neural networks hybridized with genetic algorithms [19] has shown promising results. In this Section, we will briefly introduce the mentioned techniques to address the expansion problem in chronological order. Garver [14] was the first author that systematically addressed the line expansion problem from a mathematical programming point of view. The steps that he took to solve the problem are quoted from [14] as follows: 1) Formulate the power flow equations as a linear minimization problem. 2) Use linear programming to solve the minimization problem for the needed power movements. This result is called a linear flow estimate. 3) Select a circuit addition based on the location of the largest overload in this flow estimate. 4) Repeat the flow-estimation and circuit-selection steps until no overloads remain. The most important new feature of his flow estimation technique was the fact that overloads did not appear on circuits, but on a new type of network link called an overload path. Overload path existed between every bus in the network. They were used in the linear flow estimation, where the overload paths with flows became the possible rights-ofway for new circuit additions. Lee and Hnyilicza [16] proposed a different approach that used branch-and-bound integer programming. Their method can be summarized as follows, as quoted from [16]: 1) The discrete nature of the cost function is taken into account and for each right-of-way, an optimal capacity curve is derived which recognizes space constraints and different line

240 80

Bus 5

Bus 1

4 UNITS 30 30 30

187

53

60

Bus 3
4 UNITS 60 60 120 120 40

62

51
240

Bus 2
32 4

357
3 UNITS 120 240 240

Bus 6
188
180

Bus 4

Figure 1-2 : 6 Bus System Solution.

1.2. Previous Work

Traditionally, the transmission expansion planning problem has been studied using two different techniques: techniques based on mathematical programming, in particular linear programming [14], dynamic programming1 [15], and branch-and-bound techniques [16] techniques based on sensitivity analysis [17, 18]

1. Since the number of possible network configurations for a typical power system is very large, dynamic programming is quite impractical.

(existing and future) in the system is given in Table 1-1, where the capacity of each line is determined based on thermal limitations and stability requirements. The solution obtained by Garver in [14] without generation rescheduling (generator outputs 1, 3, and 6 are 50, 165 and 545 MW respectively) is shown in Figure 1-2. The arrows indicate the value and direction of the active power flow across the line(s).The optimal solution has a cost of 200 monetary units, and circuit additions are: n 26 = 4 circuits, n 35 = 1 circuit, and n 46 = 2 circuits as shown in Figure 1-2.

Bus: from/to 1/2 1/4 1/5 2/3 2/4 2/6 3/5 4/6 5/6

Cost (units) 40 60 20 20 40 30 20 30 61

Suscept. (1/ ) 2.50 1.67 5.00 5.00 2.50 3.33 5.00 3.33 1.64

Capacity (MW) 100 80 100 100 100 100 100 100 78

Table 1-1. 6 Bus System Circuit Data

Should we allow generation rescheduling, i.e. the real power generation ranging from 0 to the maximum generation available (150, 360, and 600 MW respectively), the optimal solution had a cost of 130 monetary units, and circuit additions are: n 26 = 3 circuits, and n 35 = 2 circuits.

240 80

Bus 5
100

Bus 1

4 UNITS 30 30 30 60

100 4 UNITS 60 60 120 120 40 80 100 100

Bus 3

Bus 2
240 100

3 UNITS 120 240 240

Bus 6
180

Bus 4

LEGEND Existing line Possible line addition Figure 1-1 : 6 Bus Example.

We assume that transmission line cost is 1 monetary unit/mile. Its initial configuration has 5 nodes: 1, 2, 3, 4, and 5 and 6 branches: 1-2, 1-4, 1-5, 2-3, 2-4, and 3-5, as shown in Figure 1-1 and Table 1-1. When the system expands, there is a new bus: bus 6 and 4 new possible rights-of-way: 2-6, 3-5, 4-6 and 5-6. The connection between any two buses is allowed with a limit of 4 parallel paths in each right-of-way. The data of the different lines 5

AT = P

(1-2)

where A is the node-branch incidence matrix, T is the branch power flows vector, and P is the net power injections vector. (b) Limits in branch power flow: T T max ( Z j ) where T max is the branch power flow limit vector. In a DC load flow model, each element of the branch power flow vector T in constraint (12) can be described as follows: Zj T j = ---- ( j l ) xj (1-3)

(1-4)

where Z j is the variable representing the total number of parallel links of line j, x j is the reactance of a link of branch j and j and l are the voltages angles of the terminal buses of branch j. Then, constraint (1-2) becomes: B ( Zj ) = P (1-5)

1 where B ( Z j ) is the susceptance matrix whose elements are: B kl = ----- for the off-diagx kl onal terms, and B kk = B kl for the diagonal terms, x kl is the total reactance of branch (k, l), l k are the branches connected to bus k, and is the vector of nodal voltage angles. Finally, constraint (1-3) becomes: B L A T max ( Z j ) where B L is a diagonal matrix whose elements are: Z j x j . Let us illustrate the problem formulation by introducing the classical 6 bus system described in [14], as shown in Figure 1-1. We will use this simple example in the following sections to simulate coalition formation schemes and to allocate total expansion costs. 4
t l

(1-6)

This dissertation presents a cooperative game theory model of planning within a multiagent system environment, where the agents cooperate with each other to achieve the optimal common expansion goal. Here, the agents have to fulfill certain number of tasks, i.e., adding new lines, and they want to cooperate by means of forming coalitions to reduce overall costs. Each agent is rational, in the sense of being a utility maximizer, and she is an independently motivated agent, not willing to settle for a plan generated by a central planner [5].

1.1. The Transmission Expansion Planning Problem

The objective of the mathematical model of a transmission expansion planning problem is minimizing the capital and operating costs associated with the system expansion over the planning horizon. The constraints associated with this model are the physical and economical constraints that are important when attempting to expand a utility system at a minimum cost and yet meet all economic and demand restrictions that are placed upon the system. We can formulate the transmission expansion planning problem in the following terms: min

iN

C i P Gi +

jA

Kj ( Z j Z j )

(1-1)

where C i is the cost per unit power at node i for the whole planning period, P Gi is the real power injected into the network by generators at node i, K j is the construction investment cost per parallel link of line j, Z j is the variable representing the total number of parallel links of line j, Z j is the initial number of parallel links of line j, N g is the set of generator nodes, and A n is the set of possible lines. Assuming a DC load flow model for simplicity, the problem is subject to the following constraints: (a) Power nodal balance at each bus, Kirchhoffs laws: 3
0

to solve the day-to-day problems that a utility company has to address. The basis for cooperation and coalition formation among entities that share common interests is therefore purely economical, but the results benefit the entire network, since the resulting system is more reliable when the added lines are collectively agreed upon. Game theory (GT) is sometimes described as multiperson decision theory or the analysis of conflict. One of the applications of GT has been the modeling of sustaining cooperation in apparently noncooperative environments through repeated interactions. As a modeling tool, cooperative GT has been successfully applied in the power systems area to share the gains of regional cooperation in centralized planning investments [2], to allocate wheeling transaction costs [3], or to allocate cost savings in an energy brokerage system [4]. Research on Distributed Artificial Intelligence (DAI) has focused on how coalitions are formed and on negotiation algorithms amongst players of economic games. Until now, the range of applications of DAI has been restricted to other fields not related to power systems planning: stock market trading [5], cooperative databases [6], and algorithmic theory [8]. Cooperative game theory concepts have been used, but they were suited to decentralized multitask environments [5, 6, 7, 8, 9]. In the Power Systems area, Wu et al. [10] have developed a decentralized algorithm to optimize multilateral trades among generators and customers, although only looking at the operational side. Similarly, for transmission planning, Bushnell and Stoft [11], Chao and Peck [12], and Hogan [13] have described investment incentives and market mechanisms in a deregulated market, respectively. In the light of these encouraging results, we propose a combined GT/DAI approach to transmission planning that addresses and solves the pending issues of transmission expansion in a deregulated electricity industry: Determining how coalitions are formed Implementing a negotiation algorithm Allocating total expansion costs to every single agent of the transmission game 2

Chapter 1 Introduction
Power systems transmission planning addresses the problem of determining the optimal number of lines that should be added to an existing network to supply the forecasted load as economically as possible, subject to operating constraints. The objective is the minimum cost expansion plan given the base network configuration, the generation facilities, and the forecasted demands for a target year [1]. Traditionally, expansion plans were carried out by the utilities alone, without considering external factors: this was a monopoly where companies decided their own expansion plans without competing utilities that offered their services. Since the initial PURPA regulations starting in 1992, the electric utility industry is facing deregulation to allow transmission open access to suppliers and customers. In the near future, transmission expansion planning will involve decisions taken by some of the actors in the expansion scenario (suppliers, customers, and/or transmission line owners) that can and will affect decisions taken by other players. This intertwined decision process occurs when a new transmission line is built, and it is shared by several players of the expansion game. The decision whether to build the line or not, and the allocation of costs to the players who will use the line is still an open issue in a decentralized environment. Traditional planning tools that have been used in the utility industry have to be revised and adapted to the new competitive environment. We envision the new planning environment as a group of loosely linked entities, the players of the expansion game, that cooperate on the basis of mutual benefit. This means that current widespread planning tools, optimal power flow and unit commitment are two examples, have to fit the new demands that a decentralized and competitive environment presents, but without losing their power 1

A mi madre, con cario

ix

And last but not least, my family. My mother has been the only solid pillar I have had during times of trouble and anxiety. My wonderful sister Mara del Carmen and her husband Jos Antonio have always encouraged me to pursue higher ideals and do it with all my heart. I do not want to forget my sister Mara Teresa and my lovely nephew and nieces: Ignacio, Cristina, and Tineke. Thanks to all.

viii

Acknowledgments
This dissertation would not be possible without the support and collaboration of many persons in the EECS Department. First, I would like to thank Professor Felix F. Wu for his continuous advice and mentorship. His valuable comments during this period have been invaluable and worth for a lifetime. This Thesis owes to him a sense of ambition and innovation that would be impossible to attain without his leadership. I also thank Professors Shmuel Oren and Pravin P. Varaiya for their patience, support and wise counseling before and after the Qualifying Examination. Their knowledge and encouragement have been a key factor during my graduate student years. Thanks also to the EEE Department of the University of Hong Kong, who provided me with a fantastic working environment to pursue my goals. My special thanks to Professors Mario Russo and Arturo Losi, Dr. Matthias Klusch and Dr. Onn Shehory, and all the LG201 students at HKU. I do not want to forget my great office mates, first at 341 Cory, and later at the new 273 location. Thanks to Linda Kamas, Heath Hoffmann, Jeff Kao, Tetiana Lo, Liam Murphy, Zhigang Qin, S.Venkatesan, Angela Chuang, and Sandra Ellis. I am also really proud to be a friend of Reinaldo Garca, Jos Antonio Ortega Osona, Mara del Mar Fernndez Vega, Carlos Mora, and Leonor Chico. I will never forget the evenings we remembered our beloved Spain (and Recife too!). Generous funding was provided by the Spanish Ministry of Education and Science, first with a Fulbright Scholarship and later with a Doctores y Tecnlogos Scholarship. Without their economic support, this Thesis would never be written. I also want to thank the EECS Department, and Professor Wu in particular, for their funding during my last year of graduate studies. vii

List of Tables
Table 1-1 Table 3-1 Table 3-2 Table 4-1 6 Bus System Circuit Data................................................................. 6 6 Bus System Circuit Data................................................................ 46 Coalition Expansion Costs................................................................ 56 Coalition Expansion Costs................................................................ 64

vi

List of Figures
Figure 1-1 Figure 1-2 Figure 2-1 Figure 2-2 Figure 3-1 Figure 3-2 Figure 3-3 Figure 3-4 Figure 4-1 Figure 4-2 Figure 4-3 Figure 4-4 Figure 5-1 Figure 5-2 Figure 5-3 Figure 5-4 Figure 5-5 Figure 5-6 Figure 5-7 Figure 5-8 Figure 5-9 Figure 5-10 6 Bus Example ................................................................................... 5 6 Bus System Solution........................................................................ 7 The Core of a Three Person Game.................................................... 20 The Nucleolus of a Three Person Game ........................................... 24 Coordinating Plans for Agents.......................................................... 43 6 Bus Problem................................................................................... 47 Two Examples of Autonomous Coalitions ....................................... 50 Overall Coalition Formation Framework ......................................... 55 Cost Allocation Procedure ................................................................ 70 Overall Coalition Formation and Cost Allocation Procedure........... 71 BSV versus SV Allocations .............................................................. 72 Core, Nucleolus, and Shapley Value Calculations ........................... 74 DistOpt Palette.................................................................................. 85 Measures Galaxy .......................................................................... 86 APP_3 Galaxy.................................................................................. 87 Final3 Universe............................................................................. 87 IEEE-57 bus Case Subproblem 1 Objective Function...................... 89 IEEE-57 bus Case Subproblem 2 Objective Function...................... 90 IEEE-57 bus Case Subproblem 3 Objective Function...................... 90 IEEE-57 bus Case Subproblem 1 Maximum Error .......................... 91 IEEE-57 bus Case Subproblem 2 Maximum Error .......................... 91 IEEE-57 bus Case Subproblem 3 Maximum Error .......................... 92

Chapter 4. A Cost Allocation Method: Backward Induction Algorithm ............. 61 4.1. 4.2. 4.3. 4.4. 4.5. Introduction ...............................................................................................61 The Backward Induction Algorithm..........................................................62 Coalition Stability......................................................................................66 Simulation Results .....................................................................................68 Discussion..................................................................................................71

Chapter 5. DistOpt: A Distributed Optimization Software Environment .......... 76 5.1. 5.2. 5.3. 5.4. 5.5. 5.6. 5.7. Introduction ...............................................................................................76 The Auxiliary Problem Principle and Problem Transformation................77 Ptolemy Environment ................................................................................81 DistOpt Structure .......................................................................................83 DistOpt Simulation ....................................................................................88 An Optimal Power Flow Application ........................................................89 Conclusions ...............................................................................................92

Chapter 6. Conclusions and Future Work ............................................................. 94 6.1. 6.2. 6.3. Contributions .............................................................................................94 Future Research .........................................................................................94 Concluding Remarks .................................................................................95

BIBLIOGRAPHY .................................................................................................. 96

iv

Table of Contents

List of Figures ........................................................................................................... v List of Tables ........................................................................................................... vi Acknowledgments .................................................................................................. vii Chapter 1. Introduction .............................................................................................. 1 1.1. 1.2. 1.3. 1.4. The Transmission Expansion Planning Problem.........................................3 Previous Work .............................................................................................7 Thesis Contributions..................................................................................11 Thesis Outline............................................................................................13

Chapter 2. Cooperative Game Theory: An Overview............................................ 14 2.1. 2.2. 2.3. 2.4. 2.5. 2.6. 2.7. 2.8. 2.9. Introduction ...............................................................................................14 N-Person Cooperative Games....................................................................15 Characteristic Function and Imputation ....................................................17 Core, Stable Set, and Bargaining Set.........................................................18 Excess Theories: Kernel and Nucleolus ....................................................22 Shapley value.............................................................................................25 Equal Excess Theory Model......................................................................27 Dynamic Coalition Formation Models: Transfer Schemes .......................28 Summary....................................................................................................31

Chapter 3. A Coalition Formation Algorithm in Transmission Planning............ 32 3.1. 3.2. 3.3. 3.4. 3.5. 3.6. 3.6.1 3.6.2. 3.6.3. 3.6.4. 3.7. 3.8. New Policies in a Deregulated Environment.............................................32 Transmission Planning under the Open Access Paradigm ........................37 Multiagent Environments in Power Systems.............................................42 Simplified Network Expansion Model ......................................................45 Coalitions in Expansion Planning..............................................................48 Four Phases of the Coalition Formation Algorithm: .................................50 Self-Calculation Phase ............................................................................52 Communication Phase.............................................................................53 Bilateral Shapley Value Calculation Phase.............................................53 Bilateral Negotiation Phase.....................................................................54 Simulation Results .....................................................................................55 Discussion..................................................................................................59

iii

Abstract A Cooperative Game Theory Approach to Transmission Planning in Power Systems by Javier Contreras Doctor of Philosophy in Electrical Engineering and Computer Sciences University of California at Berkeley Professor Felix F. Wu, Chair

The rapid restructuring of the electric power industry from a vertically integrated entity into a decentralized industry has given arise to complex problems. In particular, the transmission component of the electric power system requires new methodologies to fully capture this emerging competitive industry. Game theory models are used to model strategic interactions in a competitive environment. This thesis presents a new decentralized framework to study the transmission network expansion problem using cooperative game theory. First, the players and the rules of the game are defined. Second, a coalition formation scheme is developed. Finally, the optimized cost of expansion is allocated based on the history of the coalition formation.

________________________ Chairman

A Cooperative Game Theory Approach to Transmission Planning in Power Systems Copyright 1997 by Javier Contreras All rights reserved

The dissertation of Javier Contreras is approved:

Chair

Date

Date

Date

University of California at Berkeley 1997

A Cooperative Game Theory Approach to Transmission Planning in Power Systems by Javier Contreras

B.S. (Universidad de Zaragoza, Spain) 1989 M.S. (University of Southern California) 1992

A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Engineering-Electrical Engineering and Computer Sciences in the GRADUATE DIVISION of the UNIVERSITY of CALIFORNIA, BERKELEY

Committee in charge: Professor Felix F. Wu, Chair Professor Pravin P. Varaiya Professor Shmuel Oren

1997