756

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 9, NO. 5, SEPTEMBER 1998

A Genetic-Based Neuro-Fuzzy Approach for Modeling and Control of Dynamical Systems

Wael A. Farag, Victor H. Quintana, and Germano Lambert-Torres
Abstract Linguistic modeling of complex irregular systems constitutes the heart of many control and decision making systems, and fuzzy logic represents one of the most effective algorithms to build such linguistic models. In this paper, a linguistic (qualitative) modeling approach is proposed. The approach combines the merits of the fuzzy logic theory, neural networks, and genetic algorithms (GAs). The proposed model is presented in a fuzzy-neural network (FNN) form which can handle both quantitative (numerical) and qualitative (linguistic) knowledge. The learning algorithm of an FNN is composed of three phases. The rst phase is used to nd the initial membership functions of the fuzzy model. In the second phase, a new algorithm is developed and used to extract the linguistic-fuzzy rules. In the third phase, a multiresolutional dynamic genetic algorithm (MRD-GA) is proposed and used for optimized tuning of membership functions of the proposed model. Two well-known benchmarks are used to evaluate the performance of the proposed modeling approach, and compare it with other modeling approaches. Index TermsDynamic control, fuzzy logic, genetic algorithms, modeling, neural networks.

The knowledge representation in fuzzy modeling can be viewed as having two classes. The rst (class A) as suggested by Takagi and Sugeno in [11] can represent a general class of static or dynamic nonlinear systems. It is based on fuzzy partition of input space and it can be viewed as the expansion of piecewise linear partition represented as If is then where and is and is (1)

I. INTRODUCTION HE principle of incompatibility, formulated by Zadeh [1], explains the inadequacy of traditional quantitative techniques when used to describe complex systems. Zadeh has suggested a linguistic (qualitative) analysis for these systems in place of a quantitative analysis. Accordingly, linguistic modeling of complex systems has become one of the most important issues [2][5]. A linguistic model is a knowledgebased representation of a system; its rules and inputoutput variables are described in a linguistic form which can be easily understood and handled by a human operator; in other words, this kind of representation of information in linguistic models imitates the mechanism of approximate reasoning performed in the human mind. The fuzzy set theory formulated by Zadeh [6] has been considered an appropriate presentation method for linguistic terms and human concepts since Mamdanis pioneering work in fuzzy control [7]. This work has motivated many researchers to pursue their research in fuzzy modeling [2][5], [8][13]. Fuzzy modeling uses a natural description language to form a system model based on fuzzy logic with fuzzy predicates.
Manuscript received February 28, 1997; revised October 10, 1997. W. A. Farag and V. H. Quintana are with the Electrical and Computer Engineering Department, University of Waterloo, Waterloo, Ontario, Canada, N2L 3G1. G. Lambert-Torres is with Escola Federal de Engenharia de Itjuba, Itajuba, MG, Brazil. Publisher Item Identier S 1045-9227(98)06185-2.

denotes the th fuzzy rule, and is the input and is the output of the . , are fuzzy fuzzy rule membership functions which can be bell-shaped, trapezoidal, or triangular, etc., and usually they are not associated with linguistic terms. From (1), it is noted that Takagi and Sugeno approach approximates a nonlinear system with a combination of several linear systems by decomposing the whole input space into several partial fuzzy spaces and representing each output space with a linear equation. This type of knowledge representation does not allow the output variables to be described in linguistic terms which is one of the drawbacks of this approach. Another drawback is that the parameter identication of this model is carried out iteratively using a nonlinear optimization method [10], [11]. The implementation of this method is not an easy task [8], [12], [13], because the problem of determining the optimal membership parameters involve a nonlinear programming problem. The second class of knowledge representation (class B) in fuzzy models is developed by Mamdani [14] and used by Lin and Lee [15] and Sugeno and Yasukawa [5]. The knowledge is presented in these models as If is and then is is and is (2)

are fuzzy memwhere , bership functions which are bell-shaped, trapezoidal, or triangular, etc., and usually associated with linguistic terms. This approach has some advantages over the rst approach. The consequent parts are presented by linguistic terms, which makes this model more intuitive and understandable and gives more insight into the model structure. Also, this modeling approach is easier to implement than the rst approach [13]. This second form (class B) of knowledge representation will be adopted throughout this paper as we are more concerned with the linguistic modeling approaches.

10459227/98$10.00 1998 IEEE

FARAG et al.: GENETIC-BASED NEURO-FUZZY APPROACH

757

Many studies regarding nding the rules and tuning the membership function parameters of fuzzy models have been reported [2][5], [8][13]. Neural networks are integrated with fuzzy logic in a form of fuzzy neural networks (FNNs) and used to build fuzzy models [15][22]. Many algorithms have been proposed to train these FNNs [15][22], and Jang et al. [19] have reviewed the fundamental and advanced developments in neuro-fuzzy synergisms for modeling and control. Lin and Lee [15] have proposed a three-phase learning algorithm. In the rst phase, they have used the self-organizing feature map algorithm for coarse-identication of fuzzy model parameters. In the second phase, they have used a competitive learning technique to nd the rules. And in the third phase they have used the backpropagation algorithm for ne-tuning the parameters. In this paper, we propose a new approach for building linguistic models for complex dynamical systems. The structure of the model is formed using a ve-layer fuzzy neural network. The parameter identication of the fuzzy model is composed of three phases. The rst phase uses the Kohonens self organizing feature map algorithm to nd the initial parameters of the membership functions. A new algorithm is proposed, in the second phase, to nd the linguistic rules. The third phase ne-tunes the membership functions parameters using a new genetic algorithm (GA) called proposed multiresolutional dynamic genetic algorithm (MRD-GA). The method used in this work builds a linguistic model in a general framework known as the black box approach in systems theory. That is, the model is built for a system without a priori knowledge about the system provided that numerical inputoutput data is given. This paper is organized as follows. In Section II, a brief overview on conventional genetic algorithms is given. Section III illustrates the structure of the neuro-fuzzy model. Section IV describes the hybrid learning algorithm. Sections V and VI present the simulation results of two benchmarks. Section VII concludes the work done in this paper. II. OVERVIEW
OF

where each gene can assume a nite number of values (alleles). A population consists of a nite number of chromosomes. The genetic algorithm evaluates a population and generates a new one iteratively, with each successive population referred to as a generation. Given an initial population , the based on the previous GA generates a new generation as follows [24]: generation Population at time Initialize Evaluate While (not terminate-condition) do begin :Increment generation from select recombine :apply genetic operators (crossover, mutation) evaluate end end. The GA uses three basic operators to manipulate the genetic composition of a population: reproduction, crossover, and mutation. Reproduction is a process by which the most highly rated chromosomes in the current generation are reproduced in the new generation. Crossover operator provides a mechanism for chromosomes to mix and match attributes through random processes. For example, if two chromosomes (parents) are selected at random (such as and ) and an arbitrary crossover site is selected (such as 3), then the resulting two and chromosomes (offspring) will be after the crossover operation takes place. Mutation is a random alteration of some gene values in a chromosome. Every gene in each chromosome is a candidate for mutation, and its selection is determined by the mutation probability. GAs provide a means to optimize ill-dened, irregular problems. They can be tailored to the needs of different situations. Because of their robustness, GAs have been successfully applied to generate ifthen rules and adjust membership functions of fuzzy systems [25][28]. The GA described above is a conventional GA, meaning the one in which the parameters are kept constant while the optimization process is running (static GA). In our approach, we introduce a new dynamic GA (MRD-GA) in which some of its parameters as well as the problem conguration change from one generation to next (while the optimization process is running) as will be discussed later in Section IV-C. III. THE NEURO-FUZZY (NF) MODEL TOPOLOGY The NF model is built using a multilayer fuzzy neural network shown in Fig. 1. The system has a total of ve layers as proposed by Lin and Lee [15]. A model with two inputs and a single output is considered here for convenience. Accordingly, there are two nodes in layer 1 and one node in layer 5. Nodes in layer 1 are input nodes that directly transmit input signals to the next layer. Layer 5 is the output layer. Nodes in layers 2 and 4 are term nodes and they act as membership

GENETIC ALGORITHMS

GAs are powerful search optimization algorithms based on the mechanics of natural selection and natural genetics. GAs can be characterized by the following features [23], [24]: a scheme for encoding solutions to the problem, referred to as chromosomes or strings; an evaluation function (referred to as a tness function) that rates each chromosome relative to the others in the current set of chromosomes (referred to as a population); an initialization procedure for a population of chromosomes (strings); a set of operators which are used to manipulate the genetic composition of the population (such as recombination, mutation, crossover, etc.); a set of parameters that provide initial settings for the algorithm and operators as well as the algorithms termination conditions. A candidate solution (in a GA) for a specic problem is called a chromosome and consists of a linear list of genes,

758

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 9, NO. 5, SEPTEMBER 1998

Layer 2: The nodes of this layer act as membership functions to express the linguistic terms of input variables. For a bell-shaped function, they are for for for (5) Note that layer 2 links are all set to unity. Layer 3: The links in this layer are used to perform precondition matching of fuzzy rules. Thus, each node has two input values from layer 2. The correlation-minimum inference procedure is utilized here to determine the ring strengths of each rule. The output of the nodes in this layer is determined by fuzzy AND operation. Hence, the functions of the layer are as follows: for for (6) (7) (4)

Fig. 1. Topology of the neuro-fuzzy model.

functions to express the inputoutput fuzzy linguistic variables. A bell-shaped function is adopted to represent the membership and the variance are function, in which the mean value adjusted through the learning process. The two fuzzy sets of and the rst and the second input variables consist of linguistic terms, respectively. The linguistic terms, such as positive large (PL), positive medium (PM), positive small (PS), zero (ZE), negative small (NS), negative medium (NM), negative large (NL), are numbered in descending order in the nodes and nodes are included term nodes. Hence, in layers 2 and 4, respectively, to indicate the inputoutput linguistic variables. Each node of layer 3 is a rule node and represents a nodes single fuzzy control rule. In total, there are in layer 3 to form a fuzzy rule base for two linguistic input variables. The links of layers 3 and 4 dene the preconditions and consequences of the rule nodes, respectively. For each rule node, there are two xed links from the input term nodes. Layer 4 links, encircled in dotted line, are adjusted in response to varying control situations. By contrast, the links of layers 2 and 5 remain xed between the inputoutput nodes and their corresponding term nodes. The NF model can adjust the fuzzy rules and their membership functions by modifying layer 4 links and the parameters that represent the bell-shaped membership functions for each node in layers 2 and 4. For convenience we use the following notation to describe the functions of the nodes in each of the ve layers. the net input value to the th node in layer . the output value of the th node in layer . the mean and variance of the bell-shaped function of the th node in layer . the link that connects the output of the th node in layer 3 with the input to the th node in layer 4. Layer 1: The nodes of this layer directly transmit input signals to the next layer. That is (3)

The link weights in this layer are also set to unity. Layer 4: Each node of this layer performs the fuzzy OR operation to integrate the eld rules leading to the same output linguistic variable. The functions of the layer are expressed as follows: (8) for (9)

in this layer expresses the association of The link weight the th rule with the th output linguistic variable. It can take only two values; either one or zero. Layer 5: The node in this layer computes the output signal of the NF model. The output node together with layer 5 links act as a defuzzier. The center of area defuzzication scheme, used in this model, can be simulated by (10) (11)

Hence, the th link weight in this layer is

IV. THE PROPOSED HYBRID LEARNING ALGORITHM In this section we present a three-phase learning scheme for the above NF model. The proposed scheme is an extension of the ideas presented in [15] and [21]. It is quite convenient to divide the task of constructing the fuzzy model into the following subtasks: locating initial membership functions, nding the ifthen rules, and optimal tuning of the membership functions. In phase one and two of the proposed scheme, unsupervised learning algorithms are used to perform the

FARAG et al.: GENETIC-BASED NEURO-FUZZY APPROACH

759

rst and the second subtasks. In phase three, a supervised learning scheme is used to perform the third subtask. To initiate the learning scheme, training data and the desired or selected coarse of fuzzy partition (i.e., the size of the term set of each inputoutput linguistic variable) must be provided from the outside world. For more details about the structure identication of fuzzy models refer to Sugeno et al. [29]. A. Learning-Phase One The problem for the self-organized learning can be stated as: , the desired Given the training input data , the fuzzy partitions output value and , and the desired shapes of membership functions, we want to locate the membership functions. In this phase, the network works in a two-sided manner; that is, the nodes and the links at layer 5 are in the updown transmission mode manner (follow the dotted lines in Fig. 1) so that the training input and output data are fed into this network from both sides. The centers (or means) and the widths (or variances) of the membership functions are determined by a self-organized learning technique that is analogous to statistical clustering. This serves to allocate network resources efciently by placing the domains of membership functions covering only those regions of the inputoutput space where data are present. Kohonens self-organized feature-map (SOM) algorithm is of the membership function [15] adapted to nd the center (12)

B. Learning-Phase Two After the parameters of the membership functions have been found, the training signals from both external sides can reach the outputs of the term nodes at layer two and layer four. Furthermore, the outputs of term nodes at layer two can be transmitted to rule-nodes through the initial architecture of layer-three links. Thus we can get the ring strength of each rule-node. Based on these rule-ring strengths (denoted as s) and the outputs of term-nodes at layer four (denoted s), we want to decide the correct consequence-link as layer-four-links) for each rule node (from the connected rules. A new algorithm is proposed here to nd the to perform this task. We refer to this algorithm as maximum matching-factor algorithm (MMFA). The MMFA is described as follows. Step 1: For each layer-three-rule node we construct matching factors. In this case, we have matching , where the factors. Each matching factor is denoted as ), and subscript is the rule-node-index ( the subscript is the output-linguistic-variable-index (output). term-node-index) ( is calculated according to the following pseuStep 2: docode:

(no. of available training examples) if is the maximum element in the set Otherwise. end end end. s using the previous Step 3: After calculating all the code considering all the available training patterns, the rules consequences can be determined form these factors according to the following pseudocode: For Find the maximum matching-factor from the set Find the corresponding term-node index of Delete all the layer-four-links of the th rule-node except the one connecting it , with the term-node of index end Step 4: From the above algorithm, only one term in the output linguistic variables term set can become the consequence of a certain fuzzy-logic rule. If all the matching factors of a rule-node are very small (meaning that this rule has small or no effect on the output), then all the corresponding links are deleted, and this rule-node is eliminated.

for

(13) (14)

is a monotonically decreasing scalar learning rate, where . This adaptive formulation runs independently and for each input and output linguistic variable. The determination s is can be accomplished in constant of which of time via a winner-take-all circuit [15]. Once the centers of membership functions are found, their widths can be determined by the -nearest-neighbors heuristic, by minimizing the following objective function with respect to the widths s:

(15)

where is an overlap parameter that usually ranges from 1.0 to 2.0. Since our third learning phase will optimally adjust the centers and the widths of the membership functions, the widths can be simply determined by the rst-nearest-neighbor heuristic at this stage as (16)

760

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 9, NO. 5, SEPTEMBER 1998

C. Learning-Phase Three After the fuzzy rules are found, the whole network structure is established, and the third-learning phase is started in order to optimally adjust the parameters of the membership functions. Optimization, in the most general form, involves nding the most optimum solution from a family of reasonable solutions according to an optimization criterion. For all but a few trivial problems, nding the global optimum can never be guaranteed [30, ch. 14]. Hence, optimization in the last three decades has focused on methods to achieve the best solution per unit computational cost. The problem for this supervised learning phase can be , stated as: Given the training input data the desired output value , the fuzzy and , the desired shapes of membership partitions functions, and the fuzzy rules, adjust the parameters of the membership functions optimally. In this phase, the network works in the feedforward manner; that is, the nodes and the links at layer four are in the down-up transmission mode. A new approach is proposed here to use GAs for the optimization of the membership functions parameters. Problemspecic knowledge is used to tailor a GA to the needs of this learning phase. The main attribute of the proposed approach is that the fuzzy-model conguration is dynamically-adapted while the optimization process is running. Accordingly, the MRD-GA changes its search space with the change of the problem conguration and with the advance of generations. The GA search space is monotonically getting narrower and narrower, while the model parameters are getting closer and closer to the optimal values. . The GA is coded using the well-structured language C The program allows the user to dene the values for population size (pop size), maximum number of generations (max gen), probability of crossover (pcross), and probability of mutation (pmut). In order to select the individuals for the next generation, the tournament selection method is used. In this method, two members of the population are selected at random and their tness values compared. The member with higher tness advances to the next generation. An advantage of this method is that it needs less computational effort than other methods. Also, it does not need a scaling process (like the roulette wheel selection). However, the particulars of the reproduction scheme are not critical to the performance of the GA; virtually, any reproduction scheme that biases the population toward the tter strings works well [25]. The MRD-GA uses decimal-integer strings to encode the model parameters. The decimal strings are considered a more suitable representative method than the binary strings. This representation allows the use of a more compact-size strings. The number of alleles (individual locations which make up the string) is determined from the total number of fuzzy sets used to partition the spaces of the inputoutput variables. For the model conguration shown in Fig. 1, we have ( ) membership functions. Each bell-shaped membership function is dened by two parameters (the center , and the width ). To optimize the membership functions, we have to optimize ( ) parameters. Thus, the GA uses

strings of length alleles. It is allowed for each allele ]. To convert the to take any value in the set [ allele-value to a new center or width of a certain membership function, we use the following procedure. Step 1: The initial values of the centers and widths of the fuzzy controller are entered to the GA program, for example, ) and ( ). ( Step 2: The new centers and widths are calculated from the allele values as (17) (18) and are the new center and width values, respecwhere is the value of the th allele in the string, and tively, and are the offsets of the centers and widths, respectively. It is recommended to set these offsets to very small values (around 0.001). This allows a more stable convergence of the MRD-GA. Step 3: If the allele value of any center or width equals is greater than ve then ve then no change occurs. If positive change occurs (the center or width increases). If it is less than ve then negative change occurs (the center or width decreases). The MRD-GA uses the mean squared error (MSE) (the error is the difference between the actual output and the estimated output by the fuzzy model) as a tness function. Simply, for each chromosome (1/MSE) is considered as the tness data points as measure of it. The MSE is calculated from MSE (19)

is the actual value, and is the estimated value. where and ) decrease Each generations, the offset values ( according to the following decaying functions: (20) (21) and are the modifying factors for the centers where and widths, respectively. The decaying functions can take any decaying shape such as, for example, an exponential decay. The usual GA terminating condition is a maximum allowable generations or a certain value of MSE required to be reached. In this GA algorithm, the stopping criteria is the execution of a certain number of generations without any improvement in the best tness value. In this criteria, you do not need to specify a required MSE value (which usually unknown in advance) or a required number of generations (where there is no grantee that this number will produce an appropriate solution). The MRD-GA pseudocode is shown at the bottom of the next page. This GA offers exciting advantages over the conventional GA [31][33]. It allows a dynamic increase in the resolution and ) as the model of the search space (by decreasing parameters approach their optimal values. It also changes the nature of the model-identication problem from a static type to and continuously) which a dynamic type (by adapting decreases the chances of the GA premature convergence. The MRD-GA has also advantages over the backpropagation (BP)

FARAG et al.: GENETIC-BASED NEURO-FUZZY APPROACH

761

algorithm [15], [21]. The MRD-GA allows to obtain intermediate solutions, which the BP usually cannot offer; also, the GA does not suffer from convergence problems with the same degree that the BP suffers (i.e., the MRD-GA is more robust).

TABLE I THE COMPLETE FUZZY ASSOCIATIVE MEMORY MATRIX WITH THE RULES

V. THE FIRST NUMERICAL EXAMPLE The proposed modeling algorithm is examined using the well-known example of system identication given by Box and Jenkins [34]. The process is a gas furnace with a single and a single output : gas ow rate and CO coninput pairs centration, respectively. The data set consists of of inputoutput measurements. The sampling interval is 9 s. The inputs to the fuzzy model are selected to be and , respectively [35]. The input variable , where is modied to be is the average of all the s. Each of the input variables is intuitively partitioned into seven linguistic sets ). The output of the fuzzy model is , where ( . The model the actual process output is is partitioned into nine linguistic sets ( ). output The gas furnace is modeled using the fuzzy-neural network shown in Fig. 1. The SOM algorithm (Section IV-A) is used to determine the initial centers and widths of the 23 member functions of the inputoutput variables. The three scaling factors of this model are determined as Gu 0.658, Gy 0.227, and Go 7.909. The resultant membership functions after nishing this learning phase are shown in Fig. 2. The MMFA algorithm (Section IV-B) is used to nd the linguistic fuzzy rules of the gas furnace model. Out of the 49 rules of the fuzzy model, 37 rules are only considered. The rest are deleted because they have very small matching factors (less than 1% of the highest matching factor of all the rules). The 37 rules are shown in Table I. The MRD-GA (Section IV-C) is then applied to optimize the parameters of the gas furnace model. The algorithm 0.9, pmut 0.1, parameters are set as follows: pcross

TABLE II THE COMPUTATION TIME OF EXAMPLE 1

chromosome-length 46, , , , , and . To study the pop size effect on the search performance of the proposed GA, the GA is applied four times with different population sizes (pop size 80, 50, 30, and 12). After nishing the second learning phase and before applying the GA, the model has an MSE value of 0.937. This MSE value is decreased to 0.111 after 4972 generations of the MRD-GA using pop_size 50 (note that the MSE value reached 0.15 after only 900 generations). The computation time elapsed to perform the whole learning scheme is roughly determined as shown in Table II. The resultant membership functions and the model output are shown in Figs. 3 and 4, respectively. The MSE decrease rates using different population sizes are also shown in Fig. 5. In Table III, our fuzzy model is compared with other models identied from the same data. It can be seen that our model outperforms all the other models in its class (class B). In

Initialize . Population at time . :The best tness value. Initialize best t = 0. . Evaluate and assign best t to it. Search for the best tness of While (not terminate-condition) do Begin :Increment generation. then modify (decrease) and as given in (20), and (21). If from using tournament selection criteria. Select :apply genetic operators (crossover, mutation). Recombine . Evaluate and compare it with best t, if larger then do Search for the best tness of Begin . Assign best t to the best tness value of and Adapt the centers and the widths to the state of the chromosome having the best tness using (17), (18). End End End.

according

762

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 9, NO. 5, SEPTEMBER 1998

Fig. 2. The normalized membership functions after SOM.

Fig. 3. The optimized membership functions after MRD-GA.

comparison with class A models, Sugenos model [10] has less MSE value using six inputs but, in the same time, has much higher MSE value using the same inputs used by our , and ). Also, This model is quite model ( difcult to build [8], [12], [13]. The most difcult aspect lies in the identication of the premise structure, mainly the membership functions of the input variables. For each

membership function, at least two or three parameters have to be calculated through a nonlinear programming procedure. The choice and computation of these membership functions are rather tricky and subjective so that it is possible for different designers to sometimes get completely different results. Wangs model (class A) [8] has comparable results and less number of rules; however, the number of rules does

FARAG et al.: GENETIC-BASED NEURO-FUZZY APPROACH

763

Fig. 4. Output of the gas furnace fuzzy model.

Fig. 5. The MRD-GA convergence rates with different population sizes.

not necessarily give a reliable indication of the number of unknown parameters of the model. For example, in Wangs model [8] ve class A rules are used with two inputs, the number of unknown parameters in this case (in both the premise and consequent parts) are 35. In our model, 37 class B rules are used with two inputs and 46 unknown parameters. Bearing in mind that our model shows about 30% reduction in the MSE value and provides a linguistic description for the gas furnace system; these two advantages, in our view, compensate for the difference in the number of parameters (46 versus 35). VI. THE SECOND NUMERICAL EXAMPLE This example is taken from Narendra et al. [36] in which the plant to be identied is given by the second-order highly

nonlinear difference equation (22) Training data of 500 points are generated from the plant model, assuming a random input signal uniformly dis]. This data is used to build a tributed in the interval [ linguistic-fuzzy model for this plant. The plant is modeled using the FNN described in , and , Section III. The model has three inputs , and are intuitively and a single output . The inputs partitioned into ve fuzzy linguistic spaces NL, NS, ZE, PS, is partitioned into three fuzzy spaces N, PL , the input is partitioned into 11 fuzzy spaces Z, P and the output NVL, NL, NM, NS, NVS, ZE, PVS, PS, PM, PL, PVL .

764

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 9, NO. 5, SEPTEMBER 1998

COMPARISON

OF

TABLE III OUR MODEL WITH OTHER MODELS

The SOM algorithm described in Section IV-A is used to determine the initial centers and widths of the 24 membership functions of the inputoutput variables of the fuzzy model. The four scaling factors of this fuzzy model are determined 0.7476, Gy 0.4727, from this learning phase as Gu 0.6261, and Go 5.5781. Gyy According to the structure of this fuzzy-neural network, the . number of rules (rule nodes in the third layer) is The MMFA described in Section IV-B is used to nd the 75 rules of this fuzzy model and the results are shown in Table IV. The MRD-GA (Section IV-C) is applied to optimize the parameters of the dynamic-system model. The algorithm pa, pmut 0.05, rameters are set as follows: pop size , , , chromosome-length , , and . After nishing the second learning phase and before applying the MRD-GA, the model has an MSE value of 0.2058. This MSE value is decreased to 0.0374 after 3517 generations using a single point crossover with pcross value of 0.9 (note that the MSE value reached 0.06 after only 470 generations). The computation time used to perform this learning process is illustrated in Table V. The MSE decay rates using different crossover probabilities are shown in Fig. 6.

After the learning process is nished, the model is tested by applying a sinusoidal input signal to the fuzzy model. The output of both the fuzzy model and the actual model are shown in Fig. 7. The fuzzy model has a good match with the actual model with a MSE of 0.0403. Another test . is carried out using an input signal The result is shown in Fig. 8 and the MSE in this case is 0.0369. After extensive testing and simulations, the fuzzy model proved a good performance in forecasting the output of the complex-dynamic plant. Remember that in this example only 500 data points are used to build the model; while in [36], 100 000 data points have been used to identify a neural network model. It can be expected that the performance of the identied fuzzy model may be further improved if the number of data points used to build the model is increased. In order to compare our modeling approach with that of Sugenos [10], [11] and Wangs [8] approaches, both of these approaches are implemented. The Sugenos approach is implemented using the MATLAB fuzzy-logic tool box. The approach [37] applies the least-squares algorithm (LSA) and the backpropagation gradient descent method for identifying linear (consequent) and nonlinear (premise) parameters of the class A fuzzy rules, respectively. The core function of

FARAG et al.: GENETIC-BASED NEURO-FUZZY APPROACH

765

Fig. 6. The MRD-GA convergence rates with different crossover rates.

Fig. 7. Testing of the fuzzy model versus the actual model.

this algorithm is implemented using an-optimized-for-speed C code. Wangs approach is implemented using the C programming language. The approach uses the fuzzy C-means (FCM) clustering algorithm [38] to nd the premise parameters of the class A fuzzy rules, then it applies the least squares algorithm to nd the consequent linear parameters of the rules.

Table VI compares our modeling approach with both of Sugenos and Wangs approaches. The models are learned from the previously generated 500 data pairs and tested by . All the applying a sinusoidal input signal experiments are carried out on a Pentium 166-MHz PC. The comparison shows the advantages of our modeling approach.

766

IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 9, NO. 5, SEPTEMBER 1998

Fig. 8. Testing of the fuzzy model versus the actual model.

TABLE IV THE COMPLETE FAM MATRICES WITH THE FUZZY RULES

TABLE V THE COMPUTATION TIME OF EXAMPLE 2

TABLE VI A MODELING COMPARISON USING EXAMPLE 2

VII. CONCLUSION This paper presents a neuro-fuzzy approach for linguistic modeling of complex-dynamical systems, in which a proposed genetic algorithm plays a central role. One of the advantages of the method presented is that it divides the learning algorithm into three phases. The initial membership functions are found in the rst phase using Kohonens self organizing feature maps. In the second phase, a new algorithm is proposed to extract and optimize the fuzzy rules for the neuro-fuzzy model.

In the third phase, the membership functions are ne-tuned using the proposed MRD-GA. The performance of the neurofuzzy approach is tested using two benchmarks, and compared with other models. The approach shows a good performance in building accurate linguistic fuzzy models. ACKNOWLEDGMENT The authors wish to thank the anonymous reviewers for their valuable and meticulous comments. The guidance and help of Dr. A. Y. Tawk at Wilfrid Laurier University are also gratefully acknowledged.

FARAG et al.: GENETIC-BASED NEURO-FUZZY APPROACH

767

REFERENCES
[1] L. Zadeh, Outline of a new approach to analysis of complex systems and decision processes, IEEE Trans. Syst., Man, Cybern., vol. SMC-3, pp. 2844, 1973. [2] R. M. Tong, The construction and evaluation of fuzzy models, Advances in Fuzzy Set Theory and Applications, M. M. Gupta, R. K. Ragade, and R. R. Yager, Eds. Amesterdam, The Netherlands: NorthHolland, 1979, pp. 559576. [3] W. Pedrycz, An identication algorithm in fuzzy relational systems, Fuzzy Sets Syst., vol. 13, pp. 153167, 1984. [4] C. W. Xu and Y. Z. Lu, Fuzzy model identication and self-learning for dynamic systems, IEEE Trans. Syst., Man, Cybern., vol. 17, pp. 683689, 1987. [5] M. Sugeno and T. Yasukawa, A fuzzy-logic-based approach to qualitative modeling, IEEE Trans. Fuzzy Syst., vol. 1, pp. 731, Feb. 1993. [6] L. A. Zadeh, Fuzzy sets, Inform. Contr., vol. 8, pp. 338353, 1965. [7] E. H. Mamdani and S. Assilian, An experiment in linguistic synthesis with a fuzzy logic controller, Int. J. ManMachine Studies, vol. 7, no. 1, pp. 113, 1975. [8] L. Wang and R. Langari, Complex systems modeling via fuzzy logic, IEEE Trans. Syst., Man, Cybern., vol. 26, pp. 100106, Feb. 1996. [9] W. Pedrycz and J. V. de Oliveira, Optimization of fuzzy models, IEEE Trans. Syst., Man, Cybern., vol. 26, no. 4, pp. 627636, Aug. 1996. [10] M. Sugeno and K. Tanaka, Successive identication of a fuzzy model and its application to prediction of a complex system, Fuzzy Sets Syst., vol. 42, pp. 315334, 1991. [11] T. Takagi and M. Sugeno, Fuzzy identication of systems and its applications to modeling and control, IEEE Trans. Syst., Man, Cybern., vol. 15, 1985. [12] L. Wang and R. Langari, Building Sugeno-type models using fuzzy discretization and orthogonal parameter estimation techniques, IEEE Trans. Fuzzy Syst., vol. 3, pp. 454458, Nov. 1995. [13] E. Kim, M. Park, S. Ji, and M. Park, A new approach to fuzzy modeling, IEEE Trans. Fuzzy Syst., vol. 5, pp. 328337, Aug. 1997. [14] E. Mamdani, Advances in the linguistic synthesis of fuzzy controllers, Int. J. ManMachine Studies, vol. 8, pp. 669678, 1976. [15] C. T. Lin and C. S. G. Lee, Neural-network-based fuzzy logic control and decision system, IEEE Trans. Comput., vol. 40, pp. 13201336, Dec. 1991. [16] H. Nomura, I. Hayashi, and N. Wakami, A self-tunning method of fuzzy control by descent method, Int. Fuzzy Syst. Assoc., Brussels, Belgium, 1991, pp. 155158. [17] L. X. Wang and J. Mendel, Generating fuzzy rules by learning from examples, IEEE Trans. Syst., Man, Cybern., vol. 22, pp. 14141427, 1992. [18] C. C. Hung, Building a neuro-fuzzy learning control system, AI Expert, pp. 4049, Nov. 93. [19] J. S. R. Jang and C. T. Sun, Neuro-fuzzy modeling and control, Proc. IEEE, vol. 83, no. 3, pp. 378406, Mar. 1995. [20] Y. Lin and G. A. Cunningham, A new approach to fuzzy-neural system modeling, IEEE Trans. Fuzzy Syst., vol. 3, pp. 190198, May 1995. [21] W. A. Farag, V. H. Quintana, and G. Lambert-Torres, Neural-networkbased self-organizing fuzzy-logic automatic voltage regulator for a synchronous generator, in Proc. 28th Annu. NAPS, Cambridge, MA, Nov. 1012, 1996, pp. 289296. [22] R. Langari and L. Wang, Fuzzy models, modular networks, and hybrid learning, Fuzzy Sets Syst., vol. 79, pp. 141150, 1996. [23] D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, MA: Addison-Wesley, 1989. [24] Z. Miachalewicz, GA + Data Structure = Evolution Programming. New York: Springer-Verlag, 1994. [25] C. L. Karr and E. J. Gentry, Fuzzy control of pH using genetic algorithms, IEEE Trans. Fuzzy Syst., vol. 1, pp. 4653, Feb. 1993. [26] D. Park, A. Kandel, and G. Langholz, Genetic-based new fuzzy reasoning models with application to fuzzy control, IEEE Trans. Syst., Man, Cybern., vol. 24, pp. 3947, Jan. 1994. [27] A. Homaifar and E. McCormick, Simultaneous design of membership functions and rule sets for fuzzy controllers using genetic algorithms, IEEE Trans. Fuzzy Syst., vol. 3, May 1995. [28] H. Ishibuchi, K. Nozaki, N. Yamamoto, and H. Tanaka, Selecting fuzzy ifthen rules for classication problems using genetic algorithms, IEEE Trans. Fuzzy Syst., vol. 3, pp. 260270, Aug. 1995.

[29] M. Sugeno and G. T. Kang, Structure identication of fuzzy model, Fuzzy Sets Syst., vol. 28, pp. 1533, 1988. [30] T. J. Ross, Fuzzy Logic with Engineering Applications. New York: McGraw-Hill, 1995. [31] M. Lee and H. Takagi, Dynamic control of genetic algorithms using fuzzy logic techniques, in Proc. 5th Int. Conf. Genetic Algorithms, Univ. Illinois, Urbana-Champaign, July 1721, 1993, pp. 7683. [32] A. N. Aizawa and B. W. Wah, Dynamic control of genetic algorithms in a noisy environment, Proc. 5th Int. Conf. Genetic Algorithms, Univ. Illinois, Urbana-Champaign, July 1721, 1993, pp. 4849. [33] J. Kim and B. P. Zeigler, Designing fuzzy logic controllers using a multiresolutional search paradigm, IEEE Trans. Fuzzy Syst., vol. 4, pp. 213226, Aug. 1996. [34] G. E. P. Box and G. M. Jenkins, Time Series Analysis: Forecasting and Control, 2nd ed. San Francisco, CA: Holden-Day, 1976. [35] R. M. Tong, Synthesis of fuzzy models for industrial processesSome recent results, Int. J. General Syst., vol. 4, pp. 143162, 1978. [36] K. S. Narendra and K. Parthasarathy, Identication and control of dynamical systems using neural networks, IEEE Trans. Neural Networks, vol. 1, Mar. 1990. [37] J.-S. R. Jang, ANFIS: Adaptive-network-based fuzzy inference system, IEEE Trans. Syst., Man, Cybern., vol. 23, pp. 665685, 1993. [38] J. C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algorithms. New York: Plenum, 1981.

Wael A. Farag received the B.Sc. and M.Sc. degrees from Cairo University, Egypt, in 1990 and 1993, respectively, both in electrical engineering. Currently, he is a Ph.D. candidate in the Electrical and Computer Engineering Department at the University of Waterloo, Ontario, Canada. From 1990 to 1994, he worked as an Assistant Lecturer in the Electrical Engineering Department at Cairo University. His research interests include microprocessor-based systems, fuzzy logic and neural networks, evolutionary algorithms, and control systems.

Victor H. Quintana (M73SM80) received the Dipl. Ing. degree from the State Technical University of Chile in 1959, the M.Sc. degree in electrical engineering from University of Wisconsin, Madison, in 1965, and the Ph.D. degree in electrical engineering from the University of Toronto, Ontario, Canada, in 1970, respectively. Since 1973, he has been with the University of Waterloo, Department of Electrical and Computer Engineering, where he is currently a Full Professor. His main research interests are in the areas of numerical optimization techniques, state estimation, and control theory as applied to power systems. Dr. Quintana is an Associate Editor of the International Journal of Energy Systems and a member of the Association of Professional Engineers of the Province of Ontario.

Germano Lambert-Torres received the Ph.D. degree in electrical engineering from Ecole Polytechnique de Montr al, Canada. e Since 1983, he has been with the Escola Federal de Engenharia de Itajuba (EFEI), Brazil, where he is a Full Professor and Associate Chairman of graduate Studies. During the 1995 to 1996 academic year, he was a Visiting Professor at University of Waterloo, Canada. He has served as a consultant for many power industries in South-America countries. He has published more than 200 chapter books, journal articles, and conference papers on intelligent systems applied to power system problem solving.