Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
He Zhenya, Wei Chengjian, Yang Luxi, Gao xiqi and Yao susu Southeast University, Dept. o f Radio Ensneering Nanjing 210096, P.R. China Russell C. Eberhart and Yuhui Shi Purdue School o f Engineering and Technology 799 W. Michigan St., ET215 Indianupolis, IN 46202
Abstract
A four-layer fuzzy neural network is presented to realise
knowledge acquisition from input-output samples. The network parameters including the necessary membership functions of the input variables artd the consequent parameters are tuned and identiyied using a modiyied particle swarm algorithm which uses each particles best current performance of its neighbows to replace the besf prwious one and uses a not accumulative rate o f change to replace the accumulative one for accelerating search procedure. The trained network i s then pruned so that the general rules can be extracted and explained. The experimental results have shown that the similar classification rules can be obtained in comparison to that o f otherfuzzy neural approaches.
algorithm for training and pruning the network The conventional training algorithms are gradient decent method and evolutionary computing method The goal of network pruning is to make a fat network slim and make the network manageable for knowledge extracting task. The third phase is to acquire knowledge and explain rules after pruning the network. In this paper, we propose a new method for extracting knowledge using an adaptive fuzzy neural network. A modified version of particle swarm optimisation is apglied to train the fuzzy neural network, which uses each particles best current performance of its neighbows to replace the best previous one and uses a not accumulative rate of change to replace the accumulative one to accelerate search procedure. The @cIe swarm algorithm like evolutionary computing methods searches for the optimal solutions from multiple directions but requires only primitive mathematical operators and is computationally inexpensive in terms of both memory requirements and speed. The network is then pruned by selecting stronger finng values of the consequent parameters. Finally, the proposed method was examined for the bench mark example classification of Iris species. The knowledge of classification was succesmly extracted. In the following several sections, the f u z z y neural network architecture, particle swarm learning algorithm and experimental results are described in details.
2. Fuzzy NeuraI Network
1. Introduction
Extracting general rules from input -output samples has being an interesting problem in many research fields, such as classification and pattern recognition. Over the last few years, a number of methods of extracting symbolic rules Using artificial neural network methods have been reported. Yoon et a1 proposed a rule extraction method using neural network by destructive learning [11. Hayashi et ai tried to extract fuzzy rules of k conjuncts [2]. Horikawa et ai [3] presented a fuzzy modelling method based on if-then rules using gradient descent learning. Feldman [4], Karr [SI used genetic algorithm for generating if-then rule and tuning the membership functions of antecedent and consequent fuzzy sets. Generally, extracting knowledge based on artificial neural network or fuzzy neural network consists of three main phases. One is to design an adequate network structure that may contain possible configurations of all rules and easy to be trained. Due to fuzzy systems are designed to operate w i t h linguistic variables, the combination of fuzzy systems and neural networks has shown their effectiveness in knowledge acquisition. Another phase is to use an
74
a manner more like humans, a better way is to incorporate the concept of fuzzy logic into the neural network to construct fuzzy network. Several authors have studied fuzzy neural network structures and learning algorithms [6][7].Jang [8] proposed an adaptive-network-based fuzzy inference system, which serves as a basis for generating a set of fuzzy if-then rules. Horikawa et a i presented three types of fuzzy neural network with sigmoid active functions [3]. These networks realise fuzzy implication and reasoning, which are based on Takagi and Sugenos method [9].They proposed a mathematical tool to build a fuzzy model of a system, which can be eqressed by
rules.
where, X, ; . -, Xt are input variables. A, 7 - A, denote fuzzy sets with appropriate membership functions, which are also referred to as premise parameters. po;-, p k represent parameters of consequence. This fuzzy model has been successfully used for fuzzy identification and non-linear modelling. For classification, more than one output node is needed depending upon the number of objects to be classified. A simple case is only to idenw po of the consequent part. Based on the above fuzzy system the corresponding fuzzy neural network can be constructed Fig.1 gives an architecture of fuzzy network for classification. For simplicity, only the case of two input variables and two output nodes is shown. This fuzzy network is composed of four layers containing fuzzification of the input variables, fuzzy rule operation and defuzzification functions.
A(x) =
where x is the input variable and a, b, ,Cl are the premise parameters. The shape and position of the membership function will change with any of a, , b, or C ,. Layer 2 performs and operation of incoming signals. It is equivalent to the meaning of the firing strength in fuzzy system.
to the sum of all the
V[][d]=ACC-CONST*rand()*(PBESTx[][d]-
Layer 4 is the output layer, the number of nodes correspnds to that of objects to be classified Each node in this layer sums all the action values from all the inference
and got a better version of particle swarm search. The following two test functions are used to demonstrate the effectiveness of the modified version:
75
IF high2 AND low3 AND low4 THEN class 1 IF high2 AND high3 AND low4 THEN class2 IF high2 AND high3 AND high4 THEN class3
5. Conclusions
We have described the architecture of fuzzy neural network for classification. The fuzzy implication and reasoning are realised using the proposed fuzzy network. To overcome the drawbacks of gradient information and evolutionary computing based learning algorithm, we apply a modified version of particle swarm algorithm to tune the premise parameters and consequent parameters. The m o d i f i e d version of particle swarm algorithm uses each particle's best current performance of its neighburs to replace the best previous one and uses a not acmulative rate of change to replace the accumulative one which accelerate the seasch procedure.
~
3. Knowledge Acquisition
After training fuzzy neural network, the knowledge is permeated into its own particular knowledge structure. That is, knowledge is contained in network topology and parameter sets. In general, this knowledge is very diacult to interpret because there exist redundant rules or rules overlapping. Before e'xtracting rules from the trained network we have to prune redudant components fiom the network to make the network manageable for the d e extracting goal. We can take advantage of a small network and extract a relatively small d e set. The strategy of pruning network used in this work is very simple. That is, for all the remaining links in the layer four, set those weaker firing strengths under a certain threshold zero and retrain the network After replacing the firing strength, the network error will be increased. Therefore the all the parameters of the fuzzy neural network including premise and consequent parameter sets need to be retrained using particle swam learning algorithm. The firing strength for each rule is distributed again among the rule links. This process repeats until a relatively small rule set and minima network error are achieved.
References
1.
learning, The 1994 Intemational Conference on Neural Networks, Ireland, Florida, pp. 1176-1771. 2. Y. Hayashi, et ai., Hybrid expert system with fuzzy ifthen rule generation, Proceedings of the Intemational Joint ConferenceJ 994.on Neural Networks, Beijing, China,pp.m:liO 1-506. 3. S. Honlcawa, et al., On fuzzy modelling using fuzzy neural networks w i t h the back-propagation algorithm, IEEE Trans. Neural Networks, 3(5), pp.801-806, 1992. 4. D.S. Feldman, Fuzzy network synthesis with genetic t h ICGA, University of Illinois at algorithm, Proc. 5 Urbana Champaign 312-317,1993. 5. C.L. K m , Design of an adaptive fizzy logic controller uskg a genetic algorithm, Proc. 4th ICGA, San Diego,
Ca, USA, 450457,199 1.
6.
7.
J.J. Buckley and Y. Hayashi, Fuzzy neural nets and applications, Fuzzy Systems and AI, 3, pp. 11-11, 1992. L.X. Wang and J.M. Mendel, Fuzzy basis fimctions, universal approximation and orthogonal least squares learning,IEEE Trans. Neural Networks, 3(5), pp.807-8 14, 1992. R. Jang, ANFIS: Adaptive-network-based fuzzy inference system, IEEE Trans. on Systems, Man and Cybernetics, Vol. 23, pp.665-683, 1993. T. Takagi and M. Sugeno, Fuzzy identification of systems and its applications to modeling and control, IEEE Trans.
on System, Man, and Cybernetics, Vol. SMC-15, No.1, 1985. 10. R.C. Eberhart, A new optimizer using particle swarm theory, Sixth International Symposium on Micro Machine and Human Science, 39-43. 11. J. Kennedy and R.C. E k M , Particle s w a optimisation, IEEE Conference on Neural networks, 1995, A d a . 12. J. Kennedy, The particle swarm: social adaptation knowledge, ICEC97 303-308.
p ~ 116-132, .
13. J. Kennedy and R.C. Eberhart, A discrete binary version of the particle swarm a l g o r i h IEEE Conhence on System, Man, Cybernetics, 1997. 14. Ehderson, The irises of the gaspe ~leninsula, Bill. Pamer. Iris Soc.59(1935) 2-5. 15. S.K. Haigamuge, et al, A new method for generating fuzzy classification systems using rbf neurons w i t h extended rce learning, The 1994 Intematio,nalConference on Neural Networks, Orlando, Florida, pp. 1589-1594.
Y,
Y 2
E
L
1os
.- - I
---0
100 200 300
.
I
I .
m
iterations
Q)
400
IU
100
200
Iterations
300
400
77