Fuzzy Rule Mining Using Genetic Algorithms

Introduction: Classification is a popular task in data mining and machine learning [1]. In a classification problem, if a priori probabilities and conditional probability density of all classes are assumed to be known then statistical classifier like Bayesian classifier [5] is well suited to solve these problems. If these probabilities of the all classes are not known then methods like maximum likelihood [7], k-nearest neighbor [7,8], decision tree [2], instance-based algorithm [4] and soft computing tools such as neural network [6,9], support vector machine [3] and fuzzy sets[10-12] and genetic algorithms[13] are used. The objective of classification is to design a function, termed as classifier, from data, to allocate a predefined set of class labels to the set of objects correctly. The classification problem can be defined as follows: For given data consisting of vectors describing the objects, each belongs to a particular class Ci , from a set of predefined classes C= {C1, C2, .., Cn}. Each vector n x is consists of n features x1 , x 2 , , x n and generates a n-dimensional space R , called feature space . A classifier F is a mapping which maps the feature space R n to C. That is F: R n C. Every crisp or non-fuzzy or hard classifier believes that any pattern x , belongs to a particular class basing only on specified criteria. So these classifiers are easy to implement and used to classify the classes which are well defined, well separable and distinguishable boundaries. But in for the problems, which are vague and having overlapping boundaries such classifiers are not so good. Fuzzy classifiers [10, 14], which uses fuzzy logic [17], handle these problems very efficiently as they allows imprecise class definition and can recognize patterns with overlapping classes [15, 16], with different degree of membership values. Fuzzy classifification system is a rule-based approach which uses IF-THEN rules to classify the data. So popularly it is known as fuzzy rule-based systems. In literature a good range of approaches [11, 12, 18-22] have been proposed for generating and learning fuzzy if-then rules from numerical data for classification problems. The primary objectives of fuzzy rule-based systems are: maximization of performance and comprehensibility. The comprehensibility of fuzzy classification system related to the factors like: i) Clarity of fuzzy partitions, which include number of fuzzy sets used to describe each variable, the explanation of the linguistic variable used for each fuzzy set, the number of fuzzy sets used to describe each variable etc.

ii) iii) iv)

Minimalism of fuzzy rule-based systems, which relates to the use of number of input variables and number of fuzzy if-then rule. Clarity of fuzzy if-then rules, which depends on type of fuzzy if-then rules used and number of antecedent conditions used in each fuzzy ifthen rule. Easiness of fuzzy reasoning, which relate to the selection of a single winner rule.

Optimal fuzzy Classifier design and finding an optimal solution in a highdimensional search space in which every point characterize a rule set, membership functions and the performance of the subsequent scheme are equivalent task. Searching optimal solutions in complex spaces can be handled very efficiently by Genetic Algorithms (GAs). So Genetic Algorithms [23] are used to handled the task like, fuzzy rule-base generation, fuzzy rule-base optimization and both generation and tuning of membership functions. The use of genetic algorithms in design and optimization of fuzzy systems is known as Genetic Fuzzy Systems [24]. FUZZY CALSSIFICATION SYSTEM The pattern classification problem is a C-class problem in the n-dimensional pattern space [0, 1]n with continuous attributes. Classical or crisp classification rule system constitute a number of IF-THEN rules, which are of the form Reference: [1]Duda, R. O., Hart, P.E. and Stork, D.G. (2001): Pattern Classification. 2nd Edition, John Wiley. [2] Quinlan, J.R. (1993): C4.5: Programs for Machine learning. San Mateo-California: Morgan Kaufmann Publishers. [3] Vapnik, V. N. (1999): The Nature of Statistical Learning Theory. 2nd Edition, Sringer. [4] Aha, D. W., Kibler, D. and Albert, M. K. (1991): Instance-based Learning Algorithms. Machine Learning, Vol. 6, No.1, pp. 37-66. [5] John, G. H. and Langley, P. (1995): Estimating continuous distributions in Bayesian Classifiers. In Proceccedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, San Mateo: Morgan Kaufmann, pp.338-345. [6] Haykin, S. (1998): Neural Networks: A Comprehensive Foundation.2nd Edition, Prentice-Hall, Englewood Cliffs, NJ.

[7] Duda, R.O., Hart, P. E. and Stork, D. G. (2001): Pattern Classification. 2nd Edition, Wiley Interscience Publication. [8] Tou, J. T. and Gonzalez, R. C. (1974): Pattern Recognition Principles. AdditionWesley, Reading, MA. [9] Ripley, B. D. (1996): Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge. [10] Zadeh, L. A. (1965): Fuzzy Sets. Information Control, Vol. 8, pp. 338-353. [11] Zimmermann, H. J. (2001): Fuzzy Set Theory and Applications. 4th Edition, Kluwer Academic Publishers, Dordrecht. [12] Klir, G. J. and Yuan, B. (1997): Fuzzy Sets and Fuzzy Logic: Theory and Applications, Prentice-Hall, India. [13] Goldberg, D. E. (1989): Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, Reading, MA. [14] Kuncheva, L. I. (2000): Fuzzy Classifiers Design. Springer, Berlin. [15] Maseli, F., Rodolfi, A. and Copnese, C. (1996): Fuzzy Classification of Spatially Degraded Thematic Mapper Data for the Estimation of Sub-pixel Components. International Journal of Remote Sensing, Vol. 17, No.3, pp. 537-551. [16] Mandal, D.P. Murthy, C. A. and Pal, S. K. (1992): Formulation of Multivalued Recognition System. IEEE Transaction on System, Man and Cybernetics, Vol. 22, No. 4, 607-620. [17] Meier, W., Weber, R. and Zimmermann, H. J. (1994): Fuzzy Data Analysis Methods and Industrial Applications. Fuzzy Sets and Systems, Vol. 24, pp.141-160. [18] Ishibuchi, H., Nozaki, K. and Tanaka, H. (1992): Distributed Representation of Fuzzy Rules and its application to Pattern Classification. Fuzzy Sets and Systems, Vol. 52, No.1, pp. 21-32. [19] Abe, S. and Lan, M. S. (1995): A Method for Fuzzy Rules Extraction Directly From Numerical Data and its Application to Pattern Classification. IEEE Transactions on Fuzzy Systems, Vol. 3, No.1, pp. 18-28. [20] Ishibuchi, H. and Yamamoto, T. (2004): Comparison of Heuristics Criteria for Fuzzy Rule selection in Classification Problems. Fuzzy Optimization and Decision Making, Vol. 3, No. 2, pp.119-139.

[21] Bardossy, A. and Samaniego, L. (2002): Fuzzy Rule-based Classification of Remotely Sensed Imagery. IEEE Transaction on Geoscience and Remote Sensing, Vol. 40, No.2, pp.362-374. [22] Chen, Y. and Wang, J. Z. (2003): Support Vector Learning of Fuzzy Rule-based Classification Systems. IEEE Transaction on Fuzzy Systems, Vol.11, No.6. pp. 716-728. [23] Goldberg, D. E. (1989): Genetic Algorithms in Search, Optimization and Machine Learning. Adission Wesley. [24] Herrera, F. and Magdalena, L. (1997): Genetic Fuzzy Systems. Tatra Mountains Mathematical Publications, Vol. 13, pp. 93-121.