Redefinition of Decision Rules Based on the Importance of Elementary Conditions Evaluation

• Autorzy: Sikora M.
• Języki publikacji: EN

Abstrakty

EN

The paper presents an algorithm of decision rules redefinition that is based on evaluation of the importance of elementary conditions occurring in induced rules. Standard and simplified (heuristic) indices of elementary condition importance evaluation are described. There is a comparison of the results obtained by both indices concerning classifiers quality and elementary condition rankings estimated by the indices. The efficiency of the proposed algorithm has been verified on 21 benchmark data sets. Moreover, an analysis of practical applications of the proposed methods for biomedical and medical data analysis is presented. The obtained results show that the redefinition reduces considerably a rule set needed to describe each decision class. Additionally, after the rule set redefinition negated elementary conditions may also occur in new rules.

Słowa kluczowe

EN

decision rules; importance of elementary conditions; rule quality measures; knowledge; discovery; classification

• Wydawca: IOS Press
• Czasopismo: Fundamenta Informaticae
• Rocznik: 2013
• Tom: Vol. 123, nr 2
• Strony: 171--197
• Opis fizyczny: Bibliogr. 66 poz., tab., wykr.

Twórcy

autor

Sikora M.

• Institute of Informatics, Silesian University of Technology, Akademicka 16, 44-100 Gliwice, Poland Institute of Innovative Technologies EMAG, Leopolda 31, 40-189 Katowice, Poland

Bibliografia

• [1] Agotnes T., Komorowski J., Løken T.: Taming large rule models in rough set approaches, Lecture Notes in Artificial Intelligence 1704, 1999, 193–203.
• [2] Agrawal R., Srikant R.: Fast Algorithms for Mining Association Rules, Proc. of the 20th VLDB Conference, Santiago, Chile, 2004.
• [3] An A., Cercone N.: Rule quality measures for rule induction systems – description and evaluation, Computational Intelligence 17, 2001, 409–424.
• [4] Ashbumer M., Ball C.A., Blake J.A., Botstein D., Butler H., Cherry J.M., et al.: Gene Ontology: Tool for the unification of biology, The Gene Ontology consortium. Nature Genetics 25, 2000, 25–29.
• [5] Banzhaf J. F.: Weighted voting doesnt work: A mathematical analysis, Rutgers Law Review 19, 1965, 317–343.
• [6] Bazan J., Szczuka M., Wróblewski J.: A new version of rough set exploration system, Lecture Notes in Computer Sciences, 2475, 2002, 397–404.
• [7] Bruha I., Tkadlec J.: Rule quality for multiple–rule classifier - Empirical expertise and theoretical methodology, Intelligent Data Analysis 7, 2003, 99–124.
• [8] Brzezińska I., Słowiński R., Greco S.: Mining Pareto–optimal rules with respect to support and confirmation or support and anti–support, Engineering Applications of Artificial Intelligence 20(5), 2007, 587–600.
• [9] Chateaneuf A., Jaffray J.: Some characterizations of Lower Probabilities and other Monotone Capacities through the use of Mbius Inversion, Mathematical Social Science 17, 1989, 263–283.
• [10] Clark P., Niblett T.: The CN2 induction algorithm, Machine Learning 3, 1989, 261–283.
• [11] Cohen W.W.: Fast effective rule induction, Proc. of the twelfth Int. Conference ICML95, 1995, 115–123.
• [12] Dembczyński K., Kotłowski W., Słowiński R.: ENDER a statistical framework for boosting decision rules, Data Mining and Knowledge Discovery 21, 2010, 52–90.
• [13] Derrac J., Garcia S., Herrera F.: IFS-CoCo: Instance and feature selection based on cooperative coevalution with nearest neighbor rule, Pattern Recognition 43(6), 2010, 2083–2105.
• [14] Duch W., Adamczak R., Grabczewski K.: A New methodology of extraction, optimization and application of crisp and fuzzy logical rules, IEEE Transaction on Neural Networks 11(2), 2000, 1–31.
• [15] Eisen M.B., Spellman P.T., Brown P.O., Botstein D.: Cluster analysis and display of genome-wide expression patterns, . Proc. Natl. Acad. Sci. USA 95, 1998, 14863–14868.
• [16] Fayyad U.M., Piatetsky-Shapiro G., Smyth P., Uthurusamy R.: From data mining to knowledge discovery, in: Advances in knowledge discovery and data mining , Cambridge, MIT–Press, 1996, 37–58.
• [17] Freitas A.A.: On rule interestingness measures, Knowledge-Based Systems 12(5–6), 1999, 309–315.
• [18] Fürnkranz, J.: Separate-and-conquer rule learning, Artificial Intelligence Review 13, 1999, 3–54.
• [19] Fürnkranz J., Flach P.A.: ROC ’n’ Rule Learning – Towards a Better Understanding of Covering Algorithms, Machine Learning 58, 2005, 39–77.
• [20] Geng L., Hamilton H.J.: Interestingness measures for data mining A survey, ACM Computing Surveys 38(3) Article 9, 2006, 1–32.
• [21] Grabisch M.: k-order additive discrete fuzzy measures and their representation, Fuzzy Sets and Systems 89, 1997, 445–456.
• [22] Greco S., Matarazzo B., Słowiński R.: Fuzzy measures as a technique for rough set analysis, Proc. 6 th European Congress on Intelligent Techniques & Soft Computing, 1998, 99–103.
• [23] Greco S., Matarazzo B., Słowiński R.: The use of rough sets and fuzzy sets in MCDM, in: Advances in Multiple Criteria Decision Making (T. Gal, T. Hanne and T. Stewart Eds.), Kluwer Academic Publishers, 1999, 14.1–14.59.
• [24] Greco S., Matarazzo B., Słowiński R., Stefanowski J.: Importances interaction of conditions in decision rules, Lecture Notes in Artificial Intelligence 2475, 2002, 255-262.
• [25] Greco S., Słowiński R., Stefanowski J.: Evaluating importances of conditions in the set of discovered rules, Lecture Notes in Artificial Intelligence 4482, 2007, 314-321.
• [26] Grzymała-Busse J.W., Ziarko W.: Data mining based on rough sets, in: Data Mining Opportunities and Challenges (J. Wang Ed.), IGI Publishing, Hershey, PA, USA, 2003, 142-173.
• [27] G´ora G., Wojna A.G.: RIONA: a new classification system combining rule induction and instance-based learning, Fundamenta Informaticae, 51(4), 2002, 369–390.
• [28] Guillet F., Hamilton H.J: Quality measures in data mining, Studies in Computational Intelligence , 43 Springer-Verlag Berlin, Heidelberg, 2007.
• [29] Hidenao A., Tsumoto S.: Analyzing behavior of objective rule evaluation indices based on Pearson productmoment correlation coefficient, Lecture Notes in Artificial Intelligence 4994, 2008, 84–89.
• [30] Huynh X.H., Guillet F., Blanchard J., Kuntz P., Briand H., Gras R.: A Graph-based Clustering Approach to Evaluate Interestingness Measures: A Tool and a Comparative Study, in: [27].
• [31] Iyer V.R., Eisen M.B., Ross D.T., Moore G.T., Lee J.C., et al.: The transcriptional program in the response of human fibroblasts to serum, Science 283, 1999, 83–87.
• [32] Janssen F., Fürnkranz J.: On the quest for optimal rule learning heuristics, Machine Learning 78, 2010, 343–379.
• [33] Janusz A.: Discovering rules-based similarity in microarray data, Lecture Notes in Artificial Intelligence 6178, 2010, 49–58.
• [34] Kałwak K., Porwolik J., Mielcarek M., Gorczyńska E., et al.: Higher CD34+ and CD3+ cell doses in the graft promote long-term survival, and have no impact on the incidence of serve acute or chronic Graft-versus-host disease after in vivo T cell-depleted unrelated donor hematopoietic stem cell transplantation in children, American Society for Blood and Marrow Transplantation. Biology of Blood Marrow Transplantation 16, 2010, 1388–1401.
• [35] Kavsek B., Lavrac N.: APRIORI-SD: Adapting association rule learning to subgroup discovery, Applied Artificial Intelligence 20, 2006, 543–583.
• [36] Lavrac N., Kavsek B., Flach P.: Subgroup discovery with CN2-SD, Journal of Machine Learning Research 5, 2004, 153–188.
• [37] Li J., Cercone N.: A method of discovering important rules using rules as attributes, International Journal of Intelligent Systems 25(2), 2010, 180–206.
• [38] McGarry K.: A survey of interestingness measures for knowledge discovery, The Knowledge Engineering Review 20(1), 2005, 39–61.
• [39] Michalski R. S., Mozetic I., Hong J., Lavrac N.: The AQ15 inductive learning system: An overview and experiments, ISG Report No. 20. , Department of Computer Sciences, University of Illinois at Urbana-Champaign, 1986.
• [40] Michalski, R. S., Bratko, I. Kubar, M.: Machine learning and data mining , John Wiley and Sons, 1998.
• [41] Mrozek A.: Rough Sets in Computer Implementation of Rule-based Control of Industrial Processes, in: Intelligent Decision Support: Handbook of Applications and Advances of the Rough Set Theory (Słowiński R. Ed.), Kluwer Academic Publishers, Dordrecht, 1992, 19–31.
• [42] Nguyen H. S., Nguyen S. H.: Some efficient algorithms for rough set methods, Proceedings of the Sixth International Conference, Information Processing and Management of Uncertainty in Knowledge-Based Systems 2, July 1-5, Granada, Spain, 1996, 1451–1456.
• [43] Ohsaki M., Hidenao A., Tsumoto S., Yokoi H., Yamaguchi T.: Evaluation of rule interestingness measures in medical knowledge discovery in databases, Artificial Intelligence in Medicine 41, 2007, 127–196.
• [44] Pawlak Z.: Rough sets: Theoretical aspects of reasoning about data , Dordrecht Kluwer, 1991.
• [45] Robnik-Sikonja M., Kononenko I.: Explaining classifications for individual instances, IEEE Trans. On Knowledge and Data Engineering 20, 2008, 589–600.
• [46] Shaharanee I.Z.M., Hadzic F., Dillon T.S.: Interestingness measures for association rules based on statistical validity, Knowledge-Based Systems 23(3), 2011, 386–389.
• [47] Shapley L. S., A value for n-person games, in: Contributions to the Theory of Games II (H. W. Kuhn, A. W. Tucker Eds.), Princeton University Press, Princeton, 1953, 307317.
• [48] Sikora M.: Decision rules based data models using TRS and NetTRS – methods and algorithms, Transaction on Rough Sets XI, LNCS 5946, 2010, 130–160.
• [49] Sikora M., Gruca A.: Quality improvement of rule-based gene group descriptions using information about GO terms importance occurring in premises of determined rules, International Journal of Applied Mathematics and Computer Sciences 20(3), 2010, 555–570.
• [50] Sikora M., Gruca A.: Induction and selection of the most interesting Gene Ontology based multiattribute rules for descriptions of gene groups, Pattern Recognition Letters 32, 2011, 258–269.
• [51] Sikora M.: Induction and pruning of classification rules for prediction of microseismic hazards in coal mines, Expert Systems with Applications 38(6), 2011, 6748–6758.
• [52] Sikora, M., Wróbel, Ł.: Data-driven adaptive selection of rule quality measures for improving the rules induction algorithm, Lecture Notes in Artificial Intelligence 6743, 2011, 278–285.
• [53] Sikora M, Wróbel Ł.: Data-driven Adaptive Selection of rule quality measures for improving rule induction and filtration algorithms, International Journal of General Systems (to appear).
• [54] Skowron A., Rauszer C.: The Discernibility Matrices and Functions in Information systems, in: Intelligent Decision Support. Handbook of Applications and Advances of the Rough Sets Theory (R. Słowiński Ed.), Dordrecht, Kluwer, 1992, 331–362.
• [55] Skowron A., Wang, H., Wojna A., Bazan J.G.: A Hierarchical Approach to Multimodal Classification, Lecture Notes in Artificial Intelligence 3642, 2005, 119–127.
• [56] Stefanowski J.: Rough set based rule induction techniques for classification problems, Proc. of the 6th European Congress of Intelligent Techniques and Soft Computing vol.1 , Achen, Germany, Sept. 7-10, 1998, 107–119.
• [57] Stefanowski J., Vanderpooten D.: Induction of Decision Rules in Classification and Discovery-Oriented Perspectives, International Journal of Intelligent Systems 16, 2001, 13–27.
• [58] Stefanowski J.: The Bagging and n2-Classifiers Based on Rules Induced by MODLEM, Lecture Notes in Computer Sciences 3066, 2004, 488–497.
• [59] Strumbelj E., Kononeko I.: An efficient explanation of individual classifications using game theory, Journal of Machine Learning Research 11, 2010, 1–18.
• [60] Suzuki E.: Pitfalls for Categorizations of Objective Interestingness Measures for Rule Discovery, in: Statistical Implicative Analysis: Theory and Applications (R. Gras, E. Suzuki, F. Guillet, F. Spagnolo Eds.), Springer-Veralg, 2008, 383395.
• [61] Tsumoto S., Hirano S.: Visualization of similarities and dissimilarities in rules using multidimensional scaling, Lecture Notes in Artificial Intelligence 3488, 2005, 38–46.
• [62] Webb G.I.: Further experimental evidence against the utility of Occam‘s razor, Journal of Artificial Intelligence Research 4, 1996, 397–417.
• [63] Webb G.I.: Discovering significant patterns, Machine Learning 68(1), 2007, 1–33.
• [64] Witten I.H., Frank E.: Data mining: practical machine learning tools and techniques, Morgan Kaufmann 2005.
• [65] Wnek J., Michalski R.S.: Hypothesis-driven Constructive Induction in AQ17-HCI: A Method and Experiments, Machine Learning , 14(2), 1994, 139–168.
• [66] Yao Y., Zhou B.: Micro and Macro Evaluation of Classification Rules, Proc. 7th IEEE Conf. on Cognitive Informatics, 2008, 441–448.