Feature Selection Algorithm for Multiple Classifier Systems: A Hybrid Approach

• Autorzy: Delimata P. , Suraj Z.
• Języki publikacji: EN

Abstrakty

EN

Many problems in pattern classification and knowledge discovery require a selection of a subset of attributes or features to represent the patterns to be classified. The approach presented in this paper is designed mostly for multiple classifier systems with homogeneous (identical) classifiers. Such systems require many different subsets of the data set. The problem of finding the best subsets of a given feature set is of exponential complexity. The main aim of this paper is to present ways to improve RBFS algorithm which is a feature selection algorithm. RBFS algorithm is computationally quite complex because it uses all decision-relative reducts of a given decision table. In order to increase its speed, we propose a new algorithm called ARS algorithm. The task of this algorithm is to decrease the number of the decision-relative reducts for a decision table. Experiments have shown that ARS has greatly improved the execution time of the RBFS algorithm. A small loss on the classification accuracy of the multiple classifier used on the subset created by this algorithm has also been observed. To improve classification accuracy the simplified version of the bagging algorithm has been applied. Algorithms have been tested on some benchmarks.

Słowa kluczowe

EN

feature selection; multiple classifier systems; reduct; diversity measures

• Wydawca: IOS Press
• Czasopismo: Fundamenta Informaticae
• Rocznik: 2008
• Tom: Vol. 85, nr 1-4
• Strony: 97--110
• Opis fizyczny: bibliogr. 34 poz., tab., wykr.

Twórcy

autor

Delimata P.

autor

Suraj Z.

• Chair of Computer Science, University of Rzeszów, Rejtana 16A, 35-310Rzeszów, Poland,

Bibliografia

• [1] Bazan, J. G., Szczuka, M., Wojna, A. G., Wojnarski, M.: On the evolution of Rough Set Exploration System, in: Proceedings of the Fourth International Conference on Rough Sets and Current Trends in Computing, volume 3066 of Lectures Notes in Artificial Intelligence, pp. 592 - 601, Uppsala, Sweden, 2004. Springer-Verlag.
• [2] Blake, C.L., Merz, C.J.: UCI repository of machine learning databases, Department of Information and Computer Science, University of California, Irvine, CA, 1998. (http://www.ics.uci.edu/mlearn/mlrepository.html)
• [3] Breiman, L., Bagging predictors, Machine Learning, 24 (2), 1996, pp. 123 - 140.
• [4] Chan, P.K., Stolfo, S.J.: A comparative evaluation of voting and meta-learning on partitioned data, in: Proceedings of the 12th International Conference on Machine Learning, San Francisco, 1995, pp. 90 - 98.
• [5] Chan, P.K., Stolfo, S.J.: Experiments on multistrategy learning by meta-learning, in: Proceedings of the Second International Conference on Information and Knowledge Management, 1993, pp. 314 - 323.
• [6] Cios, K.J., Pedrycz,W., ´Swiniarski, R.W.: DataMining.Methods for Knowledge Discovery, Kluwer Academic Publishers, Dordrecht 1998.
• [7] Dasarathy, B., Sanchez, J., Townsend, S.: Nearest Neighbour Editing and Condensing Tools - Synergy Exploitation, Pattern Analysis and Applications 3(1), 2000, pp. 19 - 30.
• [8] Delimata, P., Suraj, Z.: On k-NN Method with Preprocessing. Fundamenta Informaticae, vol. 69 (3), IOS Press, Amsterdam, 2006, pp. 343 - 358.
• [9] Foroutan, I., Sklansky, J.: Feature selection for automatic classification of non-gaussian data, IEEE Transactions on Systems, Man and Cybernetics, 17:187 - 198, 1987.
• [10] Fukunaga, K.: Introduction to statistical pattern recognition, Academic Press, New York, 1990.
• [11] Haindl, M., Kittler, J., Roli, F., (Eds.), Proceedings of the Multiple Classifier Systems 7th International Workshop, MCS 2007, Prague, Czech Republic, May 23-25, 2007, Lecture Notes in Computer Science, Vol.4472.
• [12] Ilczuk, G., Wakulicz-Deja, A.: Attribute Selection and Rule Generation Techniques for Medical Diagnosis Systems, in: Proceedings of the RSFDGrC, (2) 2005, pp. 352 - 361.
• [13] Liu, H., Motoda, H.: Feature Extraction, Construction and Selection: A Data Mining Perspective. Springer, Berlin 1998.
• [14] Liu, H., Motoda, H.: Feature Selection for Knowledge Discovery and Data Mining. Springer, Berlin 1998.
• [15] Michalski, R.S., Tecuci, G.: A multistrategy approach, Machine Learning, Volume IV. Morgan Kaufmann, 1994.
• [16] Mitche D., Spiegielhalter C.J., Taylor C.: Neural and Statistical Classification, Machine Learning, Ellis Horwood 1994.
• [17] Motwani, R., Raghavan, P.: Randomized algorithms, ACM Computing Surveys, 28(1):33 - 37, 1996.
• [18] Pawlak, Z.: Rough sets: Theoretical aspects of reasoning about data, Kluwer Academic Publishers, Dordrecht 1991.
• [19] Pawlak, Z., Skowron, A.: Rough sets and boolean reasoning. Information Sciences 177(1), pp. 41 - 73
• [20] Pawlak, Z., Skowron, A.: Rough sets: Some extensions. Information Sciences 177(1), pp. 28 - 40.
• [21] Pawlak, Z., Skowron, A.: Rudiments of rough sets, Information Sciences, 2007, 177(1), pp. 3 - 27.
• [22] Pramod Kumar Sharma, Multiple Classifiers for Unconstrained Offline Handwritten Numeral Recognition, Proceedings of the International Conference on Computational Intelligence and Multimedia Applications (ICCIMA 2007) - Vol. 2, pp. 344 - 348.
• [23] Punch,W., Goodman, E., Pei, M., Chia-Shun, L., Hovland, P., Enbody, R.: Further research on feature selection and classification using genetic algorithms, in: Proceedings of the International Conference on Genetic Algorithms, pp. 557 - 564, Springer 1993.
• [24] Sheinvald, J., Dom, B., Niblack,W.: A modelling approach to feature selection, in: Proceedings of the Tenth International Conference on Pattern Recognition, pp. 535 - 539, 1990.
• [25] Skowron, A., Pal, SK.,(eds) Special volume: Rough sets, pattern recognition and data mining, Pattern Recognition Letters, vol. 24(6), 2003.
• [26] Suraj, Z., Delimata, P.: Data Mining Exploration System for Feature Selection Tasks. In: Proceedings of the International Conference on Hybrid Information Technology (ICHIT 2006), Korea, 2006, IEEE CS, 2006, pp. 284 - 286.
• [27] Suraj, Z., Neamat El Gayar, Delimata, P.: A Rough Set Approach toMultiple Classifier Systems. Fundamenta Informaticae, vol. 72(1-3), IOS Press, Amsterdam, 2006, pp. 393 - 406.
• [28] Stanfill, C., Waltz, D.: Toward memory - based reasonning, Communications of the ACM, vol.29, no.12, pp. 1213 - 1228, 1986
• [29] Stefanowski, J., Nowaczyk, S. On Using Rule Introduction in Multiple Classifiers with a Combiner Aggregation Strategy, Proceedings of the Fifth International Conference on Intelligent Systems Design and Applications (ISDA'2005),Wroclaw, Poland, September 8-10, 2005, pages: 432 - 437.
• [30] Swiniarski, R., Skowron, A., Rough sets methods in feature selection and extraction, Pattern Recognition Letters, 2003, 24(6), pp. 833 - 849.
• [31] Wolpert, D.: Stacked Generalisation, Neural Networks, No. 5, 1992, pp. 241 - 259.
• [32] Wojna, A., Analogy-Based Reasoning in Classifier Construction (full phd dissertation), Transactions on Rough Sets IV, Lecture Notes in Computer Science, 3700, Springer-Verlag 2005, pages 277 - 374.
• [33] Yule, G., U., On the association of attributes in statistics. Phil. Trans., A, 194:257 - 319, 1900.
• [34] Zhi-Hua Zhou, Yang Yu: Ensembling local learners ThroughMultimodal perturbation IEEE Transactions on Systems, Man and Cybernetics, Part B, vol.35, Issue 4, Aug. 2005 pp. 725 - 735.