Recursive-rule extraction algorithm with J48graft and applications to generating credit scores

• Autorzy: Tanaka Y. , Takagi T. , Saito T. , Iiduka H. , Kikuchi H. , Bologna G. , Mitra S.

Identyfikatory

DOI

10.1515/jaiscr-2016-0004

• Języki publikacji: EN

Abstrakty

EN

The purpose of this study was to generate more concise rule extraction from the Recursive- Rule Extraction (Re-RX) algorithm by replacing the C4.5 program currently employed in Re-RX with the J48graft algorithm. Experiments were subsequently conducted to determine rules for six different two-class mixed datasets having discrete and continuous attributes and to compare the resulting accuracy, comprehensibility and conciseness. When working with the CARD1, CARD2, CARD3, German, Bene1 and Bene2 datasets, Re-RX with J48graft provided more concise rules than the original Re-RX algorithm. The use of Re-RX with J48graft resulted in 43.2%, 37% and 21% reductions in rules in the case of the German, Bene1 and Bene2 datasets compared to Re-RX. Furthermore, the Re-RX with J48graft showed 8.87% better accuracy than the Re-RX algorithm for the German dataset. These results confirm that the application of Re-RX in conjunction with J48graft has the capacity to facilitate migration from existing data systems toward new concise analytic systems and Big Data.

Słowa kluczowe

EN

Rule Extraction; Credit Scoring; Re-RX algorithm; J48graft

• Wydawca: University of Social Sciences
• Czasopismo: Journal of Artificial Intelligence and Soft Computing Research
• Rocznik: 2016
• Tom: Vol. 6, No. 1
• Strony: 35--44
• Opis fizyczny: Bibliogr. 33 poz., tab.

Twórcy

autor

Tanaka Y.

• Department of Computer Science, Meiji University Kawasaki 214-8571, Japan

autor

Takagi T.

• Department of Computer Science, Meiji University Kawasaki 214-8571, Japan

autor

Saito T.

• Department of Computer Science, Meiji University Kawasaki 214-8571, Japan

autor

Iiduka H.

• Department of Computer Science, Meiji University Kawasaki 214-8571, Japan

autor

Kikuchi H.

• Department of Frontier Media Science,,Meiji University Nakano-ku, Tokyo 164-8525, Japan

autor

Bologna G.

• Department of Information Technology, University of Applied Sciences of Western Switzerland Rue de la prairie 4, 1204 Geneva, Switzerland

autor

Mitra S.

• Sushmita Mitra Machine Intelligence Unit, Indian Statistical Institute 203 B.T. Road, Kolkata 700 108, India

Bibliografia

• [1] Marqus, A.I., Garca, V., & Snchez, J.S. On the suitability of resampling techniques for the class imbalance problem in credit scoring. Journal of the Operational Research Society 64, pp. 1060–1070, 2013.
• [2] Zhao Z., Xu, S., Kang, B. H., Kabir, M. M. J., & Liu, Y. Investigation and improvement of multi layer perceptron neural networks for credit scoring Expert Systems with Applications 42, pp. 3508–3516, 2015
• [3] Finlay, S. M. Multiple classifier architectures and heir applications to credit risk assessment. European Journal of Operational Research 210, pp. 368–378, 2011.
• [4] Quinlan, J.R. Programs for Machine Learning. Morgan Kaufmann Series in Machine Learning, San Mateo, CA, 1993, Morgan Kaufman.
• [5] Martens, D., Baesens, B., Van Gestel, T., & Vanthienen, J. Comprehensible credit scoring models using support vector machines. European Journal of Operational Research 183, pp. 1497–1488, 2007.
• [6] Baesens, B., et al. Using neural network rule extraction and decision tables for credit-risk evaluation. Management Science 49, No. 3, pp. 312–329, 2004.
• [7] Abellan, J., and Mantas, C. Improving experimental studies about ensembles of classifiers for bankruptcy prediction and credit scoring. Expert Systems with Applications, pp. 3825–3830, 2014.
• [8] Finlay, S. M. Multiple classifier architectures and their applications to credit risk assessment. European Journal of Operational Research 210, pp. 368–378, 2011.
• [9] Braket, N., and Bradely, A.P. Rule extraction from support vector machine: a review. Neurocomputing 74, pp. 178–190, 2010.
• [10] Setiono, R., and Liu, H. Neurolinear: From neural networks to oblique decision rules. Neurocomputing 17, No. 1, pp. 1–24, 1997.
• [11] Setiono, R. and Liu, H. A connectionist approach to generating oblique decision trees. IEEE Trans. Syst., Man, Cybern. B, Cybern. Vol. 29, No. 3, pp. 440–444, Jun. 1999.
• [12] Setiono, R., Baesens, B. & Mues, C. A note on knowledge discovery using neural Setiono networks and its application to credit card screening. European Journal of Operational Research 192, pp.326-332, 2009.
• [13] Setiono, R., et al. Recursive neural network rule extraction for data with mixed attributes. IEEE Trans. Neural Netw. 19, No. 2, pp. 299–307, 2008.
• [14] Setiono, R., and Liu, H. Symbolic representation of neural networks. IEEE Computer 29, No. 3, pp. 71–77, 1996.
• [15] Gupta, A, Park, S. and Lam, S.M. Generalized analytic rule extraction for feedforward neural networks, IEEE Trans. Knowledge and Data Engineering, 11, pp.985-991, 1999.
• [16] Etchell, T.A. and Lisboa, J.P.G., Orthogonal search-based rule extraction (OSRE) for trained neural-networks: A practical and efficient approach, IEEE Trans. Neural Networks 17, pp.374-384, 2006.
• [17] Hansen, L.K., and Salamon, P., Neural network ensembles. IEEE Trans. Patter Analysis and Machine Intelligence 12, pp. 993–1001, 1990.
• [18] Igelnik, S., Pao, Y.-H., LeClair, S. R., and Shen, C. Y. The ensemble approach to neural-network learning and generalization. IEEE Trans. Neural Networks 10, pp. 19–30, 1999.
• [19] Liao, J.-J., Shih, C.-H., Chen, T.-F., and Hsu, M.-F. An example-based model for two-class imbalanced financial problem. Economic Modelling 37, pp. 175–183, 2014.
• [20] Setiono R. et al., Rule extraction from minimal neural networks for credit card screening, Inter. J. of Neural Systems., Vol. 21, No. 4, pp. 265–276, 2011.
• [21] Setiono R. et al., A note on knowledge discovery using neural networks and its application to credit card screening, European J. Operational Research, Vol. 192, pp. 326–332, 2009.
• [22] Hayashi Y. et al., Understanding consumer heterogeneity: A business intelligence application of neural networks, Knowledge-Based Systems, Vol. 23, No. 8, pp. 856–863, 2010.
• [23] Bologna G., Is it worth generating rules from neural network ensemble?, J. of Applied Logic, Vol. 2, pp. 325–348, 2004.
• [24] Zhou, Z.-H. Extracting symbolic rules from trained neural network ensembles. AI Communications 16, pp. 3–15, 2003.
• [25] http://fiji.sc/javadoc/weka/classifiers/trees/J48graft.html
• [26] Setiono R. et al., A penalty-function approach for pruning feedforward neural networks, Neural Comp., Vol. 9, No. 1, pp. 185–204, 1997.
• [27] Witten, I.H. and Frank, E., Data Ming: Practical Machine Learning Tools With Java Implementations. San Francisco, CA: Morgan Kaufmann, 1999.
• [28] Quinlan J.R., Induction of decision trees, Machine Learning, Vol.1, pp.81-106, 1986.
• [29] Webb. G.I., Decision Tree Grafting from the All-Tests-But-One Partition, in Proc. 16th International Joint Conference on Artificial Intelligence (IJCAI), Vol. 2, pp. 702–707, 1999.
• [30] Webb, G. I., Decision Tree Grafting, Learining, IJCAI’ 97 Proceedings of the 15th International Conference on Artificial Intelligence, Vol.2, pp. 846-885, 1997.
• [31] Frank, A. & Asuncion, A. University of California. Irvine Machine Learning Repository. http://archive.ics.uci.edu/ml/, 2010
• [32] Prechelt, L. Proben1 — A set of benchmarks and benchmarking rules for neural network training algorithms, Technical Report 21/94, Fakultt fr Informatik, Universitt Karlsruhe, Germany. Anonymous ftp available from ftp://pub/papers/techreport/1994/1994-21.ps.gz on ftp.ira.uka.de , 1994.
• [33] Smith, M. Neural Networks for Statistical Modeling, New York: Van Nostrand Reinhold, 1993.

Uwagi

PL

Opracowanie ze środków MNiSW w ramach umowy 812/P-DUN/2016 na działalność upowszechniającą naukę.