On Certain Limitations of Recursive Representation Model

• Autorzy: Jastrzębski S. , Sieradzki I.

Identyfikatory

DOI

10.4467/20838476SI.16.003.6184

• Języki publikacji: EN

Abstrakty

EN

There is a strong research eort towards developing models that can achieve state-of-the-art results without sacricing interpretability and simplicity. One of such is recently proposed Recursive Random Support Vector Machine (R2SVM) model, which is composed of stacked linear models. R2SVM was reported to learn deep representations outperforming many strong classi-ers like Deep Convolutional Neural Network. In this paper we try to analyze it both from theoretical and empirical perspective and show its important limitations. Analysis of similar model Deep Representation Extreme Learning Machine (DrELM) is also included. It is concluded that models in its current form achieves lower accuracy scores than Support Vector Machine with Radial Basis Function kernel.

Słowa kluczowe

EN

support vector machine; random recursive support vector machine; extreme learning machine; representation learning; stacked generalization

• Wydawca: Wydawnictwo Uniwersytetu Jagiellońskiego
• Czasopismo: Schedae Informaticae
• Rocznik: 2016
• Tom: Vol. 25
• Strony: 37--47
• Opis fizyczny: Bibliogr. 13 poz., rys.

Twórcy

autor

Jastrzębski S.

• Uniwersytet Jagielloński w Krakowie, Polska, Gołębia 24, 31-007 Kraków

autor

Sieradzki I.

• Jagiellonian University in Kraków

Bibliografia

• [1] Bengio Y., Learning deep architectures for AI. Foundations and Trends in Machine Learning, 2009, 2.
• [2] Podolak I.T., Roman A., Theoretical foundations and experimental results for a hierarchical classifer with overlapping clusters. Computational Intelligence, 2013, 29 (2), pp. 357 388.
• [3] Czarnecki W.M., Tabor J., Extreme entropy machines: Robust information theoretic classification. arXiv preprint arXiv:1501.05279, 2015.
• [4] Huang G.B., Zhu Q.Y., Siew C.K., Extreme learning machine: Theory and applications. Neurocomputing, 2006, 70 (1 3), pp. 489-501.
• [5] Tissera M., McDonnell M., Deep extreme learning machines for classification. In: Proceedings of ELM-2014 Volume 1. vol. 3 of Proceedings in Adaptation, Learning and Optimization. Springer International Publishing 2015, pp. 345-354.
• [6] Vinyals O., Jia Y., Deng L., Darrell T., Learning with recursive perceptual representations. In: Advances in Neural Information Processing Systems 25. Curran Associates, Inc. 2012, pp. 2825-2833.47
• [7] Yu W., Zhuang F., He Q., Shi Z., Learning deep representations via extreme learning machines. Neurocomputing, 2015, 149, pp. 308-315.
• [8] Wolpert D.H., Stacked generalization. Neural Networks, 1992, 5, pp. 241-259.
• [9] Cortes C., Vapnik V., Support-vector networks. Machine Learning, 1995, 20 (3), pp. 273-297.
• [10] Tang Y., Deep learning using linear support vector machines. In: In ICML, 2013.
• [11] Lichman M., UCI machine learning repository, 2013.
• [12] Chang C.C., Lin C.J., LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2011, 2, pp. 27:1-27:27.
• [13] Pedregosa F., Varoquaux G., Gramfort A., Michel V., Thirion B., Grisel O., Blondel M., Prettenhofer P., Weiss R., Dubourg V., Vanderplas J., Passos A., Cournapeau D., Brucher M., Perrot M., Duchesnay E., Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 2011, 12, pp. 2825-2830.

Uwagi

PL

Opracowanie ze środków MNiSW w ramach umowy 812/P-DUN/2016 na działalność upowszechniającą naukę (zadania 2017).