Accelerating the relevance vector machine via data partitioning

• Autorzy: Ben-Shimon D. , Shmilovici A.
• Konferencja: ADBIS Workshop on Data Mining and Knowledge Discovery (ADMKD'2005) / sympozjum [1st; September 15-16, 2005; Tallinn, Estonia]
• Języki publikacji: EN

Abstrakty

EN

The Relevance Vector Machine (RVM) is a method for training sparse generalized linear models, and its accuracy is comparably to other machine learning techniques. For a dataset of size N the runtime complexity of the RVM is O(NJ) and its space complexity is O(N2) which makes it too expensive for moderately sized problems. We suggest three different algorithms which reduce the runtime complexity to O(N") via partitioning the dataset into small chunks of size P. A heuristic is presented for selecting the chunk size. Extensive experiments with benchmark datasets indicate that the partition algorithms can significantly reduce the complexity of the RVM while retaining the attractive attributes of the original solution.

Słowa kluczowe

EN

machine learning; data mining; relevance vector machine

• Wydawca: Wydawnictwo Politechniki Poznańskiej
• Czasopismo: Foundations of Computing and Decision Sciences
• Rocznik: 2006
• Tom: Vol. 31, No. 1
• Strony: 27--41
• Opis fizyczny: Bibliogr. 7 poz.

Twórcy

autor

Ben-Shimon D.

autor

Shmilovici A.

• Department of Information Systems Engineering, Ben-Gurion University, P.O. Box 653, 84105 Beer-Sheva, Israel,

Bibliografia

• [1] Cristianini N., Shawe-Taylor J., An Introduction to Support Vector Machines and other kernel-based learning methods, Cambridge University Press, New York. NY, 1999.
• [2] Down T.A., Hubbard T.J.P., Relevance Vector Machines for Classifying Points and Regions in Biological Sequences, Wellcome Trust, Sanger Institute, 2003.
• [3] Rasmussen C. E., Quinonero-Candela J., Healing the Relevance Vector Machine through Augmentation. Proceedings of the 22nd International Conference on Machine Learning, August 7-11, Bonn, Germany, 2005.
• [4] D'Souza A., Vijayakumar S., Schaal S., The Bayesian Backfitting Relevance Vector Machine, Proceedings of the 21st International conference on Machine Learning, July 4-8, Baniff, Canada, 2004.
• [5] Tipping M. E., Sparse Bayesian Learning and the Relevance Vector Machine. Journal of Machine Learning Research 1. 2001,211 -244.
• [6] Tipping M. E., Paul A., Fast Marginal Likelihood Maximization for Sparse Bayesian Models. Proceedings of the 9th International workshop on Artificial Intelligence and Statistics, January 3-6, Key West. Florida, 2003.
• [7] Wipf D., Palmer J., Rao B., Perspectives on Sparse Bayesian Learning. Advances in Neural Information Processing systems, 16. Cambridge, Massachussettes, MIT Press, 2004.