Tenfold bootstrap procedure for support vector machines

• Autorzy: Vrigazova Borislava , Ivanov Ivan

Identyfikatory

DOI

10.7494/csci.2020.21.2.3634

• Języki publikacji: EN

Abstrakty

EN

Cross validation is often used to split input data into training and test sets in support vector machines. The two most commonly used cross validation versions are the tenfold and leave-one-out cross validation. Another commonly used resampling method is the random test/train split. The advantage of these methods is that they avoid overfitting in a model and perform model selection. However, they can increase the computational time for fitting support vector machines by increasing the size of the dataset. In this research, we propose an alternative for fitting SVM, which we call the tenfold bootstrap for support vector machines. This resampling procedure can significantly reduce execution time despite the large number of observations, while preserving a model’s accuracy. With this finding, we propose a solution to the problem of slow execution time when fitting support vector machines on big datasets.

Słowa kluczowe

EN

support vector machine; bootstrap; cross validation

• Wydawca: Wydawnictwa AGH
• Czasopismo: Computer Science
• Rocznik: 2020
• Tom: T. 21 (2)
• Strony: 253--268
• Opis fizyczny: Bibliogr. 29 poz., tab.

Twórcy

autor

Vrigazova Borislava

• Sofia University, Bulgaria

autor

Ivanov Ivan

• Sofia University, Bulgaria

Bibliografia

• [1] Bootstrapped 0.0.2. https://pypi.org/project/bootstrapped/. Accessed: 2019– 12–15.
• [2] Barboza F., Kimura H., Altman E.: Machine Learning Models and Bankruptcy Prediction, Expert Systems with Applications, vol. 83, pp. 405–417, 2017.
• [3] Berrar D.: Introduction to the Non-Parametric Bootstrap. In: Encyclopedia of Bioinformatics and Computational Biology, Volume 1, Elsevier, pp. 766–773, 2019.
• [4] Breiman L.: The Little Bootstrap and Other Methods for Dimensionality Selection in Regression: X-fixed Prediction Error, Journal of American Statistical Association, vol. 87, pp. 738–754, 1994.
• [5] Breiman L.: Better Subset Regression Using the Nonnegative Garrote, Technometrics, vol. 37, pp. 373–384, 1995.
• [6] Chatzis S., Siakoulis V., Petropoulos A., Stavroulakis E., Vlachogiannakis N.: Forecasting stock market crisis events using deep and statistical machine learning techniques, Expert Systems with Applications, vol. 112, pp. 353–371, 2018.
• [7] Cortes C., Vapnik V.: Support-vector networks, Machine Learning, vol. 20, pp. 273–297, 1995.
• [8] Efron B.: Bootstrap Methods: Another Look at the Jackknife, The Annals of Statistics, vol. 7(1), pp. 1–26, 1979. https://www.jstor.org/stable/2958830
• [9] Efron B., Tibshirani R.: Improvements on Cross-Validation: The .632+ Bootstrap Method, Journal of the American Statistical Association, vol. 92, pp. 548–560, 1997.
• [10] Frunza M.C.: Solving Modern Crime in Financial Markets, Academic Press, Chapter 2I – Support Vector Machines, 2016.
• [11] Hoerl A.E., Kennard R.W.: Ridge Regression. Applications to nonorthogonal problems, Technometrics, vol. 12(1), pp. 69–82, 1970.
• [12] James G., Witten D., Hastie T., Tibshirani R.: An Introduction to Statistical Learning, Springer, 2013.
• [13] Khairunnahar L., Hasib M.A., Rezanur R.H.B., Islam M.R., Hosain M.K.: Classification of malignant and benign tissue with logistic regression, Informatics in Medicine Unlocked, vol. 16, 2019.
• [14] Krstajic D., Buturovic L.J., Leahy E., Thomas S.: Cross-validation pitfalls when selecting and assessing regression and classification models, Journal of Cheminformatics, vol. 6, p. 10, 2014.
• [15] Luo X., Zhu X., Lim E.G.: A parametric bootstrap algorithm for cluster number determination of load pattern categorization, Energy, vol. 180, pp. 50–60, 2019.
• [16] MacKinnon J.G.: Bootstrap Inference in Econometrics, The Canadian Journal of Economics, vol. 35, pp. 615–645, 2002.
• [17] Maldonado S., P´erez J., Weber R., Labb´e M.: Feature selection for Support Vector Machines via Mixed Integer Linear Programming, Information Sciences, vol. 279, pp. 163–175, 2014.
• [18] Morais C.L.M., Lima K.M.G., Martin F.L.: Uncertainty estimation and misclassification probability for classification models based on discriminant analysis and support vector machines, Analytica Chimica Acta, vol. 1063, pp. 40–46, 2019.
• [19] Olejnik S., Mills J., Keselman H.: Using Wherry’s Adjusted R2 and Mallow’s Cp for Model Selection from All Possible Regressions, The Journal of Experimental Education, vol. 68, pp. 365–380, 2000.
• [20] Ozcan B., Ozturk I.: Renewable energy consumption-economic growth nexus in emerging countries: A bootstrap panel causality test, Renewable and Sustainable Energy Reviews, vol. 104, pp. 30–37, 2019.
• [21] Tibshirani R.: Regression shrinkage and selection via the lasso: a retrospective, Journal of the Royal Statistical Society Series B (Statistical Methodology), vol. 73(3), pp. 273–282, 2011.
• [22] Vrigazova B.: Nonnegative Garrote as a Variable Selection Method in Panel Data, International Journal of Computer Science and Information Security, vol. 16(1), pp. 95–106, 2018.
• [23] Vrigazova B., Ivanov I.: Optimization of the ANOVA Procedure for Support Vector Machines, International Journal of Recent Technology and Engineering, vol. 8(4), pp. 5160–5165, 2019.
• [24] Vrigazova B., Ivanov I.: The bootstrap procedure in classification problems, International Journal of Data Mining, Modelling and Management, vol. 12, 2020, in press.
• [25] Wong T.-T.: Performance evaluation of classification algorithms by k-fold and leave-one-out cross validation, Pattern Recognition, vol. 48(9), pp. 2839–2846, 2015.
• [26] Xie Z., Chen S.: Exchange rates and fundamentals: A bootstrap panel data analysis, Economic Modelling, vol. 78, pp. 209–224, 2019.
• [27] Zhang Y., Yang Y.: Cross-validation for selecting a model selection procedure, Journal of Econometrics, vol. 1, pp. 95–112, 2015.
• [28] Zou H.: The Adaptive Lasso and Its Oracle Properties, Journal of the American Statistical Association, vol. 101, 2006.
• [29] Zou N., Politis D.N.: Bootstrap seasonal unit root test under periodic variation, Econometrics and Statistics, 2020.