Defect Backlog Size Prediction for Open-Source Projects with the Autoregressive Moving Average and Exponential Smoothing Models

• Autorzy: Anioła Paulina , Pandey Sushant Kumar , Staron Miroslaw , Ochodek Mirosław

Identyfikatory

DOI

10.15439/2023F5474

• Języki publikacji: EN

Abstrakty

EN

Context: predicting the number of defects in a defect backlog in a given time horizon can help allocate project resources and organize software development. Goal: to compare the accuracy of three defect backlog prediction methods in the context of large open-source (OSS) projects, i.e., ARIMA, Exponential Smoothing (ETS), and the state-of-the-art method developed at Ericsson AB (SM). Method: we perform a simulation study on a sample of 20 open-source projects to compare the prediction accuracy of the methods. Also, we use the Na\"{\i}ve prediction method as a baseline for sanity check. We use statistical inference tests and effect size coefficients to compare the prediction errors. Results: ARIMA, ETS, and SM were more accurate than the Na\"{\i}ve method. Also, the prediction errors were statistically lower for ETS than for SM (however, the effect size was negligible). Conclusions: ETS seems slightly more accurate than SM when predicting defect backlog size of OSS projects.

Słowa kluczowe

EN

computer science; smoothing methods; computational modeling; predictive models; software; autoregressive process

PL

informatyka; wygładzanie; modelowanie obliczeniowe; modele predykcyjne; oprogramowanie; proces autoregresyjny

• Wydawca: Polskie Towarzystwo Informatyczne
• Czasopismo: Annals of Computer Science and Information Systems
• Rocznik: 2023
• Tom: Vol. 35
• Strony: 83--92
• Opis fizyczny: Bibliogr. 17 poz., wz., tab., wykr.

Twórcy

autor

Anioła Paulina

autor

Pandey Sushant Kumar

• Dept. of CSE Chalmers
• University of Gothenburg, Sweden

• https://orcid.org/0000-0003-1882-2435

autor

Staron Miroslaw

• Dept. of CSE Chalmers
• University of Gothenburg, Sweden

• https://orcid.org/0000-0002-9052-0864

autor

Ochodek Mirosław

• Poznan University of Technology, ul. Piotrowo 2, 60-695 Poznan, Poland

• https://orcid.org/0000-0002-9103-717X

Bibliografia

• 1. P. Sielicka, “Defect backlog size prediction with the autoregressive moving average and exponential smoothing models,” Master’s thesis, Poznan University of Technology, Poland, 2019.
• 2. W. Meding, “Effective monitoring of progress of agile software development teams in modern software companies: an industrial case study,” in Proceedings of the 27th International Workshop on Software Measurement and 12th International Conference on Software Process and Product Measurement, 2017, pp. 23–32.
• 3. M. Staron and W. Meding, “A method for forecasting defect backlog in large streamline software development projects and its industrial evaluation,” Information and Software Technology, vol. 52, no. 10, pp. 1069–1079, 2010.
• 4. M. Staron and W. Meding, “Defect inflow prediction in large software project,” e-Informatica Software Engineering Journal, vol. 4, no. 1, pp. 89–107, 2010.
• 5. M. Staron and W. Meding, “Predicting weekly defect inflow in large software projects based on project planning and test status,” Information and Software Technology, vol. 50, no. 7, pp. 782–796, 2008.
• 6. R. Rana, M. Staron, C. Berger, and J. Hansson, “Analysing defect inflow distribution of automotive software projects.” PROMISE ’14’ - 10th International Conference on Predictive Models in Software Engineering, 2013.
• 7. G. E. Box, G. M. Jenkins, G. C. Reinsel, and G. M. Ljung, Time series analysis: forecasting and control. John Wiley & Sons, 2015.
• 8. C. C. Holt, “Forecasting seasonals and trends by exponentially weighted moving averages,” International journal of forecasting, vol. 20, no. 1, pp. 5–10, 2004.
• 9. P. R. Winters, “Forecasting sales by exponentially weighted moving averages,” Management science, vol. 6, no. 3, pp. 324–342, 1960.
• 10. R. J. Hyndman and G. Athanasopoulos, Forecasting: principles and practice, 2nd edition. OTexts: Melbourne, Australia, 2013, oTexts.com/fpp2. Accessed on 03.06.2019.
• 11. R. J. Hyndman and Y. Khandakar, “Automatic time series forecasting: the forecast package for R,” Journal of Statistical Software, vol. 26, no. 3, pp. 1–22, 2008.
• 12. R. Rana, “Software defect prediction techniques in automotive domain: Evaluation, selection and adoption,” Ph.D. dissertation, University of Gothenburg, 2015.
• 13. H. D. T.J. Yu, V.Y. Shen, “An analysis of several software defect models,” IEEE Transactions on Software Engineering, vol. 14, no. 9, pp. 1261 – 1270, 1998.
• 14. B. Bernard Nicolau de França and G. Horta Travassos, “Simulation based studies in software engineering: A matter of validity,” CLEI electronic journal, vol. 18, no. 1, pp. 5–5, 2015.
• 15. M. Shepperd and S. MacDonell, “Evaluating prediction systems in software project estimation,” Information and Software Technology, vol. 54, no. 8, pp. 820–827, 2012.
• 16. B. Kitchenham, L. Madeyski, D. Budgen, J. Keung, P. Brereton, S. Charters, S. Gibbs, and A. Pohthong, “Robust statistical methods for empirical software engineering,” Empirical Software Engineering, vol. 22, no. 2, 2017.
• 17. C. Wohlin and et al, Experimentation in Software Engineering: An Introduction. Kluwer Academic Publisher, Boston, MA, 2000.

Uwagi

1. Main Track Regular Papers

2. Opracowanie rekordu ze środków MEiN, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2024).