Performance measurement with high-performance computer using HW-GA anomaly-detection algorithms for streaming data

• Autorzy: Fondaj Jakup , Hasani Zirije , Krrabaj Samedin

Identyfikatory

DOI

10.7494/csci.2022.23.3.4389

• Języki publikacji: EN

Abstrakty

EN

Anomaly detection for streaming real-time data is very important; more significant is the performance of an algorithm in order to meet real-time requirements. Anomaly detection is very crucial in every sector because, by knowing what is going wrong with data/digital systems, we can make decisions to help in every sector. Dealing with real-time data requires speed; for this reason, the aim of this paper is to measure the performance of our proposed Holt–Winters genetic algorithm (HW-GA) as compared to other anomaly-detection algorithms with a large amount of data as well as to measure how other factors such as visualization and the performance of the testing environment affect the algorithm’s performance. The experiments will be done in R with different data sets such as the as real COVID-19 and IoT sensor data that we collected from Smart Agriculture Libelium sensors and e-dnevnik as well as three benchmarks from the Numenta data sets. The real data has no known anomalies, but the anomalies are known in the benchmark data; this was done in order to evaluate how the algorithm works in both situations. The novelty of this paper is that the performance will be tested on three different computers (in which one is a high-performance computer); also, a large amount of data will be used for our testing, as will how the visualization phase affects the algorithm’s performance.

Słowa kluczowe

EN

time-series data; HW-GA; anomaly detection; big streaming data; Numenta; COVID-19 data set; high-performance computer; Libelium sensor data; e-dnevnik

• Wydawca: Wydawnictwa AGH
• Czasopismo: Computer Science
• Rocznik: 2022
• Tom: T. 23 (3)
• Strony: 395--410
• Opis fizyczny: Bibliogr. 15 poz., rys., tab.

Twórcy

autor

Fondaj Jakup

autor

Hasani Zirije

autor

Krrabaj Samedin

Bibliografia

• [1] Contributed packages, CRAN. https://cran.r-project.org/web/packages/.
• [2] Boldt M., Borg A., Ickin S., Gustafsson J.: Anomaly detection of event sequences using multiple temporal resolutions and Markov chains, Knowledge and Information Systems, vol. 62, 2020. doi: 10.1007/s10115-019-01365-y.
• [3] Ekberg J., Ylinen J., Loula P.: Network behaviour anomaly detection using HoltWinters algorithm. In: 2011 International Conference for Internet Technology and Secured Transactions, pp. 627–631, 2011.
• [4] Falcao F., Zoppi T., Barbosa Vieira da Silva C., Santos A., Fonseca B., Ceccarelli A., Bondavalli A.: Quantitative comparison of unsupervised anomaly detection algorithms for intrusion detection. In: SAC ’19: Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing, pp. 318–327, 2019. doi: 10.1145/3297280.3297314.
• [5] Hasani Z.: Robust anomaly detection algorithms for real-time big data: Comparison of algorithms. In: 2017 6th Mediterranean Conference on Embedded Computing (MECO), pp. 1–6, 2017. doi: 10.1109/MECO.2017.7977130.
• [6] Hasani Z.: Anomaly Detection Algorithms for Streaming Data: Performance Comparison, Journal of Computer Science, vol. 16(7), pp. 950–955, 2020. doi: 10.3844/jcssp.2020.950.955.
• [7] Hasani Z., Jakimovski B., Velinov G., Kon-Popovska M.: An Adaptive Anomaly Detection Algorithm for Periodic Data Streams: 19th International Conference, Madrid, Spain, November 21–23, 2018, Proceedings, Part I, pp. 385–397, 2018. doi: 10.1007/978-3-030-03493-1˙41.
• [8] Jankov D., Sikdar S., Mukherjee R., Teymourian K., Jermaine C.: Real-Time High Performance Anomaly Detection over Data Streams: Grand Challenge. In: Proceedings of the 11th ACM International Conference on Distributed and EventBased Systems, p. 292–297, DEBS ’17, Association for Computing Machinery, New York, NY, USA, 2017. doi: 10.1145/3093742.3095102.
• [9] Kasunic M., McCurley J., Goldenson D., Zubrow D.: An Investigation of Techniques for Detecting Data Anomalies in Earned Value Management Data Software Engineering Measurement and Analysis (SEMA), 2011.
• [10] Lavin A., Ahmad S.: Evaluating Real-Time Anomaly Detection Algorithms – The Numenta Anomaly Benchmark. In: 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA), pp. 38–44, 2015. doi: 10.1109/ ICMLA.2015.141.
• [11] Srk: Novel Corona virus 2019 dataset, 2021. https://www.kaggle.com/datasets/ sudalairajkumar/novel-corona-virus-2019-dataset.
• [12] Wang Z., Zhou Y., Li G.: Anomaly detection for machinery by using Big Data Real-Time processing and clustering technique. In: Proceedings of the 2019 3rd International Conference on Big Data Research, p. 30–36, ICBDR 2019, Association for Computing Machinery, New York, NY, USA, 2019. doi: 10.1145/3372454. 3372480.
• [13] Yuanyan L., Xuehui D., Yi S.: Data streams anomaly detection algorithm based on self-set threshold. In: Proceedings of the 4th International Conference on Communication and Information Processing, pp. 18–26, ICCIP ’18, Association for Computing Machinery, New York, NY, USA, 2018. doi: 10.1145/3290420. 3290451.
• [14] Zhang L., Zhao J., Li W.: Online and Unsupervised Anomaly Detection for Streaming Data Using an Array of Sliding Windows and PDDs, IEEE Transactions on Cybernetics, vol. 51(4), pp. 2284–2289, 2019. doi: 10.1109/TCYB.2019.2935066.
• [15] Zhu H., Liu B., Lu Y., Li W., Yu N.: Real-time Anomaly Detection with HMOF Feature. In: ICVIP 2018: Proceedings of the 2018 the 2nd International Conference on Video and Image Processing, pp. 49–54, 2018. doi: 10.1145/3301506. 3301510.

Uwagi

PL

Opracowanie rekordu ze środków MEiN, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2022-2023).